The Distributed Dragon Forge: A Hands-On OpenTelemetry Adventure with Quarkus
#java#quarkus#opentelemetry
Markus

Markus @myfear

About: Markus is a Java Champion, former Java EE Expert Group member, founder of JavaLand, reputed speaker at Java conferences around the world, and a very well known figure in the Enterprise Java world.

Location:
Munich, Germany
Joined:
Mar 26, 2024

The Distributed Dragon Forge: A Hands-On OpenTelemetry Adventure with Quarkus

Publish Date: Jul 11
0 0

Distributed Dragon Forge

Most observability tutorials start with something uninspiring: a monolith logging its own method calls. But real microservices don’t live in a vacuum. They talk to each other, rely on external systems, and sometimes fail silently unless you have proper distributed tracing in place.

In this hands-on adventure, you'll build two cooperating Quarkus microservices: a Quest Service that sends a crafting request to a Forge Service. We'll use OpenTelemetry for distributed tracing and Jaeger as our visualization tool. The traces will span service boundaries and show a waterfall of every detail—from start to sword.

Let’s light the forge.

Prerequisites

  • JDK 17+ installed

  • Apache Maven 3.9.x

  • Podman and podman-compose (or Docker)

  • cURL or Postman

If you can’t wait to get started, the project source can be found in my Github repository.

Set Up the World (The Project)

First, we'll create a multi-module Maven project to house our two services.

Create a parent project:

mvn io.quarkus:quarkus-maven-plugin:create \
    -DprojectGroupId=dev.adventure \
    -DprojectArtifactId=quarkus-otel-adventure

cd quarkus-otel-adventure
Enter fullscreen mode Exit fullscreen mode

Remove the generated sources. The parent project doesn’t need them:

rm -rf src
Enter fullscreen mode Exit fullscreen mode

Open the pom.xml and add the packaging tag underneath the version:

<packaging>pom</packaging>
Enter fullscreen mode Exit fullscreen mode

Now we need to create the two microservices using the Quarkus Maven Plugin:

The Forge Service (our downstream worker):

mvn io.quarkus:quarkus-maven-plugin:create \
    -DprojectGroupId=dev.adventure \
    -DprojectArtifactId=forge-service \
    -Dextensions="rest,opentelemetry"
Enter fullscreen mode Exit fullscreen mode

The Quest Service (our upstream client):

mvn io.quarkus:quarkus-maven-plugin:create \
    -DprojectGroupId=dev.adventure \
    -DprojectArtifactId=quest-service \
    -Dextensions="rest,opentelemetry,rest-client"
Enter fullscreen mode Exit fullscreen mode

Your project structure should now look like this:

quarkus-otel-adventure/
├── pom.xml
├── forge-service/
└── quest-service/
Enter fullscreen mode Exit fullscreen mode

Light the Forge (Forge Service)

The forge-service will have one job: to craft a sword, which will take a simulated amount of time.

Replace the contents of forge-service/src/main/java/dev/adventure/GreetingResource.java with the following. We'll rename the file to ForgeResource too.

package dev.adventure;

import io.opentelemetry.api.trace.Span;
import io.opentelemetry.instrumentation.annotations.WithSpan;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;

@Path("/forge")
public class ForgeResource {

    @POST
    @Produces(MediaType.TEXT_PLAIN)
    public String craftSword() throws InterruptedException {
        // This custom span will appear nested inside the JAX-RS span
        workTheBellows();
        Thread.sleep(250); // Simulating hard work
        Span.current().addEvent("Sword is quenched.");
        return "Legendary Sword forged!";
    }

    @WithSpan("heating-the-metal") // Creates a new span for this method
    void workTheBellows() throws InterruptedException {
        Span.current().setAttribute("forge.temperature", "1315°C");
        Thread.sleep(150);
    }
}
Enter fullscreen mode Exit fullscreen mode

  1. Here, @WithSpan creates a detailed child span, and Span.current().addEvent(...) adds a specific log point within the parent span's timeline.

  2. Configure the Forge Service:

# Run on port 8081
quarkus.http.port=8081

# OpenTelemetry Configuration
quarkus.application.name=forge-service
quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317
Enter fullscreen mode Exit fullscreen mode

Begin the Quest (Quest Service)

The quest-service will initiate the action by making a REST call to the forge-service.

The ForgeClient interface will define how we call the forge-service.

package dev.adventure;

import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;

@Path("/forge")
@RegisterRestClient(configKey="forge-api")
public interface ForgeClient {

    @POST
    @Produces(MediaType.TEXT_PLAIN)
    String craftSword();
}
Enter fullscreen mode Exit fullscreen mode

This is the entry point for our adventure. Replace the default GreetingResource and rename the file to QuestResource.

package dev.adventure;

import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import org.eclipse.microprofile.rest.client.inject.RestClient;
import org.jboss.logging.Logger;

@Path("/quests")
public class QuestResource {

    private static final Logger LOG = Logger.getLogger(QuestResource.class);

    @Inject
    @RestClient
    ForgeClient forgeClient;

    @GET
    @Path("/start")
    public String startQuest() {
        LOG.info("Quest starting! We need a sword.");
        String result = forgeClient.craftSword();
        LOG.info("Quest update: " + result);
        return "Quest Started! Acquired: " + result;
    }
}
Enter fullscreen mode Exit fullscreen mode

Configure the Quest Service:

Update quest-service/src/main/resources/application.properties to define the location of the forge and connect to Jaeger.

# OpenTelemetry Configuration
quarkus.application.name=quest-service
quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317

# REST Client Configuration
quarkus.rest-client.forge-api.url=http://localhost:8081
Enter fullscreen mode Exit fullscreen mode

Run the Adventure & See the Magic

Now, let's bring our world to life.

Create a compose-devservices.yml file in the root quarkus-otel-adventure directory.

version: '3'
services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686" # Jaeger UI
      - "4317:4317" # OTLP gRPC receiver
Enter fullscreen mode Exit fullscreen mode

Run Jaeger:

podman-compose up
Enter fullscreen mode Exit fullscreen mode

Run the Microservices:

Open two separate terminals in the project's root directory.

Terminal 1 (Forge):

cd forge-service
quarkus:dev
Enter fullscreen mode Exit fullscreen mode

Terminal 2 (Quest):

cd quest-service
quarkus:dev
Enter fullscreen mode Exit fullscreen mode

Trigger the Quest:

In a third terminal, use curl to start the quest.

curl http://localhost:8080/quests/start
Enter fullscreen mode Exit fullscreen mode

You should get the response:

Quest Started! Acquired: Legendary Sword forged!
Enter fullscreen mode Exit fullscreen mode

Visualize the Trace in Jaeger:

Open your web browser and go to http://localhost:16686.

  • In the "Service" dropdown, select quest-service.

  • Click "Find Traces".

Jaeger View

You'll see a single trace for your /quests/start operation. Clicking on it reveals the entire distributed journey! You'll see a waterfall diagram showing:

  • The total time taken by the quest-service.

  • A child span for the REST client call to forge-service.

  • The time spent inside the forge-service processing the request.

  • Our custom heating-the-metal span, showing exactly how long that sub-task took.

  • The event "Sword is quenched" annotated on the timeline.

This provides a complete, end-to-end view of the request, proving that the trace context was automatically propagated from the quest-service to the forge-service across the network.

What You Learned

You just built a distributed microservice setup with end-to-end observability. Along the way, you learned:

  • How to trace cross-service communication in Quarkus using OpenTelemetry.

  • How to use @WithSpan to annotate custom spans.

  • How span context propagates automatically via the REST client.

  • How to visualize traces with Jaeger.

These same principles apply to any real-world system involving service orchestration, background jobs, or long chains of API calls.

In the world of observability, you are now a blacksmith and a warrior.

Next Quests

  • Read more about Quarkus and Observability

  • Read more about OpenTelemetry and Quarkus

  • Add baggage or trace attributes like quest.id or user.id.

  • Introduce failures and retry logic to trace error handling.

  • Chain more services (e.g., EnchanterService, DeliveryService) and visualize deeper trees.

Your forge is hot. Keep crafting.

Comments 0 total

    Add comment