Collecting and transporting JFR dumps from containerized environments

Authored by: Aditi Srinivas, Gireesh Punathil

This article was originally published on IBM Developer.

JDK Flight Recorder (JFR) was introduced in IBM Semeru Runtimes v11.0.27, v17.0.15 and v21.0.7 releases. JFR provides an industry standard, low overhead, continuous workload monitoring experience to the users of Semeru-based Java workloads that are running in containerized or conventional deployment targets.

In containerized environments like Red Hat OpenShift, diagnosing performance issues in Java applications often requires capturing JFR data. However, collecting and transporting these recordings from ephemeral, isolated containers poses logistical and technical challenges.

Traditional monitoring tools often assume direct or SSH access to the host, which is typically restricted in modern cloud-native environments. In addition, many container environments inhibit persistent volumes, so the diagnostic data needs to be taken out through manual means, before the container is recycled. Without streamlined methods to securely extract and manage JFR files, root cause analysis can be delayed, affecting application reliability and supportability.

To address these challenges, we need a well-defined set of tasks to be carried out to perform the application monitoring and performance analysis seamlessly. This article provides the steps to configure and record a JFR dump from a running Semeru application in a containerized environment (specifically an OpenShift environment) and transport it to your local system, with the assumption that you do not have SSH access to the pod that is running the container.

Set up your environment

Make sure rsync is installed beforehand. Perform this step in the Dockerfile itself while defining the image. This step is only required if your JFR dump is abnormally large (more than 2GB).

# sudo apt install rsync

Next, make sure you have the ability to execute shell scripts in the remote OpenShift container.

# oc login --token=<your_token> --server=<your_cluster_api>

Locate the target pod:

# oc get pods -n <namespace>

Locate the container within the pod:

# oc get pod <pod> -n <namespace> -o jsonpath='{.spec.containers[*].name}'

Issue sample commands or shell scripts inside the container:

# oc exec -n <namespace> <pod> -c <container> -- /bin/sh -c <cmd>

For example:

To list files in the root directory:

# oc exec -n my-namespace my-pod -c my-container -- /bin/sh -c "ls /"

Print environment variables:

oc exec -n my-namespace my-pod -c my-container -- /bin/sh -c "env"

Collect a JFR dump

JFR capabilities are enabled by default in most JVMs, so you don’t need to pass any special arguments to activate them.

To initiate a JFR recording, you can use the jcmd utility, which allows you to interact with the running JVM process.

Note down the JVM ID which will be used later using jcmd -l command.

# oc exec -n <namespace> <pod> -c <container> -- /bin/sh -c "jcmd -l"

Make sure to embed the commands in double quotes to avoid truncation.

Continue reading on IBM Developer.

IBM Developer @ibmdeveloper-staff