How to obtain CPU-bound runnable thread details from specific SAS® Viya® Java based pod services


Complete the following steps to obtain CPU-bound runnable thread details from specific SAS Viya Java based pod services:

  1. Locate the name of the pod that runs the Java application for which you need the thread dump. List pods via the following command:

    kubectl get pods

  2. Access the pod's shell by executing a shell inside the target pod. Replace <POD_NAME> with the actual pod name:

    kubectl exec -it sas-risk-pipeline-5bb6ffb5f8-5q7cm -- /bin/bash

    Note: You might use sh instead of bash, depending on the container image.

  3. Identify the Java process ID (PID). Once inside the pod, find the PID of the currently running Java application. Submit commands similar to the following to locate this information:

    ps aux | grep java 

    USERPID %CPU %MEM VSZRSSTTY STAT  START TIMECOMMAND
    sas70.1 0.5 3249788     778392    ?SlJan26 262:06/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el9.x86_64/bin/java

    Make note of the PID.

    In addition, gather the output of the following command:

    ps aux --sort=-%cpu | head -n 10  > /tmp/top10-pidcpu.txt

  4. Gather top output for the duration of the time that you force thread dumps against a specific PID.

    For example, if you force thread dumps every two minutes and gather five thread dumps, the total duration is ten minutes of CPU data. The command for this scenario looks similar to the following against PID 7:

    top -bcH -d 420 -p 7 -n 5 > /tmp/topoutput.txt

  5. Generate thread dumps. If installed, use the jstack tool to generate the thread dump every few seconds or minutes (depending on how long the process is suspended). Replace <PID> with the Java process ID and <FILENAME> with your desired output file name. Replace <n> with an iterative incremental number for as many thread dumps as you take:

    jstack <PID> > <FILENAME>.tdump

    Using the previous scenario, here is an example ps aux output:

    jstack 7 > threaddump<n>.tdump

    If you do not have the jstack tool, send a kill -3 <PID> signal every few seconds or minutes (depending on how long the process is suspended), which prints the thread dump to the application pod logs file.

  6. Exit the pod bash or shell session. Then, gather the pod log and other generated files within the /tmp folder, which are topoutput.txt, and top10-pidcpu.txt files. Lastly, upload the log to SASTSDrive for further review.

    Note: Include the jstack dumps if you generated the dumps with the jstack tool.

    Here is an example:

    sudo kubectl logs sas-risk-pipeline-5bb6ffb5f8-5q7cm -n viya >> sas-risk-pipeline-5bb6ffb5f8-5q7cm.log