Skip to content

Serialize stale jmethodID frames as <unloaded> instead of jvmtiError - #744

Merged
jbachorik merged 1 commit into
mainfrom
fix/jvmtiError
Aug 20, 2026
Merged

Serialize stale jmethodID frames as <unloaded> instead of jvmtiError#744
jbachorik merged 1 commit into
mainfrom
fix/jvmtiError

Conversation

@jbachorik

@jbachorik jbachorik commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?:
Remaps the dump-time JVMTI-resolution failure branch in Lookup::fillJavaMethodInfo (flightRecorder.cpp) to serialize the frame as <unloaded> instead of the synthetic jvmtiError method name. Removes the now-dead jvmtiError assertions from SmokeCpuTest, SmokeWallTest, CTimerSamplerTest, and RemoteSymbolicationTest.

Motivation:
Profiles in hotdog (non-remotesym) showed all leaf frames serialized as jvmtiError during the early class-churn window (~first 10-15 min), a regression vs v1.49.0.

Root cause is JDK-8268406 "Deallocate jmethodID native memory" (integrated in JDK 26, build b05): jmethodID memory is now freed on class unload (pre-26 it was kept alive — the leak this change fixed). The profiler's stale-jmethodID detection relies on dereferencing the jmethodID to verify it still points to the right Method*:

  • VMMethod::check_jmethodID_hotspot (vmStructs.cpp): if (hotspot_version() > 25) return true; — unconditional, no validation.
  • _can_dereference_jmethod_id = _has_method_structs && hotspot_version() <= 25 (vmStructs.cpp): validatedId() skips the deref.

Both are disabled for JDK > 25 because the deref no longer aliases a kept-alive JNIMethodBlock — it would fault on freed memory. So a stale jmethodID from an unloaded/purged class passes all capture-time gates, reaches fillJavaMethodInfo at dump time, GetMethodDeclaringClass returns JVMTI_ERROR_INVALID_METHODID, and the frame was serialized as jvmtiError.

Per the JVMTI spec, retransform does not invalidate jmethodIDs (the original ID is repurposed to the new method version). What invalidates them is class unload — and JDK-8268406 changed what "invalidated" means at the memory level. The early-run correlation is class churn (generated accessors, lambda forms, Mockito-generated classes, retransform obsolete-method purge via the JDK-8313816 path) which settles after warmup.

<unloaded> is the correct, safe label: the method was present at sample time, its metadata was reclaimed before the JFR dump. It matches the existing angle-bracket synthetic-frame convention (<remote>, <vtable_receiver>, <unresolved_vtable_receiver>). The capture-time unknown path (JMETHODID_NOT_WALKABLE / null jmethodID) stays distinct, so the two failure modes remain distinguishable in profiles:

  • unknown — capture-time validation failure (walker couldn't trust the ID).
  • <unloaded> — dump-time resolution failure (ID was valid at capture, metadata reclaimed by dump).

Additional Notes:

  • The deref-based stale check is unrevivable on JDK ≥ 26 (the jmethodID memory is freed), so this remap is the right fix, not a stopgap. Retaining the raw Method* (the alternative) is crash-prone on class unload and reverts Avoid retaining invalid VM method pointers #699's safety work; on JDK 26 the Method* is freed too.
  • The jvmtiError label is no longer emitted by any code path, so the per-test assertions against it were dead checks that could never fail — removed rather than kept as sentinels.

How to test the change?:
SmokeCpuTest + 391 gtests pass (0 failures) on Corretto 26. A reproducing test that forces a class unload during profiling to lock in the <unloaded> behavior can be added as a follow-up.

For Datadog employees:

  • This PR doesn't touch any of that.
  • JIRA: (none)

@jbachorik jbachorik added the AI label Aug 19, 2026
@dd-octo-sts

dd-octo-sts Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmzvulz
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Thu Aug 20 09:14:41 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs1
Logic error
Dereference of null pointer1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Logic errorDereference of null pointerprofiler.hfindLibraryByAddress52328

@jbachorik
jbachorik marked this pull request as ready for review August 19, 2026 15:35
@jbachorik
jbachorik requested a review from a team as a code owner August 19, 2026 15:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3bacdea1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp
Comment thread ddprof-test/src/test/java/com/datadoghq/profiler/cpu/CTimerSamplerTest.java Outdated
@dd-octo-sts

dd-octo-sts Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 54a8d9de

@datadog-datadog-prod-us1-2

This comment has been minimized.

@jbachorik
jbachorik force-pushed the fix/jvmtiError branch 2 times, most recently from 79540b0 to b21f7cd Compare August 19, 2026 16:36
@dd-octo-sts

dd-octo-sts Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #32352690944 | Commit: db51a29 | Duration: 14m 44s (longest job)

All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - - -
8-ibm - - -
8-j9 - -
8-librca - -
8-orcl - - -
11 - - -
11-j9 - -
11-librca - -
17 - -
17-graal - -
17-j9 - -
17-librca - -
21 - -
21-graal - -
21-librca - -
25 - -
25-graal - -
25-librca - -

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-08-20 09:29:51 UTC

JDK-8268406 (JDK 26) freed jmethodID memory on class unload, so the
profiler's deref-based stale-jmethodID detection (validatedId deref +
check_jmethodID) -- disabled for JDK > 25 -- can no longer catch stale
IDs at capture time. A stale jmethodID from an unloaded/purged class
passes all capture gates, reaches fillJavaMethodInfo at dump time, and
GetMethodDeclaringClass returns JVMTI_ERROR_INVALID_METHODID. The frame
was serialized as the synthetic 'jvmtiError' method, which surfaced as
all-leaf jvmtiError frames in hotdog during the early class-churn
window (regression vs v1.49.0, which ran on JDK <= 25).

Remap the dump-time resolution failure branch to '<unloaded>', matching
the angle-bracket synthetic-frame convention (<remote>,
<vtable_receiver>). This is the correct and safe label: the method was
present at sample time, its metadata was reclaimed before the JFR dump.
The capture-time 'unknown' path (JMETHODID_NOT_WALKABLE / null
jmethodID) stays distinct.

Remove the now-dead jvmtiError assertions from SmokeCpuTest,
SmokeWallTest, CTimerSamplerTest and RemoteSymbolicationTest: the label
is no longer emitted anywhere, so those checks can never fail.

@zhengyu123 zhengyu123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jbachorik
jbachorik merged commit 382fbfd into main Aug 20, 2026
108 checks passed
@jbachorik
jbachorik deleted the fix/jvmtiError branch August 20, 2026 16:01
@github-actions github-actions Bot added this to the 1.50.0 milestone Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants