Protect HotspotSupport::resolve() with longjmp

[zhengyu123]

What does this PR do?:

Hardens HotspotSupport::resolve() — the dump-time fallback that resolves a raw Method* to a jmethodID when cstack=vm,fjmethodid=false — against crashes from stale JVM metadata, and fixes three separate bugs that caused <clinit> frames captured under that configuration to always serialize as "unknown" instead of resolving correctly:

1. JVMSupport::initExecution() short-circuited on a stale Fully_loaded state left over from a previous session, so a later fjmethodid=false start never re-evaluated whether jmethodID preloading should actually be disabled.
2. The <clinit> JVMTI fallback called GetClassMethods directly instead of going through loadMethodIDsIfNeededImpl(), skipping the JDK-8062116 classloader-data patch that every other preload path applies.
3. HotspotSupport::fillJavaFrame() read VM::arguments()._force_jmethodID — a global Arguments object that's only updated by the command-line agent-attach path, never by the JavaProfiler.execute() JNI API. Every session started via the JNI API (which is how the test suite, and most embedding use, starts the profiler) therefore saw a permanently stale _force_jmethodID, so un-preloaded methods were always treated as unwalkable and serialized as "unknown" instead of taking the intended raw-Method* fallback path. Fixed by giving Profiler its own _force_jmethodID member (mirroring the existing _cstack field) set from the actual startup Arguments.

Also adds a malloc-backed fallback for method/class/signature names that don't fit the fixed inline buffers used during crash-protected resolution (previously such names silently resolved as "unknown" with no fallback), and closes out the correctness gaps that fallback introduced: a double-free on the crash-recovery path (release() is now idempotent), a setjmp/longjmp indeterminate-value issue (the malloc'd pointers are now volatile, since they're mutated between sigsetjmp() and a possible siglongjmp() and then read back by the recovery path), and a dead-code fallback (the malloc path was unreachable for class-name/signature since their short-buffer size equaled the hard rejection ceiling — lowered to 1024 so the fallback actually engages).

Motivation:

ClinitResolutionTest was failing: a class's <clinit> frame, sampled while spinning for ~2s under cpu=1ms,cstack=vm,fjmethodid=false, never resolved to its real class/method name in the resulting JFR — it always showed as the shared "unknown" frame. Root-causing this surfaced the three bugs above, none of which were specific to <clinit> — they affect any raw-Method* resolution under fjmethodid=false, <clinit> just happened to be the scenario the test exercised.

Additional Notes:

• Fixes (1) and (2) were ported from paul.fournillon/jmethod_clinit_fix, adapted to this branch's crash-protection refactor.
• Known, lower-priority items surfaced during review but intentionally not addressed here (happy to split into follow-ups if preferred):
  • Name/signature resolution is still bounded (512/1024/4096 bytes, not the full 65535 the JVM permits) — a deliberate fidelity/complexity tradeoff, not a full fix.
  • METHOD_RESOLVE_FAULT_RECOVERED overlaps STACKWALK_LONGJMP_RECOVERED for the same fault; nothing enforces the "subtract, never sum" comment, so a naive counter-summing dashboard would double-count.
  • The <clinit> fallback's patchClassLoaderData() call (JDK 8 only) is now reachable from the dump thread as well as from ClassPrepare; a narrow race could invoke it twice for the same class.
  • resolve()'s top-level doc comment ("only resolves system-classloader methods") is slightly stale given the <clinit> fallback's load_all=true; the constraint still holds in practice via the earlier FindClass call, so this is cosmetic.

How to test the change?:

• ./gradlew :ddprof-lib:gtestDebug_hotspotSupport_ut :ddprof-lib:gtestDebug_jvmSupport_ut — new/updated unit tests for JVMSupport::initExecution()'s stale-state bug and the <clinit> fallback's loadMethodIDsIfNeededImpl() routing.
• ./gradlew testdebug --tests com.datadoghq.profiler.cpu.ClinitResolutionTest — the regression test for this bug, passes reliably on the first attempt (previously failed even after all 5 retries).
• ./gradlew testdebug --tests "com.datadoghq.profiler.cpu.*" — full cpu test package, including VtableReceiverFrameTest, all passing.

For Datadog employees:

• If this PR touches code that signs or publishes builds or packages, or handles
  credentials of any kind, I've requested a security review (run the dd:platform-security-review
  skill, or file a request via the PSEC review form).
• This PR doesn't touch any of that.
• JIRA: PROF-15786

Unsure? Have a question? Request a review!

[dd-octo-sts]

CI Test Results

Run: #32369962976 | Commit: 1d0cf59 | Duration: 14m 29s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	✅	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0

---

Updated: 2026-08-20 12:56:55 UTC

[dd-octo-sts]

Scan-Build Report

User:	runner@runnervmzvulz
Working Directory:	/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:	make -j4 all
Clang Version:	Ubuntu clang version 18.1.3 (1ubuntu1)
Date:	Thu Aug 20 12:41:19 2026

Bug Summary

Bug Type	Quantity	Display?
All Bugs	1
Logic error
Dereference of null pointer	1

Reports

Bug Group	Bug Type ▾	File	Function/Method	Line	Path Length
Logic error	Dereference of null pointer	profiler.h	findLibraryByAddress	529	28

[dd-octo-sts]

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 f74f4114

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 54e771719e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[r1viollet]

Can readMethodNames() be rewritten as a dedicated SafeAccess-only traversal, rather than installing a broad siglongjmp landing pad?

[r1viollet]

This effectively protects for crashes, but are we not worried about downstream effects of passing invalid data ? Using data that was freed would not crash but still produce invalid data.

[r1viollet]

I like the deterministic tests, though have we considered fuzzing for this type of problem ?
I can imagine we could:

• randomize the metadata graph
• fault every pointer edge
• mutate every offset, index and length
• exercise readable-but-recycled metadata
• fuzz combinations of nested signals and cleanup states

[zhengyu123]

Can readMethodNames() be rewritten as a dedicated SafeAccess-only traversal, rather than installing a broad siglongjmp landing pad?

@r1viollet The goal is the opposite - SafeAccess is very expensive, because every single plain load becomes a method call, and the cost to recover is literally the same as siglongjmp - a trip to signal handler and unwinding states.

Once completing siglongjmp protections, I would like to remove all SafeAccess inside the protected blocks, that should improve performance.

[zhengyu123]

This effectively protects for crashes, but are we not worried about downstream effects of passing invalid data ? Using data that was freed would not crash but still produce invalid data.

@r1viollet Yes, this is my main concern - this approach does mask off some bad bugs inside the protected blocks. I am think about setting up a special build, that disables siglongjmp protection for testing ...

[zhengyu123]

I like the deterministic tests, though have we considered fuzzing for this type of problem ? I can imagine we could:

• randomize the metadata graph
• fault every pointer edge
• mutate every offset, index and length
• exercise readable-but-recycled metadata
• fuzz combinations of nested signals and cleanup states

@r1viollet Yes, we may need more fuzzing tests.

I think fault-injection and fuzzing overlap in certain ways, and fault-injection has certain advantages, please refer to my reply to Thomas' comment here

Fault-injection allows fault to propagate through call graph, instead of selected points.

[jbachorik]

Nice cleanup! Looks sane to me, there is one minor issue Sphinx found, if you can address it before merge, it would be great, but it is not a high priority.