Skip to content

Stop AppSecInterceptor from silently retrying failed okhttp requests - #12242

Open
dougqh wants to merge 3 commits into
masterfrom
dougqh/fix-appsec-interceptor-duplicate-request
Open

Stop AppSecInterceptor from silently retrying failed okhttp requests#12242
dougqh wants to merge 3 commits into
masterfrom
dougqh/fix-appsec-interceptor-duplicate-request

Conversation

@dougqh

@dougqh dougqh commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Narrows the try/catch in AppSecInterceptor.intercept() (both okhttp-2.2 and okhttp-3.0) so it only guards the AppSec-specific hook logic (sampleRequest/onRequest/onResponse), not the real chain.proceed() network call. Genuine I/O failures now propagate normally on the first (and only) attempt, exactly as they would without this instrumentation.

Motivation

AppSecInterceptor.intercept() previously wrapped the real chain.proceed(request) call inside the same try/catch (Exception e) that guards the AppSec hooks. Any IOException from the actual outbound request (e.g. java.net.ConnectException on a transient network failure) was caught by that handler, logged at debug, and the request was silently retried via chain.proceed(chain.request()). This means:

  • Non-idempotent requests (POST/PUT) can be executed twice on a transient failure.
  • If the retry also fails, its IOException propagates unhandled and gets misattributed to AppSecInterceptor.intercept in error tracking, when the real cause is just an ordinary network failure.

Found via Datadog Error Tracking issue 7d2a0a46-aab1-11f0-a468-da7ad0900002 (38k+ occurrences) — automated root cause analysis confirmed the stack shows ConnectException propagating through AppSecInterceptor:57/58, matching this code path exactly. Same shape exists in the okhttp2 sibling (db728b88-b44c-11f0-a740-da7ad0900002).

Additional Notes

  • No dedicated unit test exists for AppSecInterceptor behavior itself in either module; happy to add one covering the retry regression if desired.

Test plan

  • ./gradlew :dd-java-agent:instrumentation:okhttp:okhttp-3.0:compileJava :dd-java-agent:instrumentation:okhttp:okhttp-2.2:compileJava — BUILD SUCCESSFUL
  • ./gradlew :dd-java-agent:instrumentation:okhttp:okhttp-3.0:spotlessApply :dd-java-agent:instrumentation:okhttp:okhttp-2.2:spotlessApply — applied, no additional changes
  • ./gradlew :dd-java-agent:instrumentation:okhttp:okhttp-3.0:test :dd-java-agent:instrumentation:okhttp:okhttp-2.2:test — BUILD SUCCESSFUL, existing instrumentation tests pass

🤖 Generated with Claude Code

chain.proceed(request) was wrapped in the same try/catch that guards
the AppSec request/response hooks, so any IOException from the real
network call (e.g. ConnectException) was swallowed and the request
was silently retried via chain.proceed(chain.request()). This double-
executes non-idempotent requests on transient network failures and
surfaces the retry's own failure as an unhandled error blamed on the
interceptor. Narrow the try/catch to only cover the AppSec hooks so
genuine I/O failures propagate normally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dougqh dougqh added inst: okhttp Square OkHttp instrumentation tag: ai generated Largely based on code generated by an AI or LLM type: bug fix Bug fix labels Aug 19, 2026
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 69.74% (+11.12%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: acf135f | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.99 s 13.98 s [-0.6%; +0.7%] (no difference)
startup:insecure-bank:tracing:Agent 12.89 s 12.96 s [-1.5%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 16.94 s 16.68 s [+0.5%; +2.6%] (maybe worse)
startup:petclinic:iast:Agent 16.90 s 17.01 s [-1.4%; +0.2%] (no difference)
startup:petclinic:profiling:Agent 16.75 s 16.85 s [-1.6%; +0.5%] (no difference)
startup:petclinic:sca:Agent 16.74 s 16.40 s [+1.1%; +3.0%] (significantly worse)
startup:petclinic:tracing:Agent 16.06 s 16.05 s [-1.1%; +1.1%] (no difference)

Commit: acf135f2 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh
dougqh marked this pull request as ready for review August 21, 2026 13:29
@dougqh
dougqh requested a review from a team as a code owner August 21, 2026 13:29
@dougqh
dougqh requested review from ygree and removed request for a team August 21, 2026 13:29
dougqh and others added 2 commits August 21, 2026 09:55
Covers both okhttp-2.2 and okhttp-3.0 AppSecInterceptor.intercept():
asserts an IOException from chain.proceed() propagates without being
swallowed/retried, using Mockito + AgentTracer.forceRegister instead
of Groovy/Spock.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The network call is outside both hook exception handlers. The first I/O failure now passes to the caller, with no silent retry in either OkHttp integration.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 161655c · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: okhttp Square OkHttp instrumentation tag: ai generated Largely based on code generated by an AI or LLM type: bug fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant