Skip to content

Add debug diagnostics to the dd-openfeature SDK - #13092

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
masterfrom
ethan.romero/FFL-3401/ffe-sdk-diagnostics
Oct 28, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
masterfrom
ethan.romero/FFL-3401/ffe-sdk-diagnostics

Conversation

@ethan-romero-dg

@ethan-romero-dg ethan-romero-dg commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Adds DEBUG diagnostics to the dd-openfeature SDK. Stacked on #13091, which adds the agent-side logs.

Provider. Logs the lifecycle (initializing, ready, timed out, recovered, configuration changed or unavailable, shutting down). A failed initialization logs only the exception type, because the message of a configuration error can echo the endpoint URL.

Evaluations. DebugLoggingHook logs the raw outcome of every evaluation: flag, context, default and resolved values, variant, reason, error and metadata. It is registered only when both DD_TRACE_DEBUG and the application's logger allow DEBUG, and the raw context can carry personal data, so:

  • the Javadoc says so, and an INFO notice, logged through the hook's own logger, announces it when the hook is registered;
  • when only DD_TRACE_DEBUG is on, one WARN names the logger category to enable;
  • each value is rendered within 1024 characters, to a depth of 8, and a cyclic or very deep value cannot overflow the evaluating thread's stack. A container is not walked further once the budget is spent;
  • the error message is the one the application observes, so the consent filter already applies.

The reasons an exposure is not sent are logged at TRACE.

Configuration delivery. Logs the runtime start-up (source, event transport, which pipelines are enabled) and every configuration applied or removed. For agentless delivery: the endpoint without its userinfo or query, applied and unchanged (HTTP 304) responses, each retry with its delay, and that a malformed or empty payload retains the current configuration. For Remote Configuration: the subscription and malformed documents.

Event pipelines. Logs the exposure flush (starting, delivered, or failed with the reason and the batch size), the shutdown requests, and delivered flag evaluation flushes. On master a full exposure queue dropped exposures without a log. The producer, an application thread, still does not log: it counts the drops and the serializer reports them once per flush.

Tests. The module has no SLF4J 2 compatible logging backend in its tests, so its loggers were no-ops. A small test-only SLF4J provider (testlog/CapturingServiceProvider) captures events so the new tests can assert the logs.

Motivation

Onboarding diagnostics for feature flags (FFL-3401), following the SDK logging checklist.

Additional Notes

This also addresses the review of #12897, which targeted the provider code that #12844 moved into this module:

  • Logging per exposure on the application thread: gone, see above.
  • The same raw data logged twice per evaluation: one line per evaluation, capped. The personal data is documented and announced.
  • The raw error message before the consent filter: only the filtered message is logged.
  • The HTTP status logged twice per poll and stack traces for every attempt: a successful poll logs once, and the retry lines carry no stack trace.
  • catch (Throwable): the hook catches RuntimeException | LinkageError only.
  • A hardcoded logback-classic test dependency: none is added.

Validation: dd-openfeature tests, Spotless, forbidden-APIs, SpotBugs and Javadoc pass, as do the feature-flagging-lib and feature-flagging-agent tests.

Contributor Checklist

Jira ticket: FFL-3401

🤖 Generated with Claude Code

@ethan-romero-dg ethan-romero-dg added type: feature Enhancements and improvements tag: diagnostics Diagnostics related changes tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Oct 27, 2026

ethan-romero-dg commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

@ethan-romero-dg

Copy link
Copy Markdown
Contributor Author

@AlexeyKuznetsov-DD thanks for the review on #12897. I closed that PR because #12844 moved the provider code it logged from into the dd-openfeature module. The logging is redone in #13092, and the agent-side part is in #13091. Here is how each of your comments came out:

Comment Now
P1: logging once per exposure on the application thread ExposurePipeline.accept no longer logs. It counts queue-full drops in an AtomicLong, and the serializer reports them once per flush at DEBUG. On master these drops weren't logged at all.
P2: suppressed: duplicate once per duplicate The pipeline no longer dedups, because the provider does. The equivalent "already recorded" line runs on the evaluating thread and is TRACE only, so it is off at DEBUG.
P2: the same raw data logged twice per evaluation There is one raw line per evaluation, from the hook at the finally stage. Each value is capped at 1024 characters, and nested values are rendered within that budget, to a depth of 8, and safely for cycles.
P2: raw errorMessage logged before the consent filter Only the message the application observes is logged, and the provider already consent-filters it (there's a test). The raw context is still logged by design, only when DD_TRACE_DEBUG and the logger's DEBUG are both on. The Javadoc says it can contain personal data, and an INFO line through the hook's logger says so when the hook is registered.
P3: HTTP status logged twice per poll A successful poll logs once. The "poll attempt" line is only for a retry, and a test checks it.
P3: up to 4 stack traces per failed poll The per-attempt retry lines carry no throwable (tested). The existing final failure warning keeps the stack trace once per poll cycle.
P3: catch (Throwable) The hook catches RuntimeException | LinkageError only. The renderer is bounded, so it doesn't throw StackOverflowError on a cyclic or very deep value. A test covers that case.
P3: hardcoded logback-classic:1.6.5 No dependency is added. This module's test classpath has no working SLF4J 2 backend, because logback 1.2.x is ignored by SLF4J 2. So there's a small test-only SLF4J provider that captures the logs.

@ethan-romero-dg

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T20:51:21.472550Z 5c9001a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 68.82% (+9.31%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a244377 | Docs | Give us feedback!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60e55aee92

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dd-octo-sts

dd-octo-sts Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.81 s 14.64 s [+0.2%; +2.1%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.59 s 13.64 s [-1.3%; +0.5%] (no difference)
startup:petclinic:appsec:Agent 17.70 s 17.52 s [+0.2%; +1.9%] (maybe worse)
startup:petclinic:iast:Agent 16.77 s 17.53 s [-8.5%; -0.1%] (maybe better)
startup:petclinic:profiling:Agent 17.29 s 17.20 s [-0.5%; +1.5%] (no difference)
startup:petclinic:sca:Agent 17.11 s 17.65 s [-7.1%; +1.0%] (no difference)
startup:petclinic:tracing:Agent 16.65 s 16.25 s [-1.9%; +6.8%] (no difference)

Commit: a244377c · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@ethan-romero-dg
ethan-romero-dg marked this pull request as ready for review October 27, 2026 20:45
@ethan-romero-dg
ethan-romero-dg requested a review from a team as a code owner October 27, 2026 20:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c9001a7aa

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@datadog-prod-us1-6 datadog-prod-us1-6 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

A throwing DEBUG logging backend can interrupt shutdown before workers and configuration delivery are stopped. Shutdown flags are already set, preventing subsequent attempts from completing cleanup.

Open Bits AI session

🤖 Bits Code Review · Commit 5c9001a

Base automatically changed from ethan.romero/FFL-3401/ffe-agent-diagnostics to master October 27, 2026 22:41

@datadog-prod-us1-6 datadog-prod-us1-6 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

A throwing DEBUG logging backend can interrupt runtime shutdown cleanup and permanently stop exposure delivery. Moving shutdown logs after each component’s signal still leaves the remaining runtime cleanup vulnerable.

Open Bits AI session

🤖 Bits Code Review · Commit 3bd2246

@btthomas btthomas left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix that deep-copy part.

ethan-romero-dg and others added 4 commits October 28, 2026 15:26
Help onboarding by logging what the dd-openfeature SDK does, at DEBUG.

Provider: log the lifecycle (initializing, ready, timed out, recovered,
configuration changed or unavailable, shutting down) and why an exposure
is suppressed (at TRACE). Log only the type of an initialization failure,
since its message can echo the endpoint URL.

Evaluations: log the raw outcome of every evaluation, including the
evaluation context and any personal data in it, only when DD_TRACE_DEBUG
is on and the application logger allows DEBUG. Each value is rendered
within 1024 characters and a depth limit, with cycle detection, so that
a cyclic or very deep value cannot overflow the stack of the evaluating
thread. Say so at INFO, through the logger of the hook, when the hook is
registered. When only DD_TRACE_DEBUG is on, warn which logger category
to enable.

Configuration delivery: log the runtime start-up (source, transport,
enabled pipelines) and every configuration applied or removed. For
agentless delivery, log the endpoint without its query, applied and
unchanged responses, retries with their delay, and that a malformed or
empty payload retains the current configuration. For Remote
Configuration, log the subscription and malformed documents.

Event pipelines: log the exposure flush lifecycle (with the failure
reason and the number of dropped exposures), the exposures dropped by a
full queue (counted on the application thread and reported once per
flush, so the producer never logs), the shutdown requests, and delivered
flag evaluation flushes.

The module has no SLF4J 2 compatible logging backend in tests, so add a
test-only SLF4J provider that captures events.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Address the review of the SDK diagnostics.

Log only the scheme, host and port of the agentless endpoint when the
configuration source starts. The path of a custom base URL can carry a
credential, as its userinfo and query can.

Log that the provider shut down after its cleanup, so that a failing
logging backend cannot keep the metrics and the evaluator from being
shut down.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A debug log that ran after a state flag was set but before the work it
describes could, with a failing logging backend, skip that work, and the
flag then kept it from being retried. Log after the work instead:

- apply a configuration, or its removal, and notify the listeners before
  logging it;
- signal the exposure and flag evaluation workers before logging the
  shutdown request;
- cancel the polling and shut the executor down before logging that the
  agentless source stopped;
- log that the agentless source started after its first poll and the
  scheduling of the next ones.

The provider lifecycle logs cannot be moved after their work without
restructuring the initialization, so they are contained instead: a log
that fails no longer turns a successful initialization into an error or
skips an event.

Each path has a test that makes the logging backend throw and checks the
work still happened. Add a test accessor to the flag evaluation
pipeline, like the exposure pipeline has.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The debug renderer read a Structure through keySet() and getValue().
For an immutable structure these copy every key and deep-clone the
value, so the whole context was copied before the budget applied.
Walk asUnmodifiableMap() instead, and cover immutable contexts.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ethan-romero-dg
ethan-romero-dg force-pushed the ethan.romero/FFL-3401/ffe-sdk-diagnostics branch from 3bd2246 to a244377 Compare October 28, 2026 15:27
@ethan-romero-dg

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-10-28 16:57:08 UTC ℹ️ Start processing command /merge


2026-10-28 16:57:12 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).


2026-10-28 17:46:11 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for b060e05:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.
Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

@ethan-romero-dg

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-10-28 17:48:28 UTC ℹ️ Start processing command /merge


2026-10-28 17:48:32 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).


2026-10-28 18:49:01 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 1560b11 into master Oct 28, 2026
621 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the ethan.romero/FFL-3401/ffe-sdk-diagnostics branch October 28, 2026 18:49
@github-actions github-actions Bot added this to the 1.69.0 milestone Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM tag: diagnostics Diagnostics related changes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants