Implement OpenTelemetry Observable Metrics API#10631
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits intomasterfrom Feb 19, 2026
Merged
Implement OpenTelemetry Observable Metrics API#10631gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits intomasterfrom
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits intomasterfrom
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063693
Total [baseline] (8.772 s) : 0, 8772172
Agent [candidate] (1.073 s) : 0, 1072779
Total [candidate] (8.745 s) : 0, 8745486
section iast
Agent [baseline] (1.228 s) : 0, 1228461
Total [baseline] (9.364 s) : 0, 9363728
Agent [candidate] (1.232 s) : 0, 1232378
Total [candidate] (9.371 s) : 0, 9371287
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (627.712 ms) : 0, 627712
BytebuddyAgent [candidate] (632.953 ms) : 0, 632953
AgentMeter [baseline] (29.081 ms) : 0, 29081
AgentMeter [candidate] (29.43 ms) : 0, 29430
GlobalTracer [baseline] (257.612 ms) : 0, 257612
GlobalTracer [candidate] (259.474 ms) : 0, 259474
AppSec [baseline] (33.126 ms) : 0, 33126
AppSec [candidate] (33.215 ms) : 0, 33215
Debugger [baseline] (62.87 ms) : 0, 62870
Debugger [candidate] (64.635 ms) : 0, 64635
Remote Config [baseline] (605.17 µs) : 0, 605
Remote Config [candidate] (653.982 µs) : 0, 654
Telemetry [baseline] (9.886 ms) : 0, 9886
Telemetry [candidate] (11.049 ms) : 0, 11049
Flare Poller [baseline] (5.328 ms) : 0, 5328
Flare Poller [candidate] (3.85 ms) : 0, 3850
section iast
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.21 ms) : 0, 1210
BytebuddyAgent [baseline] (793.412 ms) : 0, 793412
BytebuddyAgent [candidate] (795.763 ms) : 0, 795763
AgentMeter [baseline] (11.342 ms) : 0, 11342
AgentMeter [candidate] (11.358 ms) : 0, 11358
GlobalTracer [baseline] (247.27 ms) : 0, 247270
GlobalTracer [candidate] (248.345 ms) : 0, 248345
IAST [baseline] (27.007 ms) : 0, 27007
IAST [candidate] (26.975 ms) : 0, 26975
AppSec [baseline] (33.11 ms) : 0, 33110
AppSec [candidate] (34.004 ms) : 0, 34004
Debugger [baseline] (66.493 ms) : 0, 66493
Debugger [candidate] (65.903 ms) : 0, 65903
Remote Config [baseline] (529.538 µs) : 0, 530
Remote Config [candidate] (530.493 µs) : 0, 530
Telemetry [baseline] (8.601 ms) : 0, 8601
Telemetry [candidate] (8.643 ms) : 0, 8643
Flare Poller [baseline] (3.458 ms) : 0, 3458
Flare Poller [candidate] (3.489 ms) : 0, 3489
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1066903
Total [baseline] (10.878 s) : 0, 10877621
Agent [candidate] (1.069 s) : 0, 1069410
Total [candidate] (10.874 s) : 0, 10873891
section appsec
Agent [baseline] (1.242 s) : 0, 1241702
Total [baseline] (11.104 s) : 0, 11104150
Agent [candidate] (1.24 s) : 0, 1239842
Total [candidate] (11.093 s) : 0, 11093454
section iast
Agent [baseline] (1.231 s) : 0, 1230547
Total [baseline] (11.24 s) : 0, 11240420
Agent [candidate] (1.232 s) : 0, 1232223
Total [candidate] (11.214 s) : 0, 11213971
section profiling
Agent [baseline] (1.192 s) : 0, 1192195
Total [baseline] (10.967 s) : 0, 10966691
Agent [candidate] (1.198 s) : 0, 1198397
Total [candidate] (10.928 s) : 0, 10928474
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (627.555 ms) : 0, 627555
BytebuddyAgent [candidate] (629.383 ms) : 0, 629383
AgentMeter [baseline] (29.076 ms) : 0, 29076
AgentMeter [candidate] (29.383 ms) : 0, 29383
GlobalTracer [baseline] (257.774 ms) : 0, 257774
GlobalTracer [candidate] (259.355 ms) : 0, 259355
AppSec [baseline] (32.934 ms) : 0, 32934
AppSec [candidate] (33.197 ms) : 0, 33197
Debugger [baseline] (68.622 ms) : 0, 68622
Debugger [candidate] (65.072 ms) : 0, 65072
Remote Config [baseline] (623.192 µs) : 0, 623
Remote Config [candidate] (614.151 µs) : 0, 614
Telemetry [baseline] (9.158 ms) : 0, 9158
Telemetry [candidate] (11.307 ms) : 0, 11307
Flare Poller [baseline] (3.792 ms) : 0, 3792
Flare Poller [candidate] (3.745 ms) : 0, 3745
section appsec
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.21 ms) : 0, 1210
BytebuddyAgent [baseline] (659.233 ms) : 0, 659233
BytebuddyAgent [candidate] (658.564 ms) : 0, 658564
AgentMeter [baseline] (11.984 ms) : 0, 11984
AgentMeter [candidate] (11.998 ms) : 0, 11998
GlobalTracer [baseline] (258.43 ms) : 0, 258430
GlobalTracer [candidate] (258.622 ms) : 0, 258622
AppSec [baseline] (167.991 ms) : 0, 167991
AppSec [candidate] (168.1 ms) : 0, 168100
Debugger [baseline] (67.064 ms) : 0, 67064
Debugger [candidate] (66.146 ms) : 0, 66146
Remote Config [baseline] (664.738 µs) : 0, 665
Remote Config [candidate] (656.537 µs) : 0, 657
Telemetry [baseline] (9.631 ms) : 0, 9631
Telemetry [candidate] (9.326 ms) : 0, 9326
Flare Poller [baseline] (3.788 ms) : 0, 3788
Flare Poller [candidate] (3.696 ms) : 0, 3696
IAST [baseline] (25.609 ms) : 0, 25609
IAST [candidate] (25.437 ms) : 0, 25437
section iast
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (794.081 ms) : 0, 794081
BytebuddyAgent [candidate] (795.479 ms) : 0, 795479
AgentMeter [baseline] (11.301 ms) : 0, 11301
AgentMeter [candidate] (11.31 ms) : 0, 11310
GlobalTracer [baseline] (247.532 ms) : 0, 247532
GlobalTracer [candidate] (247.939 ms) : 0, 247939
AppSec [baseline] (33.867 ms) : 0, 33867
AppSec [candidate] (33.847 ms) : 0, 33847
Debugger [baseline] (66.941 ms) : 0, 66941
Debugger [candidate] (66.723 ms) : 0, 66723
Remote Config [baseline] (540.681 µs) : 0, 541
Remote Config [candidate] (551.082 µs) : 0, 551
Telemetry [baseline] (8.619 ms) : 0, 8619
Telemetry [candidate] (8.649 ms) : 0, 8649
Flare Poller [baseline] (3.442 ms) : 0, 3442
Flare Poller [candidate] (3.415 ms) : 0, 3415
IAST [baseline] (26.964 ms) : 0, 26964
IAST [candidate] (26.992 ms) : 0, 26992
section profiling
ProfilingAgent [baseline] (99.63 ms) : 0, 99630
ProfilingAgent [candidate] (99.729 ms) : 0, 99729
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.201 ms) : 0, 1201
BytebuddyAgent [baseline] (683.05 ms) : 0, 683050
BytebuddyAgent [candidate] (685.62 ms) : 0, 685620
AgentMeter [baseline] (8.535 ms) : 0, 8535
AgentMeter [candidate] (8.569 ms) : 0, 8569
GlobalTracer [baseline] (215.726 ms) : 0, 215726
GlobalTracer [candidate] (217.129 ms) : 0, 217129
AppSec [baseline] (32.536 ms) : 0, 32536
AppSec [candidate] (32.824 ms) : 0, 32824
Debugger [baseline] (67.537 ms) : 0, 67537
Debugger [candidate] (68.634 ms) : 0, 68634
Remote Config [baseline] (622.057 µs) : 0, 622
Remote Config [candidate] (643.697 µs) : 0, 644
Telemetry [baseline] (8.866 ms) : 0, 8866
Telemetry [candidate] (9.128 ms) : 0, 9128
Flare Poller [baseline] (3.762 ms) : 0, 3762
Flare Poller [candidate] (3.785 ms) : 0, 3785
Profiling [baseline] (100.198 ms) : 0, 100198
Profiling [candidate] (100.323 ms) : 0, 100323
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section baseline
no_agent (19.178 ms) : 18978, 19379
. : milestone, 19178,
appsec (18.275 ms) : 18094, 18456
. : milestone, 18275,
code_origins (17.939 ms) : 17759, 18120
. : milestone, 17939,
iast (18.557 ms) : 18371, 18742
. : milestone, 18557,
profiling (19.193 ms) : 18997, 19389
. : milestone, 19193,
tracing (17.562 ms) : 17386, 17738
. : milestone, 17562,
section candidate
no_agent (18.186 ms) : 17999, 18374
. : milestone, 18186,
appsec (18.394 ms) : 18208, 18580
. : milestone, 18394,
code_origins (17.671 ms) : 17497, 17844
. : milestone, 17671,
iast (18.383 ms) : 18202, 18563
. : milestone, 18383,
profiling (18.634 ms) : 18449, 18818
. : milestone, 18634,
tracing (17.431 ms) : 17259, 17604
. : milestone, 17431,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section baseline
no_agent (1.17 ms) : 1159, 1181
. : milestone, 1170,
iast (3.065 ms) : 3027, 3104
. : milestone, 3065,
iast_FULL (5.96 ms) : 5898, 6021
. : milestone, 5960,
iast_GLOBAL (3.612 ms) : 3558, 3666
. : milestone, 3612,
profiling (1.983 ms) : 1965, 2000
. : milestone, 1983,
tracing (1.771 ms) : 1756, 1786
. : milestone, 1771,
section candidate
no_agent (1.191 ms) : 1179, 1203
. : milestone, 1191,
iast (3.244 ms) : 3199, 3289
. : milestone, 3244,
iast_FULL (5.682 ms) : 5626, 5738
. : milestone, 5682,
iast_GLOBAL (3.684 ms) : 3630, 3738
. : milestone, 3684,
profiling (2.102 ms) : 2081, 2124
. : milestone, 2102,
tracing (1.75 ms) : 1736, 1764
. : milestone, 1750,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section baseline
no_agent (15.285 s) : 15285000, 15285000
. : milestone, 15285000,
appsec (14.881 s) : 14881000, 14881000
. : milestone, 14881000,
iast (18.072 s) : 18072000, 18072000
. : milestone, 18072000,
iast_GLOBAL (17.783 s) : 17783000, 17783000
. : milestone, 17783000,
profiling (15.258 s) : 15258000, 15258000
. : milestone, 15258000,
tracing (14.754 s) : 14754000, 14754000
. : milestone, 14754000,
section candidate
no_agent (14.971 s) : 14971000, 14971000
. : milestone, 14971000,
appsec (14.807 s) : 14807000, 14807000
. : milestone, 14807000,
iast (17.913 s) : 17913000, 17913000
. : milestone, 17913000,
iast_GLOBAL (18.044 s) : 18044000, 18044000
. : milestone, 18044000,
profiling (15.347 s) : 15347000, 15347000
. : milestone, 15347000,
tracing (14.73 s) : 14730000, 14730000
. : milestone, 14730000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4da900ef0f, baseline=1.60.0-SNAPSHOT~8203a22d82
dateFormat X
axisFormat %s
section baseline
no_agent (1.479 ms) : 1467, 1490
. : milestone, 1479,
appsec (3.783 ms) : 3562, 4003
. : milestone, 3783,
iast (2.261 ms) : 2192, 2331
. : milestone, 2261,
iast_GLOBAL (2.307 ms) : 2237, 2377
. : milestone, 2307,
profiling (2.128 ms) : 2071, 2185
. : milestone, 2128,
tracing (2.076 ms) : 2022, 2130
. : milestone, 2076,
section candidate
no_agent (1.475 ms) : 1463, 1486
. : milestone, 1475,
appsec (3.79 ms) : 3569, 4011
. : milestone, 3790,
iast (2.264 ms) : 2194, 2333
. : milestone, 2264,
iast_GLOBAL (2.304 ms) : 2234, 2374
. : milestone, 2304,
profiling (2.073 ms) : 2018, 2128
. : milestone, 2073,
tracing (2.071 ms) : 2018, 2125
. : milestone, 2071,
|
mcculls
force-pushed
the
mcculls/otel-metrics-async-storage
branch
from
9b76ed7 to
9623466
Compare
PerfectSlayer
approved these changes
Feb 19, 2026
Contributor
There was a problem hiding this comment.
All looking good, left very minor comments. @mhlidd feel free to have a look too 😉
🎯 suggestion: About testing, should we test for NaN values and for throwing callbacks? I saw there were handled but could be interesting to check we don't break their support later?
| BatchCallback { | ||
|
|
||
| private static final Logger LOGGER = LoggerFactory.getLogger(OtelObservableCallback.class); | ||
| private static final RatelimitedLogger RATELIMITED_LOGGER = |
Contributor
There was a problem hiding this comment.
🎯 suggestion: Typo? As the linter does not seem happy about 🤷 Same with the one from OtelObservableMeasurement
Suggested change
| private static final RatelimitedLogger RATELIMITED_LOGGER = | |
| private static final RatelimitedLogger RATE_LIMITED_LOGGER = |
Contributor
Author
There was a problem hiding this comment.
Ratelimited is supposed to be one word in the original class though, so RATELIMITED matches that
...a-agent/agent-otel/otel-shim/src/main/java/datadog/opentelemetry/shim/metrics/OtelMeter.java
Outdated
Show resolved
Hide resolved
|
|
||
| private void logNotActive() { | ||
| if (LOGGER.isDebugEnabled()) { | ||
| RATELIMITED_LOGGER.warn( |
Contributor
There was a problem hiding this comment.
❔ question: Is warn expected here? The idea would be that it should be bring to attention only if debug logs are enable. Did I get it right?
Contributor
Author
There was a problem hiding this comment.
It is expected - basically to match OTel behaviour we only want to let users know about this when debugging, but we also want to avoid a mass of repeated logs when doing that.
Since RatelimitedLogger only logs at one level the simplest solution is to check if debug is enabled first - that way users can get the same feedback in the same mode.
...t-otel/otel-shim/src/main/java/datadog/opentelemetry/shim/metrics/OtelInstrumentBuilder.java
Outdated
Show resolved
Hide resolved
gh-worker-dd-mergequeue-cf854d
bot
deleted the
mcculls/otel-metrics-async-storage
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Additional Notes
Also add tests to exercise both the observable and plain metrics API.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.