Track external accumulators in tracer instead of using SparkInfo values by charlesmyu · Pull Request #10553 · DataDog/dd-trace-java

charlesmyu · 2026-02-09T05:33:49Z

What Does This Do

Updates the metrics in the _dd.spark.sql_plan meta field to use distributions calculated from individual task metrics, rather than the naively summed metrics provided by the StageInfo objects from Spark. This is because StageInfo naively sums all accumulators, even though that may not make sense for certain Spark SQL metrics (e.g. avg hash probes per key for aggr operations). Instead, we should accumulate those ourselves into distribution metrics and emit them accordingly.

Currently in the UI, this is only used in one place (in the Spark SQL metrics in the DJM product), so we're not too worried about changing the format here. UI update to follow.

Motivation

We'd like accurate metrics for Spark SQL operations that can reflect task-level characteristics as a distribution. This brings us more in line with what is shown in the Spark UI:

Additional Notes

We can't get rid of the original map that tracks accumulators to stages as we still use that to associate Spark SQL operations to stages. However, we can avoid storing the entire accumulator now, and instead just store a simple map of accumulator ID to stage ID. This will be done in a followup PR: #10645

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-02-09T06:22:38Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date	1771530866	1771538498
git_commit_sha	`af8b844`	`e413d1d`
release_version	1.60.0-SNAPSHOT~af8b84438c	1.60.0-SNAPSHOT~e413d1d9cf

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1771540273	1771540273
ci_job_id	1441486989	1441486989
ci_pipeline_id	97739279	97739279
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-evrwqy5q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-evrwqy5q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1068747
Total [baseline] (10.877 s) : 0, 10877012
Agent [candidate] (1.072 s) : 0, 1071665
Total [candidate] (10.947 s) : 0, 10947384
section appsec
Agent [baseline] (1.242 s) : 0, 1242403
Total [baseline] (11.105 s) : 0, 11104793
Agent [candidate] (1.239 s) : 0, 1238697
Total [candidate] (11.013 s) : 0, 11013326
section iast
Agent [baseline] (1.241 s) : 0, 1240914
Total [baseline] (11.166 s) : 0, 11166215
Agent [candidate] (1.233 s) : 0, 1233413
Total [candidate] (11.314 s) : 0, 11314238
section profiling
Agent [baseline] (1.192 s) : 0, 1192055
Total [baseline] (11.016 s) : 0, 11015633
Agent [candidate] (1.193 s) : 0, 1193069
Total [candidate] (10.926 s) : 0, 10926312

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.069 s	-
Agent	appsec	1.242 s	173.656 ms (16.2%)
Agent	iast	1.241 s	172.167 ms (16.1%)
Agent	profiling	1.192 s	123.307 ms (11.5%)
Total	tracing	10.877 s	-
Total	appsec	11.105 s	227.781 ms (2.1%)
Total	iast	11.166 s	289.204 ms (2.7%)
Total	profiling	11.016 s	138.622 ms (1.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.072 s	-
Agent	appsec	1.239 s	167.032 ms (15.6%)
Agent	iast	1.233 s	161.748 ms (15.1%)
Agent	profiling	1.193 s	121.404 ms (11.3%)
Total	tracing	10.947 s	-
Total	appsec	11.013 s	65.941 ms (0.6%)
Total	iast	11.314 s	366.853 ms (3.4%)
Total	profiling	10.926 s	-21.072 ms (-0.2%)

gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (630.407 ms) : 0, 630407
BytebuddyAgent [candidate] (631.711 ms) : 0, 631711
AgentMeter [baseline] (29.14 ms) : 0, 29140
AgentMeter [candidate] (29.266 ms) : 0, 29266
GlobalTracer [baseline] (258.376 ms) : 0, 258376
GlobalTracer [candidate] (259.238 ms) : 0, 259238
AppSec [baseline] (33.007 ms) : 0, 33007
AppSec [candidate] (33.397 ms) : 0, 33397
Debugger [baseline] (63.938 ms) : 0, 63938
Debugger [candidate] (64.774 ms) : 0, 64774
Remote Config [baseline] (623.525 µs) : 0, 624
Remote Config [candidate] (629.253 µs) : 0, 629
Telemetry [baseline] (9.851 ms) : 0, 9851
Telemetry [candidate] (9.183 ms) : 0, 9183
Flare Poller [baseline] (6.018 ms) : 0, 6018
Flare Poller [candidate] (6.155 ms) : 0, 6155
section appsec
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (660.33 ms) : 0, 660330
BytebuddyAgent [candidate] (658.016 ms) : 0, 658016
AgentMeter [baseline] (11.975 ms) : 0, 11975
AgentMeter [candidate] (11.995 ms) : 0, 11995
GlobalTracer [baseline] (258.71 ms) : 0, 258710
GlobalTracer [candidate] (258.255 ms) : 0, 258255
AppSec [baseline] (168.177 ms) : 0, 168177
AppSec [candidate] (167.759 ms) : 0, 167759
Debugger [baseline] (66.817 ms) : 0, 66817
Debugger [candidate] (66.46 ms) : 0, 66460
Remote Config [baseline] (671.681 µs) : 0, 672
Remote Config [candidate] (660.821 µs) : 0, 661
Telemetry [baseline] (9.388 ms) : 0, 9388
Telemetry [candidate] (9.401 ms) : 0, 9401
Flare Poller [baseline] (3.728 ms) : 0, 3728
Flare Poller [candidate] (3.634 ms) : 0, 3634
IAST [baseline] (25.387 ms) : 0, 25387
IAST [candidate] (25.327 ms) : 0, 25327
section iast
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (802.838 ms) : 0, 802838
BytebuddyAgent [candidate] (795.852 ms) : 0, 795852
AgentMeter [baseline] (11.348 ms) : 0, 11348
AgentMeter [candidate] (11.334 ms) : 0, 11334
GlobalTracer [baseline] (248.652 ms) : 0, 248652
GlobalTracer [candidate] (247.611 ms) : 0, 247611
AppSec [baseline] (33.467 ms) : 0, 33467
AppSec [candidate] (35.325 ms) : 0, 35325
Debugger [baseline] (67.399 ms) : 0, 67399
Debugger [candidate] (66.485 ms) : 0, 66485
Remote Config [baseline] (545.028 µs) : 0, 545
Remote Config [candidate] (544.189 µs) : 0, 544
Telemetry [baseline] (8.703 ms) : 0, 8703
Telemetry [candidate] (8.694 ms) : 0, 8694
Flare Poller [baseline] (3.466 ms) : 0, 3466
Flare Poller [candidate] (3.511 ms) : 0, 3511
IAST [baseline] (27.153 ms) : 0, 27153
IAST [candidate] (26.985 ms) : 0, 26985
section profiling
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (681.844 ms) : 0, 681844
BytebuddyAgent [candidate] (682.574 ms) : 0, 682574
AgentMeter [baseline] (8.55 ms) : 0, 8550
AgentMeter [candidate] (8.61 ms) : 0, 8610
GlobalTracer [baseline] (216.071 ms) : 0, 216071
GlobalTracer [candidate] (216.458 ms) : 0, 216458
AppSec [baseline] (32.719 ms) : 0, 32719
AppSec [candidate] (32.857 ms) : 0, 32857
Debugger [baseline] (67.175 ms) : 0, 67175
Debugger [candidate] (67.463 ms) : 0, 67463
Remote Config [baseline] (625.542 µs) : 0, 626
Remote Config [candidate] (635.346 µs) : 0, 635
Telemetry [baseline] (8.965 ms) : 0, 8965
Telemetry [candidate] (9.071 ms) : 0, 9071
Flare Poller [baseline] (3.832 ms) : 0, 3832
Flare Poller [candidate] (3.754 ms) : 0, 3754
ProfilingAgent [baseline] (100.414 ms) : 0, 100414
ProfilingAgent [candidate] (99.836 ms) : 0, 99836
Profiling [baseline] (101.002 ms) : 0, 101002
Profiling [candidate] (100.416 ms) : 0, 100416

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065499
Total [baseline] (8.72 s) : 0, 8719840
Agent [candidate] (1.07 s) : 0, 1070364
Total [candidate] (8.757 s) : 0, 8757404
section iast
Agent [baseline] (1.229 s) : 0, 1229421
Total [baseline] (9.374 s) : 0, 9373703
Agent [candidate] (1.232 s) : 0, 1231501
Total [candidate] (9.363 s) : 0, 9363281

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.229 s	163.922 ms (15.4%)
Total	tracing	8.72 s	-
Total	iast	9.374 s	653.862 ms (7.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.07 s	-
Agent	iast	1.232 s	161.138 ms (15.1%)
Total	tracing	8.757 s	-
Total	iast	9.363 s	605.877 ms (6.9%)

gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (627.579 ms) : 0, 627579
BytebuddyAgent [candidate] (633.164 ms) : 0, 633164
AgentMeter [baseline] (29.087 ms) : 0, 29087
AgentMeter [candidate] (29.316 ms) : 0, 29316
GlobalTracer [baseline] (257.55 ms) : 0, 257550
GlobalTracer [candidate] (258.005 ms) : 0, 258005
AppSec [baseline] (33.031 ms) : 0, 33031
AppSec [candidate] (32.95 ms) : 0, 32950
Debugger [baseline] (65.271 ms) : 0, 65271
Debugger [candidate] (64.388 ms) : 0, 64388
Remote Config [baseline] (631.745 µs) : 0, 632
Remote Config [candidate] (632.461 µs) : 0, 632
Telemetry [baseline] (10.562 ms) : 0, 10562
Telemetry [candidate] (10.706 ms) : 0, 10706
Flare Poller [baseline] (4.489 ms) : 0, 4489
Flare Poller [candidate] (3.772 ms) : 0, 3772
section iast
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (794.652 ms) : 0, 794652
BytebuddyAgent [candidate] (794.916 ms) : 0, 794916
AgentMeter [baseline] (11.317 ms) : 0, 11317
AgentMeter [candidate] (11.31 ms) : 0, 11310
GlobalTracer [baseline] (247.38 ms) : 0, 247380
GlobalTracer [candidate] (247.923 ms) : 0, 247923
AppSec [baseline] (33.164 ms) : 0, 33164
AppSec [candidate] (31.633 ms) : 0, 31633
Debugger [baseline] (66.051 ms) : 0, 66051
Debugger [candidate] (69.152 ms) : 0, 69152
Remote Config [baseline] (548.871 µs) : 0, 549
Remote Config [candidate] (535.661 µs) : 0, 536
Telemetry [baseline] (8.601 ms) : 0, 8601
Telemetry [candidate] (8.479 ms) : 0, 8479
Flare Poller [baseline] (3.392 ms) : 0, 3392
Flare Poller [candidate] (3.492 ms) : 0, 3492
IAST [baseline] (27.062 ms) : 0, 27062
IAST [candidate] (26.846 ms) : 0, 26846

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date	1771530866	1771538498
git_commit_sha	`af8b844`	`e413d1d`
release_version	1.60.0-SNAPSHOT~af8b84438c	1.60.0-SNAPSHOT~e413d1d9cf

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1771540759	1771540759
ci_job_id	1441486990	1441486990
ci_pipeline_id	97739279	97739279
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ryeu22h0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ryeu22h0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 4 performance improvements and 2 performance regressions! Performance is the same for 11 metrics, 19 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-153.608µs; -79.681µs] or [-6.166%; -3.198%]	unsure [-361.222µs; -1.211µs] or [-4.924%; -0.017%]	unstable [-92.553op/s; +196.241op/s] or [-6.517%; +13.819%]	2.375ms	7.154ms	1471.969op/s	2.491ms	7.335ms	1420.125op/s
scenario:load:insecure-bank:iast_FULL:high_load	worse [+203.960µs; +422.246µs] or [+4.031%; +8.345%]	worse [+486.099µs; +1115.204µs] or [+4.012%; +9.205%]	unstable [-121.886op/s; +28.323op/s] or [-15.027%; +3.492%]	5.373ms	12.916ms	764.344op/s	5.060ms	12.116ms	811.125op/s
scenario:load:petclinic:tracing:high_load	better [-2.160ms; -0.977ms] or [-11.656%; -5.272%]	better [-3.535ms; -0.979ms] or [-11.621%; -3.217%]	unstable [-18.385op/s; +36.984op/s] or [-7.198%; +14.480%]	16.966ms	28.164ms	264.719op/s	18.535ms	30.421ms	255.419op/s
scenario:load:petclinic:iast:high_load	better [-1294.145µs; -439.921µs] or [-7.067%; -2.402%]	same [-1649.780µs; +251.561µs] or [-5.547%; +0.846%]	unstable [-6.853op/s; +47.105op/s] or [-2.742%; +18.849%]	17.447ms	29.042ms	270.032op/s	18.314ms	29.741ms	249.906op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.435 ms) : 18238, 18632
.   : milestone, 18435,
appsec (18.754 ms) : 18565, 18942
.   : milestone, 18754,
code_origins (17.827 ms) : 17646, 18007
.   : milestone, 17827,
iast (18.683 ms) : 18493, 18872
.   : milestone, 18683,
profiling (18.458 ms) : 18272, 18644
.   : milestone, 18458,
tracing (18.859 ms) : 18667, 19051
.   : milestone, 18859,
section candidate
no_agent (18.972 ms) : 18777, 19167
.   : milestone, 18972,
appsec (18.727 ms) : 18535, 18918
.   : milestone, 18727,
code_origins (18.668 ms) : 18478, 18857
.   : milestone, 18668,
iast (17.836 ms) : 17656, 18017
.   : milestone, 17836,
profiling (18.732 ms) : 18544, 18920
.   : milestone, 18732,
tracing (17.625 ms) : 17450, 17800
.   : milestone, 17625,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.435 ms [18.238 ms, 18.632 ms]	-
appsec	18.754 ms [18.565 ms, 18.942 ms]	318.687 µs (1.7%)
code_origins	17.827 ms [17.646 ms, 18.007 ms]	-608.38 µs (-3.3%)
iast	18.683 ms [18.493 ms, 18.872 ms]	247.694 µs (1.3%)
profiling	18.458 ms [18.272 ms, 18.644 ms]	23.238 µs (0.1%)
tracing	18.859 ms [18.667 ms, 19.051 ms]	424.403 µs (2.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.972 ms [18.777 ms, 19.167 ms]	-
appsec	18.727 ms [18.535 ms, 18.918 ms]	-245.57 µs (-1.3%)
code_origins	18.668 ms [18.478 ms, 18.857 ms]	-304.348 µs (-1.6%)
iast	17.836 ms [17.656 ms, 18.017 ms]	-1.136 ms (-6.0%)
profiling	18.732 ms [18.544 ms, 18.92 ms]	-239.889 µs (-1.3%)
tracing	17.625 ms [17.45 ms, 17.8 ms]	-1.347 ms (-7.1%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.175 ms) : 1163, 1186
.   : milestone, 1175,
iast (3.223 ms) : 3176, 3269
.   : milestone, 3223,
iast_FULL (5.7 ms) : 5644, 5756
.   : milestone, 5700,
iast_GLOBAL (3.512 ms) : 3455, 3569
.   : milestone, 3512,
profiling (2.07 ms) : 2052, 2089
.   : milestone, 2070,
tracing (1.827 ms) : 1811, 1844
.   : milestone, 1827,
section candidate
no_agent (1.185 ms) : 1173, 1197
.   : milestone, 1185,
iast (3.106 ms) : 3064, 3149
.   : milestone, 3106,
iast_FULL (6.052 ms) : 5989, 6115
.   : milestone, 6052,
iast_GLOBAL (3.475 ms) : 3419, 3531
.   : milestone, 3475,
profiling (2.064 ms) : 2044, 2084
.   : milestone, 2064,
tracing (1.843 ms) : 1827, 1859
.   : milestone, 1843,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.175 ms [1.163 ms, 1.186 ms]	-
iast	3.223 ms [3.176 ms, 3.269 ms]	2.048 ms (174.3%)
iast_FULL	5.7 ms [5.644 ms, 5.756 ms]	4.525 ms (385.3%)
iast_GLOBAL	3.512 ms [3.455 ms, 3.569 ms]	2.337 ms (199.0%)
profiling	2.07 ms [2.052 ms, 2.089 ms]	895.689 µs (76.3%)
tracing	1.827 ms [1.811 ms, 1.844 ms]	652.724 µs (55.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.185 ms [1.173 ms, 1.197 ms]	-
iast	3.106 ms [3.064 ms, 3.149 ms]	1.921 ms (162.1%)
iast_FULL	6.052 ms [5.989 ms, 6.115 ms]	4.867 ms (410.7%)
iast_GLOBAL	3.475 ms [3.419 ms, 3.531 ms]	2.29 ms (193.2%)
profiling	2.064 ms [2.044 ms, 2.084 ms]	878.832 µs (74.2%)
tracing	1.843 ms [1.827 ms, 1.859 ms]	658.296 µs (55.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date	1771530866	1771538498
git_commit_sha	`af8b844`	`e413d1d`
release_version	1.60.0-SNAPSHOT~af8b84438c	1.60.0-SNAPSHOT~e413d1d9cf

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1771540481	1771540481
ci_job_id	1441486992	1441486992
ci_pipeline_id	97739279	97739279
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-91pcj3k6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-91pcj3k6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (3.758 ms) : 3536, 3980
.   : milestone, 3758,
iast (2.258 ms) : 2188, 2328
.   : milestone, 2258,
iast_GLOBAL (2.293 ms) : 2223, 2363
.   : milestone, 2293,
profiling (2.104 ms) : 2047, 2161
.   : milestone, 2104,
tracing (2.054 ms) : 2000, 2109
.   : milestone, 2054,
section candidate
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (3.725 ms) : 3506, 3944
.   : milestone, 3725,
iast (2.246 ms) : 2177, 2315
.   : milestone, 2246,
iast_GLOBAL (2.29 ms) : 2221, 2360
.   : milestone, 2290,
profiling (2.099 ms) : 2042, 2156
.   : milestone, 2099,
tracing (2.047 ms) : 1993, 2100
.   : milestone, 2047,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.47 ms [1.458 ms, 1.481 ms]	-
appsec	3.758 ms [3.536 ms, 3.98 ms]	2.288 ms (155.7%)
iast	2.258 ms [2.188 ms, 2.328 ms]	787.913 µs (53.6%)
iast_GLOBAL	2.293 ms [2.223 ms, 2.363 ms]	823.276 µs (56.0%)
profiling	2.104 ms [2.047 ms, 2.161 ms]	634.236 µs (43.2%)
tracing	2.054 ms [2.0 ms, 2.109 ms]	584.569 µs (39.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.454 ms, 1.477 ms]	-
appsec	3.725 ms [3.506 ms, 3.944 ms]	2.26 ms (154.2%)
iast	2.246 ms [2.177 ms, 2.315 ms]	780.485 µs (53.3%)
iast_GLOBAL	2.29 ms [2.221 ms, 2.36 ms]	825.017 µs (56.3%)
profiling	2.099 ms [2.042 ms, 2.156 ms]	633.576 µs (43.2%)
tracing	2.047 ms [1.993 ms, 2.1 ms]	581.162 µs (39.7%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~e413d1d9cf, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.497 s) : 15497000, 15497000
.   : milestone, 15497000,
appsec (14.671 s) : 14671000, 14671000
.   : milestone, 14671000,
iast (18.213 s) : 18213000, 18213000
.   : milestone, 18213000,
iast_GLOBAL (17.808 s) : 17808000, 17808000
.   : milestone, 17808000,
profiling (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,
tracing (14.585 s) : 14585000, 14585000
.   : milestone, 14585000,
section candidate
no_agent (15.673 s) : 15673000, 15673000
.   : milestone, 15673000,
appsec (15.003 s) : 15003000, 15003000
.   : milestone, 15003000,
iast (18.349 s) : 18349000, 18349000
.   : milestone, 18349000,
iast_GLOBAL (18.184 s) : 18184000, 18184000
.   : milestone, 18184000,
profiling (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
tracing (14.568 s) : 14568000, 14568000
.   : milestone, 14568000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.497 s [15.497 s, 15.497 s]	-
appsec	14.671 s [14.671 s, 14.671 s]	-826.0 ms (-5.3%)
iast	18.213 s [18.213 s, 18.213 s]	2.716 s (17.5%)
iast_GLOBAL	17.808 s [17.808 s, 17.808 s]	2.311 s (14.9%)
profiling	14.972 s [14.972 s, 14.972 s]	-525.0 ms (-3.4%)
tracing	14.585 s [14.585 s, 14.585 s]	-912.0 ms (-5.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.673 s [15.673 s, 15.673 s]	-
appsec	15.003 s [15.003 s, 15.003 s]	-670.0 ms (-4.3%)
iast	18.349 s [18.349 s, 18.349 s]	2.676 s (17.1%)
iast_GLOBAL	18.184 s [18.184 s, 18.184 s]	2.511 s (16.0%)
profiling	14.984 s [14.984 s, 14.984 s]	-689.0 ms (-4.4%)
tracing	14.568 s [14.568 s, 14.568 s]	-1.105 s (-7.1%)

charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from 4e5bdc7 to ba09c80 Compare February 9, 2026 14:48

charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch 5 times, most recently from cde7981 to e52fbc5 Compare February 19, 2026 21:41

charlesmyu mentioned this pull request Feb 19, 2026

Store accumulator-stage lookups directly #10645

Draft

Track external accumulators in tracer instead of using SparkInfo values

e413d1d

charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from e52fbc5 to e413d1d Compare February 19, 2026 22:02

charlesmyu added the inst: apache spark Apache Spark instrumentation label Feb 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Track external accumulators in tracer instead of using SparkInfo values#10553

Track external accumulators in tracer instead of using SparkInfo values#10553
charlesmyu wants to merge 1 commit intomasterfrom
charles.yu/djm-0000/fix-spark-plan-metrics

charlesmyu commented Feb 9, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Feb 9, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

Conversation

charlesmyu commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

charlesmyu commented Feb 9, 2026 •

edited

Loading

pr-commenter bot commented Feb 9, 2026 •

edited

Loading