Update startup benchmark SLO by sarahchen6 · Pull Request #11221 · DataDog/dd-trace-java

sarahchen6 · 2026-04-28T15:24:29Z

What Does This Do

Update startup benchmark failure threshold based on automatically generated SLOs

Motivation

The startup benchmarks have been stable since they were defined in #10090. The new threshold is set automatically based on the latest data on master (pipeline) and according to https://github.com/DataDog/relenv-benchmark-analyzer/blob/main/README.md#generate, as recommended by the edited file.

Additional Notes

To check our startup benchmark history, take a look at the Perf SLOs dashboard and Legacy dashboard

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-04-28T19:31:29Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-startup-slo
git_commit_date	1777379592	1777469046
git_commit_sha	`9d73760`	`01ca5aa`
release_version	1.62.0-SNAPSHOT~9d737609c4	1.62.0-SNAPSHOT~01ca5aab16

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1777470791	1777470791
ci_job_id	1642192980	1642192980
ci_pipeline_id	110430925	110430925
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-43mv8ou3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-43mv8ou3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066311
Total [baseline] (8.841 s) : 0, 8840808
Agent [candidate] (1.071 s) : 0, 1070603
Total [candidate] (8.882 s) : 0, 8882053
section iast
Agent [baseline] (1.26 s) : 0, 1260110
Total [baseline] (9.59 s) : 0, 9589540
Agent [candidate] (1.251 s) : 0, 1251268
Total [candidate] (9.634 s) : 0, 9633979

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	iast	1.26 s	193.8 ms (18.2%)
Total	tracing	8.841 s	-
Total	iast	9.59 s	748.732 ms (8.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	iast	1.251 s	180.665 ms (16.9%)
Total	tracing	8.882 s	-
Total	iast	9.634 s	751.926 ms (8.5%)

gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.259 ms) : 0, 1259
BytebuddyAgent [baseline] (636.853 ms) : 0, 636853
BytebuddyAgent [candidate] (640.414 ms) : 0, 640414
AgentMeter [baseline] (29.426 ms) : 0, 29426
AgentMeter [candidate] (29.77 ms) : 0, 29770
GlobalTracer [baseline] (249.297 ms) : 0, 249297
GlobalTracer [candidate] (250.889 ms) : 0, 250889
AppSec [baseline] (32.899 ms) : 0, 32899
AppSec [candidate] (32.962 ms) : 0, 32962
Debugger [baseline] (59.943 ms) : 0, 59943
Debugger [candidate] (60.277 ms) : 0, 60277
Remote Config [baseline] (603.394 µs) : 0, 603
Remote Config [candidate] (610.048 µs) : 0, 610
Telemetry [baseline] (9.966 ms) : 0, 9966
Telemetry [candidate] (9.149 ms) : 0, 9149
Flare Poller [baseline] (10.002 ms) : 0, 10002
Flare Poller [candidate] (9.014 ms) : 0, 9014
section iast
crashtracking [baseline] (1.271 ms) : 0, 1271
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (834.614 ms) : 0, 834614
BytebuddyAgent [candidate] (825.989 ms) : 0, 825989
AgentMeter [baseline] (11.685 ms) : 0, 11685
AgentMeter [candidate] (11.421 ms) : 0, 11421
GlobalTracer [baseline] (240.192 ms) : 0, 240192
GlobalTracer [candidate] (241.314 ms) : 0, 241314
IAST [baseline] (27.744 ms) : 0, 27744
IAST [candidate] (28.015 ms) : 0, 28015
AppSec [baseline] (32.809 ms) : 0, 32809
AppSec [candidate] (32.005 ms) : 0, 32005
Debugger [baseline] (63.344 ms) : 0, 63344
Debugger [candidate] (63.143 ms) : 0, 63143
Remote Config [baseline] (539.465 µs) : 0, 539
Remote Config [candidate] (546.253 µs) : 0, 546
Telemetry [baseline] (8.067 ms) : 0, 8067
Telemetry [candidate] (8.074 ms) : 0, 8074
Flare Poller [baseline] (3.456 ms) : 0, 3456
Flare Poller [candidate] (3.399 ms) : 0, 3399

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1075540
Total [baseline] (11.217 s) : 0, 11216898
Agent [candidate] (1.065 s) : 0, 1064804
Total [candidate] (11.123 s) : 0, 11122830
section appsec
Agent [baseline] (1.269 s) : 0, 1268948
Total [baseline] (11.075 s) : 0, 11074896
Agent [candidate] (1.269 s) : 0, 1268854
Total [candidate] (11.067 s) : 0, 11067437
section iast
Agent [baseline] (1.249 s) : 0, 1248912
Total [baseline] (11.325 s) : 0, 11324636
Agent [candidate] (1.248 s) : 0, 1248165
Total [candidate] (11.393 s) : 0, 11392879
section profiling
Agent [baseline] (1.186 s) : 0, 1186346
Total [baseline] (11.029 s) : 0, 11029139
Agent [candidate] (1.189 s) : 0, 1188936
Total [candidate] (10.947 s) : 0, 10947450

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.076 s	-
Agent	appsec	1.269 s	193.408 ms (18.0%)
Agent	iast	1.249 s	173.372 ms (16.1%)
Agent	profiling	1.186 s	110.806 ms (10.3%)
Total	tracing	11.217 s	-
Total	appsec	11.075 s	-142.002 ms (-1.3%)
Total	iast	11.325 s	107.738 ms (1.0%)
Total	profiling	11.029 s	-187.759 ms (-1.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.269 s	204.05 ms (19.2%)
Agent	iast	1.248 s	183.361 ms (17.2%)
Agent	profiling	1.189 s	124.132 ms (11.7%)
Total	tracing	11.123 s	-
Total	appsec	11.067 s	-55.394 ms (-0.5%)
Total	iast	11.393 s	270.049 ms (2.4%)
Total	profiling	10.947 s	-175.38 ms (-1.6%)

gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.248 ms) : 0, 1248
crashtracking [candidate] (1.228 ms) : 0, 1228
BytebuddyAgent [baseline] (642.413 ms) : 0, 642413
BytebuddyAgent [candidate] (637.052 ms) : 0, 637052
AgentMeter [baseline] (29.966 ms) : 0, 29966
AgentMeter [candidate] (29.494 ms) : 0, 29494
GlobalTracer [baseline] (251.558 ms) : 0, 251558
GlobalTracer [candidate] (249.032 ms) : 0, 249032
AppSec [baseline] (33.014 ms) : 0, 33014
AppSec [candidate] (32.71 ms) : 0, 32710
Debugger [baseline] (61.299 ms) : 0, 61299
Debugger [candidate] (60.545 ms) : 0, 60545
Remote Config [baseline] (606.731 µs) : 0, 607
Remote Config [candidate] (609.859 µs) : 0, 610
Telemetry [baseline] (9.31 ms) : 0, 9310
Telemetry [candidate] (8.358 ms) : 0, 8358
Flare Poller [baseline] (9.836 ms) : 0, 9836
Flare Poller [candidate] (9.656 ms) : 0, 9656
section appsec
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (675.589 ms) : 0, 675589
BytebuddyAgent [candidate] (676.755 ms) : 0, 676755
AgentMeter [baseline] (12.197 ms) : 0, 12197
AgentMeter [candidate] (12.25 ms) : 0, 12250
GlobalTracer [baseline] (251.261 ms) : 0, 251261
GlobalTracer [candidate] (250.125 ms) : 0, 250125
AppSec [baseline] (186.554 ms) : 0, 186554
AppSec [candidate] (185.656 ms) : 0, 185656
Debugger [baseline] (65.588 ms) : 0, 65588
Debugger [candidate] (64.849 ms) : 0, 64849
Remote Config [baseline] (578.739 µs) : 0, 579
Remote Config [candidate] (566.527 µs) : 0, 567
Telemetry [baseline] (7.942 ms) : 0, 7942
Telemetry [candidate] (7.866 ms) : 0, 7866
Flare Poller [baseline] (6.225 ms) : 0, 6225
Flare Poller [candidate] (7.408 ms) : 0, 7408
IAST [baseline] (25.011 ms) : 0, 25011
IAST [candidate] (24.856 ms) : 0, 24856
section iast
crashtracking [baseline] (1.24 ms) : 0, 1240
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (824.908 ms) : 0, 824908
BytebuddyAgent [candidate] (824.723 ms) : 0, 824723
AgentMeter [baseline] (11.288 ms) : 0, 11288
AgentMeter [candidate] (11.405 ms) : 0, 11405
GlobalTracer [baseline] (239.339 ms) : 0, 239339
GlobalTracer [candidate] (240.086 ms) : 0, 240086
AppSec [baseline] (33.48 ms) : 0, 33480
AppSec [candidate] (30.659 ms) : 0, 30659
Debugger [baseline] (63.59 ms) : 0, 63590
Debugger [candidate] (63.568 ms) : 0, 63568
Remote Config [baseline] (545.31 µs) : 0, 545
Remote Config [candidate] (529.566 µs) : 0, 530
Telemetry [baseline] (8.031 ms) : 0, 8031
Telemetry [candidate] (7.981 ms) : 0, 7981
Flare Poller [baseline] (3.482 ms) : 0, 3482
Flare Poller [candidate] (3.507 ms) : 0, 3507
IAST [baseline] (26.805 ms) : 0, 26805
IAST [candidate] (28.341 ms) : 0, 28341
section profiling
ProfilingAgent [baseline] (94.298 ms) : 0, 94298
ProfilingAgent [candidate] (94.212 ms) : 0, 94212
crashtracking [baseline] (1.181 ms) : 0, 1181
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (691.342 ms) : 0, 691342
BytebuddyAgent [candidate] (693.369 ms) : 0, 693369
AgentMeter [baseline] (8.911 ms) : 0, 8911
AgentMeter [candidate] (9.002 ms) : 0, 9002
GlobalTracer [baseline] (208.216 ms) : 0, 208216
GlobalTracer [candidate] (208.408 ms) : 0, 208408
AppSec [baseline] (32.712 ms) : 0, 32712
AppSec [candidate] (32.812 ms) : 0, 32812
Debugger [baseline] (65.913 ms) : 0, 65913
Debugger [candidate] (65.934 ms) : 0, 65934
Remote Config [baseline] (571.905 µs) : 0, 572
Remote Config [candidate] (585.103 µs) : 0, 585
Telemetry [baseline] (8.115 ms) : 0, 8115
Telemetry [candidate] (8.107 ms) : 0, 8107
Flare Poller [baseline] (3.562 ms) : 0, 3562
Flare Poller [candidate] (3.586 ms) : 0, 3586
Profiling [baseline] (94.857 ms) : 0, 94857
Profiling [candidate] (94.76 ms) : 0, 94760

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-startup-slo
git_commit_date	1777379592	1777469046
git_commit_sha	`9d73760`	`01ca5aa`
release_version	1.62.0-SNAPSHOT~9d737609c4	1.62.0-SNAPSHOT~01ca5aab16

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1777471372	1777471372
ci_job_id	1642192981	1642192981
ci_pipeline_id	110430925	110430925
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-cnpebupo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-cnpebupo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 5 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-239.400µs; -159.862µs] or [-9.191%; -6.138%]	better [-704.095µs; -305.694µs] or [-9.188%; -3.989%]	unstable [-23.173op/s; +241.048op/s] or [-1.703%; +17.719%]	2.405ms	7.159ms	1469.344op/s	2.605ms	7.664ms	1360.406op/s
scenario:load:insecure-bank:iast_FULL:high_load	better [-454.838µs; -252.659µs] or [-8.001%; -4.445%]	better [-1216.616µs; -539.229µs] or [-8.973%; -3.977%]	unstable [-23.952op/s; +110.202op/s] or [-3.282%; +15.099%]	5.331ms	12.681ms	772.969op/s	5.685ms	13.559ms	729.844op/s
scenario:load:petclinic:iast:high_load	better [-1221.920µs; -412.663µs] or [-6.609%; -2.232%]	unsure [-1383.005µs; -13.926µs] or [-4.693%; -0.047%]	unstable [-17.572op/s; +38.322op/s] or [-7.067%; +15.412%]	17.672ms	28.770ms	259.031op/s	18.489ms	29.468ms	248.656op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.378 ms) : 19177, 19578
.   : milestone, 19378,
appsec (20.007 ms) : 19802, 20213
.   : milestone, 20007,
code_origins (18.068 ms) : 17888, 18248
.   : milestone, 18068,
iast (18.767 ms) : 18583, 18951
.   : milestone, 18767,
profiling (18.433 ms) : 18250, 18616
.   : milestone, 18433,
tracing (17.916 ms) : 17737, 18094
.   : milestone, 17916,
section candidate
no_agent (18.101 ms) : 17917, 18286
.   : milestone, 18101,
appsec (19.327 ms) : 19135, 19519
.   : milestone, 19327,
code_origins (17.833 ms) : 17658, 18009
.   : milestone, 17833,
iast (18.008 ms) : 17830, 18186
.   : milestone, 18008,
profiling (18.546 ms) : 18358, 18734
.   : milestone, 18546,
tracing (18.185 ms) : 18005, 18364
.   : milestone, 18185,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.378 ms [19.177 ms, 19.578 ms]	-
appsec	20.007 ms [19.802 ms, 20.213 ms]	629.589 µs (3.2%)
code_origins	18.068 ms [17.888 ms, 18.248 ms]	-1.309 ms (-6.8%)
iast	18.767 ms [18.583 ms, 18.951 ms]	-610.5 µs (-3.2%)
profiling	18.433 ms [18.25 ms, 18.616 ms]	-944.622 µs (-4.9%)
tracing	17.916 ms [17.737 ms, 18.094 ms]	-1.462 ms (-7.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.101 ms [17.917 ms, 18.286 ms]	-
appsec	19.327 ms [19.135 ms, 19.519 ms]	1.225 ms (6.8%)
code_origins	17.833 ms [17.658 ms, 18.009 ms]	-268.19 µs (-1.5%)
iast	18.008 ms [17.83 ms, 18.186 ms]	-93.453 µs (-0.5%)
profiling	18.546 ms [18.358 ms, 18.734 ms]	444.506 µs (2.5%)
tracing	18.185 ms [18.005 ms, 18.364 ms]	83.333 µs (0.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.263 ms) : 1250, 1276
.   : milestone, 1263,
iast (3.365 ms) : 3314, 3416
.   : milestone, 3365,
iast_FULL (6.341 ms) : 6276, 6407
.   : milestone, 6341,
iast_GLOBAL (3.673 ms) : 3614, 3732
.   : milestone, 3673,
profiling (2.662 ms) : 2632, 2691
.   : milestone, 2662,
tracing (1.932 ms) : 1916, 1948
.   : milestone, 1932,
section candidate
no_agent (1.241 ms) : 1228, 1253
.   : milestone, 1241,
iast (3.111 ms) : 3073, 3149
.   : milestone, 3111,
iast_FULL (5.982 ms) : 5921, 6042
.   : milestone, 5982,
iast_GLOBAL (3.617 ms) : 3562, 3673
.   : milestone, 3617,
profiling (2.422 ms) : 2398, 2446
.   : milestone, 2422,
tracing (1.889 ms) : 1874, 1905
.   : milestone, 1889,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.263 ms [1.25 ms, 1.276 ms]	-
iast	3.365 ms [3.314 ms, 3.416 ms]	2.102 ms (166.4%)
iast_FULL	6.341 ms [6.276 ms, 6.407 ms]	5.078 ms (402.0%)
iast_GLOBAL	3.673 ms [3.614 ms, 3.732 ms]	2.41 ms (190.8%)
profiling	2.662 ms [2.632 ms, 2.691 ms]	1.399 ms (110.7%)
tracing	1.932 ms [1.916 ms, 1.948 ms]	669.154 µs (53.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.241 ms [1.228 ms, 1.253 ms]	-
iast	3.111 ms [3.073 ms, 3.149 ms]	1.87 ms (150.7%)
iast_FULL	5.982 ms [5.921 ms, 6.042 ms]	4.741 ms (382.1%)
iast_GLOBAL	3.617 ms [3.562 ms, 3.673 ms]	2.376 ms (191.5%)
profiling	2.422 ms [2.398 ms, 2.446 ms]	1.181 ms (95.2%)
tracing	1.889 ms [1.874 ms, 1.905 ms]	648.439 µs (52.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-startup-slo
git_commit_date	1777379592	1777469046
git_commit_sha	`9d73760`	`01ca5aa`
release_version	1.62.0-SNAPSHOT~9d737609c4	1.62.0-SNAPSHOT~01ca5aab16

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1777471008	1777471008
ci_job_id	1642192982	1642192982
ci_pipeline_id	110430925	110430925
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-s5lmcpui 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-s5lmcpui 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.374 s) : 15374000, 15374000
.   : milestone, 15374000,
appsec (14.715 s) : 14715000, 14715000
.   : milestone, 14715000,
iast (18.636 s) : 18636000, 18636000
.   : milestone, 18636000,
iast_GLOBAL (17.997 s) : 17997000, 17997000
.   : milestone, 17997000,
profiling (15.435 s) : 15435000, 15435000
.   : milestone, 15435000,
tracing (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
section candidate
no_agent (15.523 s) : 15523000, 15523000
.   : milestone, 15523000,
appsec (15.102 s) : 15102000, 15102000
.   : milestone, 15102000,
iast (18.29 s) : 18290000, 18290000
.   : milestone, 18290000,
iast_GLOBAL (17.991 s) : 17991000, 17991000
.   : milestone, 17991000,
profiling (15.519 s) : 15519000, 15519000
.   : milestone, 15519000,
tracing (14.736 s) : 14736000, 14736000
.   : milestone, 14736000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.374 s [15.374 s, 15.374 s]	-
appsec	14.715 s [14.715 s, 14.715 s]	-659.0 ms (-4.3%)
iast	18.636 s [18.636 s, 18.636 s]	3.262 s (21.2%)
iast_GLOBAL	17.997 s [17.997 s, 17.997 s]	2.623 s (17.1%)
profiling	15.435 s [15.435 s, 15.435 s]	61.0 ms (0.4%)
tracing	14.965 s [14.965 s, 14.965 s]	-409.0 ms (-2.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.523 s [15.523 s, 15.523 s]	-
appsec	15.102 s [15.102 s, 15.102 s]	-421.0 ms (-2.7%)
iast	18.29 s [18.29 s, 18.29 s]	2.767 s (17.8%)
iast_GLOBAL	17.991 s [17.991 s, 17.991 s]	2.468 s (15.9%)
profiling	15.519 s [15.519 s, 15.519 s]	-4.0 ms (-0.0%)
tracing	14.736 s [14.736 s, 14.736 s]	-787.0 ms (-5.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~01ca5aab16, baseline=1.62.0-SNAPSHOT~9d737609c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.488 ms) : 1476, 1499
.   : milestone, 1488,
appsec (3.752 ms) : 3535, 3968
.   : milestone, 3752,
iast (2.28 ms) : 2210, 2350
.   : milestone, 2280,
iast_GLOBAL (2.326 ms) : 2255, 2396
.   : milestone, 2326,
profiling (2.107 ms) : 2052, 2162
.   : milestone, 2107,
tracing (2.094 ms) : 2040, 2148
.   : milestone, 2094,
section candidate
no_agent (1.487 ms) : 1476, 1499
.   : milestone, 1487,
appsec (3.824 ms) : 3603, 4046
.   : milestone, 3824,
iast (2.276 ms) : 2206, 2345
.   : milestone, 2276,
iast_GLOBAL (2.332 ms) : 2261, 2402
.   : milestone, 2332,
profiling (2.496 ms) : 2341, 2651
.   : milestone, 2496,
tracing (2.078 ms) : 2024, 2131
.   : milestone, 2078,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.476 ms, 1.499 ms]	-
appsec	3.752 ms [3.535 ms, 3.968 ms]	2.264 ms (152.1%)
iast	2.28 ms [2.21 ms, 2.35 ms]	792.34 µs (53.3%)
iast_GLOBAL	2.326 ms [2.255 ms, 2.396 ms]	837.887 µs (56.3%)
profiling	2.107 ms [2.052 ms, 2.162 ms]	619.272 µs (41.6%)
tracing	2.094 ms [2.04 ms, 2.148 ms]	606.019 µs (40.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.487 ms [1.476 ms, 1.499 ms]	-
appsec	3.824 ms [3.603 ms, 4.046 ms]	2.337 ms (157.2%)
iast	2.276 ms [2.206 ms, 2.345 ms]	788.372 µs (53.0%)
iast_GLOBAL	2.332 ms [2.261 ms, 2.402 ms]	844.412 µs (56.8%)
profiling	2.496 ms [2.341 ms, 2.651 ms]	1.009 ms (67.9%)
tracing	2.078 ms [2.024 ms, 2.131 ms]	590.581 µs (39.7%)

sarahchen6 · 2026-04-29T17:07:12Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-04-29T17:07:17Z

View all feedbacks in Devflow UI.

2026-04-29 17:07:16 UTC ℹ️ Start processing command /merge

2026-04-29 17:07:21 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-04-29 18:19:41 UTC ℹ️ MergeQueue: This merge request was merged

Update startup benchmark SLO

1e3abe4

sarahchen6 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Apr 28, 2026

Recalculate both startup thresholds off latest master data

86e101a

Update with latest results

01ca5aa

sarahchen6 marked this pull request as ready for review April 29, 2026 13:28

sarahchen6 requested a review from a team as a code owner April 29, 2026 13:28

sarahchen6 requested review from AlexeyKuznetsov-DD, PerfectSlayer and erikayasuda and removed request for a team April 29, 2026 13:28

AlexeyKuznetsov-DD approved these changes Apr 29, 2026

View reviewed changes

gh-worker-dd-mergequeue-cf854d Bot merged commit b868f3c into master Apr 29, 2026
579 of 582 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the sarahchen6/update-startup-slo branch April 29, 2026 18:19

github-actions Bot added this to the 1.62.0 milestone Apr 29, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update startup benchmark SLO#11221

Update startup benchmark SLO#11221
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits intomasterfrom
sarahchen6/update-startup-slo

sarahchen6 commented Apr 28, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

sarahchen6 commented Apr 29, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Apr 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sarahchen6 commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

sarahchen6 commented Apr 29, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sarahchen6 commented Apr 28, 2026 •

edited

Loading

pr-commenter Bot commented Apr 28, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Apr 29, 2026 •

edited

Loading