[POC] Bench: Add Dabao (Baochip-1x) benchmarks - #1448
mkannwischer wants to merge 1 commit into
Conversation
Add a test platform for the Dabao board (Baochip-1x, RV32IMAC at 350 MHz) and run the benchmarks on it in CI. Test binaries are built with a soft-float RV32IMAC newlib toolchain (new `.#dabao` dev shell) and linked into the xous-core `baremetal` image, which provides the USB serial console. link.py builds and signs that image with a pinned, patched xous-core; exec_wrapper.py flashes it through the boot1 console, runs the test and reports its exit code. Signed-off-by: Matthias J. Kannwischer <matthias@zerorisc.com>
There was a problem hiding this comment.
Mac Mini (M1, 2020) benchmarks (opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
46491 cycles |
46494 cycles |
1.00 |
ML-DSA-44 sign |
131379 cycles |
131434 cycles |
1.00 |
ML-DSA-44 verify |
47328 cycles |
47376 cycles |
1.00 |
ML-DSA-65 keypair |
81700 cycles |
81719 cycles |
1.00 |
ML-DSA-65 sign |
215297 cycles |
215302 cycles |
1.00 |
ML-DSA-65 verify |
79317 cycles |
79322 cycles |
1.00 |
ML-DSA-87 keypair |
132366 cycles |
132372 cycles |
1.00 |
ML-DSA-87 sign |
277449 cycles |
277416 cycles |
1.00 |
ML-DSA-87 verify |
137025 cycles |
136996 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Mac Mini (M1, 2020) benchmarks (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
113520 cycles |
112841 cycles |
1.01 |
ML-DSA-44 sign |
405441 cycles |
402663 cycles |
1.01 |
ML-DSA-44 verify |
120169 cycles |
119436 cycles |
1.01 |
ML-DSA-65 keypair |
193745 cycles |
193013 cycles |
1.00 |
ML-DSA-65 sign |
654209 cycles |
651616 cycles |
1.00 |
ML-DSA-65 verify |
193705 cycles |
192988 cycles |
1.00 |
ML-DSA-87 keypair |
318890 cycles |
318884 cycles |
1.00 |
ML-DSA-87 sign |
831198 cycles |
831107 cycles |
1.00 |
ML-DSA-87 verify |
325785 cycles |
325770 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Arm Cortex-A76 (Raspberry Pi 5) benchmarks (opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
112268 cycles |
112281 cycles |
1.00 |
ML-DSA-44 sign |
354108 cycles |
354075 cycles |
1.00 |
ML-DSA-44 verify |
117045 cycles |
117165 cycles |
1.00 |
ML-DSA-65 keypair |
194692 cycles |
194672 cycles |
1.00 |
ML-DSA-65 sign |
583444 cycles |
583313 cycles |
1.00 |
ML-DSA-65 verify |
192940 cycles |
193133 cycles |
1.00 |
ML-DSA-87 keypair |
320562 cycles |
321007 cycles |
1.00 |
ML-DSA-87 sign |
747067 cycles |
747555 cycles |
1.00 |
ML-DSA-87 verify |
318257 cycles |
318676 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Arm Cortex-A76 (Raspberry Pi 5) benchmarks (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
211931 cycles |
212004 cycles |
1.00 |
ML-DSA-44 sign |
760430 cycles |
760098 cycles |
1.00 |
ML-DSA-44 verify |
229710 cycles |
229440 cycles |
1.00 |
ML-DSA-65 keypair |
375838 cycles |
376081 cycles |
1.00 |
ML-DSA-65 sign |
1245711 cycles |
1245097 cycles |
1.00 |
ML-DSA-65 verify |
371812 cycles |
371792 cycles |
1.00 |
ML-DSA-87 keypair |
601673 cycles |
601811 cycles |
1.00 |
ML-DSA-87 sign |
1585077 cycles |
1583956 cycles |
1.00 |
ML-DSA-87 verify |
617440 cycles |
617448 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Arm Cortex-A72 (Raspberry Pi 4) benchmarks (opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
218382 cycles |
215465 cycles |
1.01 |
ML-DSA-44 sign |
600412 cycles |
599960 cycles |
1.00 |
ML-DSA-44 verify |
215741 cycles |
215886 cycles |
1.00 |
ML-DSA-65 keypair |
383044 cycles |
380665 cycles |
1.01 |
ML-DSA-65 sign |
991660 cycles |
995589 cycles |
1.00 |
ML-DSA-65 verify |
367305 cycles |
365798 cycles |
1.00 |
ML-DSA-87 keypair |
646829 cycles |
642746 cycles |
1.01 |
ML-DSA-87 sign |
1329897 cycles |
1328306 cycles |
1.00 |
ML-DSA-87 verify |
628922 cycles |
623047 cycles |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Intel Xeon 4th gen (c7i)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
43340 cycles |
43335 cycles |
1.00 |
ML-DSA-44 sign |
130247 cycles |
130054 cycles |
1.00 |
ML-DSA-44 verify |
45253 cycles |
45274 cycles |
1.00 |
ML-DSA-65 keypair |
75857 cycles |
76106 cycles |
1.00 |
ML-DSA-65 sign |
213779 cycles |
213867 cycles |
1.00 |
ML-DSA-65 verify |
74904 cycles |
74645 cycles |
1.00 |
ML-DSA-87 keypair |
123505 cycles |
123276 cycles |
1.00 |
ML-DSA-87 sign |
270748 cycles |
270714 cycles |
1.00 |
ML-DSA-87 verify |
120734 cycles |
120652 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Intel Xeon 4th gen (c7i) (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
92178 cycles |
92092 cycles |
1.00 |
ML-DSA-44 sign |
348951 cycles |
348083 cycles |
1.00 |
ML-DSA-44 verify |
99497 cycles |
99491 cycles |
1.00 |
ML-DSA-65 keypair |
154055 cycles |
154151 cycles |
1.00 |
ML-DSA-65 sign |
564933 cycles |
565525 cycles |
1.00 |
ML-DSA-65 verify |
159268 cycles |
159287 cycles |
1.00 |
ML-DSA-87 keypair |
255997 cycles |
255425 cycles |
1.00 |
ML-DSA-87 sign |
724691 cycles |
724512 cycles |
1.00 |
ML-DSA-87 verify |
264060 cycles |
263806 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
AMD EPYC 4th gen (c7a)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
47641 cycles |
47153 cycles |
1.01 |
ML-DSA-44 sign |
139565 cycles |
140214 cycles |
1.00 |
ML-DSA-44 verify |
49662 cycles |
50124 cycles |
0.99 |
ML-DSA-65 keypair |
81945 cycles |
82197 cycles |
1.00 |
ML-DSA-65 sign |
228505 cycles |
229690 cycles |
0.99 |
ML-DSA-65 verify |
82565 cycles |
84336 cycles |
0.98 |
ML-DSA-87 keypair |
130664 cycles |
130415 cycles |
1.00 |
ML-DSA-87 sign |
280357 cycles |
280238 cycles |
1.00 |
ML-DSA-87 verify |
128424 cycles |
128337 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton5
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
51447 cycles |
51544 cycles |
1.00 |
ML-DSA-44 sign |
162190 cycles |
162240 cycles |
1.00 |
ML-DSA-44 verify |
54804 cycles |
54695 cycles |
1.00 |
ML-DSA-65 keypair |
90193 cycles |
90158 cycles |
1.00 |
ML-DSA-65 sign |
266245 cycles |
266210 cycles |
1.00 |
ML-DSA-65 verify |
89984 cycles |
89808 cycles |
1.00 |
ML-DSA-87 keypair |
145990 cycles |
145901 cycles |
1.00 |
ML-DSA-87 sign |
334511 cycles |
335555 cycles |
1.00 |
ML-DSA-87 verify |
144887 cycles |
145479 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
AMD EPYC 4th gen (c7a) (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
118934 cycles |
119113 cycles |
1.00 |
ML-DSA-44 sign |
458308 cycles |
459803 cycles |
1.00 |
ML-DSA-44 verify |
131602 cycles |
131354 cycles |
1.00 |
ML-DSA-65 keypair |
201101 cycles |
201239 cycles |
1.00 |
ML-DSA-65 sign |
747563 cycles |
742008 cycles |
1.01 |
ML-DSA-65 verify |
210192 cycles |
210141 cycles |
1.00 |
ML-DSA-87 keypair |
331258 cycles |
332735 cycles |
1.00 |
ML-DSA-87 sign |
941447 cycles |
943014 cycles |
1.00 |
ML-DSA-87 verify |
344794 cycles |
346024 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
AMD EPYC 3rd gen (c6a)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
52235 cycles |
52012 cycles |
1.00 |
ML-DSA-44 sign |
156653 cycles |
155876 cycles |
1.00 |
ML-DSA-44 verify |
54462 cycles |
54350 cycles |
1.00 |
ML-DSA-65 keypair |
89985 cycles |
89686 cycles |
1.00 |
ML-DSA-65 sign |
253797 cycles |
253118 cycles |
1.00 |
ML-DSA-65 verify |
90323 cycles |
89910 cycles |
1.00 |
ML-DSA-87 keypair |
143615 cycles |
143380 cycles |
1.00 |
ML-DSA-87 sign |
310837 cycles |
309922 cycles |
1.00 |
ML-DSA-87 verify |
138829 cycles |
138722 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton5 (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
100210 cycles |
99994 cycles |
1.00 |
ML-DSA-44 sign |
369798 cycles |
369411 cycles |
1.00 |
ML-DSA-44 verify |
110532 cycles |
110169 cycles |
1.00 |
ML-DSA-65 keypair |
175570 cycles |
176078 cycles |
1.00 |
ML-DSA-65 sign |
595149 cycles |
596508 cycles |
1.00 |
ML-DSA-65 verify |
177598 cycles |
177442 cycles |
1.00 |
ML-DSA-87 keypair |
285023 cycles |
286625 cycles |
0.99 |
ML-DSA-87 sign |
757785 cycles |
757839 cycles |
1.00 |
ML-DSA-87 verify |
296702 cycles |
295774 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
AMD EPYC 3rd gen (c6a) (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
133760 cycles |
133339 cycles |
1.00 |
ML-DSA-44 sign |
521437 cycles |
519956 cycles |
1.00 |
ML-DSA-44 verify |
147050 cycles |
146742 cycles |
1.00 |
ML-DSA-65 keypair |
224336 cycles |
224146 cycles |
1.00 |
ML-DSA-65 sign |
848581 cycles |
847357 cycles |
1.00 |
ML-DSA-65 verify |
235579 cycles |
234420 cycles |
1.00 |
ML-DSA-87 keypair |
369081 cycles |
367973 cycles |
1.00 |
ML-DSA-87 sign |
1065583 cycles |
1064638 cycles |
1.00 |
ML-DSA-87 verify |
382215 cycles |
381485 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Intel Xeon 3rd gen (c6i)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
61780 cycles |
61663 cycles |
1.00 |
ML-DSA-44 sign |
188687 cycles |
188790 cycles |
1.00 |
ML-DSA-44 verify |
66126 cycles |
66187 cycles |
1.00 |
ML-DSA-65 keypair |
108095 cycles |
108346 cycles |
1.00 |
ML-DSA-65 sign |
311306 cycles |
311940 cycles |
1.00 |
ML-DSA-65 verify |
108586 cycles |
109108 cycles |
1.00 |
ML-DSA-87 keypair |
170515 cycles |
172400 cycles |
0.99 |
ML-DSA-87 sign |
378902 cycles |
383035 cycles |
0.99 |
ML-DSA-87 verify |
170152 cycles |
171307 cycles |
0.99 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Arm Cortex-A72 (Raspberry Pi 4) benchmarks (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
300040 cycles |
299209 cycles |
1.00 |
ML-DSA-44 sign |
1132984 cycles |
1128871 cycles |
1.00 |
ML-DSA-44 verify |
330605 cycles |
329148 cycles |
1.00 |
ML-DSA-65 keypair |
549720 cycles |
546676 cycles |
1.01 |
ML-DSA-65 sign |
1868692 cycles |
1860299 cycles |
1.00 |
ML-DSA-65 verify |
530524 cycles |
527863 cycles |
1.01 |
ML-DSA-87 keypair |
851734 cycles |
848296 cycles |
1.00 |
ML-DSA-87 sign |
2376484 cycles |
2384110 cycles |
1.00 |
ML-DSA-87 verify |
880414 cycles |
879177 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Intel Xeon 3rd gen (c6i) (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
153848 cycles |
156139 cycles |
0.99 |
ML-DSA-44 sign |
588313 cycles |
594995 cycles |
0.99 |
ML-DSA-44 verify |
169080 cycles |
171509 cycles |
0.99 |
ML-DSA-65 keypair |
262325 cycles |
262241 cycles |
1.00 |
ML-DSA-65 sign |
958413 cycles |
964309 cycles |
0.99 |
ML-DSA-65 verify |
271882 cycles |
272016 cycles |
1.00 |
ML-DSA-87 keypair |
431233 cycles |
432829 cycles |
1.00 |
ML-DSA-87 sign |
1210420 cycles |
1213001 cycles |
1.00 |
ML-DSA-87 verify |
446721 cycles |
447738 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton4
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
67382 cycles |
67188 cycles |
1.00 |
ML-DSA-44 sign |
198582 cycles |
199058 cycles |
1.00 |
ML-DSA-44 verify |
70337 cycles |
70294 cycles |
1.00 |
ML-DSA-65 keypair |
119365 cycles |
119349 cycles |
1.00 |
ML-DSA-65 sign |
325608 cycles |
325681 cycles |
1.00 |
ML-DSA-65 verify |
116876 cycles |
117069 cycles |
1.00 |
ML-DSA-87 keypair |
196651 cycles |
196230 cycles |
1.00 |
ML-DSA-87 sign |
421761 cycles |
420592 cycles |
1.00 |
ML-DSA-87 verify |
193332 cycles |
192796 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton3
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
71588 cycles |
71502 cycles |
1.00 |
ML-DSA-44 sign |
210232 cycles |
210202 cycles |
1.00 |
ML-DSA-44 verify |
74910 cycles |
74960 cycles |
1.00 |
ML-DSA-65 keypair |
126070 cycles |
126199 cycles |
1.00 |
ML-DSA-65 sign |
344764 cycles |
345032 cycles |
1.00 |
ML-DSA-65 verify |
124137 cycles |
124351 cycles |
1.00 |
ML-DSA-87 keypair |
206726 cycles |
207013 cycles |
1.00 |
ML-DSA-87 sign |
439860 cycles |
444604 cycles |
0.99 |
ML-DSA-87 verify |
204791 cycles |
204509 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton4 (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
127800 cycles |
127847 cycles |
1.00 |
ML-DSA-44 sign |
441873 cycles |
442380 cycles |
1.00 |
ML-DSA-44 verify |
136459 cycles |
136598 cycles |
1.00 |
ML-DSA-65 keypair |
223044 cycles |
223363 cycles |
1.00 |
ML-DSA-65 sign |
714348 cycles |
714348 cycles |
1 |
ML-DSA-65 verify |
220910 cycles |
220979 cycles |
1.00 |
ML-DSA-87 keypair |
365028 cycles |
365561 cycles |
1.00 |
ML-DSA-87 sign |
916025 cycles |
958782 cycles |
0.96 |
ML-DSA-87 verify |
371402 cycles |
371548 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
oqs-bot
left a comment
There was a problem hiding this comment.
Graviton3 (no-opt)
Details
| Benchmark suite | Current: bcba070 | Previous: 14d195a | Ratio |
|---|---|---|---|
ML-DSA-44 keypair |
138397 cycles |
138360 cycles |
1.00 |
ML-DSA-44 sign |
488095 cycles |
488372 cycles |
1.00 |
ML-DSA-44 verify |
149570 cycles |
149676 cycles |
1.00 |
ML-DSA-65 keypair |
242068 cycles |
242151 cycles |
1.00 |
ML-DSA-65 sign |
793052 cycles |
792429 cycles |
1.00 |
ML-DSA-65 verify |
242155 cycles |
241998 cycles |
1.00 |
ML-DSA-87 keypair |
395952 cycles |
396891 cycles |
1.00 |
ML-DSA-87 sign |
1014800 cycles |
1060851 cycles |
0.96 |
ML-DSA-87 verify |
405100 cycles |
404926 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
CBMC Results (ML-DSA-44, REDUCE-RAM)
Full Results (229 configurations)
|
CBMC Results (ML-DSA-65, REDUCE-RAM)
Full Results (229 configurations)
|
CBMC Results (ML-DSA-87, REDUCE-RAM)
Full Results (229 configurations)
|
CBMC Results (ML-DSA-44)
Full Results (229 configurations)
|
CBMC Results (ML-DSA-65)
Full Results (229 configurations)
|
CBMC Results (ML-DSA-87)
Full Results (229 configurations)
|
Add a test platform for the Dabao board (Baochip-1x, RV32IMAC at 350 MHz) and run the benchmarks on it in CI.
Test binaries are built with a soft-float RV32IMAC newlib toolchain (new
.#dabaodev shell) and linked into the xous-corebaremetalimage, which provides the USB serial console. link.py builds and signs that image with a pinned, patched xous-core; exec_wrapper.py flashes it through the boot1 console, runs the test and reports its exit code.Warning
Nowhere near ready to be merged.