Reuse pre-keyed AES-GCM contexts for ledger encryption - #8170
Closed
Eddy Ashton (eddyashton) with Copilot wants to merge 2 commits into
Closed
Reuse pre-keyed AES-GCM contexts for ledger encryption#8170Eddy Ashton (eddyashton) with Copilot wants to merge 2 commits into
Eddy Ashton (eddyashton) with Copilot wants to merge 2 commits into
Conversation
Co-authored-by: eddyashton <6000239+eddyashton@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Reuse initialised AES-GCM contexts for ledger encryption
Reuse pre-keyed AES-GCM contexts for ledger encryption
Aug 18, 2026
DescriptionComparing 1 available run from this branch (#8170) against the trend of the last 30 Each chart plots every benchmark as an axis, with values normalized so 100 is the EWMA baseline of recent Axis labels show the latest branch value and its difference from the main EWMA baseline, where 0% is on the baseline. They are coloured green where the latest run improves on the baseline, red where it regresses, and grey where the difference is within one std dev of the baseline (within noise). Higher is better for throughput and rate, lower for latency and memory. Throughput (tx/s)---
config:
radar:
width: 620
height: 620
marginTop: 90
marginRight: 220
marginBottom: 60
marginLeft: 220
axisLabelFactor: 1.12
curveTension: 0.08
theme: base
themeCSS: |
.radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
.radarCurve-4{stroke-width:1.75px!important;stroke-opacity:1.00!important}
.radarAxisLabel:nth-of-type(1){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(2){fill:#808A94!important}
.radarAxisLabel:nth-of-type(3){fill:#808A94!important}
.radarAxisLabel:nth-of-type(4){fill:#808A94!important}
.radarAxisLabel:nth-of-type(5){fill:#808A94!important}
.radarAxisLabel:nth-of-type(6){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(7){fill:#2DA44E!important}
themeVariables:
cScale0: "#62B5E5"
cScale1: "#62B5E5"
cScale2: "#62B5E5"
cScale3: "#62B5E5"
cScale4: "#F97316"
radar:
axisColor: "#9CA3AF"
graticuleColor: "#E5E7EB"
graticuleOpacity: 0
axisStrokeWidth: 1
curveOpacity: 0
---
radar-beta
axis b0["Basic: 75,568 tx/s ▲ 12%"]
axis b1["Basic Blocking: 1,015 tx/s ▬ 0%"]
axis b2["Basic JS: 4,821 tx/s ▬ +1%"]
axis b3["Basic Multi-Threaded: 85,817 tx/s ▬ -1%"]
axis b4["Historical Queries: 207,361 tx/s ▬ 0%"]
axis b5["Logging: 69,857 tx/s ▲ 13%"]
axis b6["Logging JWT: 10,580 tx/s ▲ 4%"]
curve stddev2_high["main EWMA + 2 std dev"]{103.44, 103.18, 103.47, 105.57, 108.77, 106.32, 105.30}
curve stddev1_high["main EWMA + 1 std dev"]{101.72, 101.59, 101.74, 102.78, 104.39, 103.16, 102.65}
curve stddev1_low["main EWMA - 1 std dev"]{98.28, 98.41, 98.26, 97.22, 95.61, 96.84, 97.35}
curve stddev2_low["main EWMA - 2 std dev"]{96.56, 96.82, 96.53, 94.43, 91.23, 93.68, 94.70}
curve branch_0["#8170"]{111.80, 99.61, 100.84, 99.42, 100.50, 112.85, 103.98}
graticule polygon
max 121
min 83
ticks 0
showLegend false
Latency (ms)---
config:
radar:
width: 620
height: 620
marginTop: 90
marginRight: 220
marginBottom: 60
marginLeft: 220
axisLabelFactor: 1.12
curveTension: 0.08
theme: base
themeCSS: |
.radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
.radarCurve-4{stroke-width:1.75px!important;stroke-opacity:1.00!important}
.radarAxisLabel:nth-of-type(1){fill:#808A94!important}
.radarAxisLabel:nth-of-type(2){fill:#808A94!important}
.radarAxisLabel:nth-of-type(3){fill:#2DA44E!important}
themeVariables:
cScale0: "#62B5E5"
cScale1: "#62B5E5"
cScale2: "#62B5E5"
cScale3: "#62B5E5"
cScale4: "#F97316"
radar:
axisColor: "#9CA3AF"
graticuleColor: "#E5E7EB"
graticuleOpacity: 0
axisStrokeWidth: 1
curveOpacity: 0
---
radar-beta
axis b0["Commit Latency 16ms: 5.31 ms ▬ 0%"]
axis b1["Commit Latency 1ms: 1.87 ms ▬ -4%"]
axis b2["Commit Latency 256ms: 202 ms ▼ 1%"]
curve stddev2_high["main EWMA + 2 std dev"]{157.66, 110.07, 101.24}
curve stddev1_high["main EWMA + 1 std dev"]{128.83, 105.03, 100.62}
curve stddev1_low["main EWMA - 1 std dev"]{71.17, 94.97, 99.38}
curve stddev2_low["main EWMA - 2 std dev"]{42.34, 89.93, 98.76}
curve branch_0["#8170"]{99.77, 96.31, 99.33}
graticule polygon
max 199
min 1
ticks 0
showLegend false
Memory (bytes)---
config:
radar:
width: 620
height: 620
marginTop: 90
marginRight: 220
marginBottom: 60
marginLeft: 220
axisLabelFactor: 1.12
curveTension: 0.08
theme: base
themeCSS: |
.radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
.radarCurve-4{stroke-width:1.75px!important;stroke-opacity:1.00!important}
.radarAxisLabel:nth-of-type(1){fill:#808A94!important}
.radarAxisLabel:nth-of-type(2){fill:#808A94!important}
.radarAxisLabel:nth-of-type(3){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(4){fill:#808A94!important}
.radarAxisLabel:nth-of-type(5){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(6){fill:#808A94!important}
themeVariables:
cScale0: "#62B5E5"
cScale1: "#62B5E5"
cScale2: "#62B5E5"
cScale3: "#62B5E5"
cScale4: "#F97316"
radar:
axisColor: "#9CA3AF"
graticuleColor: "#E5E7EB"
graticuleOpacity: 0
axisStrokeWidth: 1
curveOpacity: 0
---
radar-beta
axis b0["Basic: 86.4 MiB ▬ 0%"]
axis b1["Basic Blocking: 71.7 MiB ▬ 0%"]
axis b2["Basic JS: 69.5 MiB ▼ 3%"]
axis b3["Basic Multi-Threaded: 89.9 MiB ▬ +1%"]
axis b4["Logging: 75.4 MiB ▼ 1%"]
axis b5["Logging JWT: 69.1 MiB ▬ +1%"]
curve stddev2_high["main EWMA + 2 std dev"]{106.89, 100.63, 103.61, 102.32, 101.57, 101.41}
curve stddev1_high["main EWMA + 1 std dev"]{103.44, 100.31, 101.80, 101.16, 100.79, 100.71}
curve stddev1_low["main EWMA - 1 std dev"]{96.56, 99.69, 98.20, 98.84, 99.21, 99.29}
curve stddev2_low["main EWMA - 2 std dev"]{93.11, 99.37, 96.39, 97.68, 98.43, 98.59}
curve branch_0["#8170"]{99.89, 100.20, 97.47, 100.80, 98.94, 100.58}
graticule polygon
max 112
min 88
ticks 0
showLegend false
Rate (ops/s)---
config:
radar:
width: 620
height: 620
marginTop: 90
marginRight: 220
marginBottom: 60
marginLeft: 220
axisLabelFactor: 1.12
curveTension: 0.08
theme: base
themeCSS: |
.radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
.radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
.radarCurve-4{stroke-width:1.75px!important;stroke-opacity:1.00!important}
.radarAxisLabel:nth-of-type(1){fill:#808A94!important}
.radarAxisLabel:nth-of-type(2){fill:#E5484D!important}
.radarAxisLabel:nth-of-type(3){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(4){fill:#2DA44E!important}
.radarAxisLabel:nth-of-type(5){fill:#808A94!important}
.radarAxisLabel:nth-of-type(6){fill:#808A94!important}
themeVariables:
cScale0: "#62B5E5"
cScale1: "#62B5E5"
cScale2: "#62B5E5"
cScale3: "#62B5E5"
cScale4: "#F97316"
radar:
axisColor: "#9CA3AF"
graticuleColor: "#E5E7EB"
graticuleOpacity: 0
axisStrokeWidth: 1
curveOpacity: 0
---
radar-beta
axis b0["CHAMP get: 38,009,688 ops/s ▬ 0%"]
axis b1["CHAMP put: 4,922,059 ops/s ▼ 10%"]
axis b2["KV deserialisation: 1,700,970 ops/s ▲ 5%"]
axis b3["KV serialisation: 1,600,256 ops/s ▲ 11%"]
axis b4["KV snapshot deserialisa...: 4,197 ops/s ▬ 0%"]
axis b5["KV snapshot serialisation: 4,702 ops/s ▬ +3%"]
curve stddev2_high["main EWMA + 2 std dev"]{104.38, 102.05, 102.33, 102.37, 102.27, 109.48}
curve stddev1_high["main EWMA + 1 std dev"]{102.19, 101.03, 101.17, 101.18, 101.13, 104.74}
curve stddev1_low["main EWMA - 1 std dev"]{97.81, 98.97, 98.83, 98.82, 98.87, 95.26}
curve stddev2_low["main EWMA - 2 std dev"]{95.62, 97.95, 97.67, 97.63, 97.73, 90.52}
curve branch_0["#8170"]{100.40, 89.72, 104.73, 110.55, 100.50, 103.39}
graticule polygon
max 118
min 82
ticks 0
showLegend false
|
Comment on lines
+196
to
+197
| auto lease = context_pools->encrypt.acquire(evp_cipher, key); | ||
| auto* ctx = lease.get(); |
There was a problem hiding this comment.
So I've gone back and forth on whether this is the right approach. The subtle detail is that EVP_CIPHER_CTX is not thread-safe, but the existing API (std::shared_ptr<KeyAesGcm>->encrypt) is safe to call from multiple threads, because it creates this context locally (at great cost). If we want to reuse that context, we need to handle the thread-safety somewhere. Roughly, we can either do that way down here inside, and not touch the external APIs at all, or create some kind of AesGcmContext object at the external API, and make management (and thread-safe access to it) the caller's responsibility. Since the affinity is both thread- and instance- specific (we have multiple keys for old ledger secrets!), we don't get help from thread_local. We've actually provided both methods already - for SHA its a single global context that's expensive to lookup, so trivial to cache statically for everyone. For TLS, there are many contexts that are externally managed, and we ensure non-concurrent access to these. This ContextPool is local and well-contained, but uncomfortably complex.
Member
|
Preferring #8178 - less internal magic, potentially even greater perf wins. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ledger encryption recreated and fully initialized an OpenSSL cipher context for every transaction, making provider and key setup a significant CPU cost.
Context reuse
Lifecycle and concurrency
Coverage
Microbenchmark