Repository navigation
feat: enforce signed V3 release evidence gates - #312
Draft
EvanProgramming wants to merge 35 commits into
Draft
EvanProgramming wants to merge 35 commits into
EvanProgramming wants to merge 35 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
EvanProgramming
force-pushed
the
Evan/v3-release-evidence
branch
15 times, most recently
from
October 22, 2026 19:17
7276872 to
992b027
Compare
EvanProgramming
force-pushed
the
Evan/v3-release-evidence
branch
3 times, most recently
from
October 22, 2026 22:03
22543f7 to
068b8f0
Compare
EvanProgramming
force-pushed
the
Evan/v3-release-evidence
branch
from
October 22, 2026 22:19
068b8f0 to
b9ff1e4
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracks #268; stacked on #311. This PR closes #268 only after every release gate has signed, candidate-bound evidence.
Root cause: release gates lacked a machine-enforced evidence chain bound to the exact clean candidate SHA, and the six-platform results had no reproducible collector. Partial or stale evidence could otherwise be published as a release record.
Approach: validate signed gate records against the exact candidate, configured GPG signer, fixture/command hashes, result-artifact hashes, classifications, and six-platform R266 matrix. Evidence and manifest paths inside the candidate checkout are rejected. Signature and manifest publication do not overwrite existing files, and partial publication removes the signature. CI uploads platform logs, environment and dependency metadata; the collector rejects stale, incomplete, escaping, or failed results. Linux and macOS run the full unit and gateway integration suites. Under the agreed Windows policy, Windows runs supported-feature agent/sub-agent acceptance and Go TUI tests/build; POSIX-dependent Python suites and interactive PTY acceptance are explicitly recorded as unavailable. Every platform still requires acceptance evidence and a 30-minute soak.
Validation: exact-candidate CI run 40063606173 passed ordinary CI and all six platform matrix jobs on candidate
9c6ef88f05ec006069205a07cf4cb5c971ae8cb4. The six-platform R266 bundle passed record, artifact-hash, and pinned-GPG signature validation; its detached-signed record SHA-256 isd29abbcdc209d0b1c9acbb7b0e117fe7813dc1f2ca104982728b67306380062d. Current evidence bundle has valid signatures and exact candidate binding for R250–R266: R250–R258, R260–R261, R265, and R266 validate as passed. R259 and R262–R264 remain signed inconclusive because the matched live evaluation is incomplete.The initial live run with
gemini-3.5-flash-litereached 133/180 arm rows and 2,207 calls before Panstar returnedAI_RELAY_UPSTREAM_ERROR, leaving 11 unknown-usage reservations. A first attempted switch togemini-3.1-flash-litewas mislabeled: the report label changed but the relay module remained ongemini-3.5-flash-lite; the ledger shows all 10 requests used 3.5, so that attempt is excluded as 3.1 evidence while its full spend/reservation remains counted. With both module constants corrected, a freshgemini-3.1-flash-literun reached 1/180 arm rows beforeAI_RELAY_STREAM_INCOMPLETE; its one request has unknown usage and remains reserved. Across the attempts, cumulative exposure is US$10.520449970 and US$9.479550030 remains under the US$20 cap. The matched evaluation is incomplete and no model cohorts are pooled. No aggregate release manifest is available; the validator correctly rejects the inconclusive gates.Risks and follow-up: #266/#267 remain inconclusive because the complete matched cohort, full usage accounting, and statistical gates are not met. #268 remains blocked because four required gates are inconclusive. Keep this PR draft until the matched cohort and every signed release gate pass.