Make the UK eFRS parity re-pin one reviewed command; hold every identity mirror in lockstep (companion to #747) - #749
Draft
MaxGhenis wants to merge 4 commits into
Draft
Make the UK eFRS parity re-pin one reviewed command; hold every identity mirror in lockstep (companion to #747)#749MaxGhenis wants to merge 4 commits into
MaxGhenis wants to merge 4 commits into
Conversation
The reference artifact's identity (HF revision, sha256, size) is mirrored in eight committed files and attested by the totals digest and four gate-battery digests; #723 moved it by hand. tools/repin_uk_efrs_parity_reference.py makes the move one reviewed command: resolve the release tag through the HF API, verify the licensed bytes, rewrite every mirror in lockstep and refuse leftovers, regenerate both instruments with the committed tools, emit the totals sidecar outside the repository, move totals_sha256 and re-cut the battery digests, write a disclosure-safe receipt, run the lockstep tests. --dry-run edits nothing; the recompute path must reproduce the committed digests before it may mint new ones. test_uk_efrs_repin_tool.py holds every identity / totals-digest / battery-digest mirror equal to the committed parity reference, so a drift in any one of them fails PR CI. Prepares the post-uk-data#462 re-pin (v1.56.14 -> v1.56.16 or later) for #686. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed-in candidate evidence The coverage manifest is rebuilt from the reference plus the committed candidate evidence; if a release changes the reference's populated-column surface the evidence must be refreshed with the certified candidate first, so the tool stops instead of writing a manifest over stale evidence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The regenerated reference records the installed policyengine-uk version; a re-pin that silently changed the engine would conflate an artifact move with an engine-surface move. Refuse unless --allow-engine-change is passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_VERSION #747 records the uk-data release tag beside the byte identity (SOURCE_VERSION / source.version). A regeneration for another release must not inherit the committed tag, so the identity now carries the tag (from --release, or --version with explicit pins), the in-memory patch sets it, and the on-disk move is anchored to the SOURCE_VERSION assignment — never a global replacement of a tag literal, which also appears in unrelated pins such as the registry-parity pinned_version. The lockstep test binds the committed reference's source.version to the tool's constant once it exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
uk-efrs-parity-repin-post-462
branch
from
August 23, 2026 10:21
c5341bd to
34ba133
Compare
MaxGhenis
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the UK enhanced-FRS parity re-pin one reviewed command, and holds every committed mirror of the reference identity in lockstep under PR CI. Companion to #747, which carries the actual 1.56.14 → 1.56.16 move (uk-data#462's benunit-sort fix) as a hand-made eleven-file commit for #686: this PR's tool reproduces that commit exactly (see the cross-check below), so the next re-pin — a later uk-data release bundling the take-up mask and the pension split, if that is the decision — is a single command on top of #747. Nothing here regenerates a licensed instrument; the committed pins on this branch are main's (v1.56.14) until #747 lands.
What lands
tools/repin_uk_efrs_parity_reference.py— resolves an HF release tag to its commit and the artifact's LFS sha256/size, verifies the licensed bytes (HF cache /--input-h5/--download), rewrites every committed mirror of the identity in lockstep and refuses leftovers, regeneratesuk/efrs_parity_reference.jsonanduk/release_input_coverage_manifest.jsonwith the committed tools (engine held at uv.lock's policyengine-uk unless--allow-engine-change), emits the gate's totals sidecar outside the repository (UK parity increment 4: port the weighted integrity gates (input-mass parity + QRF tail concentration) into the UK terminal battery (#578) #609), movestotals_sha256(the sidecar's canonical evidence digest), re-cuts the four gate-battery digests that attest the moved spec, writes a disclosure-safe receipt, and runs the lockstep tests.--dry-runcomputes everything into a scratch directory and edits nothing. Guards: the recompute path must reproduce the committed digests before it may mint new ones; a release whose populated-column surface differs from the checked-in candidate evidence is refused until--candidate-h5refreshes it; the release tag moves only through the anchoredSOURCE_VERSIONassignment (Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747's constant), never a global literal replacement;frs_release.json's raw-zip acquisition revision is left alone unless--move-frs-release-acquisitionis passed and the zip's LFS sha is verified identical at the new revision.packages/microcosm-build/tests/test_uk_efrs_repin_tool.py— holds the eight identity mirrors, the fourtotals_sha256mirrors and the gate-battery digest mirrors equal to the committed parity reference and to each other, bindssource.versiontoSOURCE_VERSIONonce that constant exists, proves the digest recompute reproduces the committed pins, and covers the literal-move machinery hermetically. A drift in any one mirror now fails PR CI.Pin inventory (every committed place the reference identity lives)
uk/efrs_parity_reference.jsonsource.{revision,sha256,size_bytes,url[,version]}, 145nonzero_shares,entity_stats, engine listsuk/release_input_coverage_manifest.jsonreference.{revision,sha256,populated_input_columns}; columns/counts derived--candidate-h5only if the surface moves)uk/gates.jsonuk_input_mass_parity→reference_registry.efrs-post-calibration.identity.{revision,sha256},totals_sha256uk_runtime/weighted_integrity.pyUK_INPUT_MASS_REFERENCE_EVIDENCE_SHA256,_UK_INPUT_MASS_REFERENCE_DESCRIPTORmicrocosm-data …/contract.py_UK_INPUT_MASS_REFERENCE_IDENTITY,_UK_INPUT_MASS_REFERENCE_EVIDENCE_SHA256,_UK_GATE_BATTERY_{POLICY_SHA256,GATES_MANIFEST_SHA256,SPEC_FINGERPRINT,INPUT_MASS_EVIDENCE_SHA256}_UK_GATE_BATTERY_DEGENERATE_EVIDENCE_SHA256is adjacent and unaffected)tools/build_uk_efrs_parity_reference.pySOURCE_REVISION,SOURCE_SHA256,SOURCE_SIZE_BYTES(+SOURCE_VERSIONafter #747)test_uk_parity_reference.py,test_uk_terminal_gates.py,test_uk_weighted_integrity.py,microcosm-data/tests/test_contract.py--emit-weighted-totals)totals_sha256(weighted_integrity._input_mass_reference_evidence_sha256, not file bytes)Not the artifact identity, left as is:
uk/frs_release.jsonacquisition.huggingface_revision(the raw FRS zip; same LFS sha05dd0069…at both HF revisions; validated by nothing);uk/take_up_contract.jsoncommit: ebf733c×7 (uk-data'sparameters/tree andutils/takeup.pyare unchanged between the tags);uk_national_targets.jsonregistry_parity(already 12a1e028 = v1.56.16);brma_rent_counts.json(VOA CSV provenance);UK_REFERENCE_DATASET_NAME(a label). The UKspec_sha256does not move: the instruments arelegacy_jsonresources, which project nothing onto the normative spec surface (checked empirically on 057dd95: swapping the three files leaves8bf62b6e…unchanged; only the unpinnedpackage_fingerprintmoves). Prose that still names the 2023-24 identity — the opening paragraph ofUK_COVERAGE_PROGRESS.md, atest_us_plan.pycomment about the pre-1.56.16 registry accounting — is journal/comment text, not a pin; historicize when convenient.Cross-check against #747
--release 1.56.14 --dry-runis an exact no-op — reference and manifest byte-identical to the committed files, totals digeste70a4538…reproduced canonically, all four battery digests reproduced.89b1f8e8):--release 1.56.16 --dry-runis an exact no-op — identitya9e52499…/e433e532…/ 126,553,300,source.version 1.56.16, the regenerated reference and coverage manifest byte-identical to Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747's committed files,totals_sha256 fd41cb5f…, and the four battery digests623f340d…/f2cc2af4…/3601b4c7…/16093e86…all reproduced; the lockstep test passes on that tree. Two independent paths (hand edits in Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747; this tool from the committed extraction code) agree bit for bit.transport_consumption);would_claim_uc0.550594 → 0.550692;is_marriedunchanged (nonzero shares are permutation-blind to the benunit sort, which is why the weighted side matters). Weighted totals: 131 columns, 128 defined ratios; median 0.8%, p90 ≈ 9%, 20 columns beyond 5% — all person-level, mostly reported-benefit and education-support columns (max 2.2×,is_before_universal_credit_qualifying_young_person_terminal_date); benunit-level ≤ 3.5%, household-level ≤ 4.3%. The two uk-data changes between the tags (Entropy-calibration ideas from weighted Monte Carlo (Avellaneda et al. 2001): KL objective arm, ESS diagnostic, per-target tolerances, target sensitivities #458's recalibration and Build M live default: non_sch_d_capital_gains lands at $74.6B vs its $10.2B SOI target (+635%); Schedule-D CGD route is empty #462's sort) are not separated by this measurement.What a re-pin does not re-adjudicate
uk_input_mass_parity's tolerance (4.5218…, minted as the worst surviving drift against the 2023-24 reference per the #630 receipts) and its two reviewed exclusions (charitable_investment_gifts→ 2027-02-20,owned_land→ 2026-09-20) are bound to the reference name, not its identity; a same-name re-pin carries them mechanically and the gate re-evaluates them (stale / expired entries fail) at the next licensed run. Re-arming is #686's step (#747 "held behind the review gate"), not a side effect of moving the pins. #733's nonzero-share screen of the microcosm spine (27 columns beyond ±0.02) was taken against the v1.56.14 reference; #747 re-runs it (26 columns).Provenance
Implemented by Claude (Fable) in session. Adversarial verification: a Claude Workflow was rejected on the session limit, so six sol lanes (read-only) read the instruments / gates / contract / release object / target surface, and six sol lanes refuted them; all verdicts adjudicated in session, and every mechanism claim above re-opened against the code. Licensed artifacts read from the local HF cache; nothing downloaded, nothing published; the totals sidecars stayed outside every checkout.
🤖 Generated with Claude Code