Repository navigation
Make set_input on a branch drop values calculated from the input it replaces - #560
Merged
Merged
Conversation
A branch starts with every value its parent holds, and set_input on a branch used to keep everything calculated from the value it replaced, so a value the parent calculated before branching answered for the branch whatever its inputs said. Every stored array now carries a sequence number from one process-wide counter (store order), and storages record which values are inputs; both are copied with clones. A simulation family shares a StoreHistory of the first store of each variable and period (including values a holder does not keep) and of the first uprated or carried-over value of each variable. set_input on a branch drops the branch's non-input values stored at or after the earliest of those for an overlapping period; when there is none, nothing is dropped. Simulation.drop_computed_arrays() drops every non-input value, for branches whose policy changes; _invalidate_all_caches now uses the same input flags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Neither is stored, but values calculated from them are, so set_input on a branch must see them in the store history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ypothesis The country-package smoke job installs no dev dependencies, so the module-level hypothesis import failed its collection. The synthetic system the examples and the property share moves to tests/fixtures/branch_input_invalidation.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each simulation now keeps its own store history: a branch copies its parent's, and a formula that calculates in another simulation takes in that simulation's history when calculate returns (fast-cache hits included). A drop prunes the records it made obsolete and records the kept inputs again. An unrelated store in a sibling branch, or in the other arm of a reform/baseline pair, no longer lowers a branch's threshold, so MTR-style and baseline branches calculate shared values once. Also from the soundness and code reviews: - a branch whose input dropped values stops reading macro-cache files (they are keyed by branch and period, not inputs); drop_computed_arrays too; - requires_computation_after counts a prerequisite requested before a drop; - values a custom set_input handler calculates are not inputs; - disk storage writes a new file per store, so a recalculation no longer changes what child branches or same-named branches read; - a value written into storage without a number counts as a dependency; - storages pickled before numbering still store (__setstate__); - derivative keeps inputs set after the simulation was built; - property test: an independent model of divided inputs, reused and forgotten branch names, a blacklist mode, derandomized CI examples with an environment soak; examples for each finding; - docs and docstrings say what the code does, with the remaining limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the second soundness review (in progress): - A drop while a formula is still running in the simulation (its own set_input, or drop_computed_arrays) no longer prunes the history: the formula may hold values it read before the drop and store a result from them. Calculations in flight are counted per simulation; a clone starts at zero. - Merging is incremental: each history keeps a journal of its changes since it was created or pruned, so repeated calls into a simulation whose history keeps growing no longer re-read every record (quadratic before). - The merge map is keyed weakly, so temporary branches' entries go with them, and is left out of pickles (weak references do not pickle). - Unpickling a storage or history advances the process's sequence counter past the numbers it carries; disk restore keeps each key's most recently written file (numbers restart in every process). - Tests: drops inside a running formula, an uprated value handed back by a temporary branch (also in the property's scenarios), cross-process unpickling, restore order. Docs: the in-flight rule and the thread limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… success From the second soundness review: - A branch calculated from a thread with no formula context (a worker a formula starts without copying its context) hands its store history to every ancestor with a calculation running, so the waiting formula's result keeps the dependency. Unrelated simulations are not touched. - Disk files carry a per-process token as well as the sequence number: numbers restart in each process, so a later process could otherwise overwrite a file a restored view still maps. - requires_computation_after counts a prerequisite only once its calculation succeeded. - Docs and comments: the soundness argument states records for overlapping periods (summed and divided values), disk files stay until the directory is removed, the thread limit covers only unrelated simulations, and the history is copied into branches, not shared. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dation Conflicts kept both sides: InMemoryStorage keeps #556's shared-array clone and also copies the sequence numbers and input flags; drop_computed also discards dropped keys from _shared; get_branch's docstring keeps both paragraphs. Two #556 tests encoded the shadowing this branch fixes: - test_set_input_on_branch_leaves_parent now expects the branch's income_tax to follow its new salary (the parent stays unchanged); - test_branch_copies_only_what_it_reads accepts that the input drops, rather than copies, values calculated after salary was first stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sk names From the round-3 soundness review: - A calculation still running when an input set on its simulation dropped values returns its result but does not cache it (an input epoch), so the next calculation uses the new input. - A calculation that raises still hands its store history to the calling formula: whether it fails can depend on what it read. - A merge records how far it read the source before reading, so a record added meanwhile (another thread) is read next time. - dump_simulation records each variable's input periods (inputs.txt); restore_simulation restores inputs as inputs, then every other value under one later number, so a branch input drops what it may have fed. Older dumps restore every value as an input, as before. - Disk restore reads the previous `<key>.<number>.npy` names, prefers this process's file on a timestamp tie, and a forked child gets its own token. - StoreHistory pickles from before journals load with empty journals. - Tests for each; docs updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pickle test merged a history that died at once, so the weak map was empty when pickled. Keep the source alive so the test covers the case the __getstate__ exists for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A process pool forked from a multi-threaded process (the test process can have threads by then) can deadlock; the child now only writes its token to a pipe and exits. Still fails without the at-fork hook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ro reads From the round-4 soundness review (on bff21a7): - A result refused because an input changed while it was calculated left its period unknown, so later carry-over found nothing (0 instead of 3). calculate now runs such a calculation once more from the new inputs and keeps that result; a second change still returns without keeping. - An input a formula sets for the very period it calculates is the result and is no longer overwritten by the formula's return value. - A dump does not say what each value was calculated from, nor what was read without being kept, and a macro-cache value's sources were never calculated here: the store history now records the number from which values may depend on anything (restored values, the first macro-cache read), so an input for any variable on a branch drops them. - A custom set_input handler that calculates between its own stores: the branch drops again, by the same rule, once the handler returns. - dump_simulation on a branch dumps the values the branch reads (its own and inherited) instead of the default branch's, which could not be restored. - calculate ends the calculation (in-flight count, tracer) even if handing its history back fails. - The property test also dumps and restores simulations, including a chunk that calculates, dumps, branches the restored simulation and overrides an input; it catches a history without the new record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 2, 2026
Follow-up to the review of the first commit: - Holder.delete_arrays no longer looks through the whole record. It compares the periods memory stores for the branch before and after the deletion and discards exactly those entries, so its cost does not grow with the record. Country marginal-rate code deletes every variable on a branch: with 9,000 entries and 3,024 variables the loop took 0.73 s with the scan and takes 0.018 s now (0.004 s without any pruning). A holder with disk storage, which cannot list its periods for every branch name, still looks through the record for the variable's entries in the deleted periods and drops those whose value neither storage holds. - Holder._set records the period as storage keys the value: eternity for an eternal variable whatever period it was set for, and a Period for a handler that passes a string. Each entry names one stored value, so a string-period input is exported and deleting an eternal input drops its entry. - put_in_cache stores with is_input=False (the same keyword as #560), so values a custom set_input handler calculates are formula results, which apply_reform recalculates, not inputs. - subsample starts the record again before it rebuilds the simulation, so it records only what the rebuild stores. Tests: a guard that a memory-only deletion never iterates the record; the eternal, string-period, calculating-handler, disk and subsample cases; the property's model now includes a handler that calculates and stores months under string periods. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… earlier files It faked a later process by restarting the counter only, so with the same token its first store could reuse the writer's first file name; and it aged only the writer's last file, so on a coarse file clock (Windows) the other four could tie with the later write. Passes with every write given the same timestamp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ted) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They found the cause: test_disk_restore_reads_the_latest_file_of_each_key failed on Windows' coarse file clock (fixed in acc5f58), and its rerun by pytest-rerunfailures dropped the module's setup-state entry without running its finalizers, so the module-scoped tax_benefit_system leaked into later modules; test_branch_shared_arrays traced it, and test_parameters and test_reforms then got a traced parameter tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… handlers From the round-5 soundness review (on 2b31dcb); all were also on master: - A branch stops reading macro-cache files as soon as an input is set on it, not only when the input drops something: a file for its name may come from another simulation's branch of the same name, read before any record exists. - An input set during a calculation for its own period wins under any branch name the simulation reads (Holder.set_input stores under "default"), and also when the formula returns None and core would fall back to the default for want of an earlier period. - A direct calculate_add sums again when a term changed an input while summing (an earlier term could be obsolete). - A custom set_input handler that calculated and then raised: the branch still drops again (the inputs it stored stay). The drop after a handler now runs only if the handler calculated anything. - calculate runs a calculation again until a run changes no input, at most ten times (was once), so a formula changing a second input on its rerun no longer leaves the period unknown to carry-over. - Docs: the above, and that disk-backed branches whose names contain "_" cannot be dumped (OnDiskStorage key parsing, as on master; #552 fixes it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne rerun budget From the round-6 soundness review (on 9f8d513): - Direct calculate_divide/calculate_add could return an input stored before the calculation (they do not look in storage first), after any unrelated input change: 99 instead of 10, 100 instead of 12. An input now wins only if its sequence number is after the calculation began. - Only the outermost calculation in a simulation (calculate or a direct calculate_add) runs again after an input change; inner ones run with it. The ten-rerun budget no longer multiplies through nesting (11/121/1,331 leaf runs at depths 1/2/3, now 11 each). - A result calculated across an input change is no longer written to the macro cache (only kept results are), so a branch of the same name elsewhere cannot read it (also on master). - The post-handler drop counts _calculate, not only calculate, as calculating (a handler's private carry-over calculation was missed). - Docs: the above, and the rerun cap's consequence for carry-over. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…imulations From the round-7 soundness review (on ed843df): - A parent formula calling into a branch whose formula calls back into the parent and then changes the branch's input: the parent kept the branch's refused inner result (2 instead of 6; also through calculate_add and calculate_divide). Drops made while calculations run now also bump a process-wide count, and no result calculated across one, in any simulation, is kept. Which calculation runs again is still decided per simulation: the outermost one in the simulation whose input changed, so a temporary branch settles locally and nesting does not multiply reruns. - A direct calculate_add counts as in flight while it sums, so its terms do not each get their own reruns (121 runs, now 11). - Inputs are numbered when stored, even if the caller passes a number (a private _set with an earlier number lost its input to "input wins"). - Tests for each; docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… family From the round-8 soundness review (on 116b018): refusing to cache any result calculated across an in-flight drop anywhere in the process let an unrelated simulation's input change leave a period uncached, so a later carry-over returned 0 instead of 6 (master, 9f8d513 and ed843df give 6). A result calculated across an input change is now kept out of caches only where it can matter: - in the simulation whose input changed (its own epoch, as before); - in any calculation that received a refused result, directly or through other calculations, in any simulation (a per-frame flag in _calculation_frames, passed to the caller when a frame returns); - in calculations of the same family still running in this context, which may hold values read before the drop (their cache epoch is bumped). Unrelated simulations and other threads keep caching. Reruns stay with the outermost calculation in the simulation whose input changed; calculate_add and calculate_divide run in frames too. Docs: the scope, and that values a formula already read from a branch before a formula there changed the branch's input stay read (its result is returned but not kept). Tests isolate each part (taint across families, the family bump, the unrelated family); new mutants M38/M39 are caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the round-9 soundness review (on 359c97f): - A calculation that changed its simulation's input and then left before any cache decision (a carry-over default returned early, an exception) did not tell its caller, which kept an obsolete result (0 instead of 6). - A formula that read another simulation (another family, a root clone, its own branch from a plain thread) and then calculated there something that changed that simulation's input kept its old result (2 on the second request instead of 6). - The family bump refused a settled, correct result, so a later carry-over found no period (0 instead of 6; master gives 6). One rule replaces the taint flag and the family bump: each calculation frame records, for every other simulation it got a value from (directly or through its callees), that simulation's _input_epoch when the value's calculation began; it keeps its result only if neither its own simulation's epoch nor any of those has changed. A callee hands its reads to its caller however it ends (value, early default, error), its own simulation counted at its start; a rerun that settles restarts the count, so a settled call leaves its caller clean. Fast-cache hits from another simulation are reads too. Docs updated ("values already read stay read" now covers any simulation and caught errors). Tests for each witness; mutants M39 (no reads) and M41 (a read counted from return, not start) are caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 8, 2026
Keep restored input provenance in the input export registry while excluding values listed in derived_periods.txt. Add a differential restore and subsample regression without relying on PR #560. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
US + core hub benchmark (the review's P1), with policyengine-core at this PR's head against its master merge base. Each run is a full policyengine-us
This is a blocking performance regression. A fix round is investigating it. |
MaxGhenis
added a commit
that referenced
this pull request
Oct 9, 2026
* Keep the record of set_input values in step with storage Simulation._user_input_keys records each (variable, branch, period) stored through set_input. _invalidate_all_caches (run by apply_reform) keeps the values it names, to_input_dataframe exports them, and country packages read it to tell an entered value from a calculated one. It drifted from storage in two ways (#559): - delete_arrays deleted the values but kept their entries, so a formula result calculated later for the same period counted as an input: it survived apply_reform and was exported. Holder.delete_arrays, which Simulation.delete_arrays calls for each branch it deletes from, now drops the entries for the variable, that branch and the periods in-memory storage deletes (all of them for an eternal variable). Code that deletes through the holder, as country packages do when they move an input to another variable, is covered too. Disk storage deletes only the period asked for, so the entry for a value it still holds is kept. - clone (so also get_branch) shared the record between simulations that store their values separately, so an input set on a clone, on a branch's parent after the branch was made, or on the original after cloning was recorded for both. The copy now gets its own record, and its own empty list of running set_input calls. Tests: example regressions (11 of 12 fail before the fix; the twelfth guards against dropping too much), and a Hypothesis property that runs random set_input / calculate / delete_arrays / clone / get_branch / _invalidate_all_caches sequences against a reference model of each simulation's inputs. Fixes #559 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Prune the record from what storage deleted; record each value once Follow-up to the review of the first commit: - Holder.delete_arrays no longer looks through the whole record. It compares the periods memory stores for the branch before and after the deletion and discards exactly those entries, so its cost does not grow with the record. Country marginal-rate code deletes every variable on a branch: with 9,000 entries and 3,024 variables the loop took 0.73 s with the scan and takes 0.018 s now (0.004 s without any pruning). A holder with disk storage, which cannot list its periods for every branch name, still looks through the record for the variable's entries in the deleted periods and drops those whose value neither storage holds. - Holder._set records the period as storage keys the value: eternity for an eternal variable whatever period it was set for, and a Period for a handler that passes a string. Each entry names one stored value, so a string-period input is exported and deleting an eternal input drops its entry. - put_in_cache stores with is_input=False (the same keyword as #560), so values a custom set_input handler calculates are formula results, which apply_reform recalculates, not inputs. - subsample starts the record again before it rebuilds the simulation, so it records only what the rebuild stores. Tests: a guard that a memory-only deletion never iterates the record; the eternal, string-period, calculating-handler, disk and subsample cases; the property's model now includes a handler that calculates and stores months under string periods. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep the entry of an input a storage still holds; record twelve months as stored Round-2 review fixes: - An entry is dropped only when neither storage still holds its value, so an input deleted from memory that survives on disk stays an input. - Twelve months starting on the first of a month are recorded as the year storage keys them under, so deleting that year drops the entry. - Deleting compares the keys each storage holds before and after: it no longer goes through the record or loads files for a disk-backed holder. - The property model is seeded from the situation, checks to_input_dataframe itself, and a second property covers memory and disk storage. - Regression for the carry-over case found in the review of #562. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Find the keys a delete removed without a second snapshot Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Track actual input provenance across storage tiers and record early periods safely Preserve replayed-input tuples and master carry-over marks; add r3b soundness witnesses and value-aware disk properties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Preserve registered input tiers and master holder call semantics Seed existing recorded input locations before writes, retain positional derived semantics, and validate period aliases and early-year replay and deletion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Exercise twelve-month normalization before input handlers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Clarify custom input handler cache contract * Forget restored inputs deleted in early years * Use portable compound-period disk filenames * Replay only surviving supplied input tiers after reforms * Check live and restored input provenance against independent references --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
October 10, 2026 09:25
Contributor
Author
|
US + core hub benchmark rerun at head The slowdown reported for
Peak memory is 12.3–13.3 GB on both sides. Details are in the PR body under "Hub validation". |
Contributor
Author
|
US + core hub merge audit, core#560 at
|
MaxGhenis
added a commit
that referenced
this pull request
Oct 10, 2026
…alues (#573) * Rebuild a subsampled simulation from its inputs only subsample exported every stored value, calculated ones included, and loaded them all back as inputs. A formula result calculated before subsampling then replaced its formula for good: it was carried over past the formula's end and survived apply_reform and later set_input calls, so results after subsampling depended on what had been calculated before. It now exports the values the simulation was given (loaded from the dataset or passed to set_input), for variables with a formula too, so formula-backed structural IDs from the dataset are still kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Discard stale formula branches when subsampling Preserve the baseline policy while recreating calculation branches on the sampled population. Extend the existing differential property to formulas that create branches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Register restored source inputs before subsampling Keep restored input provenance in the input export registry while excluding values listed in derived_periods.txt. Add a differential restore and subsample regression without relying on PR #560. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Use requested-period weights only for subsample selection Calculate sampling probabilities from the requested period independently of the input export. Keep existing input-column normalization and recompute derived weights on the sample. Add focused differential and property coverage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sample missing input household IDs by recorded memberships Use the flat-file loader\x27s recorded membership labels when an input-only household ID is absent. Keep calculated default IDs out of the rebuilt dataset and verify complete household partitions with shuffled, nonconsecutive labels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin monthly sampling to existing annual weight conversion Extend differential sampling examples and properties to monthly requests. Verify annual FLOW weights divide by twelve while normalized probabilities, source inputs and the default calculation period stay consistent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Preserve subsample input provenance, structural leaves and entity weight totals * Clarify legacy dump provenance and subsample branch disposal * Handle period labels, integer roles and zero weight totals when rebuilding samples * Focus disk provenance coverage and label boundary-case variables * Reject samples that cannot preserve a positive input weight total * Normalize subsample weights by the exported membership * Skip weight fallback tests when Hypothesis is unavailable --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An input override on a branch now removes values that can depend on the replaced input while preserving supplied inputs and their individual storage-tier provenance. A branch calculation must match a fresh simulation given the same inputs first.
Simulation.drop_computed_arrays()explicitly clears calculated values after a policy change.Current head:
8c5a48a88443679bd1f6c17ed20e96490f7f51e8. This PR remains draft and unmerged under d807.Registration capture repair (review r3)
Review r3 found that replacing a named worker while a memo-producing formula runs could omit the original worker's registration from its temporary result. A second suffix call reused the old
1instead of reading the replacement's4. When the outer formula stored that second value as another branch's input, the error survived its retry.Named registrations and their registered ancestor links are now captured when a calculation begins or a fast-cache value is read. Existing
get_branchlookups also capture the name actually resolved, including aliases whose names differ from the returned branch object's own name. Nested calculations and memo recall propagate the exact captured tuples. Temporary results retain those dependencies and cannot enter or remain reusable while any captured registration names a different object. Standalone clone reads retain their snapshot identities; ordinary invalidation, supplied inputs, per-tier provenance and retry rules are preserved.Twenty-four new regression cases cover replacement inside the suffix or foreign producer, a recalled intermediary, fast and holder reads, replacement of an ancestor registration, and aliased registration names. The first twelve fail on the reviewed head with a persisted result of
1instead of4; the twelve alias cases fail on the intermediate repair before the actual-name lookup guard. All twenty-four pass after the complete repair. They also verify subsequent cached results. The existing bounded foreign-read formula-call regression is retained unchanged, including the 20-node bound of 40 credit calls (60 with another replacement).Stale-attempt foreign-read repair
Review r2 found that refusing all foreign-read caches inside a stale attempt expands shared dependencies into repeated formula work. For
credit_i = worker.source + sum(credit_j for j < i), the reviewed 20-node graph required 524,308 credit formula calls by static derivation, despite having only one outer retry.Read-only foreign results now use a cache scoped to the current attempt. Every reuse validates the calculating simulation and foreign simulations' input epochs and input-store counts, branch registrations, and context-free activity. It passes the original foreign reads back to the caller so later changes still make the caller stale. Observed mutations, raw holder writes, deletions and branch creations clear the temporary cache. Calculations with mutation, creation or untracked effects cannot enter it. Restarting or ending the outer calculation discards it; these results do not enter holder, fast or macro caches during the stale attempt.
The retained formula-count regression covers depths 1, 6, 12 and 20, direct and mediated foreign reads, and another replacement after interim graph work. It requires at most two credit evaluations per node across the stale attempt and clean retry, or three with a second replacement. The six-node test fails on the r2-reviewed head with 38 calls versus a bound of 12. At depth 20, the repaired graph stays within 40 calls, or 60 with another replacement, and returns the exact expected value. These are formula-count bounds for the eligible graph; dependency validation still visits graph edges and recorded reads.
Additional regressions cover counter-preserving raw holder writes and deletions, context-free activity between sibling calculations after the restricted child exits, replacement of a named branch by a clone with identical counters, and a direct foreign
_calculatewhose result must stay associated with its own simulation. Mutation effects still replay on retries. Existing fixed-point, persistent-branch identity, optional-fast-cache and input-precedence protections remain.Validation at this head
uv run pytest -q <file>. The stale-frame result is included once in this total.uv run pytest -q tests/core.git diff --checkpasses for this repair and against canonical master.uv sync --frozen. Existing Hypothesis settings were preserved. Evidence remains untracked outside the nested repository; no dependency lockfile or generated artifact changes.bbb7b6fb26faa68f6f09c49b5c615c41208c2dfdpassed all 19 checks; that is historical validation.Documentation review:
docs/usage/branches.mdnow explains when named registration dependencies are captured and how nested calculations and temporary reuse propagate them. It retains the cache guards, dependency replay and result-array memory limits. This repair does not change public method signatures; Simulation/dump API references continue using autodoc. The existing towncrier fragment is updated. No local documentation build was run. Impact: medium. Confidence: high within the synthetic regression and core-test scope. The pending hub A/B remains outside that scope.Master composition and limits
Merge
52474302eeb76bab1fd0717098f04d3cfcfb3f7fretains canonical mastera934dc3465cd3284b6d21fb6fd7a0506b3e6581b, including #576, #558, #561, #571 and #581. The implementation preserves chronological invalidation history, per-tier input provenance, shared_drop_calculated_overlappinghelper cleanup, option-result cache routing and dtype guards, and restoring supplied inputs before calculated values. Existing helper composition regressions cover memory/disk storage, parent isolation, replay, and years 2024/0025.The documented limits remain: policy changes require explicit cleanup, existing child branches remain snapshots, root inputs skip branch dependency invalidation, and values already cached elsewhere do not become retroactively fresh. Formulas must not mutate cached arrays in place or inspect storage directly for dependency-sensitive decisions.
Hub validation (2026-10-10, at this head)
A full-data A/B on PE-US main
fe0c3a651cwith the default dataset. It compares policyengine-core at its master merge basea934dc3465with this head8c5a48a884. Runs went one at a time under the machine's heavy-job lock, base then PR, for each year.household_net_income,income_tax,state_income_tax73b90c0fb4ran 2.1–3.5× slower in 2024 and 2028 (hub comment of 2026-10-09). That slowdown does not reproduce at this head.The PE-US partner contract tests (
policyengine_us/tests/policy/baseline/partners, 145 files) ran read-only with policyengine-core at this head and PE-US mainfe0c3a651c: 623 passed, 0 failed (2026-10-10T09:41Z).Holds
d807's sequencing condition is met: PolicyEngine/policyengine-us#9741 merged on 2026-10-06. CI at this head remains a gate. Max retains the #570/#572 decisions. d899's merge order (#576 → #561 → #571 → #581, with #558 alongside), d1024/d1026 Proposal A and the principle "make inputs as leaf-nodey as possible" remain unchanged.
axiom: n/a; this changes core cache invalidation, with inputs-first equivalence as its correctness invariant.
Retained evidence before this repair
At runtime-identical source for prior head
73b90c0fb4728e7abaae63b143eebad8a5cf93b5, a 2,000-household 2024 profile returned arrays byte-identical to master. Master executed 20,828 formula calls and the prior head executed 23,486; immediate repeats added zero calls. Their calculation CPU times were 73.43 and 74.48 seconds. Shared-host wall times did not establish a speedup. This profile predates the present cache mechanism and supplies no current-head overhead bound.The previous head passed 2,303 core tests, with 2 skipped and 3 xfailed, and all 19 CI checks. Those results are historical; validation of the repaired head is reported above. Earlier household, subsample and country-package comparisons remain limited to their recorded revisions and calculation orders.