Repository navigation
Return decile relative changes as fractions - #904
Conversation
policyengine.py computes decile relative_change as a percentage (100 * absolute_change / baseline_mean). The executor copied it into DecileOutput.relative unchanged, so decile.relative and wealth_decile.relative reached the API 100x too large. Divide by 100 so they keep the fraction contract that the previous API and the app use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
left a comment
There was a problem hiding this comment.
The fix in build_decile_output looks right. It covers both decile and wealth_decile, including segmented US runs. A synthetic check confirms the units: for a 1% gain, .py 6.7.5 gives relative_change = 1.0, and this branch returns 0.01.
One request: please add a test that runs .py's own decile calculation, rather than hand-written rows.
test_decile_relative_is_a_fraction_of_the_baseline_mean feeds in relative_change values written by hand as percentages, so it only checks the division by 100. If .py ever changes the units of relative_change, this code would divide twice (or not at all), and the test would still pass. That's how the original problem got past #502: its fixture rows were hand-written fractions, so the tests only checked that the value passed through unchanged.
A suggestion:
- Build a small baseline and reform pair with known incomes, for example on the weighted-frame fixtures. The UK geography disclosure tests already run the installed
policyengine.disclosurethis way. - Run .py's decile calculation on it:
policyengine.outputs.decile_impact.calculate_decile_impacts(baseline_simulation=..., reform_simulation=..., quantiles=2), orDecileImpact(...).run(). - Pass the result to
build_decile_output. - Assert
relative[decile] == pytest.approx(absolute_change / baseline_mean). For example, a 1% gain for every household gives0.01in each decile.
If building .py simulations is too heavy for a unit test, you could instead call _calculate_decile_impact_values on a four-household _PreparedDecileAnalysis. That still runs .py's real formula, but depends on private names.
Separately, I'm opening a Public API PR to bump its economy cache version, so the ×100 results already cached aren't served once this fix is deployed.
The earlier test fed hand-written relative_change rows, so it only checked the division by 100. This test runs policyengine.py's calculate_decile_impacts on a small in-memory baseline and reform (every household gains 1%) and checks that build_decile_output returns absolute_change / baseline_mean, for income deciles and for a precomputed (wealth) decile variable. If policyengine.py changes the units of relative_change, it fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks! Added in 3b19937:
Local checks: executor tests 1648 passed, 23 skipped; Ruff 0.15.22 and the executor Pyright check pass. Thanks for the cache-version PR on the Public API side. |
anth-volk
left a comment
There was a problem hiding this comment.
Thanks, this covers it. test_decile_relative_is_the_fraction_policyengine_computes runs .py's own calculate_decile_impacts and checks that relative equals absolute_change / baseline_mean. With the division in build_decile_output reverted, it fails (1.0 instead of 0.01). With the division in place, it passes.
On merge order: this should merge and deploy before #881. #881 moves main's .py pin to 6.10.0, so after that, a deploy would no longer rebuild the 6.7.5 app the Public API uses. The Public API's cache bump, PolicyEngine/policyengine-api#4026, then goes in once this is deployed.
Fixes #827
Summary
decile.relativeandwealth_decile.relativein economy results are percentages, but the API contract and every consumer read them as fractions. The "Relative" decile charts in the app (v2 and v3, US and UK) show values 100 times too large.policyengine.py computes
relative_change = 100 * absolute_change / baseline_mean(policyengine/outputs/decile_impact.py, checked on the pinned 6.7.5).build_decile_outputcopied that value intoDecileOutput.relativeunchanged. This PR divides it by 100, so the API returns a fraction again, as the previous API ((reform - baseline) / baseline) did.Changes
simulation_output_distribution.py:build_decile_outputdividesrelative_changeby 100. Income deciles and UK wealth deciles both use this builder, and segmented national runs build their deciles through it too.relative_changein percentages, as policyengine.py does. The expected API output (CURRENT_SINGLE_YEAR_MACRO_RESULT, the UK wealth decile test) stays the same fractions.test_decile_relative_is_a_fraction_of_the_baseline_meanuses the numbers from the issue (1.1254 → 0.011254) and checks that a missing value stays 0.0.I checked the other outputs: no other executor output reads policyengine.py's
relative_change.constituency_impact[].relative_household_income_changeand the local-authority equivalent are already fractions.Follow-up
Results that the API already computed and cached keep the ×100 values. They need a recompute or a one-off correction after this deploys.
Testing
uv run --frozen pytest tests -m "not requires_gcp"in the executor: 1646 passed, 23 skipped.format --checkandcheck .pass.python -m scripts.rules pyright --sync projects/policyengine-simulation-executor: passed (no new errors over the baseline).🤖 Generated with Claude Code