Skip to content

Return decile relative changes as fractions - #904

Merged
PavelMakarchuk merged 2 commits into
mainfrom
fix/827-decile-relative-fraction
Oct 23, 2026
Merged

PavelMakarchuk merged 2 commits into
mainfrom
fix/827-decile-relative-fraction

Conversation

@PavelMakarchuk

Copy link
Copy Markdown
Collaborator

Fixes #827

Summary

decile.relative and wealth_decile.relative in economy results are percentages, but the API contract and every consumer read them as fractions. The "Relative" decile charts in the app (v2 and v3, US and UK) show values 100 times too large.

policyengine.py computes relative_change = 100 * absolute_change / baseline_mean (policyengine/outputs/decile_impact.py, checked on the pinned 6.7.5). build_decile_output copied that value into DecileOutput.relative unchanged. This PR divides it by 100, so the API returns a fraction again, as the previous API ((reform - baseline) / baseline) did.

Changes

  • simulation_output_distribution.py: build_decile_output divides relative_change by 100. Income deciles and UK wealth deciles both use this builder, and segmented national runs build their deciles through it too.
  • Test fixtures now give relative_change in percentages, as policyengine.py does. The expected API output (CURRENT_SINGLE_YEAR_MACRO_RESULT, the UK wealth decile test) stays the same fractions.
  • New test test_decile_relative_is_a_fraction_of_the_baseline_mean uses the numbers from the issue (1.1254 → 0.011254) and checks that a missing value stays 0.0.

I checked the other outputs: no other executor output reads policyengine.py's relative_change. constituency_impact[].relative_household_income_change and the local-authority equivalent are already fractions.

Follow-up

Results that the API already computed and cached keep the ×100 values. They need a recompute or a one-off correction after this deploys.

Testing

  • uv run --frozen pytest tests -m "not requires_gcp" in the executor: 1646 passed, 23 skipped.
  • Ruff 0.15.22 (the pinned version): format --check and check . pass.
  • python -m scripts.rules pyright --sync projects/policyengine-simulation-executor: passed (no new errors over the baseline).
  • Not run: an end-to-end economy run against deployed infrastructure.

🤖 Generated with Claude Code

policyengine.py computes decile relative_change as a percentage
(100 * absolute_change / baseline_mean). The executor copied it into
DecileOutput.relative unchanged, so decile.relative and
wealth_decile.relative reached the API 100x too large. Divide by 100 so
they keep the fraction contract that the previous API and the app use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@anth-volk anth-volk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix in build_decile_output looks right. It covers both decile and wealth_decile, including segmented US runs. A synthetic check confirms the units: for a 1% gain, .py 6.7.5 gives relative_change = 1.0, and this branch returns 0.01.

One request: please add a test that runs .py's own decile calculation, rather than hand-written rows.

test_decile_relative_is_a_fraction_of_the_baseline_mean feeds in relative_change values written by hand as percentages, so it only checks the division by 100. If .py ever changes the units of relative_change, this code would divide twice (or not at all), and the test would still pass. That's how the original problem got past #502: its fixture rows were hand-written fractions, so the tests only checked that the value passed through unchanged.

A suggestion:

  1. Build a small baseline and reform pair with known incomes, for example on the weighted-frame fixtures. The UK geography disclosure tests already run the installed policyengine.disclosure this way.
  2. Run .py's decile calculation on it: policyengine.outputs.decile_impact.calculate_decile_impacts(baseline_simulation=..., reform_simulation=..., quantiles=2), or DecileImpact(...).run().
  3. Pass the result to build_decile_output.
  4. Assert relative[decile] == pytest.approx(absolute_change / baseline_mean). For example, a 1% gain for every household gives 0.01 in each decile.

If building .py simulations is too heavy for a unit test, you could instead call _calculate_decile_impact_values on a four-household _PreparedDecileAnalysis. That still runs .py's real formula, but depends on private names.

Separately, I'm opening a Public API PR to bump its economy cache version, so the ×100 results already cached aren't served once this fix is deployed.

The earlier test fed hand-written relative_change rows, so it only checked
the division by 100. This test runs policyengine.py's calculate_decile_impacts
on a small in-memory baseline and reform (every household gains 1%) and
checks that build_decile_output returns absolute_change / baseline_mean,
for income deciles and for a precomputed (wealth) decile variable. If
policyengine.py changes the units of relative_change, it fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk

PavelMakarchuk commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks! Added in 3b19937: tests/test_simulation_output_distribution.py.

Local checks: executor tests 1648 passed, 23 skipped; Ruff 0.15.22 and the executor Pyright check pass.

Thanks for the cache-version PR on the Public API side.

@anth-volk anth-volk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this covers it. test_decile_relative_is_the_fraction_policyengine_computes runs .py's own calculate_decile_impacts and checks that relative equals absolute_change / baseline_mean. With the division in build_decile_output reverted, it fails (1.0 instead of 0.01). With the division in place, it passes.

On merge order: this should merge and deploy before #881. #881 moves main's .py pin to 6.10.0, so after that, a deploy would no longer rebuild the 6.7.5 app the Public API uses. The Public API's cache bump, PolicyEngine/policyengine-api#4026, then goes in once this is deployed.

@anth-volk
anth-volk marked this pull request as ready for review October 22, 2026 21:13
@PavelMakarchuk
PavelMakarchuk merged commit 1fe1957 into main Oct 23, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Economy output: decile.relative is now a percentage (×100), not a fraction

2 participants