Skip to content

2025 baseline poverty vs HBAI: AHC rates too high for non-pensioners #367

@vahid-ahmadi

Description

@vahid-ahmadi

Summary

Ran a year-matched poverty exercise on the freshly-downloaded enhanced_frs_2023_24.h5 with pe-uk 2.88.6 / pe-uk-data 1.50.2 / policyengine-core 3.25.1. BHC rates match DWP very closely in every year tested. AHC rates for children and working-age adults are consistently 9–18 pp too high, driven by an AHC median that is ~£50–100/wk below HBAI. The gap between our BHC and AHC medians (~£200/wk) is roughly double the HBAI gap (~£90/wk), which localises the issue to housing-cost deduction being too large.

Exercise

  1. Cleared the HuggingFace cache (~/.cache/huggingface/hub/models--policyengine--policyengine-uk-data) and removed stale .h5 files from policyengine_uk_data/storage/.
  2. Re-downloaded completed datasets via storage/download_completed_datasets.py using HUGGING_FACE_TOKEN.
  3. Upgraded policyengine-uk 2.87.0 → 2.88.6 (and policyengine-core 3.23.6 → 3.25.1).
  4. Built a Microsimulation from UKSingleYearDataset(enhanced_frs_2023_24.h5) and computed for each year (2024, 2025, 2026):
    • in_relative_poverty_bhc, in_relative_poverty_ahc
    • in_poverty_bhc, in_poverty_ahc (absolute)
    • medians of equiv_hbai_household_net_income (BHC) and equiv_hbai_household_net_income_ahc (AHC)
  5. Broke down by all individuals, children (<16), working-age (16–64), pensioners (65+).
  6. Compared each pe-uk year to the corresponding official publication (same calendar year as the FYE label):
    • pe-uk 2024 ↔ HBAI FYE 2024 (published May 2025; FYE 2011 absolute base)
    • pe-uk 2025 ↔ HBAI FYE 2025 (published March 2026; abs base rebased to FYE 2025)
    • pe-uk 2026 ↔ DWP low-income poverty projections FYE 2026 (Feb 2026; children only)

Script: poverty_baseline_2026.py.

Results — year-matched

pe-uk 2024 vs HBAI FYE 2024

Medians: pe-uk BHC £704/wk, AHC £508/wk · official BHC £650/wk, AHC £562/wk

Group rel BHC rel AHC abs BHC abs AHC
All — pe-uk 16.8 28.0 13.3 30.0
All — official 17 21 15 18
gap (pp) −0.2 +7.0 −1.7 +12.0
Children — pe-uk 23.7 41.2 19.1 44.0
Children — official 23 31 20 26
gap (pp) +0.7 +10.2 −0.9 +18.0
Working-age — pe-uk 15.0 28.4 11.9 30.4
Working-age — official 15 19 13 17
gap (pp) 0.0 +9.4 −1.1 +13.4
Pensioners — pe-uk 15.7 13.4 11.9 14.8
Pensioners — official 19 16 16 13
gap (pp) −3.3 −2.6 −4.1 +1.8

pe-uk 2025 vs HBAI FYE 2025

Medians: pe-uk BHC £728/wk, AHC £521/wk · official BHC £719/wk, AHC £623/wk

Group rel BHC rel AHC abs BHC abs AHC
All — pe-uk 17.4 28.6 13.7 30.9
All — official 16 20 16 20
gap (pp) +1.4 +8.6 −2.3 +10.9
Children — pe-uk 25.2 42.5 20.1 45.5
Children — official 21 27 21 27
gap (pp) +4.2 +15.5 −0.9 +18.5
Working-age — pe-uk 15.5 28.9 12.3 31.2
Working-age — official 14 19 14 19
gap (pp) +1.5 +9.9 −1.7 +12.2
Pensioners — pe-uk 16.0 13.6 12.0 15.5
Pensioners — official 16 14 16 14
gap (pp) 0.0 −0.4 −4.0 +1.5

pe-uk 2026 vs DWP FYE 2026 child projection

Measure pe-uk 2026 DWP proj gap
child rel BHC 24.6% 25% −0.4 pp ✅
child abs BHC 20.0% 20% 0.0 pp ✅
child rel AHC 42.5% 33% +9.5 pp
child abs AHC 45.4% 26% +19.4 pp

What is off

The BHC pipeline lands close to DWP in every year and every group. The AHC pipeline is the problem, and the fingerprint is consistent:

pe-uk BHC–AHC gap official BHC–AHC gap
2024 £196/wk £88/wk
2025 £207/wk £96/wk

Our housing-cost deduction is roughly twice what HBAI reports. This single fact explains:

  • AHC medians ~£50–100/wk below official,
  • relative-AHC poverty rates ~9–16 pp too high for non-pensioners,
  • absolute-AHC poverty rates ~12–19 pp too high (the anchored threshold amplifies the median drop).

Pensioners are largely unaffected on AHC (gap ±1–2 pp) because they are predominantly outright owners with little/no housing cost — consistent with the diagnosis.

A secondary, smaller issue: pensioner BHC rates run ~3–4 pp too low across measures. Worth a separate audit after AHC is resolved.

What to check in our data

  1. Housing-cost components feeding AHC net income. Audit magnitudes of rent, mortgage_interest_repayment, water_and_sewerage_charges, ground_rent, service_charge in the enhanced FRS 2023-24 build:
    • compare weighted totals and mean-per-household against the HBAI housing-cost aggregate (HBAI Table 1.1 / 1.2);
    • check for double-counting (e.g. both rent and mortgage interest populated, or service charges included twice).
  2. AHC equivalisation scale (household_equivalisation_ahc): confirm it matches HBAI's modified-OECD after-housing-costs scale, not the BHC version.
  3. Uprating paths. Between 2023-24 and projection years, check that housing-cost components are not being uprated faster than net earnings/benefits.
  4. absolute_poverty_threshold_ahc parameter: confirm the reference year and CPI series match HBAI.
  5. Enhanced-FRS calibration targets. Check whether any calibration target pulls the housing-cost distribution upward (e.g. a rent-roll control total).
  6. Regression check: rerun the same analysis on pe-uk-data main (pre-357) to rule out the regional land-to-property PR — expected to be neutral but worth confirming.
  7. Pensioner BHC under-shoot: once AHC is sorted, check pension income components (state pension, private pension drawdown) and their uprating for 2024+.

Environment

  • policyengine-uk 2.88.6
  • policyengine-uk-data 1.50.2
  • policyengine-core 3.25.1
  • Python 3.13.5
  • Dataset: enhanced_frs_2023_24.h5 (fresh download from policyengine/policyengine-uk-data)

Scope note

BHC estimates reproduce DWP well — this issue does not indicate a problem with BHC poverty, BHC income, or benefit calculations on the BHC side. The fix is concentrated in housing-cost-related variables and their downstream consumption by the AHC income aggregate.

Sources

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions