Replication Package: "Decarbonizing Heat: The Impact of Heat Pumps and a Time-of-Use Heat Pump Tariff on Energy Demand"

Louise Bernard (Centre for Net Zero), Andy Hackett (Centre for Net Zero), Robert Metcalfe (Columbia University), Andrew R. Schein (Centre for Net Zero & NBER) — American Economic Journal: Economic Policy, AEJPol-2025-0575

This repository contains the code used to produce all tables and figures in the paper and its online appendices. It does not contain the underlying household-level data, which is confidential (see Data Availability Statement below).

Overview

The paper studies two related interventions delivered by a large UK energy retailer to residential customers: (1) the staggered installation of heat pumps, and (2) the introduction of "Cosy Octopus," a time-of-use electricity tariff designed for heat pump users. We use household half-hourly and monthly electricity and gas smart-meter data, combined with property and area-level public data, to estimate the effect of both interventions on energy demand, and to compute the marginal value of public funds of the UK's heat pump subsidy (the Boiler Upgrade Scheme).

The code is organized as a numbered pipeline of R scripts orchestrated by scripts/main.R. See Description of Programs below for what each stage does and List of Tables and Programs for how each exhibit in the paper maps to the script that produces it.

Data Availability Statement

Statement about rights

• The authors certify that they had legitimate access to all data used in this paper.
• The authors do not have the right to redistribute the confidential, household-level data described under "Restricted data" below. These data were made available to the authors through their affiliation with Centre for Net Zero (part of Octopus Energy Group Limited) under the retailer's own customer-data governance arrangements, and are subject to UK GDPR and contractual restrictions on redistribution to third parties.
• The authors have the right to redistribute, and do redistribute in this package, all code, and all publicly sourced reference data listed under "Public data" below (or, where a dataset's own license does not permit redistribution, a citation and access instructions in lieu of the file itself).

Summary of availability

• Some data cannot be made publicly available (household-level smart-meter, tariff/agreement, and survey data)
• Some data are publicly available (area- and policy-level reference data used for covariates, weighting, weather, carbon accounting, and welfare calculations)

Restricted data

Data category	Description	Source	Why it's restricted
Smart-meter consumption records	Half-hourly and monthly household electricity and gas consumption	Octopus Energy Group, via Centre for Net Zero	Personal data (UK GDPR); commercially confidential customer data; no right of redistribution
Account / tariff / agreement records	Contract start/end dates, tariff type, Cosy Octopus enrollment, heat pump installation/deal dates	Octopus Energy Group, via Centre for Net Zero	Same as above
Property attributes linked to accounts	EPC rating (linked via UPRN), floor area, predicted heat loss, property value, location, as merged to household accounts	Octopus Energy Group, via Centre for Net Zero	Personal data once linked to an identifiable household (UPRN-level linkage)
Customer survey responses	Heat pump usage survey (backup heating, comfort, NPS/feedback) fielded to Octopus Energy customers	Centre for Net Zero / Octopus Energy Group	Personal data; no independent ethics/consent framework governs redistribution (see Ethics below)
EV detection flags	Household-level electric vehicle charging detection derived from consumption patterns	Centre for Net Zero, derived from restricted consumption data	Derived from restricted data above

Access for verification/replication purposes:

• There is no established, author-independent process by which an outside researcher can request access to these data.
• Centre for Net Zero / Octopus Energy Group have indicated they are unable to provide a private, unpublished copy of the data directly to the AEA Data Editor
• Centre for Net Zero / Octopus Energy Group have designated third-party replicator for verification purposes.
• Given the above, per the AEA Data and Code Availability Policy's provisions for non-public data, the authors commit to: (i) preserving the data and code for no less than five years following publication; (ii) providing reasonable assistance to requests for clarification and replication; (iii) making all code publicly available; and (iv) publicly disclosing the source of the data with contact information (below).

Contact for data provenance questions: info@centrefornetzero.org

Public data

All of the following are cited in the paper's bibliography (overleaf-export/.../bibliography.bib); citation keys are given for cross-reference. None of these files contain personal data.

Dataset	Used for	Citation key	Redistributed in this package?
ONS mid-year income estimates (MSOA, FYE2023)	Local-area income covariate	ons_income_fye2023	Cited only — public ONS release, re-download from source rather than shipped
ONS House Price Statistics for Small Areas (HPSSA), Dataset 3	MSOA-level mean property prices	ons_hpssa3	Same as above
ONS postcode-to-geography lookups (2011/2021, 2021–2023 vintages)	Crosswalk from postcode to MSOA/LSOA/LAD	ons_pcd_msoa2011_2021, ons_pcd_msoa2021_2023	Same as above
UK ETS allowance prices	Carbon pricing scenarios (§ Operating Cost Scenarios)	UKETS	Yes/public
DESNZ Boiler Upgrade Scheme statistics	Subsidy program context	DESNZ:BUS2025	Yes/public
Greenhouse Gas Reporting: Conversion Factors (2023, 2025)	Carbon intensity of gas/electricity	ukgov2023, ukgov2025	Yes/public
Green Book & supplementary guidance (2020, 2023, 2024)	Discount rates, valuation of energy/GHG, appraisal data tables	ukgov2020, ukgov2023valuation, ukgov2024	Yes/public
MCS Data Dashboard	Average heat pump installation costs	mcs_dashboard	Yes — permitted for academic use
Delta-EE / BEIS heating measures cost report	Counterfactual gas boiler cost	delta2018	Yes/public (BEIS Research Paper 2020/028)
Rennert et al. (2022), Nature	Social cost of carbon (IWG-lineage estimate)	rennert2022	Citation only (journal article, not a data extract)
BEIS Energy Follow-Up Survey	External benchmark for backup-heating prevalence	beis2021efus	Citation only
National Grid ESO / WattTime marginal carbon intensity	Marginal emissions factors	—	Yes/public — confirmed shareable
UK & US social cost of carbon series (HMG, IWG)	Welfare/MVPF carbon valuation	—	Defra 2023 htts://www.gov.uk/government/publications/greenhouse-gas-reorting-conversion-factors-2023
Air quality cost data	Air-quality externality in MVPF	ukgov2024 (Table 15)	Yes/public, part of Green Book data tables

Note: Energy Performance Certificate (EPC) ratings are not a public-data item here — they are supplied by Octopus Energy Group linked to household accounts via UPRN, and are therefore listed under "Restricted data" above, not shared.

eff_df.csv (the electricity/gas consumption-change results reported in the paper, used as input to scripts/05_MVPF.R) is included in this repository at public_data/eff_df.csv.

scripts/05_MVPF.R's other input, HP and Cosy paper welfare analysis.xlsx (compiling most of the public reference series in the table above — DESNZ/Defra carbon intensity, Green Book Formulas tables, MCS cost data, GDP deflator, SCC), now lives at public_data/HP and Cosy paper welfare analysis.xlsx.

Ethics and human subjects

No separate IRB approval or formal Data Protection Impact Assessment was conducted for this research; the underlying customer survey and data use were treated as routine commercial customer analytics by Centre for Net Zero / Octopus Energy Group rather than as academic human-subjects research requiring independent ethics review.

Per DCAS #10/#11: the customer survey referenced in scripts/01_11_cosy_survey_figures.R (fielded to Cosy Octopus tariff adopters, and cited in the paper's "Survey Responses" section, \label{subsec:survey}) is described in full in that script's header comment (question wording and response options for all ten questions, including the branching logic).

Computational Requirements

Captured from the production Vertex AI Workbench instance via scripts/utils_session_info.R (software/hardware: 2026-08-18; runtime: 2026-09-09).

Software

• R version: 4.5.1 (2025-06-13), platform x86_64-pc-linux-gnu

• Key R packages and versions used (from scripts/main.R):

  Package	Version		Package	Version
  knitr	1.51		readxl	1.5.0
  kableExtra	1.4.0		purrr	1.2.0
  did	2.5.1		progress	1.2.3
  fixest	0.14.2		lfe	3.1.1
  data.table	1.18.0		tibble	3.3.0
  lubridate	1.9.5		didimputation	0.5.0
  dplyr	1.1.4		ggtext	0.1.2
  ggplot2	4.0.3		MatchIt	4.7.2
  RColorBrewer	1.1.3		zoo	1.8.15
  tidyr	1.3.2		patchwork	1.3.2
  scales	1.4.0		stringr	1.6.0
  readr	2.1.6		stargazer	5.2.3
  forcats	1.0.1		panelView	1.1.18
  viridis	0.6.5		arrow	22.0.0.1

• Operating system: Debian GNU/Linux 11 (bullseye), kernel 5.10.0-46-cloud-amd64

Hardware

• OS: Debian GNU/Linux 11 (bullseye)
• CPU: Intel(R) Xeon(R) @ 2.20GHz, 1 socket, 32 cores, 2 threads/core (64 logical CPUs)
• Memory: 235 GiB total
• Disk space: /home/jupyter local disk 492 GiB (149 GiB used, 343 GiB available at time of capture); GCS-mounted data bucket (/home/jupyter/gcs) reports 1.0 PiB available (shared, not dedicated)

Runtime

• Full pipeline (source("scripts/main.R")), cold start (all cached intermediates deleted first per scripts/utils_session_info.R): measured 2026-09-09 at 28,753.2s elapsed (~8h) on the hardware above (user 61,486.1s, sys 19,028.2s — reflecting parallelism across the 64 logical CPUs)
• Per-stage breakdown: not captured (optional — only needed if runtime is heterogeneous across stages)

Description of Programs

scripts/main.R is the master script: it installs/loads packages, sets global plot colors and the fixest estimation config, then sources six top-level orchestrators in strict ascending numeric order (01_00 → 02_00 → 03_00 → 04_00 → 05_MVPF → 06_...). Each orchestrator in turn sources its own numbered sub-scripts (0X_01, 0X_02, ... in order, verified directly against the source() calls in 01_00_cosy.R and 02_00_heatpump.R). Sub-scripts communicate through objects left in the global environment (not return values), and each orchestrator wipes its own environment down to a whitelist after each stage to prevent cross-contamination between stages.

Script	Purpose
01_00_cosy.R	Cosy Octopus time-of-use tariff (electricity) analysis
02_00_heatpump.R	Heat pump (gas/electricity) analysis
03_00_balance_tables_and_reweighting.R	Balance tables and sample reweighting, shared across both analyses
04_00_half_hourly_analysis.R	Half-hourly consumption analysis
05_MVPF.R	Marginal value of public funds / welfare analysis
06_sample_descriptive_statistics.R	Sample descriptive statistics

Superseded explorations and old R Markdown knit artifacts (an earlier version of 05_MVPF.R, an earlier version of the loader now in 01_01_load_data.R, code split out of 04_00_half_hourly_analysis.R, and rendered knit output) are kept locally for reference but are git-ignored and not part of this package, since some carry stale hardcoded local paths.

Key variables

See DATA_DICTIONARY.md for the full variable-level metadata (DCAS #5): what each variable means, how it's constructed, and which script introduces it.

Instructions for Data Preparation and Analysis

Data preparation: Each 0X_01_load_data.R script guards its heavy merge with if (!file.exists(...)), building and caching to <datapath>/scratch/*.RDS on first run, and reading the cache thereafter (datapath is set by main.R; see Environment below — it is not a data/ folder relative to the repo root). To force a rebuild, delete the relevant cached .RDS file (commented file.remove(...) lines at the top of each loader show which file to remove).

Analysis: From an R session with working directory set to the repository root:

source("scripts/main.R")

This runs the full pipeline end-to-end with no manual intervention required, producing all figures (graphs/) and tables (tables/) referenced in the paper. The six orchestrators are not independent of one another and should be run in this sequence, not in isolation: 03_00_balance_tables_and_reweighting.R reads cached files that only 01_00_cosy.R (scratch/aggregated_data.RDS, scratch/cosy_survey_respondent_kids.RDS) and 02_00_heatpump.R (output/hp_installed.rds, output/overall_weekly.rds, scratch/ids_cs_elec.RS, scratch/ids_cs_gas.RS) produce; 05_MVPF.R reads output/eff_df.csv, produced by 02_00_heatpump.R; and 06_sample_descriptive_statistics.R reads cached files produced by both 01_00_cosy.R (scratch/did_cosy_*_universal.RDS) and 02_00_heatpump.R (scratch/est_cs_elec_weekly.RDS, scratch/est_cs_gas_weekly.RDS). Only 01_00_cosy.R, 02_00_heatpump.R, and 04_00_half_hourly_analysis.R read no cached output from another orchestrator, so those three are the only ones that can be re-run individually once the earlier stages have completed at least once. To re-run any single stage, first run the package-loading and parameter block at the top of main.R (this defines hp_color, cosy_color, etc., and the datapath variable that later scripts depend on), then source() the relevant 0X_00_*.R file directly — but only after every orchestrator it depends on (per the list above) has already been run and its cached files exist.

Environment: datapath is read from the COSY_DATAPATH environment variable (see scripts/main.R), which must point at the mounted data directory containing input/, scratch/, and output/ -- there is no local data/ folder anywhere in this pipeline by design; all restricted input/scratch/output data lives on a separately-mounted bucket. Copy .Renviron.example to .Renviron (git-ignored) at the repository root and set COSY_DATAPATH there before running scripts/main.R. In production (GCP Vertex AI Workbench), the bucket is mounted via gcsfuse before starting R, e.g.:

gcsfuse --implicit-dirs --rename-dir-limit=100 --max-conns-per-host=100 <bucket-name> <mount-point>

then set COSY_DATAPATH=<mount-point>/cosy2 in .Renviron.

graphs/ and tables/ (repo-relative) and the entire mounted datapath tree are not version-controlled; they are populated by running the pipeline against the (restricted) input data.

List of Tables and Programs

Every table/figure below is \input{}/\includegraphics{}'d directly into overleaf-export/.../main.tex from tables/*.tex / graphs/*.png. This is the complete set of active exhibits (24 tables, 48 figures) traced to the exact script and line that produces each one; figures with subgroup-parametrized filenames (e.g. per rate-period breakdowns) are generated in a loop from a single script call, noted below where relevant. Figures referenced only in commented-out \includegraphics lines in main.tex are excluded as not currently part of the published paper.

Tables

Exhibit	Producing script
balance_table.tex	02_05_balance_table.R
balance_table_cosy.tex	01_05_balance_table.R
balance_table_cosy_survey.tex	03_00_balance_tables_and_reweighting.R
balance_table_cosy_adoption.tex	03_00_balance_tables_and_reweighting.R
balance_table_cosy_hp_random.tex	03_00_balance_tables_and_reweighting.R
balance_table_hp_adoption.tex	03_00_balance_tables_and_reweighting.R
balance_hp_matching.tex	03_00_balance_tables_and_reweighting.R
matching_hp.tex	03_00_balance_tables_and_reweighting.R
did_lcts.tex	01_07_lct_ownership_and_leavers.R
did_leavers.tex	01_07_lct_ownership_and_leavers.R
ev_charging.tex	01_07_lct_ownership_and_leavers.R
leavers_ev_numbers.tex	01_07_lct_ownership_and_leavers.R
survey_numbers.tex	01_11_cosy_survey_figures.R
survey_balance_numbers.tex	03_00_balance_tables_and_reweighting.R
calendar_last12m_summary.tex	01_06_DiD_analysis.R
did.tex	01_06_DiD_analysis.R
did_prevar.tex	01_08_heterogeneity_analysis.R
hp_did_overall_detailed.tex	02_03_DiD_analysis_outputs.R
hp_did_never_treated_detailed.tex	02_03_DiD_analysis_outputs.R
hp_did_overall_cs_gas_only.tex	02_03_DiD_analysis_outputs.R
hp_did_ev.tex	02_06_ev_ownership.R
hp_ev_charging.tex	02_06_ev_ownership.R
hp_did_solar.tex	02_10_solar_PV_analysis.R
variance_decomp.tex	02_09_engineer_variance_analysis.R
MVPF.tex	05_MVPF.R
summary_prepost.tex	06_sample_descriptive_statistics.R

Figures

Exhibit	Producing script
combined_weekly_installations_deals.png	02_04_summary_graphs.R
monthly_installation.png	02_04_summary_graphs.R
dynamic_hp_plot_combined.png	02_03_DiD_analysis_outputs.R
dynamic_hp_plot_combined_with_trends.png	02_03_DiD_analysis_outputs.R
hp_calendarplot_combined_with_annual_labels.png	02_03_DiD_analysis_outputs.R (see note below)
calendar_att_12m_with_quarter_points_and_cop.png	02_03_DiD_analysis_outputs.R (see note below)
HP_anticipation.png	02_03_DiD_analysis_outputs.R
hp_temperature_gas_elec.png	02_08_cop_analysis.R
quasi_cop.png	02_08_cop_analysis.R (see note below)
hp_event_study_overall.png	02_12_event_study.R
hp_data_availability.png	02_11_data_availability.R
hp_gas_data_availability.png	02_11_data_availability.R
engineer_vs_household_fe.png	02_09_engineer_variance_analysis.R
hp_region_combined.png	02_07_heterogeneity_analysis.R
hp_temperature_morning_off-peak.png, hp_temperature_afternoon_off-peak.png, hp_temperature_peak_rate.png, hp_temperature_other.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_epc_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_floor_area_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_heatloss_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_hs_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_income_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
hp_property_value_overall.png	02_07_heterogeneity_analysis.R (looped over rate_period)
Cosy Tariff.png, Rates_by_Rate_Period_and_GSP_Group.png, Rate_Changes_by_Period.png	01_02_rate_graphs.R
weekly_adoptions.png	01_03_summary_graphs.R
data_availability.png	01_04_data_availability.R
dynamic_att_combined.png	01_06_DiD_analysis.R
calendarplot_Morning Off-peak.png, calendarplot_Afternoon Off-peak.png, calendarplot_Peak Rate.png, calendarplot_Other.png	01_06_DiD_analysis.R (looped over rate_period)
eac_combined.png, eac_share_combined.png, cosy_epc_combined.png, floor_area_combined.png, floor_area_share_combined.png, heatloss_combined.png, heatloss_share_combined.png, income_category_combined.png, cosy_temperature_all.png, cosy_temperature_binned.png	01_08_heterogeneity_analysis.R
property_value_average_bill_saving.png	01_10_structural_winner.R
combined_impact_hourly_consumption.png	04_00_half_hourly_analysis.R
MVPF_sensitivity_heatmap.png, waterfall_hp_preferred.png	05_MVPF.R

Notes:

• MVPF.tex — an earlier, superseded version of 05_MVPF.R also wrote this filename; 05_MVPF.R (as sourced by main.R) is the authoritative producer.
• hp_calendarplot_combined_with_annual_labels.png, calendar_att_12m_with_quarter_points_and_cop.png, and quasi_cop.png can also be conditionally overwritten by 02_13_gas_only_sample_robustness_check.R, gated behind Sys.getenv("WRITE_MAIN_FILENAMES") (default off) — under a default run, the scripts named above are the effective producers.

License

This package will be deposited in the AEA Data and Code Repository (openICPSR), using openICPSR's default license (per DCAS #15) — confirmed with Centre for Net Zero/Octopus Energy Group that their IP policy requires no deviation from the default.

---

This README follows the Social Science Data Editors' template README and the Data and Code Availability Standard (DCAS) v1.0. Items marked ⚠️ TODO must be resolved before this package is submitted to the AEA Data Editor.