refactor: remove demand-side mechanisms superseded by market response - #49
Open
koen-vg wants to merge 4 commits into
Open
refactor: remove demand-side mechanisms superseded by market response#49koen-vg wants to merge 4 commits into
koen-vg wants to merge 4 commits into
Conversation
The consumer-values workflow, food_utility_piecewise, food_incentives and optimal_taxes all predate the market_response demand component, which carries the same economic content (a fitted marginal utility per (food, country) consumption) with exact calibration and literature-grounded elasticities instead of extraction-plus-tuning. The GSA and doc-figure configs move onto the demand component, which also removes their locally-solved baseline scenario (it existed only to feed consumer-value extraction). The cost calibration's tight-band solve keeps the diet pinned to baseline instead of valuing it with utility blocks, and the legacy stability calibration runs under the base config's own demand regime. Tutorial 2 is rewritten around demand response with its own tutorial-02 artefact set.
…e it Config validation demanded deviation_penalty.yaml exist whenever the calibration block was enabled, even for configs that never enable the L1 penalty. Artefact sets fit with the four-step default chain (like the tutorial-02 set) ship without the legacy stability artefact, so gate the existence check on the same condition under which the solve actually opens the file (penalty enabled, l1 mode, calibrated sentinel in use). This replaces the seeding workaround that copied the default set's yaml into fresh sets.
The legacy cost and stability calibrations of the default and gbd-anchored sets are refit under the pinned-diet cost regime and the utility-free stability main solve. The L1 costs rise substantially (default: cropland 1.38 -> 6.54, grassland 0.23 -> 0.34, feed 0.063 -> 0.50 bn USD per unit deviation): with no demand-side valuation in the legacy regime, the main solve reorganizes production much more aggressively, and holding the 5% deviation target is genuinely more expensive. Consumers of these artefacts run the same regime, so the calibration and consumption regimes match. The new tutorial-02 set backs the demand-response tutorial at its 200-region resolution, fit with the four-step default chain only (no legacy cost/stability artefacts). Demand censoring stays minimal (10 of 8403 groups, total pin slack 0.14 Mt).
koen-vg
force-pushed
the
remove-legacy-demand-mechanisms
branch
from
July 31, 2026 23:08
324b7bb to
6644153
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #48 (rebased onto its final state, including the capacity-lift refit and the legacy-chain restructure). Removes the legacy demand-side mechanisms that the
market_responsedemand component supersedes, migrates the remaining consumers, and rewrites Tutorial 2 around demand response.Removed
extract_consumer_values, theconsumer_values.smkrules, validation, comparison plot, docs page): hand-rolled PMP phase 1 for the demand side, with the level-degeneracy problem the sequential calibration was built to fix.food_utility_piecewise(+calibrate_food_utility_blocks): piecewise utility from consumer values with ad-hoc decline/width knobs -- the demand curves carry the same content with exact calibration and literature elasticities.food_incentivesand theoptimal_taxesworkflow: disabled everywhere; the equivalent quantities are directly readable off the demand curves.~1.9k lines of workflow code plus the config blocks, schema entries, solve-time prefixes, and manifest wiring. The
deviation_penaltyL1 machinery (and the legacycost+stabilitysteps) stay, as the anchor for coarser curve granularities and structurally divergent configs.Migrations
gsa.yamlanddoc_figures.yamlcomplete their move onto the demand component (started in Replace L1 stability anchoring with exactly calibrated PMP market-response curves (supply + demand) #48): the doc-figurebaselinescenario and the consumer-values figure are gone, as is the GSA's locally-pinned-solve special case for cluster runs.gsa_fixed_diet.yamlkeepscomponents.demand: false+enforce_baseline_diet: true(the schema rule from Replace L1 stability anchoring with exactly calibrated PMP market-response curves (supply + demand) #48 requires exactly one of the two).Artefact refits
defaultandgbd-anchored: the legacycostandstabilityartefacts are refit under the new regime (the four-step chain from Replace L1 stability anchoring with exactly calibrated PMP market-response curves (supply + demand) #48, including the capacity-lift market-response fit, is untouched). The calibrated L1 costs rise substantially (default: cropland 1.38 -> 6.54, grassland 0.23 -> 0.34, feed 0.06 -> 0.50 bn USD per unit deviation): without any demand-side valuation, the free-diet main solve reorganizes production much harder, and holding the 5% deviation target is genuinely more expensive. Legacy consumers run the same regime, so calibration and consumption match; expect stiffer L1 behaviour in legacy configs.tutorial-02(new): backs the demand-response tutorial at 200 regions, fit with the four-step chain only -- no legacy cost/stability artefacts ship with it. Config validation now requires the legacy stability yaml only when a solve would actually resolve the"calibrated"sentinel, which replaces the earlier seeding workaround.Tutorial 2 rewritten
Now "Demand response: letting the diet adjust to policy": the reader calibrates a dedicated 200-region artefact set (
tools/calibrate --base config/tutorial/02_demand_response.yaml, committed astutorial-02), sees the exact-calibration property live (an unpriced solve reproducing the observed food system with nothing pinning it), then runs a carbon-price ladder plus two demand-side variants: the diet frozen at baseline (what flexibility is worth) and half elasticities (how much hinges on that assumption, changeable with no refit). The companion notebook compares consumption shifts, thesupply_response/demand_responseobjective columns, and emissions across the three demand regimes.Verification
gsa.yaml,gsa_fixed_diet.yaml, anddoc_figures.yaml.tools/calibrate --checkis fully green for all three artefact sets; both legacy stability calibrations converged (default: 4 warm-started iterations, residual 0.016; gbd-anchored: residual 0.013).