Split private_pension_wealth out of corporate_wealth on the UK WAS stage (#750) - #804
Closed
juaristi22 wants to merge 5 commits into
Closed
Split private_pension_wealth out of corporate_wealth on the UK WAS stage (#750)#804juaristi22 wants to merge 5 commits into
juaristi22 wants to merge 5 commits into
Conversation
Banded measures took each band's upper edge from the lower edges of sibling specs in the live registry, so excluding a band silently widened its lower neighbour to the next surviving edge (or to infinity at the top) — the #792 movers: 12 of 361 shared measures shifted by up to +858% when the exclusion register grew 5->47, each by exactly the mass of the absorbed excluded siblings. materialize_target_bindings and resolve_target_measures now accept a band_edge_registry (default: the materialized registry, bit-identical for existing callers), and _band_bounds refuses a spec whose own lower edge is absent from the supplied edge set rather than guessing. Regression fence: survivors must be bit-identical under sibling exclusion, for numeric edges and published range labels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The seam prunes measure exclusions before materialization, so the stage only ever saw the pruned registry. run_uk_calibration now takes the compiled pre-exclusion register as band_edge_registry and refuses, before any artifact is written, a non-empty exclusion receipt without it — and a register whose extra names do not exactly reconstitute the receipt (the applier already fails exclusions that match zero specs, so pruned + receipt == compiled is exact). The registry content hash is recorded as run_config.band_edge_register_sha256, flowing into the build record and identity digest. The seam tool passes compilation.registry; census counts are unchanged — only survivors' measured values return to published band widths. The stale prose in test_uk_measure_simulation that read the widening artifact as the resolution mechanism is corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2.92.0 carries private_pension_wealth (policyengine-uk#1824): the explicit capital disregard, the total_wealth re-sum, and the corporate_sector_wealth allocation key. The #750 split emits that column from the WAS stage, and an older engine's loader would silently drop it, narrowing total_wealth and the exposure keys. Locked upgrade moves policyengine-uk alone (2.89.0 -> 2.92.0, no transitive churn). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The stage folded WAS private pension wealth less current-employment DB (totalpenr8_aggr - dvvaldbt_scaper8_aggr) into corporate_wealth, which policyengine-uk counts as capital in every means test - the largest mechanism behind the UC caseload shortfall (uk-data#452 M2). The stage now emits private_pension_wealth as its own household output, drawn first in the second chain segment with the share-like holdings conditional on it and the third segment conditional on both; corporate_wealth keeps employee shares/options, UK shares, unit and investment trusts and the stocks-and-shares-ISA fold. Row identity old corporate_wealth == corporate_wealth + private_pension_wealth holds on the donor by construction and is pinned in tests, alongside a segment-1 containment test documenting why the first segment's draws cannot move. Lockstep spec pair, support bounds (operator-measured on the pinned tab: min 0.00 over 15,128 rows, outward bound [0, 8000000], byte-verified via the tool's --check), export allow-list, stage-support gate columns and zero allowances, coverage manifest, and the moved digests (battery policy/manifest/fingerprint, spine and release_cut certification part mirrors, UK spec bundle sha) re-cut in the same change. The QRF household-tail note records the 13th unarmed household column; arming stays the microcosm#796 follow-up. Closes microcosm#750 machinery; the spine rebuild, parity re-mints and seam measurement ride the follow-up commits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2.92 engine newly recognizes three loader inputs the pinned artifacts already carry and populate - bus_fare_spending (household), employment_sector and sic_industry_division (person), previously in the reference's unknown-export-columns list - so the committed extraction evidence recorded under 2.89 no longer matches its own regeneration. The parity reference, the known-gaps candidate evidence and the coverage manifest are regenerated with their own tools from the same pinned artifacts: identities unchanged (filename/revision/sha256 identical), every pre-existing share value byte-identical, surface 145 -> 148 required, count pins and version strings move, nothing else. The June candidate and the spine both carry all three columns, so the widened contract is satisfiable on both sides. This is not the efrs-post-calibration input-mass reference re-pin - that stays deferred behind the uk-data mirror per the #750 ordering, and the input-mass totals evidence is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the microcosm side of #750 (uk-data#452 mechanism M2) — the measured lever for the 8 UC caseload/two-child cells blocking the first certified cut (#796). Stacked on #803 (the #792 band-edge fix; its two commits lead this branch and drop out of the diff when #803 merges). Model side already released: policyengine-uk 2.92.0 (policyengine-uk#1824).
The split
The WAS stage folded WAS private pension wealth less current-employment DB (
totalpenr8_aggr − dvvaldbt_scaper8_aggr) intocorporate_wealth, which policyengine-uk counts as capital in every means test — on the incumbent at 2025, 46% of UC-reporting benefit-unit records fail the £16,000 test, 96% of them oncorporate_wealthalone. The stage now emitsprivate_pension_wealthas its own household output: drawn first in chain segment 2, the share-like holdings conditional on it, segment 3 conditional on both.corporate_wealthkeeps employee shares/options, UK shares, unit/investment trusts, and the stocks-and-shares-ISA fold. Row identityold corporate_wealth == corporate_wealth + private_pension_wealthholds on the donor by construction and is pinned in tests, with a segment-1 containment test documenting why the first segment's draws cannot move.Commits:
policyengine-uk>=2.92in the threeukextras (locked upgrade, 2.89.0 → 2.92.0, zero transitive churn). Required first: an older engine's loader silently drops the new column, narrowingtotal_wealthand the exposure keys. 2.92'scorporate_sector_wealthkeeps the shareholding / corporate-land-value / employer-NI keys whole, so in classification terms only the capital tests move.18b3eb98…: donorpensions − db_pensionsmin 0.00 over 15,128 rows, zero negative rows →nonnegative_outputs; outward bound[0, 8000000], byte-verified via the tool's--check; the standing Port the WAS wealth imputation and regional property uprating as declarative source stages #714 SDC adjudication extends to the one new column) + export allow-list + stage-support gate columns/zero-allowances + coverage-manifest family outputs + the moved digests (battery policy/manifest/fingerprint, spine and release_cut certification part mirrors, UK spec bundle sha). The QRF household-tail note records the 13th unarmed household column; arming stays the WS-E spine deferrals: known bugs and signed inconsistencies awaiting their increment #796 follow-up.bus_fare_spending,employment_sector,sic_industry_division— previously in the reference's "unknown export columns" list). The parity reference, the known-gaps candidate evidence, and the coverage manifest are regenerated with their own tools from the same pinned artifacts (identities unchanged — filename/revision/sha256 all identical; every pre-existing share value byte-identical): surface 145 → 148 required, counts and version strings move, nothing else. The June candidate and the spine both carry all three columns, so the widened contract is satisfiable on both sides. This is not theefrs-post-calibrationinput-mass reference re-pin — that stays deferred behind the uk-data mirror per the UK WAS stage: split private pension wealth out of corporate_wealth (uk-data#452 M2) #750 ordering, and the input-mass totals evidence (fd41cb5f…) is untouched.What this deliberately does not do
corporate_wealth: the gate's reviewed tolerance is 4.5218 and the post-split drift is ~−0.85, comfortably inside — and the UK wrapper fails stale exclusions, so adding one would fail the gate. The drift gets recorded in the seam receipt when the split spine runs.was-wealth-qrf-incidencebounds and theprivate_pension_wealthnet-new-column entry are re-minted from the spine-j measurement in the licensed phase (register entries unused on a compared surface fail--strict, so they cannot precede the spine that carries the column).Licensed phase (rides this branch before review completes)
Spine-j + twin build under the armed battery, partial engine-drift control (segment-1 shares vs spine-i; a full commit-1-only control spine only if segment 1 moves), strict-parity re-mints, e4–e8 ladder, acceptance re-cut, then the seam run with the #792 fix in place — the 8 blocked UC cells' new relative errors are the adjudication input for the #736 queue — and the measurement receipt
experiments/452-uk-pension-wealth-split.md. The certification cut itself stays with #793/#736 adjudication per the 2026-08-27 ruling.Verification
Per-commit: ruff,
ci_test_groups.py --verify, targeted UK/spec suites; full build+data shard after the spec changes; licensed bounds--checkbyte-identity; the two licensed-cache regeneration tests (skipped in CI, run locally against the pinned HF-cache artifacts) green after commit 3. The branch is main-based, so PR CI runs the engine lanes and the wheels gate once, authoritatively; local heavy suites were not duplicated (per the 2026-08-27 compute ruling) — the full build+data shard ran locally at the pre-rebase tip, and the post-rebase delta (the #803 wheels-lane reorder) was covered by its three calibration test files plus a wheel metadata build of all five shards.🤖 Generated with Claude Code