Repository navigation
test(octopus): run the IOG planner tests on the C++ kernel - #5662
Conversation
iog_bird_in_hand ran 19 full optimise_all_windows passes on the pure-Python engine and had become the slowest test in the quick suite (32s). It and iog_early_slot_flat_battery (5.5s) now optimise on the kernel, taking the quick suite from 112.7s to 76.8s. - run_iog_skew_scenario takes kernel=False, so iog_charge_skew is unchanged - iog_bird_in_hand planner scenarios use the kernel; the c1/c2 credit checks run on both engines - iog_early_slot_flat_battery defaults to the kernel; the car case still runs on both engines The kernel picks different off-peak charge limits from the old Python run with an identical score. That is the levels pass's fast mode, not a parity gap: re-running every kernel job on the Python engine matched all 2149 five-minute jobs exactly, every mismatch was a step=30 job, and with fast mode forced off both engines choose the same plan. Recorded in the debug journal, with the dead end that enable_fast_mode_levels alone does not turn fast mode off in the region passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
springfall2008
left a comment
There was a problem hiding this comment.
Automated comment from the triage bot.
Reviewed at head e5a922f (test-only PR: three IOG test modules moved onto the C++ kernel, plus a debug-journal entry). One low-severity finding posted inline (docstring accuracy in test_iog_charge_skew.py); one body note below confirming a pre-existing issue the PR's investigation surfaced.
What I checked, and found sound:
- State leakage: _SNAPSHOT_ATTRS (test_iog_charge_skew.py:18-76) includes prediction_kernel_enable, io_confirmed_credit and every other attribute the harnesses flip, and run_iog_bird_in_hand_tests and run_iog_early_slot_flat_battery_tests restore them in finally - so the new engine flips cannot leak into later tests.
- Kernel routing: every site that flips my_predbat.prediction_kernel_enable rebuilds my_predbat.prediction afterwards, and Prediction.init (prediction.py:221-224) bakes the flag into kernel_handle at construction; kernel_supported (prediction_kernel.py:856) then gates on it - the tests exercise the engine they claim to.
- The c1/c2 credit checks now genuinely run on both engines, and the early-slot slot-vs-feed comparison compares two same-engine (kernel) plans, so it stays consistent.
- The new journal entry's claims I could verify statically are accurate: plan.py:4940 overrides fast_mode inside the region passes without consulting enable_fast_mode_levels (read at plan.py:4889, honoured only by the first pass at plan.py:4927), and the kernel always simulates at 5-minute steps (prediction_kernel.py:849-856).
apps/predbat/plan.py:4940 - enable_fast_mode_levels: False does not turn fast mode off everywhere (pre-existing, verified)
The PR flags this as a side finding not changed here; confirmed it is real. optimise_levels_pass reads enable_fast_mode_levels at plan.py:4889 and honours it for the first levels pass (fast=fast_mode at plan.py:4927), but the region passes then set fast_mode = not (region_size == min_region_size) at plan.py:4940 unconditionally, so those passes always re-simulate candidates at the coarse plan_interval_minutes step (plan.py:1103-1104) regardless of the user's setting. Only Python-engine runs are affected - with the kernel available it ignores the requested step entirely (prediction_kernel.py:851-856) - i.e. a user debugging with a stale or absent kernel binary gets coarse region scans even with the setting False. Suggest a follow-up issue so the region passes respect the setting.
| PV90 scenario; by default every scenario sees load_amount and PV90 is the nominal one. | ||
| losses: (inverter_loss, battery_loss, battery_loss_discharge), lossless by default. | ||
| prepare: optional callable run once io_adjusted is set and before the Prediction is built. | ||
| kernel: optimise with the C++ prediction kernel instead of the Python engine (much faster; kernel_parity |
There was a problem hiding this comment.
Automated comment from the triage bot.
Low severity, docstring accuracy: kernel_parity pins run_prediction-level parity only - dual_run (test_kernel_parity.py:394) compares prediction.run_prediction calls and run_kernel_parity_tests (test_kernel_parity.py:1537) never runs optimise_all_windows. Planner-level picks can still differ between the engines under the levels pass's fast mode - exactly what this PR found when moving iog_bird_in_hand onto the kernel (Python chose 10.0 off-peak limits, kernel 4.0-6.25, both plans scoring identically; the 2026-10-20 entry in tools/debug-journal.md). A future author flipping a planner scenario onto the kernel would trust this line and skip the did-the-kernel-pick-a-different-plan verification this PR had to do.
Suggest: '(much faster; kernel_parity pins run_prediction parity between the engines, but planner picks can still differ under the levels pass's fast mode - see the kernel-flip trap in tools/debug-journal.md)'.
Summary
The quick suite had slowed down again. Per-test timings showed no sleeps behind it:
iog_bird_in_handwas running 19 fulloptimise_all_windowspasses on the pure-Python engine (49krun_predictioncalls in a profile). This PR moves the IOG planner tests onto the C++ kernel.iog_bird_in_handiog_early_slot_flat_batteryChanges
run_iog_skew_scenario(test_iog_charge_skew.py) takes a newkernel=Falsekeyword, soiog_charge_skewis unchanged.iog_bird_in_hand: the planner scenarios optimise on the kernel. The c1/c2 credit checks (metric_keeplowered by stored kWh × credit, hold earns nothing) now run on both engines.iog_early_slot_flat_battery: the kernel is now the default. The car-holds-the-battery case still runs explicitly on both engines.Parity check
On the kernel, the planner picks different off-peak (23:30+) charge limits than the old Python run did. Slot 0, the slot the tests assert on, is the same in every scenario, and both plans score identically on both engines. I checked this was not a kernel parity gap:
run_prediction_kernelandrun_prediction_kernel_batchand re-ran every kernel job on the Python engine, uncached. All 2149 five-minute jobs matched exactly. All 90 mismatches werestep=30jobs from the levels pass's fast mode, which Python simulates in coarse steps and the kernel always runs at 5 minutes.fast=Falseforced intooptimise_charge_limit_price_threads, Python and the kernel chose identical plans in all six scenarios, the same plans the kernel gives now.Side finding, not changed here:
enable_fast_mode_levels: Falsedoes not turn fast mode off everywhere. The region passes inoptimise_levels_passsetfast_mode = not (region_size == min_region_size)themselves.Test plan
./run_all --quickpasses (76.8s)iog_bird_in_hand,iog_early_slot_flat_battery,iog_charge_skew,kernel_paritypass on the rebased branchFollow-ups not in this PR
annual_job~2.8s includingtime.sleep(1.1),chat~2.7s,component_base~2.4s with severalasyncio.sleep(1.0)) could save roughly 7s more.random(21.6s) anddebug_cases(11.2s) are full golden plan replays.🤖 Generated with Claude Code