Repository navigation
fix(utils): keep the newest of same-minute changes in minute_data - #5473
Merged
springfall2008 merged 3 commits intoOct 18, 2026
Merged
springfall2008 merged 3 commits into
springfall2008 merged 3 commits into
Conversation
minute_data carries the newest history value forward to "now" with `minutes < newest_age`. Timestamps are truncated to the minute, so when the last two changes of a sensor fall in the same minute both have the same age and the strict comparison keeps the older one. It is then held until the next change: a room sensor flickering 22/23 that settled on 23 was read as 22 for 10 hours (Predheat internal_temperature), while the transition minute itself already held 23. History is oldest first (HA API, Predbat's DB, GE Cloud), so `<=` makes the last of the tied items win, matching minute_data_state, which got the same fix in springfall2008#2896. Incrementing sensors are unaffected because clean_incrementing_reverse already ignores the small backwards step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
piomar123
marked this pull request as ready for review
October 10, 2026 17:30
With smoothing, forward data starts each item at the previous item's time, so the first two items always tie on age. A plain <= moved the value that fills the tail past the last forecast point from item 1's to item 2's (on a cumulative load forecast, 0.0 became 0.25 for every minute after the data). Apply the tie rule only to backwards=True, where history is oldest first, and add a forward smoothing test that fails with a plain <=. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
piomar123
added a commit
to piomar123/batpred
that referenced
this pull request
Oct 18, 2026
…ingfall2008#5473 (063549d) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
minute_datacarries the newest history value forward to "now" (the gap fill from the last sample) usingnewest_state, picked withif minutes < newest_age. Since timestamps are truncated to the minute, the last two changes of a sensor that fall in the same minute have the same age, and the strict<keeps the older one. It is then used for every minute up to now, until the sensor changes again.Seen live: a heat pump room sensor (1C resolution) flickers 22/23 every few seconds overnight and settled on 23 (
22at 02:15:46,23at 02:15:48). Predheat'sinternal_temperatureread 22 for the next 10 hours, so it planned heating all day. The transition minute itself held 23, so the series even disagreed with itself.Fix
On a tie in backwards history, the later item wins (
minutes < newest_age or (backwards and minutes == newest_age)). Everybackwards=Truesource passes history oldest first (HA API client sorts it, the DB query isORDER BY datetime, GE Cloud data is time ordered), andminute_data_statealready got the same change in #2896.Impact review
newest_stateis only used by the "fill from last sample until now" step, i.e. calls withoutto_key. Rate, carbon and tariff calls (to_key) are untouched.clean_incrementing_reversealready ignored the small backwards step the stale value caused (checked, including a same-minute reset).to_key(load forecasts, Solcast, Predheat weather): unchanged. A plain<=would have changed it: withsmoothing=Trueeach item starts at the previous item's time, so the first two items always tie, and<=moved the value that fills the tail past the last forecast point from item 1's to item 2's (on a cumulative load forecast, 0.0 became 0.25 for every minute after the data). The tie rule is therefore limited tobackwards=True.Tests
Tests 27-30 in
test_minute_data: same-minute changes with and without smoothing (fail before, pass after), a same-minute counter update that must stay unchanged, and a forward smoothed load forecast whose tail fill must stay unchanged (fails with a plain<=)../run_all --quickpasses, and pre-commit is clean.🤖 Generated with Claude Code