Repository navigation
feat(models): add school holiday feature transform - #1187
Merged
Merged
Conversation
Adds SchoolHolidayFeatureAdder, which sets is_school_holiday (or one column per holiday name) from OpenHolidays API school holidays. A transform configured with a country, subdivision, and years_ahead fetches holidays once during fit and bakes them into the checkpoint. transform makes no network calls. export_csv and from_csv (or a csv_path config field) let a fitted set of holidays be reused without contacting the API. Public client, pydantic models, and SchoolHolidayRow are exported. Data: OpenHolidays API (ODbL 1.0, commercial use allowed with attribution). Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Unit tests with a stubbed client cover fit coverage and years_ahead, transform output, one-hot-by-name columns, CSV export and load (including empty and header-only files), csv_path fit without client calls, country mismatch, and checkpoint pickling of fitted holidays. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
…ng tutorial Shows exporting school holidays to CSV once, loading them without an API call, and the one-hot-by-name option. Executed notebook outputs are included. Includes OpenHolidays API attribution. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
One-hot columns were derived from the holidays present in each fit or transform window, so the feature set could change between training and inference. Freeze the holiday names seen at fit, store them with the model, and emit them sorted so column order is stable. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Re-execute the feature engineering tutorial so the committed output shows the sorted, frozen one-hot school holiday columns. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Pass CountryAlpha2 instead of plain strings so the tests satisfy the strict type check on the from_csv signature. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Tutorial notebooks are stored without cell outputs; notebooks-check rejects committed outputs. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
The school holiday client uses requests with urllib3 Retry for backoff on transient failures. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
…ries Replace the hand-rolled urllib retry loop with a requests session mounted with urllib3 Retry. Retries apply to connection errors and 429/5xx responses only, so other 4xx responses now fail immediately instead of being retried. Backoff is exponential via retry_delay_seconds as a backoff factor; the first retry is immediate. Refs: #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Add an opt-in SchoolHolidayConfig to the models and ensemble (openstef-meta) forecasting workflow configs. Holidays are fetched once during fit, or read from csv_path, so no request is made per forecast. The config lives in the transform module so it stays self-contained. Refs #756 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Compare a GBLinear forecast with and without school holiday features over the Christmas break for a Liander feeder in Amsterdam. Also runs a placebo week and repeats each fit five times to estimate run-to-run variance. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Group holidays by their member subdivision set and emit one 0/1 column per group, numbered by index. Membership is kept in a private attribute and the CSV gains subdivisions and group_code columns. Old 7-column CSVs load in plain mode and are rejected in regional mode. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
random_state was configured on GBLinearHyperParams but never forwarded to XGBRegressor, so seeded runs were not reproducible with coord_descent. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Feature engineering tutorial plots a whole-year heatmap of regional holiday groups. The holiday forecasting tutorial uses coord_descent with fixed seeds and reports paired rMAE and rCRPS over five seeds for the Christmas week, regional groups, and a placebo week. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Tutorial notebooks are committed without outputs; notebooks-check rejects stored outputs. The .py files remain the source of truth. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
|
bartpleiter
approved these changes
Nov 13, 2026
bartpleiter
left a comment
Collaborator
There was a problem hiding this comment.
Looks good! A lot of code and tests (perhaps some tests are not really needed), but in general great addition
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What does this PR do?
Adds
SchoolHolidayFeatureAdder, a transform besideHolidayFeatureAdderthat adds school holidays as features. It fetches school holiday periods for one country (and optional subdivision), both set at configuration time, from the free OpenHolidays API.OpenHolidaysClientowns all HTTP fetching. Request and response models are pydantic, and the client can be replaced by passing your own. Fetching usesrequestswith urllib3Retry: connection errors and 429/5xx responses are retried with backoff, and other 4xx responses fail immediately.fitmakes one request for the training period plus two years ahead.transformnever touches the network.export_csvwrites the holidays for a date range to CSV in one request.from_csv(orcsv_pathin the config) loads that file, so a model can be trained without any API call.regional_groups=True): for countries whose school holidays differ by province (for example NL north vs south), holidays are grouped by their member subdivision set. Each group gets one 0/1 column, numbered by index (is_school_holiday_0,_1, ...). The model picks the region from the data, so no lat/lon-to-province lookup is needed. Group membership is kept in a private attribute, and the CSV gainssubdivisionsandgroup_codecolumns. CSVs from before this change load in plain mode and are rejected in regional mode.school_holidayson the forecasting models preset and the ensemble (openstef-meta) preset. Accepts aSchoolHolidayConfig(country, subdivision, regional groups, optionalcsv_path). With GBLinear,is_school_holiday*columns are excluded from the default feature set by regex.random_stateonGBLinearHyperParamsis now passed to XGBoost. Before, it was accepted but ignored. The defaultupdaterstaysshotgun(asynchronous, so not reproducible even when seeded).coord_descentwith a seed is deterministic. No default changed.examples/tutorials/feature_engineering.py(and.ipynb) showcases the transform, including a whole-year heatmap of regional groups for NL.examples/tutorials/school_holiday_forecasting.py(and.ipynb) forecasts a Christmas week for a Liander MV feeder with and without school holidays, usingcoord_descentand five seeds. It reports paired rMAE and rCRPS for the holiday week, the regional-group variant, and a placebo normal week, and it is linked fromdocs/source/examples.rst.Data source: OpenHolidays API, licensed under ODbL 1.0 (commercial use allowed, attribution required). The attribution is in the transform docstring and the tutorials.
Closes #756
Note: Issue #756 targets
release/v4.0.0. This PR targetsmain, as requested.Validation run locally:
pytest packages/openstef-models/tests/unit/transforms/time_domain/test_school_holiday_features_adder.py packages/openstef-models/tests/unit/models/forecasting/test_gblinear_forecaster.py packages/openstef-models/tests/unit/presets/test_forecasting_workflow_school_holidays.py: 78 passedpytest packages/openstef-models/tests packages/openstef-meta/tests: 594 passedruff check packages/openstef-models examples/tutorials: passedruff format --check packages/openstef-models examples/tutorials: 178 files already formattedjupytext --syncon both tutorial.pyfiles, thenjupyter nbconvert --execute --inplace: both notebooks execute with zero error outputsjupyter nbconvert --clear-output --inplaceon both tutorial notebooks, thentools.check_notebook_outputs: 22 notebooks checked, no outputs found (thenotebooks-checkCI step requires tutorial notebooks to be committed without outputs; the.pyfiles are the source of truth)ruff check,ruff format --check,pyproject-fmt --check,ty check tools examples packages/*/src,reuse lint: passed locallypoe tests,poe doctests,poe licensecheck,uv lock --check(no access to files.pythonhosted.org from the dev machine). CI runs these in the Quality Checks workflow.32a423c: Quality Checks passed (includes reuse, licensecheck, lint, format, typing, typing tests, tests, doctests, notebooks-check); Docs Build passed.Type of change
Breaking changes checklist
_migrate_statemigration patternMigration path for existing users (e.g. "old pickled
XScalerobjects auto-migrate on load", "users must now passXexplicitly"):N/A: this adds a new opt-in transform and opt-in preset config. No existing defaults, public API, or pickled objects change. GBLinear
random_stateis now forwarded to XGBoost, which only affects runs that set it.AI disclosure
Assisted-by: <tool name>)Checklist
poe all --checkpasses locallygit commit -s)feat: ...,fix: ...,feat!: ...for breaking changes)