Skip to content

fix(timesfm3/torch): mask covariate placeholders in mixed batches - #546

Open
rudycelekli wants to merge 1 commit into
google-research:masterfrom
rudycelekli:fix/timesfm3-mask-covariate-placeholders
Open

rudycelekli wants to merge 1 commit into
google-research:masterfrom
rudycelekli:fix/timesfm3-mask-covariate-placeholders

Conversation

@rudycelekli

Copy link
Copy Markdown

What's wrong

When only some queries in a predict_batch batch have past_only_covariates or past_future_covariates, the others get an all-zero placeholder (np.zeros_like) so the arrays can be stacked. model.decode is then called without past_only_mask / past_future_mask, so it reads the placeholder as an observed covariate that is constantly zero. As a result, a series with no covariates gets a different forecast depending on which series share its batch.

Repro (small random-weight TimesFM3Torch, per_core_batch_size=2, both contexts 64 long so padding is identical):

alone = list(f.predict_batch([a], horizon=32))[0].forecast
mixed = list(f.predict_batch([a, b], horizon=32,
                             past_future_covariates=[None, pf_b]))[0].forecast
np.abs(alone - mixed).max()

Before: 6.39. After: ~1e-5 (float noise). past_only_covariates=[None, po_b] behaves the same way.

Fix

For each covariate type present in the batch, build a bool mask with the same shape as the covariate arrays: False for real rows, True for placeholder rows. Pass it to decode as past_only_mask / past_future_mask. decode already pads these masks and ORs them into the context mask, and for past-future covariates into the horizon mask too. When a covariate type is absent, its mask stays None, as before. When every row is real, the mask is all False, which is the same as decode's default.

The two fake models in the test file now accept, and ignore, the new keyword arguments.

Testing

  • New TimesFM3ForecasterCovariatePlaceholderTest.test_no_covariate_query_unaffected_by_batch_mate checks that a series' forecast alone matches its forecast when batched with a covariate-bearing series of the same length. It covers both past-future and past-only covariates and uses a tiny random-weight TimesFM3Torch loaded through from_pretrained. Without the fix, both subtests fail (max abs diff about 0.095 against a 1e-4 tolerance). With the fix, both pass.
  • pytest src/timesfm3/torch tests: 109 passed. tests/test_model_loading.py was excluded because my local environment is missing einshape.

When only some queries in a predict_batch batch carry past-only or
past-future covariates, the others get an all-zero placeholder so the
arrays can be stacked. decode was called without past_only_mask /
past_future_mask, so the placeholder was read as an observed zero-valued
covariate and a no-covariate series' forecast changed depending on which
series it was batched with.

Build a bool mask per covariate type (True on placeholder rows) and pass
it to decode, which already ORs it into the context and horizon masks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant