[Sonica-B]

Loading TimesFM 3 from a local weights file (checkpoint_path="/path/model.safetensors" or a .pth) builds a model whose variate attention applies RoPE. The published checkpoint was trained without it. The weights load without any error, univariate forecasts are unchanged, and multivariate and covariate forecasts silently drift from the Hub-loaded model.

Problem

_init_model sends Hub ids and local directories through TimesFM3Torch.from_pretrained, which reads config.json. A local .safetensors/.pth file has no config, so _make_torch_model supplies the architecture, and its default TransformerConfig has use_rope_var=True. Everything else uses False: google/timesfm-3.0-pytorch's config.json ("use_rope_var": false), the TimesFM3Torch.__init__ defaults and the MLX backend (mlx/configs.py). RoPE has no weights, so load_state_dict succeeds. With one variate, RoPE sits at position 0, where it is the identity, which is why univariate checks did not catch this.

flowchart LR
  A["checkpoint_path"] --> B{"local file?"}
  B -- "Hub id / directory" --> C["TimesFM3Torch.from_pretrained<br/>reads config.json<br/>use_rope_var=False"]
  B -- ".safetensors / .pth" --> D["_make_torch_model defaults"]
  D -- "before" --> E["use_rope_var=True<br/>variate attention rotated by variate index"]
  D -- "after" --> F["use_rope_var=False<br/>same model as the Hub path"]

Loading

Reproduction (CPU, real weights)

The same model.safetensors is loaded twice: once from the Hub id and once from the local file path, and the two forecasts are compared. The 4/8/16-covariate cases are 256-step contexts at base level 20 plus K step covariates, horizon 48.

Before (master e51928e):

transformer config fields that differ (hub, local): {'use_rope_var': (False, True)}
identical weights: True
univariate     max|hub-local| = 0.000e+00
2-variate      max|hub-local| = 1.351e-02  (rel 0.11%)
K= 4 covariates: max|hub-local|=5.289  MAE hub=0.679  MAE local=1.825
K= 8 covariates: max|hub-local|=9.071  MAE hub=0.812  MAE local=2.254
K=16 covariates: max|hub-local|=15.841  MAE hub=2.743  MAE local=6.006

After (this PR):

transformer config fields that differ (hub, local): {}
identical weights: True
univariate     max|hub-local| = 0.000e+00
2-variate      max|hub-local| = 0.000e+00  (rel 0.00%)
K= 4 covariates: max|hub-local|=0.000  MAE hub=0.679  MAE local=0.679
K= 8 covariates: max|hub-local|=0.000  MAE hub=0.812  MAE local=0.812
K=16 covariates: max|hub-local|=0.000  MAE hub=2.743  MAE local=2.743

Using the Hub model as the reference, the local-file model was 2.2x to 2.8x worse in MAE once a few covariates were present. That is consistent with the weights having been trained without variate RoPE. The benchmark scripts take checkpoint_path from TIMESFM3_CHECKPOINT (for example timesfm3-usage/benchmarks/gift_eval/evaluate_all_gift.py), so pointing that variable at a downloaded weights file hits this path.

The issue has been there since the initial torch code (0548f53).

Change

• src/timesfm3/torch/timesfm3_forecaster.py: use_rope_var=False in _make_torch_model's default TransformerConfig, with a comment tying it to the published config.
• src/timesfm3/torch/timesfm3_forecaster_test.py: test_local_checkpoint_file_uses_published_architecture mocks TimesFM3Torch and asserts the default local-file architecture has use_rope_seq=True and use_rope_var=False. It needs no weights. It fails on master with AssertionError: True is not false and passes with the fix.

A checkpoint trained with variate RoPE can still be loaded by passing transformer_config explicitly, as any non-default architecture requires today.

Testing

Run on Windows 11, CPU only, with Python 3.11.9 and torch 2.15.0+cpu:

• python -m pytest -q src/timesfm3/torch: 50 passed
• ruff check on both files: clean. ruff format --diff reports no changes in the lines this PR touches.
• The Hub-vs-local comparison above was run with google/timesfm-3.0-pytorch on CPU.

Not run: the MLX tests (macOS only) and anything on GPU.