Skip to content

chore(ci): add dbt 1.11 to the test matrix - #5999

Open
ARAVIND281 wants to merge 1 commit into
SQLMesh:mainfrom
ARAVIND281:chore/ci-dbt-1.11
Open

chore(ci): add dbt 1.11 to the test matrix#5999
ARAVIND281 wants to merge 1 commit into
SQLMesh:mainfrom
ARAVIND281:chore/ci-dbt-1.11

Conversation

@ARAVIND281

Copy link
Copy Markdown

Closes #5992

Adds dbt 1.11 to the test-dbt-versions CI matrix.

The matrix line is the easy half. The issue flags that 1.11 "may need" the same special Makefile handling 1.10 got — it does, and without it the new matrix entry fails at install time.

Why the Makefile change is required

install-dev-dbt-% pins every dbt-* package to ~=$version on the default path. Two adapters have no 1.11 release at all:

Adapter Latest on PyPI
dbt-trino 1.10.3
dbt-clickhouse 1.10.2

So the default path asks for dbt-trino~=1.11.0 and the resolve is unsatisfiable:

$ uv pip compile  # dbt-core~=1.11.0, dbt-trino~=1.11.0, dbt-clickhouse~=1.11.0
  ... because dbt-trino>=1.11.0,<1.12.dev0 has no matching version, we can
  conclude that your requirements are unsatisfiable.

The fix is the branch that already exists for 1.10: pin dbt-core (and dbt-databricks), leave the adapters unpinned, and let the resolver find a consistent set. Extending that condition to 1.11 rather than adding a parallel branch keeps it to a two-line change, and the $$version interpolation was already there — only the hardcoded 1.10.0 in the echo needed generalising.

Resolved set with the fix:

dbt-core==1.11.8        dbt-bigquery==1.11.3     dbt-trino==1.10.3
dbt-databricks==1.11.8  dbt-snowflake==1.11.6    dbt-clickhouse==1.10.2
dbt-duckdb==1.11.0      dbt-redshift==1.10.2     dbt-athena-community==1.10.2

Gotchas from the issue, checked

  • numpy — 1.11 pulls numpy 2.2.6. The numpy<2 awk condition was not extended, because the suite passes on numpy 2; adding the constraint would have been a change with nothing behind it.

  • pydantic — installs 2.12.5 on the full dev path, no 1.6-style special casing needed.

  • dbt-duckdb / dbt-snowflake!=1.10.1 pins — both resolve cleanly under the unpinned branch.

  • No CI script changes. The version-gated shell in the job holds for 1.11: the semantic_models/metrics strip is an explicit 1.3|1.4|1.5 list, the pytest-path fork is == "1.6", and the version-params check uses sort -V, which orders 1.11 above 1.5.0 correctly (a naive string compare would not):

    dbt 1.10   sort -V first=1.5.0   >= 1.5.0 -> keeps version params
    dbt 1.11   sort -V first=1.5.0   >= 1.5.0 -> keeps version params
    
  • Version-gated tests — the only upper-bound gates in tests/dbt/ are DBT_VERSION < (1, 9, 5) in test_manifest.py, which 1.11 takes the else side of. Already correct, no change.

Verification

Locally, on Python 3.10 to match the job:

UV=1 make install-dev-dbt-1.11     # succeeds; DBT_VERSION == (1, 11, 8)
make dbt-fast-test                 # 88 passed, 84.87s
cd examples/sushi_dbt && sqlmesh info --skip-connection   # Models: 11, Macros: 0 (exit 0)

Acceptance criteria

  • '1.11' in the test-dbt-versions matrix
  • UV=1 make install-dev-dbt-1.11 succeeds locally
  • make dbt-fast-test passes
  • sqlmesh info --skip-connection works in examples/sushi_dbt
  • CI green on this PR

Adds '1.11' to the test-dbt-versions matrix, and extends the existing
1.10 branch in install-dev-dbt-% to cover it.

The Makefile change is required, not incidental. The default path pins
every dbt-* package to ~=$version, and two adapters have no 1.11 release:
dbt-trino (latest 1.10.3) and dbt-clickhouse (latest 1.10.2). Asking for
dbt-trino~=1.11.0 makes the resolve unsatisfiable, so the new matrix entry
would fail at install time.

The 1.10 branch already handles exactly this shape: pin dbt-core and
dbt-databricks, leave the adapters unpinned, and let the resolver settle
on a consistent set. Extending its condition keeps this to two lines; the
echo was the only part that hardcoded 1.10.0.

Resolved: dbt-core 1.11.8, dbt-bigquery 1.11.3, dbt-snowflake 1.11.6,
dbt-databricks 1.11.8, dbt-duckdb 1.11.0, dbt-trino 1.10.3,
dbt-clickhouse 1.10.2, dbt-redshift 1.10.2, dbt-athena-community 1.10.2.

No numpy<2 constraint was added: 1.11 pulls numpy 2.2.6 and the suite
passes on it. No CI script changes are needed either — the version-gated
shell in the job already holds for 1.11, including the sort -V comparison,
which orders 1.11 above 1.5.0 correctly.

Verified locally on Python 3.10: UV=1 make install-dev-dbt-1.11 succeeds
(DBT_VERSION == (1, 11, 8)), make dbt-fast-test passes 88 tests, and
sqlmesh info --skip-connection reports 11 models in examples/sushi_dbt.

Closes SQLMesh#5992

Signed-off-by: Aravind S <63240570+ARAVIND281@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI Upgrade - Adding in CI stage to run tests for DBT v1.11

1 participant