feat(migration): add migrate subcommand and opt-in --migrate flag - #1152
Merged
Merged
Conversation
Add `nextgen migrate` so schema changes can run and exit without starting the HTTP server. Server start no longer migrates unless `--migrate` is set; zitadel start and the image CMD still pass that flag so local and container zero-config paths keep applying schema. Co-authored-by: Silvan <adlerhurst@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
🦋 Changeset detectedLatest commit: e1b3c24 The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
There are a few concrete UX/operational issues to address (unnecessary UI builds for completion, duplicated Cobra error printing, and a misleading migrate success log ordering).
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds an operator-friendly database migration entrypoint to the Go server binary and makes migrations opt-in when starting the HTTP server, while keeping existing local/docker flows migrating by default via wrappers/entrypoints.
Changes:
- Introduces a
nextgen migratesubcommand (apply pending migrations and exit) and restructures the Cobra command tree so the binary has real subcommands. - Makes
server(and default root invocation) skip migrations unless--migrateis provided; updates internal DB startup accordingly and adds coverage for idempotency/flag behavior. - Updates dev scripts, docs, Docker image defaults, and CLI local-runtime helpers so common start paths still apply migrations.
File summaries
| File | Description |
|---|---|
| scripts/run-server.mjs | Auto-adds --migrate for dev runs and conditionally builds UIs before go run. |
| scripts/run-server-debug.mjs | Same as above for debug build/run flow. |
| examples/bootstrap-users/README.md | Updates server run example to include --migrate. |
| docs/quick-start/docker-compose.md | Updates compose example to include --migrate when overriding the command. |
| docs/operations/docker-compose.yaml | Documents --migrate in the optional bootstrap-users command override. |
| Dockerfile | Sets default CMD ["--migrate"] so the published image still migrates by default. |
| CONTRIBUTING.md | Updates contributor run/debug instructions and documents the new migrate behavior/command. |
| cmd/server/server.go | Adds Cobra root + server/migrate subcommands; makes migrations opt-in via --migrate; closes pool on migrate failure. |
| cmd/server/migrate.go | Implements migrate subcommand and migration execution. |
| cmd/server/migrate_test.go | Adds tests for help output, migrate idempotency, and --migrate gating. |
| apps/cli/tests/unit/lib/local-server/docker.test.ts | Asserts Dockerfile retains CMD ["--migrate"] for migration-on-start behavior. |
| apps/cli/tests/unit/lib/local-server/binary.test.ts | Updates expectations to ensure binary runtime includes --migrate. |
| apps/cli/src/lib/local-server/binary.ts | Ensures local binary runtime always adds --migrate when spawning the server. |
| .changeset/nextgen-migrate-command.md | Records the shipped behavior change for @zitadel/server. |
Review details
- Files reviewed: 14/14 changed files
- Comments generated: 7
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
muhlemmer
self-requested a review
September 4, 2026 13:47
adlerhurst
commented
Sep 4, 2026
Wire migrate through setUpLogging and StreamRuntime so Cloud Run jobs respect instrumentation.log. Debug-log pool.Close failures without failing the process. Silence Cobra errors on root, server, and migrate. Skip UI builds for completion in the run-server wrappers. Tighten migrate tests per review. Co-authored-by: Silvan <adlerhurst@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Contributor wrappers mishandle valid global-flag ordering, and binary runtime metadata omits the injected migration flag.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
scripts/run-server-debug.mjs:96
- This only recognizes
migratewhen it is the first argument. Because--configis inherited from the root,moon run workspace:server-debug -- --config cfg.yaml migrateis valid at the binary level, but the wrapper turns it intomigrate --migrate, which fails with an unknown flag. Resolve the first positional command after consuming root flag values, and reuse it for both migration injection and the UI-build decision.
if (args[0] === "migrate" || args[0] === "completion") {
return args;
scripts/run-server.mjs:28
- This only recognizes
migratewhen it is the first argument. Since--configis a persistent root flag,moon run workspace:server -- --config cfg.yaml migrateis a valid Cobra invocation, but this wrapper appends--migrate, producingmigrate --migrate; the migrate command then fails because it has no such flag. Resolve the first positional command after consuming root flag values, and use that result both here and in the UI-build guard.
if (args[0] === "migrate" || args[0] === "completion") {
return args;
- Files reviewed: 14/14 changed files
- Comments generated: 1
- Review effort level: Balanced
Compute spawn args once so BinaryRuntimeMetadata.command matches the process that was started, not the pre-injection argv. Co-authored-by: Silvan <adlerhurst@users.noreply.github.com>
muhlemmer
approved these changes
Sep 8, 2026
muhlemmer
enabled auto-merge (squash)
September 8, 2026 08:24
Contributor
|
Thanks! |
bastionstack
added a commit
that referenced
this pull request
Sep 8, 2026
## Summary `moon run console:dev-claim` — a dev loop for the claim page, which `dev-real` structurally cannot serve. `dev-real` boots one ordinary project and signs the console into it. The claim page is the console acting as the *platform's* claim surface, and `claim/complete` only accepts a session belonging to the platform project, so on `dev-real` the page renders and the claim always fails with "This account can't claim the project" — a harness limit that reads like a product defect. Working on that page today means booting the server by hand. `dev-claim` boots the platform project, pins the console to it, and prints a ready claim link. Opt-in rather than the default, for the reason `cli-journey-e2e/scripts/run-local.mjs` already records: pinning the console to `proj_platform` changes the standalone semantics the demo and embedded suites prove. The seeded users also stay in the project being claimed, so list screens read empty in this mode and you register through the claim page — which is the real journey anyway. Every other screen is still better served by `dev-real`. Two things came along with it: - **Vite takes its port from `CONSOLE_DEV_ORIGIN`.** The rest of the script already honours that variable; vite alone stayed pinned to its config's 5174, so a second worktree could not run beside the first without editing a checked-in file. - **`apps/console/AGENTS.md`** gains the claim-loop section and three traps that break either loop before it starts, none of which say so clearly: the CLI must be built first (an unbuilt one starts the server unmigrated and dies with `no such table: projects`), a stale `node_modules` after a rebase fails the built CLI on a missing transitive dependency, and a raw `dist/server/nextgen` needs `--migrate` since #1152. ## Validation - `moon run console:dev-claim` — booted, platform project pinned, link minted and resolvable through `claim/window` - `moon run console:typecheck` - `pnpm --filter console test` - `pnpm exec oxlint apps/console` (2 warnings, both pre-existing) ## Release notes / changeset No changeset required — no shipped behavior changed. Developer tooling and docs only. ## Notes Prompted by round-2 QA on the claim flow: verifying those fixes by hand needed a server booted by hand, and the first attempt failed on each of the three traps above in turn.
muhlemmer
added a commit
that referenced
this pull request
Sep 9, 2026
#1152 landed a `migrate` subcommand and, in the same change, made `server` stop migrating unless `--migrate` is passed. That second half breaks this configuration as written: the live service carries args ["server"] from an old console deploy, which overrides the image's CMD of ["--migrate"], so a current release would have served against an unmigrated schema. The fix is the design the job was always for, and #1152 says as much — the subcommand exists because of #734/#1138. - The job runs `args = ["migrate"]`, and carries the DSN and the same VPC access the service has. `gcloud run jobs execute --wait` in the deploy workflow makes a failed migration a failed deploy rather than a broken revision. - The service declares `args = ["server"]` explicitly and `args` comes out of ignore_changes. The contract — the job migrates, the service serves — is now stated in code rather than inherited from whatever argv a container was last left with. - Both carry NEXTGEN_SERVER_GENERATE_MASTER_KEY=false, gated with the mount. Refusing to generate is only safe once a key is actually mounted; setting it outside that gate would leave the service unable to start at all while runtime_secrets_ready is still false. The job also mounts the master key, which it does not use — `migrate` runs goose and exits without building a crypter. It is there because `migrate` loads the same configuration as `server`, and with generation disabled (#1151, landing shortly) a start that finds no key fails inside loadConfig, which `migrate` calls too. So the mount keeps the job startable rather than serving the migration, and should come off if that stops being true; the grant and the volume are commented as a pair and a follow-up issue tracks it. Validated with tofu validate and tofu fmt -check -recursive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
muhlemmer
added a commit
that referenced
this pull request
Sep 9, 2026
#1152 made `server` stop migrating unless --migrate is passed, and added the `migrate` subcommand the Cloud Run job was always meant to run. Without this step a deploy would roll a revision onto a schema that was never applied — the service in #1132 deliberately runs `args = ["server"]`, which overrides the image's CMD of ["--migrate"]. The job is pointed at the same digest as the service and executed with --wait, so a failed migration fails the deploy. Without --wait the service would roll anyway and the first broken query would be the notification. Also drops the stale comment claiming migrations run at server startup, and reports the job in the run summary so a deploy says what applied the schema. Validated by parsing the workflow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mridang
added a commit
that referenced
this pull request
Sep 21, 2026
## Summary The developer's env files now configure the local server. `zitadel start` reads `.env.local`, then `.env`, and hands every `NEXTGEN_*` variable to the server it launches. That is the server's own configuration namespace, so anything the server accepts from the environment can be set per project without touching the CLI. - `.env.local` wins over `.env`; empty values are skipped; nothing outside the prefix is passed. The address, data directory and public base the CLI sets itself always win. - Binary backend: child environment. Docker: bare `--env NAME`, value via the docker client's environment. Nothing reaches argv, `runtime.json`, logs or `--json`. - `start` and `status` report the injected names under `runtime.env.injected`. - `setup` writes a comment at the top of the scaffolded `.env.example` and `.env.local` explaining this. This does not inspect `.zitadel/` to decide what the runtime needs. Once resources declare the variables they reference (the IdP work), that introspection lands here and narrows what is injected. Part of #1049. ## Validation - `moon run cli:test` - `zitadel start` against a server built from this commit, and against `ghcr.io/zitadel/nextgen:1.0.0-alpha.22` on Docker, with a `NEXTGEN_*` variable in `.env.local`: value present in the process, absent from argv and every local file. ## Release notes / changeset `@zitadel/cli` minor: `.changeset/cli-dev-runtime-env-join.md`. ## Notes A committed `.env` can now set local server config for everyone who clones; for a dev server that is the intent. A local server binary built before #1152 rejects `--migrate`; rebuild with `moon run server:build`.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Go binary now has a
migratesubcommand so schema changes can run as their own step and exit, which is what the Cloud Run migrate job in #734/#1138 needs.nextgen migrateloads the same config asserver, connects, applies pending goose migrations, and exits 0 (including no-op reruns).nextgenandnextgen serverstill start the HTTP server. They no longer migrate unless--migrateis passed.zitadel startspawns--migrate, the DockerfileCMDis--migrate, andmoon run workspace:serverinjects the flag.Closes #1138.
Validation
go test ./cmd/server/ -run 'TestCommandHelpListsMigrate|TestMigrateCommandAppliesSchemaIdempotently|TestStartDatabaseSkipsMigrationsUnlessRequested'(pass after review follow-up; migrate logsstream=runtime)go test ./cmd/server/(full package)moon run server:formatcorepack pnpm --filter @zitadel/cli exec vitest run tests/unit/lib/local-server/binary.test.ts(status metadata includes injected--migrate)moon run server:test,moon run cli:test,node scripts/check-pr-title.mjs --title "feat(migration): add migrate subcommand and opt-in --migrate flag",corepack pnpm exec changeset status --since origin/maingo run . migrateagainst a temp SQLite dir: first run applies, second run exits 0,goose_db_versionand application tables presentgo run . --helplistsmigrate;go run . migrate --helpexposes--configRelease notes / changeset
Changeset:
.changeset/nextgen-migrate-command.md— operators can runnextgen migrateto apply schema and exit;servermigrates only with--migrate;zitadel startand the published image still migrate. Lists@zitadel/server(minor; fixed group bumps together).No extra changeset for review follow-ups: logging, Close handling, SilenceErrors, and recording
--migrateinzitadel statusmetadata are polish on the same feature. The run-servercompletionskip is contributor-script only.Notes
migratenow callssetUpLogging(same OTel metrics asrun, StreamRuntime, metrics shutdown, no HTTP). Close errors after a successful migrate arezlogdebug-only and do not change the exit code.SilenceErrors: trueon root,server, andmigrate. UI builds skipped forcompletionin the run-server wrappers. Help test asserts both themigratesubcommand and--migrateflag; skip-migrations test usest.Context(). Local binary runtime metadata now stores the same argv that was spawned, including--migrate.gcloud run jobs update/execute --wait, migratorsecretAccessor) is still follow-up after ci(deploy): add the OpenTofu configuration for the GCP dev environment #1132/ci(deploy): deploy released images to Cloud Run #1133. This PR only adds the binary command.zitadel migrate;@zitadel/server migrateworks via argv passthrough.loadConfigis reused unchanged, somigratestill createsdata_dirand may mint a local master key.