Repository navigation
fix(cli): upload server files in parallel, with timeouts and retries - #141
Merged
Merged
Conversation
Lets a caller bound one server-file upload with a timeout.
A vinext worker is hundreds of modules, and each went up as its own PUT /serverfile, one after another, with no timeout or retry. Every request waits out the API's auth and D1 round trips, so a 321-module worker took 6-15 minutes from Taiwan, and one stalled or reset connection hung or failed the whole deploy, leaving the deployment queued. Upload them 8 at a time (at most 32 MB in flight), give each attempt a timeout of 60 s plus 16 KB/s of its size, and retry a network failure, a timeout, a 429 or a 5xx up to 3 times with backoff. PUT /serverfile only writes the file to staging under its name, so a retry is safe even when the server applied the first attempt. Other 4xx still stop the upload, and a missing-route 404 still falls back to the inline bundle. Progress prints one line per tenth of the files, a warning per retry, and a summary; nothing in --json mode.
There was a problem hiding this comment.
🟢 Approval recommended
The bounded parallel upload changes have no active review issues.
No open issues 🎉
What changed in this PR
Improves CLI deployment reliability and speed for workers with many server modules by replacing sequential uploads with bounded parallel uploads.
Changes:
- Uploads up to eight files concurrently, with a 32 MB in-flight limit, size-based timeouts, and retries for transient failures.
- Adds upload progress reporting and SDK abort-signal support while preserving the legacy inline-bundle fallback and leaving the final bundle request unchanged.
- Adds tests for concurrency, retries, timeouts, failure handling, and progress output.
| File | Description |
|---|---|
packages/sdk/src/client/index.ts |
Accepts an abort signal for server-file uploads. |
packages/cli/src/utils/upload-server-files.ts |
Implements bounded parallel uploads, timeouts, and retries. |
packages/cli/src/utils/upload-server-files.test.ts |
Tests upload limits, recovery, error handling, and progress reporting. |
packages/cli/src/commands/deploy.ts |
Integrates the uploader and progress reporting into deployment staging. |
packages/cli/src/commands/deploy.test.ts |
Verifies exhausted retries propagate errors without uploading the bundle. |
Review effort: Balanced
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A customer's vinext production deploys (321 modules, 14 MB worker) hung or failed at
Uploading bundle...withfetch failed/read ETIMEDOUT/read ECONNRESET, each leaving a deployment stuck inqueued.Production data (D1 + audit log + build logs) for that project:
framework=vinextdeploy) every upload took 350-1015 s; 50 of 92 CLI deploys since then were leftqueued.Cause:
stageDeploymentBundlesent onePUT /serverfileper module, sequentially, with no timeout and no retry. Each request waits out auth + several D1 round trips to the WNAM primary (~1-3 s from Taiwan), so the upload is latency-bound (it looked like "a few KB/s"), and any stall or reset in a 10+ minute run of 300+ requests killed the deploy.Change
uploadServerFiles(packages/cli/src/utils/upload-server-files.ts): 8 requests at a time, at most 32 MB in flight (a larger single file still goes alone).PUT /serverfileonly writes staging under the file's name, so a retry is safe even when the first attempt was applied. Other 4xx stop the upload; the missing-route 404 fallback to the inline bundle is kept.fetch failed/ETIMEDOUTso the existing unreachable reporting applies.--jsonmode.uploadServerFile(..., { signal }). Release needs an sdk bump with the cli.The final
PUT /bundleis unchanged and not retried: it starts the deploy job, so it is not idempotent.Verification (live, production
api.creek.dev, 2026-11-02, from s-imac in the US)Real vinext app (
create-vinext-app, 147 routes), built once, deployed with--prod --skip-buildto a test projectvx-upload-e2eunder the admin account. 214 server modules, 6.5 MB.creek@0.4.55(sequential)--jsonThe fault proxy sat in front of
api.creek.devand, for/serverfilePUTs, (#3) reset the connection, (#7) never answered, (#11) answered 502, and (#15) forwarded to the real API and then dropped the 200. The CLI retried all four (218 PUTs for 214 files) and the deploy went active. The 62 s is the stalled request's 60 s timeout. After that deploy, all 147 routes served their own<h1>page N</h1>, and/api/helloanswered.pnpm test(3720 passed), typecheck andformat:checkpass.Not in this PR
queued, so a deployment whose upload never finished staysqueuedforever./serverfileslow from Asia.