Skip to content

feat: add bounded async task pool - #1526

Open
laxmanacharya8 wants to merge 1 commit into
google:mainfrom
laxmanacharya8:feat/async-worker-pool
Open

laxmanacharya8 wants to merge 1 commit into
google:mainfrom
laxmanacharya8:feat/async-worker-pool

Conversation

@laxmanacharya8

@laxmanacharya8 laxmanacharya8 commented Oct 14, 2026 •

Copy link
Copy Markdown

Related to #1375. API proposal following the discussion in #1393.

Chunking commands into batches leaves worker slots idle while the slowest command in each batch finishes. This adds pool(concurrency, tasks) to start the next task as soon as a slot is free:

const files = await glob('images/*.png')
const results = await pool(4, files.map(file => () => $`optipng ${file}`))

Tasks are functions because zx commands start eagerly. The helper accepts synchronous iterables, consumes them lazily, and returns results in input order. It lives in goods.ts and is available through the module and CLI globals, with type declarations and documentation.

The proposed failure behavior is to stop scheduling on the first observed error, wait for active tasks to settle, then reject with that error. It does not kill running commands. A non-exhausted iterator is closed on failure. This contract and the argument order are open for API review; maintainer agreement on the design is still needed.

Validation on macOS with Node 24.21.0 (the signed commit has an identical file tree to the tested commit):

  • npm test passes: 278 unit tests passed, one skipped; build, size, formatting, type, and license checks passed.

  • Twelve new tests cover scheduling without batch barriers, lazy iteration, result ordering, invalid limits, sync/async failures, draining active tasks, iterator cleanup, and real zx commands/CLI usage.

  • Circular-dependency and CJS/ESM smoke checks pass. The new runtime code adds about 1.2 kB uncompressed; size limits reflect the measured bundles.

  • npm run test:coverage passes: 98.31% lines/statements and 91.29% branches. An earlier run hit a transient failure in the existing network-dependent fetch test; both the final full test and coverage runs passed.

  • Standard docs:build fails with the same VitePress/esbuild target error on unchanged main and this branch. A temporary esnext build override renders the documentation successfully; repository configuration is unchanged.

  • Setup Set the latest Node.js LTS version.

  • Build Built and checked the generated bundles.

  • Tests Ran the test suite and added coverage for the new behavior.

  • Docs Added API documentation and examples.

  • Sign The commit follows conventional commits and has a verified GitHub signature.

  • CoC Changes follow the project's code style and Code of Conduct.

  • Review This PR represents original work and is not solely generated by AI tools. Implementation and tests were generated with Codex; contributor review and input are pending.

@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@google-cla

google-cla Bot commented Oct 14, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@laxmanacharya8
laxmanacharya8 marked this pull request as ready for review October 16, 2026 17:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant