Repository navigation
Conversation
3b8c3a7 to
c2ee227
Compare
|
This PR is still a draft; reviewing now because it unblocks the repo-wide red Dependency Security check (#1398). P11. npm aliases make a bundled package look like toolchain ( Gate with a fixture Trivy report (CRITICAL on Suggest recording the P21. A failed weekly scan marks the tracking issue as clear ( A transient Trivy DB or P3
Checked by running: the four independently reviewed by Kiro Crew |
c2ee227 to
7c08de2
Compare
|
Re-review at Our P1, our P2 and both P3s are fixed. No new P1 or P2. Suggest approval once CI is green and the PR leaves draft. Dependency Security passes on this head.
Checked by running: the four independently reviewed by Kiro Crew |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1400 +/- ##
=======================================
Coverage ? 91.96%
=======================================
Files ? 296
Lines ? 57085
Branches ? 9397
=======================================
Hits ? 52499
Misses ? 3218
Partials ? 1368
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Updated the PR to 7c08de2. P1, npm aliases: fixed at 7c08de2. P2, failed weekly scan clears the tracking issue: fixed at 7c08de2. The tracking step reads P3.1, template comment: fixed at 7c08de2. The P3.2, CI red on c2ee227: the |
7c08de2 to
a229b23
Compare
|
Suggest approval at All CI checks pass (28 passed; only the push-only independently reviewed by Kiro Crew |
|
This suite is unbearably slow even though this change is an improvement ... looking closer I realized the MCP Apps CI job is unnecessarily duplicating 45 minutes of unit tests that the I'm going to fix that as well as some other minor fixes to attempt to get unit tests run running in parallel ... we'll see if it makes a dent on the github hosted runners |
|
@plauzy, before you change the MCP Apps job: #1387 already removes that duplicate Python run. It moves the coverage ratchet into its own Our Suggest approval above stands for independently reviewed by Kiro Crew |
|
Thanks, @fanhongy. We missed #1387. #1400 will not touch the MCP Apps job or the coverage ratchet; it stays at Merging #1400 first should also clear #1387's only red check (Dependency Security). I'll point #1408 at #1387 for the duplicate run. Any parallel-test work there will wait for #1387 and your #1350 test-stability series (#1389, #1394, #1401, #1402, #1405, #1406), so it doesn't edit the same job or the same tests. |
awslabs#1398) Dependency Security failed every pull request on any HIGH or CRITICAL finding in any tracked lockfile, including the build, test and lint toolchain. When CVE-2026-112366 (picomatch, no fix) reached the Trivy database, it failed open PRs that do not touch picomatch, although every path to it is toolchain-only. The gate now blocks on risk that reaches users, at the same strength as before for shipped code, and reports the rest. - Measure the shipped set from the build: `uv export --locked --no-dev --all-extras` for the wheel and the optional extras users install; a Vite plugin that records what web/ and every cao_mcp_apps view bundled (modules, externals and injected helpers, plus the emitted files), each package under its folder name and, for an npm alias, its real package.json name; a webpack plugin for both Docusaurus passes, plus the build-time HTML writers; `cargo tree -e normal` for the TUI. examples/ ships nothing. - Match shipped membership by package name at any version (over-block is the safe direction). - Fail closed (exit 2) on a missing, empty or incomplete build record, a shipped file no record lists, a Trivy inventory without its sentinel packages, a lockfile with no producer, a pull-request run without its base, an invalid ignore file, or any unexpected error. - Block (exit 1) on a shipped HIGH/CRITICAL not covered by a valid entry, on malware advisories anywhere, and on a toolchain HIGH/CRITICAL with a fix that the change adds. Other toolchain findings, with dependency paths, go to the summary and a SARIF file. - Validate .github/dependency-security/trivyignore.yaml, read only by the script (Trivy still runs with its ignore file disabled): each entry needs a paths or purl scope and must match a current shipped finding, and its 14-day window is checked against the YAML date and the date git first shows it on the base. - List packages that left the shipped set compared with the base's baseline. - Test the shipped-set recorder with node --test; toolchain.sarif stays an artifact. - The weekly workflow opens or updates one tracking issue for toolchain HIGH/CRITICAL findings, in a separate job that alone holds issues: write; a run that ends in an error comments on the issue and never marks it clear. - Update the contributor gate map, its packaged mirror and SECURITY.md. Co-authored-by: Kiro Agent <244629292+kiro-agent@users.noreply.github.com>
a229b23 to
b490ebc
Compare
Closes #1398
What this does
Dependency Securitynow blocks only on findings in code CAO ships. Everything else is still scanned in full (dev dependencies and unfixed advisories included), listed in the step summary and in a SARIF file, and tracked by one weekly issue. It does not fail the pull request.package.jsonlabels.uv.lock:uv export --locked --no-dev --all-extras(the wheel's runtime closure plus the optionalaguiandotelextras, which users install; without--all-extrasthe set is 87 packages, with it 97).web/andcao_mcp_apps/: a Vite plugin (scripts/shipped-packages.mjs) records the npm packages each build bundled to<project>/.shipped/bundled-packages.<config>.json.docusaurus/: a webpack plugin records the client bundle's packages, not the server render bundle's.tui/Cargo.lock:cargo tree --locked -e normal --target all.examples/**: ships nothing.scripts/dependency_security.pyclassifies each findingshippedortoolchain. It exits 1 only on a shipped HIGH/CRITICAL not covered by a valid, unexpired entry in.github/dependency-security/trivyignore.yaml. Exit 2 (fail closed) if a producer fails or returns an empty set, if a tracked lockfile outsideexamples/has no producer, or if the ignore file is invalid. It never reads "nothing shipped" as a pass..github/dependency-security/trivyignore.yaml(a deliberate deviation from the issue's literal.trivyignore.yaml): a.trivyignore.yamlat the repository root is auto-loaded by Trivy into the separatesecurityjob, andtest_security_scan_steps.py::check_no_autoloaded_trivy_configrejects it. The file is read only byscripts/dependency_security.py; Trivy still runs with--ignorefile /dev/nulland one call, so every fix(security): enforce full dependency checks and consolidate toolchain fixes #884 argument assertion holds. Each entry needsid, an httpsurl, astatement,addedandexpired_at(at most 14 days afteradded, andaddedmay not be in the future, so the limit cannot be stretched by back-dating). An entry withoutpathscovers every lockfile and package carrying that advisory; the schema is unchanged, and the gate emits a warning in the summary for each such entry. An expired entry covers nothing and the finding blocks again.toolchain-trackingjob opens or updates one issue (labeldependency-toolchain) while toolchain HIGH/CRITICAL findings remain.issues: writeis held only by that job; the scan job keepscontents: read.cao-contributorpackage copy) describe the split.The #884 braces mitigation checks (
npm run test:dependencies, thebraces-compatadapters) are untouched and still run.Before / after (same command, today's lockfiles, Trivy 0.70.0)
Command:
python3 scripts/dependency_security.py --output-dir out(head also needs the builds, below).main(5a8dc10):head:
picomatch is in none of the five shipped sets, as #1398 predicted.
patch-package, whichpackage.jsonlists as a dependency, is correctly classed toolchain because no build bundles it.Fixture checks (in
test/test_dependency_security_shipped.py): a HIGH finding bundled intoweb/fails (exit 1); the same finding only in a devDependency passes with a warning; an empty or failed producer, a lockfile with no producer, and an invalid ignore file exit 2; missing statement, an expiry over 14 days and an expired entry are each covered.Job time (local measurement, NOT GitHub Actions)
Sequence timed on one laptop, warm npm and Trivy caches, from a clean
node_modules:mainnpm ci+test:dependenciesfor docusaurus, web, cao_mcp_appsuv sync --frozen --no-dev --no-install-projectuv exportandcargo tree)About 25 s more locally. Actions runners are slower and
cargo treethere starts with a cold registry cache, so expect more; I have not measured Actions.What we gain / what we lose
Gain
patch-package.Lose
shipped-set.jsonartifact.Cargo features
The release build is
cargo build --release --locked(scripts/build_tui.py), with no--featuresor--no-default-features, andtui/Cargo.tomldefines no[features].cargo tree --locked -e normal --target alltherefore matches the release build's default-feature normal edges, and--target allis a superset of any single cross target.Verification and what I could not run
Run locally: the new and existing dependency tests, the gate-map and packaging tests,
sync_skills.py --check,make check-agent-plugin, actionlint, zizmor 1.30.1 (offline mode, medium+: no findings), black/isort/ruff,mypy_ratchet.py, the markdown link check, and the webnpm testand MCP Apps and docs type checks.Not run: the composite action and the weekly
toolchain-trackingjob on GitHub Actions. They are covered by actionlint, zizmor and the existing workflow-shape tests only. The action's PyYAML comes fromuv sync --frozen --no-dev --no-install-project;cargo tree --lockedneeds network on the runner. Please check the first CI run of this PR forDependency Security.Existing test changed
test_ci_and_scheduled_dependency_gates_are_identical_and_unconditionalis unchanged and still green: the gate job stays byte-identical inci.ymland the schedule workflow withcontents: read. The tracking issue lives in a separatetoolchain-trackingjob, and a new test,test_issues_write_exists_only_on_the_schedule_tracking_job, asserts thatissues: writeis held only by that job, never by anything inci.yml, and that the gate steps are equal. Red check: puttingissues: writeon the gate job fails both tests. Therepositoryfixture intest_dependency_security.pynow injects a shipped set (the #884 tests run on generic lockfiles with no build). The uv calls use--lockedand the schedule workflow setsUV_LOCKED=1, astest_workflow_pins.pyrequires.patch-packagestays independencies; the build-derived shipped set makes the label irrelevant.Real bundle check
From actual builds on this head:
webbundles 13 packages (@xterm/addon-fit,@xterm/xterm,events,graphology,graphology-layout,graphology-utils,lucide-react,pandemonium,react,react-dom,scheduler,sigma,zustand); the four MCP App views bundle 3, 3, 3 and 9; the Docusaurus client bundle 37. picomatch, micromatch and patch-package are in none of them (grep -c picomatchon everybundled-packages.*.jsonis 0).Co-authored-by: Kiro Agent 244629292+kiro-agent@users.noreply.github.com