Repository navigation
docs: correct README record counts, schema version, links, competitive claims - #80
Merged
Merged
Conversation
…e claims Verification (against live AVE corpus and repo code, 2026-09-12): published records: 80 (source: dist/ave-records-latest.json) covered records: 51 (source: grep AVE-2026-NNNNN in scanner/rules/) live schema: 1.1.0 coverage gap: AVE-2026-00052..00080 (29 records) Corrected: - Record count stated as 48 / 51 / 51 in three places. Split into published (80) and covered-by-this-scanner (51), which are different quantities. Coverage gap disclosed in the AVE section, tracked in #79. - Schema v1.0.0 -> 1.1.0 (live). AVE moved to the aveproject org and aveproject.org domain; ave.bawbel.io and github.com/bawbel/ave links updated accordingly. api.piranha.bawbel.io left untouched: the scanner still hardcodes that host as the piranha_url value in every engine, so changing the README would make the docs disagree with actual scan output. That migration question is unresolved and tracked in #79, not decided here. - Removed "only open-source scanner that detects toxic flows". NVIDIA SkillSpector (17k stars, Apache 2.0) ships TT1-TT5 taint chains plus an AST8 dangerous- execution chain, including a credential exfiltration chain. Replaced with a mechanism claim: AVE capability pairs over natural-language + config, not just AST, with a stable flow_id and per-flow AIVSS. - Competitive table: added NVIDIA SkillSpector (dropped the unverifiable ClawGuard row), marked cells unverifiable from Cisco DefenseClaw's public docs as `?`, added the row where SkillSpector leads (per-scan inference_usage telemetry, tracked in #79). - Fixed internal inconsistencies: added missing Magika stage (Stage 0) to both pipeline diagrams so "six engines" matches the five previously listed plus Magika; pattern rule count 40+ -> 40 (exact, grep-verified); pre-commit rev v1.2.1 -> v1.2.3 (current tag); PROJECT_STRUCTURE.md action.yml location corrected to match reality (lives in this repo, not bawbel/integrations, which is not referenced anywhere in this repo's actual CI). - Em dashes removed throughout README.md (17 -> 0), rewritten per sentence rather than mechanically swapped for hyphens. Left unchanged (do-not-invent-a-number rule, flagged for maintainer): - bawbel/sandbox:1.2.3 pin in the sandbox section. No code reference confirms whether the sandbox image is intentionally versioned independently of the scanner (1.3.0) or just stale. VERIFY. - scanner/ave_meta.py docstring still cites schema_version 1.0.0. It's a comment, not an enforced assertion, so not a functional bug, but stale. Tracked in #79. Sources for every competitive-table cell: NVIDIA/SkillSpector, snyk/agent-scan, and cisco-ai-defense/defenseclaw READMEs, fetched 2026-09-12 (see PR description).
This was referenced Oct 11, 2026
The corpus is now 87 records, and the 51 'covered' figure was wrong: 51 is the number of records with metadata in scanner/ave_meta.py, while 43 have a detection rule (42 pattern/YARA/Semgrep, plus 00024 via Magika). The gap is 44, of which 27 are static_detection and 17 are runtime-stage records that static analysis cannot detect. - Tagline: drop the uncited 'only open-source' claim and 'Never executes code', which the optional Docker sandbox stage contradicts - Comparison table: split execution into 'starts MCP servers' and 'runs scanned skill content'; link the telemetry row to #79 - CI snippet: bawbel/scanner@v1 has no tag; pin @v1.3.0, drop the nonexistent 'output' input, upload the action's sarif-file output - Pre-commit snippet: this repo has no .pre-commit-hooks.yaml, so the documented hook could never install. Replace with a repo: local hook over scanner.bawbel_pre_commit, tested against bawbel-scanner 1.3.0 - Docker: published images are 1.2.3, not 1.3.0 - Related: replace dead api.piranha.bawbel.io link with api.aveproject.org - PROJECT_STRUCTURE.md: same action tag and pre-commit corrections Refs #79
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #79
Verification (re-measured 2026-10-11)
Script and record-by-record list: #79 (comment)
The first version of this PR said 80 published and 51 covered. The 80 went stale as the corpus grew. The 51 was wrong from the start: it counted records with metadata in
scanner/ave_meta.py, not records with a rule, and the grep overscanner/rules/it came from misses the pattern engine.Corrected
?cell (below). The execution row is split into "starts MCP servers" and "runs scanned skill content". The telemetry row, where SkillSpector leads, links to README accuracy: AVE coverage gap, telemetry, sync tooling, doc drift #79.api.piranha.bawbel.iolink in Related is replaced withapi.aveproject.org(returns 200 for 00001 through 00087).bawbel/scanner@v1points at a tag that does not exist. It now pins@v1.3.0, whose tag includesaction.yml. The snippet also passed anoutput:input the action does not declare, so the SARIF upload pointed at a file that was never written. It now uploadssteps.bawbel.outputs.sarif-file.repo: bawbel/scanner,id: bawbel-scan) could never install, because this repo has never shipped.pre-commit-hooks.yaml. It's replaced with arepo: localhook that runspython -m scanner.bawbel_pre_commit. Tested in a scratch repo againstbawbel-scanner==1.3.0from PyPI: a clean file passes, and a copy of the golden fixture blocks the commit with exit 1.latestis the same digest as1.2.3). The README claimed1.3.0and1.3.0-full, which do not exist.rule_ids).@v1.3.0. There is no.pre-commit-hooks.yamlhere, and the hook definitions in bawbel/integrations are onmainonly, with no tag.Left unchanged
api.piranha.bawbel.io: that is the value every engine emits aspiranha_urltoday. The host is now NXDOMAIN, so this is a code bug, not a doc question. The fix changes the value only, never the field name, and updates the README samples in the same PR. It is noted on README accuracy: AVE coverage gap, telemetry, sync tooling, doc drift #79.scanner/ave_meta.pydocstring still cites schema 1.0.0. It is a comment, tracked in README accuracy: AVE coverage gap, telemetry, sync tooling, doc drift #79.PIRANHA_API_URLvsBAWBEL_PIRANHA_URL: config and docs disagree. Tracked in README accuracy: AVE coverage gap, telemetry, sync tooling, doc drift #79.Sources for every competitive-table cell
2026-09-12 (execution model, taint chains TT1-TT5 + AST8, baseline suppression
glob+fingerprint no expiry,
metadata.inference_usage, SARIF 2.1.0, Apache 2.0,no AIVSS/CVSS, no MCP conformance grading, no rug-pull detection, internal rule IDs
with OSV.dev integration for CVEs).
(explicitly executes MCP stdio servers, "Toxic Flows" detection category present but
chain count not published,
--ignore-issues-codes/--ignore-riskssuppression withno expiry/audit trail, no SARIF, no MCP conformance grading, Apache-2.0).
fetched 2026-09-12. Only License (Apache-2.0) was clearly confirmable from the
excerpt; every other cell is marked
?rather than asserting an unconfirmedcapability. License re-checked 2026-10-11 via the GitHub API: Apache-2.0.
Not in scope for this PR
scanner/,config/, ortests/.developis merged into this branch and all checks pass.🤖 Generated with Claude Code