Skip to content

Add HOL coding-agent benchmark as vendor-labeled reading (closes #18) - #29

Merged
requie merged 1 commit into
mainfrom
docs/coding-agent-benchmark
Oct 1, 2026
Merged

requie merged 1 commit into
mainfrom
docs/coding-agent-benchmark

Conversation

@requie

@requie requie commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

Resolves #18 by @kantorcodes. Adds HOL's AI Coding Agent Runtime Security Benchmark as further reading in the AI Coding-Agent & CI/CD Security section.

It compares the built-in safety controls of Codex CLI, Claude Code, Cursor, Gemini CLI, and OpenCode across 11 risky scenarios (220 deterministic fixture results; JSON/CSV and methodology published).

Why it's labeled vendor-published

  • It compares those controls against HOL Guard, the publisher's own product (the suggester disclosed they're with HOL).
  • By its own description it's fixture-based: it tests documented control behavior, not live attacks, and doesn't measure exploit resistance, latency, or false positives.

The entry states this inline, matching how the guide handles other vendor-described material.

Files

  • README.md, README.es.md, README.fr.md, README.zh.md — one paragraph each (translated)
  • CHANGELOG.md, resources-validation.md (marked Vendor-described; reviewed via search results because the site is blocked from the review environment)

Verification

  • Entry appears once in each README, inside the coding-agent section
  • 0 broken internal links, no duplicate anchors, heading counts unchanged; merges cleanly into main

Closes #18

Resolves #18. Adds HOL's AI Coding Agent Runtime Security Benchmark to the
AI Coding-Agent & CI/CD Security section in all four READMEs, labeled as
vendor-published: it compares native agent controls against the
publisher's own product (HOL Guard), is fixture-based, and does not
measure exploit resistance, latency, or false positives.

Closes #18
@requie
requie merged commit 08c6449 into main Oct 1, 2026
2 checks passed
@requie
requie deleted the docs/coding-agent-benchmark branch October 28, 2026 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resource suggestion: AI Coding Agent Runtime Security Benchmark

1 participant