[Security Question]: How do you prevent malicious or supply-chain attack PRs from being merged?

[MrViSiOn]

Details

Security Concern: Preventing Malicious Pull Requests

Hi team 👋

First of all, great work on Orca — it's an impressive project.

I wanted to raise a security question that I think is important for a tool like this, which runs with elevated privileges on users' machines and integrates deeply with AI agents and code execution.

The concern

Orcas's open-source nature means that anyone can submit a Pull Request. This opens the door to potential supply-chain attacks or malicious contributions, where a bad actor could:

• Inject malicious code into core execution paths
• Introduce backdoors in agent integrations or terminal/shell runners
• Smuggle data-exfiltration logic into networking or API-related modules
• Gradually escalate privileges through seemingly innocent incremental PRs

These kinds of attacks are well-documented (e.g., the xz utils backdoor, event-stream npm incident, etc.) and are particularly dangerous for tools that:

• Execute code locally
• Have access to the filesystem
• Communicate with external APIs or AI backends

My questions

1. What is your current PR review process? Do you have a checklist or policy for reviewing contributions from unknown/new contributors?
2. Do you use any automated security scanning on PRs (e.g., CodeQL, Dependabot, Semgrep, Socket.dev for supply-chain risks)?
3. Do you enforce signed commits or any form of contributor identity verification?
4. Is there a SECURITY.md or responsible disclosure policy in place?
5. Do you have branch protection rules that require reviews from specific trusted maintainers before merging?

Suggestion

I'd strongly recommend:

• Publishing a SECURITY.md with a clear vulnerability disclosure process
• Enabling GitHub's dependency review and secret scanning
• Requiring at least 2 maintainer approvals for PRs touching sensitive areas (agent execution, networking, auth)
• Running SAST tools (CodeQL / Semgrep) automatically on every PR via GitHub Actions
• Considering a CODEOWNERS file to assign mandatory reviewers per module

Given that Orca is an AI-powered tool that executes code and interacts with the filesystem, the attack surface is significant. I'd love to understand your current security posture and how the community can help keep this project safe.

Thanks for your time and for maintaining this project!

[Pr1p]

An attempt at an answer to the supply-chain question (and a modest proposal)

This question has been sitting unanswered for a while, so let me take a shot at it — partly as an answer, and partly because working through it surfaced things the project is already doing right that nobody has written down anywhere.

First, the uncomfortable framing: orca is unusually exposed to this question. It welcomes agent-generated contributions (AGENTS.md, the required AI-review summary in the PR template), it runs ~100 PRs a day — roughly two-thirds of that the team's own work, with the outside community accounting for about a third (4,384 community PRs in Jun–Aug alone; more numbers in the data reply over at #22851) — and the entire human decision layer is four people merging after bot verdicts. Great environment for contributors. Also, if you think like an attacker, a great environment for attackers.

The good news, which I only appreciated after actually reading the workflows: most of a real defense is already built.

• The queue itself has a trust tier baked in — track-community-prs.yaml checks org membership on every opened PR and files external ones into a side board. Bots excluded.
• ~23 CI workflows that actually test things: e2e across Windows/WSL/macOS/Linux, crash survival, perf contracts — plus pullfrog and coderabbitai reviewing everything that moves.
• A release side with signing rehearsals, ref validation, and the cloud-verify/cloud-prove attestation workflows.

None of this is written down anywhere a contributor (or an auditor) could find — which is honestly most of the gap. The trust model exists, but only in YAML.

What's actually left, threat-wise, seems to be three things.

The boring one: build/release tampering. Mostly covered, with two cheap upgrades on the table — SLSA-style provenance on artifacts, and pinning third-party Actions to full SHAs. That closes the "binary ≠ source" class that CI alone can't see.

The scary one: a patient attacker writing correct code. No CI catches this — xz-utils passed every automated check in 2024 and was only caught by one engineer noticing a half-second SSH delay. The only defense is human attention pointed at the right places, which suggests two small rules: first-time contributors touching security-sensitive paths (.github/workflows/, signing, IPC/preload, auth, dependency manifests) get a human review regardless of bot verdicts — and CODEOWNERS grows from its current two entries to cover exactly those paths, so the right eyes are auto-requested.

The 2026-specific one: attention dilution as an attack surface. A malicious PR buried in 3.6k open ones, bot-approved, waiting for one tired glance — flooding works precisely because the human layer is four people. And the noise floor for that flood already exists: the community merge rate measured at 17% and falling, accounts with 30+ PRs and zero merges, and a handful of accidental million-line diffs from bad agent syncs — all the same shape a patient attacker would want to hide inside. Soft per-author caps on concurrently-open PRs and "bot-checklist-complete before the human queue" gating are the obvious mitigations, none of which need new infrastructure.

The cheapest first step, if the team wants one: a SECURITY.md that says in plain language what bots verify, what humans verify, what a first-time contributor should expect, and how someone becomes trusted. One document, answers this question permanently for every future reader, and gives maintainers a place to say "no" to the rest of it explicitly rather than by silence.

All of this is offered for the maintainers to adopt, adapt, or veto — I don't have visibility into why things are shaped the way they are, and the shipping pace this team sustains is the thing worth protecting here.

(In the spirit of this project's own disclosure norms: drafted with AI assistance, reviewed by me.)