Skip to content

Require answer evidence and block unsafe adversarial acceptances - #94

Draft
aryanorastar wants to merge 1 commit into
mainfrom
codex/answer-evidence-ci-gate
Draft

aryanorastar wants to merge 1 commit into
mainfrom
codex/answer-evidence-ci-gate

Conversation

@aryanorastar

@aryanorastar aryanorastar commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Customer problem and change

Answer-affecting changes can omit comparison evidence, and the existing eight rule-only checks do not expose several unsafe model-review acceptances. Prepare a trusted-main metadata policy with an actual current-head check and add six original fictional acceptance probes to the no-model runner.

The policy covers production root code/data/dependencies, all scripts (including ingestion), prompts/filters and workflows. It requires a current-head comparison evidence link or a concrete no-answer-change justification with an exact-head, exact-section-digest, non-author maintainer approval. Review submission/edit/dismissal events use an unprivileged relay; the privileged main workflow freshly reads open PR metadata, never branch code or artifacts. Invalid/partial metadata and races block the check.

Verification and actual blockers

  • 8/8 metadata-policy unit groups pass, including observed red-before ingestion bypass, stale/withdrawn approval, malformed inventory, pagination/races and failure check publication. Independent exact-head scoped review PASS at eeaf75c.
  • Full unit/integration suite 1,266/1,266 PASS. No provider requests.
  • Existing eight ordinary no-model checks pass. Four of six adversarial probes pass; two FAIL: unsupported refund approval survives the injected review boundary; device-specific instructions are approved with unidentified hardware.
  • Exact-head hosted CI 40918340012: database PASS; rules24 fails after all1,266 unit tests pass because the two adversarial probes fail; rules20 was cancelled by matrix fail-fast. No failed check is waived.
  • These are genuine blocking failures, not skipped/expected-failure green assertions. The existing CI no-model command now exits nonzero. This draft must not be marked ready or merged while those boundaries remain red.
  • Probes use injected transports through production owners. They do not prove live-model resistance, establish a whole-product accuracy percentage, or reveal what was in the protected comparison set.

Installation and release

The workflow and required-check enforcement are not installed on main. Fresh reads show classic branch protection absent and effective main branch rules empty. After general acceptance repairs, accuracy disposition, review and green current-head checks, the trusted workflow must be merged and its fixed Mira answer evidence head check made required, then verified. No repository settings were changed.

A comparison URL declaration is not machine grading of accuracy or evidence adequacy. Read and grade the linked record independently; the implementation documents its limits and does not claim a tamper-proof control against administrators or malicious workflow maintainers. docs/ANSWER_EVIDENCE_GATE.md has the staged installation checklist.

#90 is merged cffb46b but its provenance code does not enforce hardware applicability. Production stays the healthy a9c9508/a6ef6b22 rollback; release and the third Discord smoke remain held. No paid run, customer/protected data, Discord post, optional enablement, merge or deployment.

Answer-change evidence

Mode: pending
Head: eeaf75c
Justification: This is a blocking draft: two adversarial acceptance regressions remain unresolved and required-check installation has not occurred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant