[safe-output-health] 🏥 Safe Output Health Report - 2026-11-09 (partial: 4h window)

[github-actions[bot]]

Summary

• Period: 2026-11-08T23:39:58Z to 2026-11-09T04:17:39Z (4.0 of 24 hours, partial coverage)
• Runs Analyzed: 222 runs in the logs summary (214 executed)
• Workflows Active: not tallied for this partial window
• Safe Output Jobs Executed: 184 concluded (206 runs carry a safe_outputs job, 22 skipped)
• Safe Output Jobs Failed: 11
• Error Clusters Identified: 2 in scope
• Cadence: healthy. 24.0 hours since the previous successful audit, run §42446071860 started 2026-11-08T04:15:33Z

Key Metrics and Issues

• Overall success rate: 94.0% (173 of 184 concluded). The complete prior window was 97.2%.
• Most important finding: the Process Safe Outputs step failed in 10 runs in this 4-hour slice. Eight of those 10 runs had already recorded GitHub items (reviews or comments) before the failure, so reruns risk duplicates. The step error text is not in any local artifact, so the root cause is still open after four audits.

Detailed findings, statistics, and evidence

Safe Output Job Statistics

Job	Records	Skipped	Failed	Success rate of executed
safe_outputs (job level)	206	22	11	94.0% (173 of 184)

Per-job-type breakdown (create_issue, add_comment and so on) was not tallied in this window.

Error Clusters

Cluster 1: Process Safe Outputs step failure

• Count: 10 runs, all confirmed as failure in run_summary.json
• Affected Jobs: safe_outputs / Process Safe Outputs
• Affected Workflows: design-decision-gate, eslint-miner, pr-specification-reviewer, smoke-claude, smoke-project
• Engines: claude, copilot, pi
• Window: 00:08:49Z to 04:02:38Z
• Items recorded before failure: 8 of 10. Two runs, §42538588878 and §42556432224, recorded none.
• Sample: §42559589900. Process Safe Outputs ran 04:12:16 to 04:12:23 UTC and failed. The audit records submit_pull_request_review Cancel obsolete worker backlog during queue maintenance #76126 (review 5821579373, COMMENTED). The Upload Safe Outputs Items step still succeeded afterwards, so the failure comes after the review is written.
• Root cause: undetermined. No error text in run_summary.json, audit.json, or the audit tool output. No workflow-logs step log is present in the local logs.
• Unverified lead: in run 42559589900, both safeoutputs MCP calls (push_to_pull_request_branch and submit_pull_request_review) are recorded with status incomplete, even though a review was created. Whether this matters is not established.

Cluster 2: Process trusted work queue controls step failure

• Count: 1 run, §42555622663 (ESLint Factory Dispatcher)
• Affected Jobs: safe_outputs. The Process trusted work queue controls step failed and Process Safe Outputs was skipped.
• Items recorded: 0
• Recurrence: per cache history the same step failed twice on 2026-11-07 and twice on 2026-11-08, so this is a third consecutive day.
• Root cause: undetermined.

Out of scope (not safe output)

pre_activation Setup Scripts (8 runs), collect_failure_evidence (4 runs), and agent executor and conclusion aftermath clusters. Not analyzed here.

Root Cause Analysis

API-Related Issues

No API errors were observed in the safe-output failures. Core API quota was healthy at the end of the window (11322 of 15000 remaining).

Data Validation Issues

No validation or parsing error text was recorded for any safe-output failure, so none can be classified.

Permission and Other Issues

No permission errors seen. The Process Safe Outputs failure occurs after items are written, which points to post-write processing rather than input validation. This is an inference, not confirmed.

Recommendations

Critical Issues (Immediate Action Required)

• Process Safe Outputs fails with no recorded error. Priority: High

• Root Cause: the step does not write its error to the run summary or audit output, and the step log is not in the local logs.

• Recommended Action: pull the step log for §42559589900 from the workflow-logs artifact, then have gh-aw write a one-line error to the job summary when this step fails.

• Affected: safe_outputs jobs in design-decision-gate, eslint-miner, pr-specification-reviewer, smoke-claude, smoke-project.

• Do not rerun failed safe_outputs jobs blindly. Priority: High

• Root Cause: 8 of 10 failed runs already have items recorded.

• Recommended Action: check for existing review or comment URLs before any manual rerun.

Bug Fixes Required

• Process Safe Outputs failure after item write
  • File/Location: not yet identified; needs the step log
  • Problem: the step fails after GitHub items are created, so output is partly delivered
  • Fix: identify the failing command from the step log, then fix it or make it idempotent
  • Affected Jobs: safe_outputs

Configuration Changes

• Log collection continuation
  • Current: the continuation has prune_older_runs set to true and writes into the shared logs directory
  • Recommended: fetch the remaining window to a separate location, or confirm the prune behaviour first
  • Reason: avoids losing evidence while extending coverage

Process Improvements

• Failure visibility
  • Current State: the failing step gives no error text in any audit output
  • Proposed: write a one-line error to the job summary on failure
  • Benefits: future audits can classify the failure without downloading step logs

Work Item Plans

Work Item 1: Retrieve and document the Process Safe Outputs step log

• Type: Investigation
• Priority: High
• Description: obtain the step log for a failing run that recorded items (42559589900) and one that recorded none (42538588878), then identify the failing command.
• Acceptance Criteria:
  • Step log attached to the investigation
  • Root cause recorded in the audit cache
• Technical Approach: download the workflow-logs artifact with authenticated gh (maintainer), then read safe_outputs/(n)_Process Safe Outputs.txt
• Estimated Effort: Small
• Dependencies: maintainer GitHub auth; this agent has none

Work Item 2: Surface safe-output step errors in the job summary

• Type: Enhancement
• Priority: Medium
• Description: write a one-line error to the job summary when Process Safe Outputs or Process trusted work queue controls fails.
• Acceptance Criteria:
  • Failure reason visible in audit output for the next failing run
• Estimated Effort: Small
• Dependencies: none

Work Item 3: Investigate the Process trusted work queue controls failure

• Type: Investigation
• Priority: Medium
• Description: this step has failed on three consecutive days with zero items recorded. Identify the trigger and failing command.
• Estimated Effort: Small
• Dependencies: step log, as in Work Item 1

Historical Context

Trends

• Error rate trend: higher in this slice (6.0%, 11 of 184) than in the complete 2026-11-07 to 2026-11-08 window (2.8%, 15 of 537). Four hours is too short to call a trend.
• Most common recurring issue: Process Safe Outputs. Failure counts by audit were 6 on 2026-11-06, 16 on 2026-11-07 (partial), 12 on 2026-11-08, and 10 in this 4-hour slice.
• Improvement since last audit: none. Cadence is healthy: the 2026-11-08 successful run closes the 72-hour gap recorded in the prior audit. Scheduled outcomes for 2026-11-06 and 2026-11-07 remain unverified.

Metrics and KPIs

• Overall Safe Output Success Rate: 94.0% (partial window)
• Most Reliable Job Type: not separated in this window
• Most Problematic Job Type: safe_outputs, specifically the Process Safe Outputs step
• Average Time to Failure: not computed

Next Steps or Limitations

• Pull the Process Safe Outputs step log for §42559589900 and §42538588878 from the workflow-logs artifact (needs maintainer GitHub auth)
• Add a one-line error to the job summary when Process Safe Outputs fails
• Check items before any rerun of a failed safe_outputs job
• Fetch the remaining ~19.6 hours (end_date 2026-11-08T23:36:04Z, before_run_id 42536253196) without pruning existing logs
• Limitations: this report covers 4.0 of 24 hours. The continuation was not followed because it writes to the shared logs directory with prune_older_runs set to true. No step logs were available, so the root cause is unconfirmed. Per-job-type and per-workflow counts were not tallied.

References:

• §42559589900
• §42555622663
• §42538588878

Generated by 🔒 Safe Output Health Monitor · claude · routed: haiku55 xhigh · ~28 AIC · ⌖ ~2 AIC · ⊞ 9.1K · ◷

• expires on Nov 9, 2026, 8:31 PM UTC-08:00