Fail the workflow when ClaudeCode scan fails

[douglashill]

Problem

A failed Claude Code invocation could still leave the GitHub Action run marked as successful, resulting in nothing being posted on the PR and no visible error.

Example: https://github.com/PSPDFKit/PSPDFKit/actions/runs/31812208847/job/94805295401

claude --version raised an exec format error, the scan produced an error payload instead of a review. The failure was easy to miss.

Solution

• mark the scan as failed when the ClaudeCode step produces an error payload, no results file, an empty results file, or an invalid results payload
• fail the workflow at the end of the job when that happens, while still uploading logs and results artifacts
• skip PR review posting and slash-command completion reactions when the scan already failed
• set the published check run conclusion to failure when the scan failed
• require strict JSON parsing for Claude CLI stdout, since the CLI is already invoked with --output-format json

[github-actions]

📋 PR Summary:
This PR makes ClaudeCode scan failures visible instead of silently passing. action.yml now sets a scan_failed step output when the Python audit produces an error payload, a missing/empty results file, or a payload lacking the expected findings/pr_summary/analysis_summary keys; a new terminal step fails the job in that case, the check run conclusion is forced to failure, and PR commenting plus slash-command completion reactions are skipped. On the Python side, both the Claude CLI stdout parse and the legacy result-field parse were switched from parse_json_with_fallbacks to strict json.loads, with a new unit test asserting mixed stdout is rejected.

3 files reviewed

File	Changes
action.yml	Adds scan_failed output, payload validation, and job-failing step
claudecode/github_action_audit.py	Strict JSON parsing of Claude CLI stdout and result field
claudecode/test_claude_runner.py	Test that mixed stdout with embedded JSON is rejected

---

Found 3 reliability and correctness issues. Consider addressing the suggestions in the comments.

[github-actions]

🤖 Code Review Finding: Strict stdout parsing drops the raw output from the error log, making the failures this PR surfaces undiagnosable

Severity: MEDIUM
Category: reliability

Impact: A maintainer investigating the newly-failing workflow sees Code review failed: Failed to parse Claude output with no stdout, no stderr excerpt, and no decoder message, and must reproduce the run locally to learn what the CLI actually emitted. This directly undercuts the PR's stated goal of making scan errors visible and actionable.

Recommendation: Log the decode error and a truncated repr(result.stdout) to stderr (which is captured into claudecode-error.log) when strict parsing fails, and/or include a stdout excerpt in the returned error message at line 610 the same way the non-zero-returncode branch already does.

Suggested change

	# Parse JSON output strictly. Claude CLI was invoked with --output-format json,
	# so any non-JSON stdout is treated as a hard failure.
	try:
	parsed_result = json.loads(result.stdout)
	success = True
	except json.JSONDecodeError:
	success = False
	parsed_result = None
	# Parse JSON output strictly. Claude CLI was invoked with --output-format json,
	# so any non-JSON stdout is treated as a hard failure.
	try:
	parsed_result = json.loads(result.stdout)
	success = True
	except json.JSONDecodeError as parse_error:
	success = False
	parsed_result = None
	print(
	f"[Error] Failed to parse Claude stdout as JSON ({parse_error}). "
	f"Raw output (first 2000 chars): {result.stdout[:2000]!r}",
	file=sys.stderr,
	)

[douglashill]

Addressed in 4858426.

Added logging the JSON decode error and truncated raw stdout to stderr, which ends up in claudecode-error.log. The final returned error message also includes the last parse-failure details after retries.

[github-actions]

🤖 Code Review Finding: Newly-failed runs cannot be recovered by re-running: the SHA reservation marker is cached before the scan

Severity: MEDIUM
Category: reliability

Impact: After an infrastructure-level scan failure (the claude --version exec-format error from the PR description), re-running the workflow reports success without producing a review, and the failure check run published by this PR is never updated - so a required Code Review check stays red on the commit with no way to clear it short of pushing a new commit or filing a GitHub review request (the only trigger exempted from dedup).

Recommendation: Only persist the reservation marker for a successful scan (move the cache-save step after the scan and gate it on scan_failed != 'true'), or record the outcome in the marker (status) and have determine-claudecode-enablement.sh skip dedup for markers that are still reserved/failed. At minimum, document that a failed run requires a GitHub review request to re-trigger.

[douglashill]

Addressed in 4858426.

🤖 says: The reservation marker is no longer saved to cache before the scan runs. Instead, we only mark it as completed and save it to cache after a successful scan. That means infrastructure-level failures on a given SHA no longer poison deduplication for reruns.

[github-actions]

🤖 Code Review Finding: Legacy result-field fallback now rejects markdown-fenced JSON, converting recoverable reviews into hard workflow failures

Severity: MEDIUM
Category: correctness

Impact: If structured_output is null or missing (older CLI, or the model not conforming to --json-schema) and the model fences its JSON, the review burns three full Claude invocations and then hard-fails. With this PR that no longer degrades to an empty review - it fails the whole workflow and marks the check run red.

Recommendation: Keep strict parsing for the CLI envelope (stdout) but retain the fence-tolerant extraction for the nested result text, e.g. re-import extract_json_from_text from claudecode.json_parser and fall back to it when json.loads(result_text) raises.

[douglashill]

Didn’t address.

🤖 says: This one feels more like a behavior choice than a clear bug in the same category as the other two. The goal of this PR is to make broken Claude CLI runs fail clearly instead of being silently treated as success. For the outer CLI stdout path, strict JSON parsing is important for that. For the nested legacy result field, reintroducing fence-tolerant extraction would increase compatibility, but it would also partially reintroduce the “recover malformed output” behavior that this PR is intentionally moving away from.