Skip to content

fix: retry punctuation-only summary completions - #1256

Open
igs-rogenlo wants to merge 4 commits into
Martian-Engineering:mainfrom
igs-rogenlo:fix/summary-sanitization-diagnostics
Open

igs-rogenlo wants to merge 4 commits into
Martian-Engineering:mainfrom
igs-rogenlo:fix/summary-sanitization-diagnostics

Conversation

@igs-rogenlo

@igs-rogenlo igs-rogenlo commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Summary completions containing only punctuation can pass the non-empty check and be stored as successful summaries. This occurs after removable reasoning blocks, such as <think>...</think>., and also with bare output such as ..., where the host may already have separated reasoning.

A downstream observation found a stored ... summary over substantive input after the previous, narrower fix. The raw provider envelope is unavailable, so this does not attribute the response shape to a particular provider or transport.

Change

Treat a body containing only Unicode punctuation, separators, and control characters as empty output regardless of whether this sanitizer removed a reasoning block. Reuse existing bounded retry, envelope recovery, and deterministic fallback. Preserve short CJK text, letters, numbers, and symbols without a minimum-length heuristic.

The change covers initial and retry content/envelope extraction. Metadata diagnostics remain; no summary or reasoning bodies are logged.

Validation

  • The previous PR head 933e83c fails five additional regressions and passes four substantive controls. Failures cover both extraction paths on the initial/retry completion and the real compaction engine with SQLite persistence.
  • The follow-up candidate passes 130 tests across summarization, summarizer-reasoning, and compaction-work-accounting. Typecheck, plugin build, and whitespace checks pass.
  • The same additional regressions on unmodified latest upstream main c3745b0e6ee5e23dcce8c55cca5c3f9e8ec51d62 fail five cases and pass four controls. The latest-main candidate passes all 126 tests across the same three suites.

The completions in these tests are controlled fixtures; persistence uses real SQLite. Historical summaries have not been rewritten. Earlier live-provider checks exercised the narrower sanitizer and are not acceptance evidence for this follow-up.

Additional downstream validation (2026-11-05)

The sanitizer compiled from this follow-up was applied as the sole function change to the current customized 1.2.0 bundle (-18 bytes), preserving every other runtime byte. The full before/candidate bundles passed 26 controls, including actual compaction and file-backed SQLite: the predecessor stored ..., while the candidate retried and stored recovered substantive text.

Two bounded real-provider completions passed: one ordinary summary and one recovery completion after a deliberately injected bare-punctuation first response. The recovered 703-character summary was read back from the isolated SQLite store. This probe directly used the existing Qwen HTTP route; it does not establish host runtime.llm.complete normalization coverage.

The downstream Gateway subsequently loaded the exact compiled sanitizer after a native restart. The plugin was active, readiness returned HTTP 200, and 13,709 integrity checks had no mismatches. A read-only sample of the first 12 new natural summaries after confirmed activation contained no empty or punctuation-only bodies. Busy/CPU warnings remained and are tracked separately; healthy readiness is not full performance acceptance.

The bad first completion was injected. These observations do not claim a natural bad provider response was rejected and recovered by the live Gateway, nor that all downstream conversations or other PRs have been accepted.

@igs-rogenlo igs-rogenlo changed the title fix: log metadata after summary reasoning sanitization fix: reject punctuation-only summaries after reasoning sanitization Nov 4, 2026
@igs-rogenlo igs-rogenlo changed the title fix: reject punctuation-only summaries after reasoning sanitization fix: retry punctuation-only summary completions Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant