Repository navigation
fix(sentinel): bound triage calls per run and keep Sentinel off the GPU around chat turns (#866) - #875
Merged
Conversation
…PU around chat turns (#866) On a single-GPU host where one Ollama server serves chat and Sentinel, a run with many findings sent one triage call per finding back to back; a voice turn arriving mid-burst cancelled the call in flight, which can wedge the Ollama runner and stall the voice turn for minutes. - New Sentinel Advanced option "Triage calls per run" (sentinel_triage_max_per_run, default 5); the rest of a run's findings pass through untriaged (triage_cap). Calls are 1 s apart, and once chat takes the model during a run the run's remaining findings pass through. - No Sentinel LLM call starts within 30 s of a chat turn ending on an edge deployment (SENTINEL_CHAT_GRACE_S), so a conversation's follow-up turns never land on a call started between them. - The stalled-provider probe (#828) follows the same gate and is cancelled by chat like any other background call; closes the TODOS entry "Stall probes skip the chat-priority gate". - sentinel_health: triage_calls_last_run, triage_passed_through_last_run; triage_suppress_rate counts only findings the model saw. - Docs, strings (en/cs/zh-Hans), CHANGELOG. Part of #866. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFSMNhTRnAfuCqngGL7
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code |
…race and stall probe (#866) Three gstack review cycles (specialists, Claude adversarial, Codex, Red Team) on PR #875. The fixes: - The new option was missing from the resolver allowlist, so a saved cap never reached runtime; added, with a resolver test. - The same limit now caps explanation calls per run, on every dispatch path, and a finding that passed through triage gets no explanation; the 1 s gap is measured from a shared last-call clock that triage, explanations and discovery all wait on. Cap and gap apply only to a local model shared with chat: a cloud provider, or a server marked model_provider_uncontended, is never limited. - sentinel_admission defers at once while chat is active and waits only for video; the 30 s chat grace yields, with a warning, after a streak of deferrals no more than a grace apart that lasts 300 s, so chat turns every few seconds cannot hold Sentinel off for good while a quiet house keeps its grace. model_provider_uncontended is now a module flag that bypasses every local gate, as documented. A chat or video turn from before a reload no longer touches the counters the reload reset. - A triage deferral is signalled by the gate's exception, never by the model's reason code; reserved triage_* codes in a reply are normalised. An interrupted call stays counted; a refused one does not. - The stall probe is gated at start only (active chat or video; the grace does not apply), retried 10 s after a skipped round, never cancelled once sent, and a recovery recorded by another call lets it finish; a re-stall reuses the live probe loop; skipped rounds are counted on the health sensor and refresh it once per streak. - triage_suppress_rate counts only findings the model decided on. - Test-only APIs removed (run_stats, max_per_run, sentinel_may_start, run_tracked_sentinel_call, retry_in_s); docs, strings and CHANGELOG follow the final behaviour; TODOS.md gains the value-ordered budget. Part of #866. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFSMNhTRnAfuCqngGL7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of the plan on #866 (part of #866; the model selector follows as its own PR).
What happens
On a single-GPU host where one Ollama server serves both chat and Sentinel, a run with many findings sends one triage call per finding, back to back. A voice turn arriving mid-burst cancels the call in flight; on Ollama a cancelled request can wedge the runner, and the voice turn that cancelled it then stalls for minutes. The reporter saw 17 triage calls in a couple of seconds,
triage_suppress_rate: 0across ~500 findings, and a multi-minute voice stall.What changes
sentinel_triage_max_per_run, default 5, 1–50). On a local model shared with chat, a run sends at most that many findings to the triage model; the rest pass through untriaged withreason_code: triage_cap. The same limit caps the run's explanation calls, and a finding that passed through gets no explanation. A cloud provider, or a server markedmodel_provider_uncontended, is never limited.SENTINEL_TRIAGE_CALL_GAP_S, 1 s) measured from a shared clock of the last Sentinel model call of any kind (triage, explanation, discovery), across runs, so a chat turn arriving between two calls starts without cancelling anything.triage_deferred) instead of resuming between voice turns. The deferral is signalled by the admission gate's exception, never by the model's reason code.SENTINEL_CHAT_GRACE_S, 30 s): on a shared local model no Sentinel LLM call starts within 30 s of a chat turn ending, so a conversation's follow-up turns never land on a call started between them. Admission defers at once while chat is active or in its grace, and waits only for video. The grace yields, with a warning, after a streak of deferrals (no more than a grace apart) that lasts 300 s, so an automation callingconversation.processevery few seconds cannot hold Sentinel off for good, while a quiet house keeps its grace.model_provider_uncontendednow bypasses every local gate as documented (chat session, video session, Sentinel admission and grace, probe gate). Before, chat still marked itself active and Sentinel still deferred with it on.stalled_model_providersshowsprobe_rounds_skipped. Closes the TODOS entry "Stall probes skip the chat-priority gate", the other way round from what it asked: the review showed cancelling the probe is the hazard.triage_calls_last_runandtriage_passed_through_last_run;triage_suppress_ratenow counts only findings the model decided on.model_provider_uncontendedoff on a shared-GPU host, and a cancel cannot un-wedge an Ollama runner. TODOS.md gains an entry for a value-ordered / rate-based triage budget.Review
gstack
review, three fix cycles (testing, maintainability, security, performance, simplification specialists; Red Team; Claude adversarial; Codexgpt-6-astra), about 40 findings fixed, including a resolver allowlist omission that made the option a no-op, an explanation burst the cap did not cover, two wrong clocks for the grace yield, a model-spoofable deferral code, and a duplicate probe loop. Record endsconverged: falseby protocol (edits in cycle 3), with no known open defect. QA evidence in.gstack/qa-reports/review-866/.Tests
New: cap and pass-through,
begin_runrenewal, the gap within and across runs and before explanations, the per-run short-circuit after a deferral, a model reply that cannot fake a deferral, an interrupted call staying counted, cloud deployments never capped, the grace (deferred at once, streak-based yield with its warning, stale deferrals ignored, chat arriving during the video wait), uncontended bypassing every gate, reload-generation safety for chat and video turns, the probe runner's branches (cloud, uncontended, gate closed, grace ignored, a sent probe finishing), a skipped probe round retried and counted, a re-stall reusing the live loop, a recovery leaving a sent probe alone, explanation cap/skip/gap and the option-to-limit path, the explainer's deferred flag, KPI exclusion, the resolver propagating the option, the sensor allow-list, and_run_oncerenewing both budgets each run.make lint,make test,make typecheckclean.🤖 Generated with Claude Code
https://claude.ai/code/session_01QytwFSMNhTRnAfuCqngGL7