Repository navigation
fix(embeddings): hold unchecked fallbacks, skip rejected texts within a cap, retry the record check (#857 items 1–3) - #874
Merged
Conversation
…ip rejected texts (#857) - An Ollama embedding fallback whose build setup could not compare with the store's no longer answers blind: the chain holds it until its build is read and matches the primary's (or the record's), at its first call or the record check, whichever comes first. Another build removes it. - A record check the database does not answer is retried with backoff. - A text the provider keeps rejecting no longer stops the re-embed at the same batch on every resume: its old vector is removed and logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ence, tool index re-arm - A text counts as rejected only after two failures, a probe embedding after each, and neither a timeout nor a rate limit; anything else stops the re-embed with every vector kept. - The fallback verifier reads the primary's build and the gate after its waits, holds (not removes) on a record mismatch while the primary is up, ignores a replaced chain, contains its own errors and logs a hold once. - Admitting a held fallback lets a tool index that failed meanwhile build again; setup logs a held fallback instead of announcing it. - Tests for the hold wiring, the gated record retry, the interleavings and each fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFT8XpNscXgkJDZcdrG
…tatus (#857) - With the primary down, the 'fell back' notice for a held fallback is sent once the chain admits it, instead of never. - A rate limit is told by its type or HTTP status, not by '429' in the message, so an over-length error quoting a token count is still a rejection. - A fallback held on a record mismatch logs a warning once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFT8XpNscXgkJDZcdrG
- A re-embed text is rejected (its vector dropped) only when the provider answers 400, 413 or 422 twice, read from status_code, response.status_code or Google's `code`. A timeout, rate limit (Gemini's `.code` 429 included), 5xx, dropped connection or a status-less error stops the run with every vector kept. - The fallback verifier reads the fallback's build fresh; setup's read no longer stands in for a failed one. - A chain whose only tryable members are held raises EmbeddingsUnavailableError, so memory search falls back to recent memories. - A held fallback admitted while the tool index is being written marks it, so a failure of that write re-arms instead of latching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFT8XpNscXgkJDZcdrG
…ons to their reference (#857) Second review of #857 (3 cycles): - A vector is deleted only for a text the provider rejects with 400, 413 or 422, at most 10 per repair: the count rides in the saved cursor, so a resume after a reload or restart goes on with what is left. A run over the cap stops before the rest of its batch, is not resumed by setup (its target joins the store's writers, read under the row lock), and logs how many vectors it removed. The primary's build is read again before each delete; a batch rejected for its input skips the retry waits. - Null, empty or non-finite vectors are an error that stops the run with every vector kept, never a rejection or a size mismatch. - The record check reads every fallback's build fresh. A fallback admitted against the store's record is held again once the primary's build is read, and a call in flight on a member held again discards its answer. - A chain with a held member, or none left, raises the unavailable error, so memory search uses recent memories. - The tool index re-arm flag is spent with the turn's pass; a refused member's backoff is read before the check lock; the embedding deployment is set once a held fallback is admitted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytwFT8XpNscXgkJDZcdrG
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #857: fixes its items 1–3. Items 4–8 stay open there.
What changes
1. Unchecked Ollama embedding fallbacks are held until their build is checked. A fallback whose build setup could not compare with the store's answers no embedding call until its build is read now and matches the primary's, or, while that is unknown, the build the store's record names. Another build removes it from the chain with the other-build repair issue. A fallback admitted against the record is checked again once the primary's build is read, and a call already in flight on a member held again discards its answer. While only held fallbacks are left, memory search uses recent memories. With the primary down and nothing to compare with, saving memories fails, as with no fallback (decided trade-off). The cases where the hold never ends are #873.
2. A text the provider keeps rejecting no longer blocks the re-embed. A batch that keeps failing is embedded one text at a time. A text's old vector is removed only if the provider twice answers that the input itself is bad (HTTP 400, 413 or 422, read from the status however the SDK carries it, including Gemini's
code) while a short probe text embeds after each failure. That happens at most 10 times per repair: the count is saved with the run's place, so a resume goes on with what is left. A run over the cap stops, is not resumed by setup, and the repair asks again. The primary's build is read again before each delete. A timeout, rate limit, 5xx, dropped connection, or a null, empty or NaN vector stops the run with every vector kept.3. A record check the database doesn't answer is retried for about 20 minutes. Once the checks run out, an error tells the user to reload.
Also: a tool index that failed while the fallback was held is built again when the fallback is admitted.
Review
Two gstack
/reviewruns, each through all 3 fix cycles. Readers: 6 specialists, Red Team, Claude adversarial and Codex (detached). The second run fixed 6 critical findings: 4 data-loss paths in the delete, and 2 ways two builds could mix in the store. No critical finding is open. It is recorded as not converged, because the third cycle also needed fixes and the skill allows no fourth.Deferred:
Testing
make test: 6580 passed.make typecheck: 0 errors.make lint: clean.CHANGELOG
[Unreleased]anddocs/configuration.mdare updated. This is for the 3.45.0 bundle.🤖 Generated with Claude Code
https://claude.ai/code/session_01QytwFT8XpNscXgkJDZcdrG