Repository navigation
Embedding model record: remaining ways mixed vectors or a stuck repair go unflagged #857
Copy link
Copy link
Open
Description
What happens
#855 records which embedding model filled the memory store, and its repair re-embeds the store when the model changes. The review of that PR ran three cycles and left these cases open. In each one, vectors of another model or build can still end up in the store with no repair issue, or a repair cannot finish:
- An Ollama fallback whose build is unknown can write into the store. A fallback that was down at startup, or whose
/api/showdid not name its weights, is kept in the embedding chain. When the circuit breaker switches to it later, it writes its build's vectors, nothing reads its build then, and the record does not note it as a writer. The same holds while a repair runs: the repair uses the primary, but conversations and camera captions still embed through the chain. - A batch that can never be embedded blocks the repair. One text the provider always rejects (over its input limit, for example) fails every retry. The run stops at the same place, the issue comes back, and every resume hits the same batch.
- A record check that fails once loses the history. The check runs in the background at the end of setup. If the database times out, the new model starts writing before the record notes it, and switching back later can clear the issue while those vectors stay.
- Dropping the vector table can raise a needless repair. After
DROP TABLE store_vectorsand a reload (the documented way to change the size, Embedding store size mismatch with non-Ollama providers, and a missing store table after setup, fail silently #760), the tool index can write new vectors before the record check runs, which then sees vectors under a record that names the old model. - A build pulled while Home Assistant runs is noticed only at the next startup (or at the end of a repair).
- After a host crash, the repair's database lock stays held until PostgreSQL drops the dead connection (about two hours with default keepalives), and a resume meanwhile says another run is re-embedding.
- A second entry on the same database and model shows the repair issue while the first entry's run holds the lock.
- After a full re-embed, the vector search index is about twice its size until PostgreSQL vacuums it, and recall can suffer meanwhile; nothing runs a REINDEX.
Expected
Each case either flags the store (a repair issue, or the writer noted in the record) or lets the repair finish.
Notes
- Found in the gstack review of the Memory store: nothing records which embedding model filled it, so a model change silently mixes vectors #855 fix (three cycles, not converged).
- Related: Memory store: nothing records which embedding model filled it, so a model change silently mixes vectors #855, Embedding fallback to a different model mixes vector spaces in stored memories #751, Embedding store size mismatch with non-Ollama providers, and a missing store table after setup, fail silently #760.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
goruck commentedon Oct 25, 2026
One more scenario for item 1 (from the #860 review, PR for it is open):
A kept Ollama fallback whose build setup could not read is checked again in the background up to ~55 s after startup. If that fallback answers an embedding call inside that window, the chain remembers it in memory (
served) so the writer can be noted in the store's model record when the build turns out to differ. A reload inside that window drops that in-memory state, and the vectors it wrote go unnoted. Same shape as the other items here: a reload while something is still in flight loses evidence that only lived in memory.Merge pull request #874 from goruck/fix/embedding-record-gaps-857
goruck commentedon Oct 27, 2026
Items 1–3 are fixed by #874, merged today. They ship in the 3.45.0 bundle.
Items 4–8 are still open, so this issue stays open for them.