Skip to content

Embedding model record: remaining ways mixed vectors or a stuck repair go unflagged #857

Open

Description

@goruck

What happens

#855 records which embedding model filled the memory store, and its repair re-embeds the store when the model changes. The review of that PR ran three cycles and left these cases open. In each one, vectors of another model or build can still end up in the store with no repair issue, or a repair cannot finish:

  1. An Ollama fallback whose build is unknown can write into the store. A fallback that was down at startup, or whose /api/show did not name its weights, is kept in the embedding chain. When the circuit breaker switches to it later, it writes its build's vectors, nothing reads its build then, and the record does not note it as a writer. The same holds while a repair runs: the repair uses the primary, but conversations and camera captions still embed through the chain.
  2. A batch that can never be embedded blocks the repair. One text the provider always rejects (over its input limit, for example) fails every retry. The run stops at the same place, the issue comes back, and every resume hits the same batch.
  3. A record check that fails once loses the history. The check runs in the background at the end of setup. If the database times out, the new model starts writing before the record notes it, and switching back later can clear the issue while those vectors stay.
  4. Dropping the vector table can raise a needless repair. After DROP TABLE store_vectors and a reload (the documented way to change the size, Embedding store size mismatch with non-Ollama providers, and a missing store table after setup, fail silently #760), the tool index can write new vectors before the record check runs, which then sees vectors under a record that names the old model.
  5. A build pulled while Home Assistant runs is noticed only at the next startup (or at the end of a repair).
  6. After a host crash, the repair's database lock stays held until PostgreSQL drops the dead connection (about two hours with default keepalives), and a resume meanwhile says another run is re-embedding.
  7. A second entry on the same database and model shows the repair issue while the first entry's run holds the lock.
  8. After a full re-embed, the vector search index is about twice its size until PostgreSQL vacuums it, and recall can suffer meanwhile; nothing runs a REINDEX.

Expected

Each case either flags the store (a repair issue, or the writer noted in the record) or lets the repair finish.

Notes

Activity

  1. goruck commented on Oct 25, 2026

    @goruck
    OwnerAuthor

    One more scenario for item 1 (from the #860 review, PR for it is open):

    A kept Ollama fallback whose build setup could not read is checked again in the background up to ~55 s after startup. If that fallback answers an embedding call inside that window, the chain remembers it in memory (served) so the writer can be noted in the store's model record when the build turns out to differ. A reload inside that window drops that in-memory state, and the vectors it wrote go unnoted. Same shape as the other items here: a reload while something is still in flight loses evidence that only lived in memory.

  2. added a commit that references this issue on Oct 27, 2026
  3. goruck commented on Oct 27, 2026

    @goruck
    OwnerAuthor

    Items 1–3 are fixed by #874, merged today. They ship in the 3.45.0 bundle.

    1. An Ollama fallback whose build can't be compared is held: it answers no embedding call until its build is read and matches the primary's, or the build the store's record names. The cases where that hold never ends are Embedding failover stays off, with only a log line, when a fallback's build can't be compared #873.
    2. A text the provider keeps rejecting no longer blocks the repair. Its old vector is removed only after the provider twice answers that the input itself is bad (HTTP 400, 413 or 422), at most 10 times per repair. Any other failure stops the run with every vector kept.
    3. A record check the database doesn't answer is retried for about 20 minutes, then an error says to reload.

    Items 4–8 are still open, so this issue stays open for them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions