Repository navigation
fix(agent): one embedder per PC, on the GPU, shown in Settings - #8368
kovtcharov wants to merge 15 commits into
Conversation
…y chat On a PC set up for the NPU, every chat still embedded documents and memory with the GPU embedder, because the embedder came from a session's device and the app never sets one. Each turn then loaded it on Vulkan next to the FastFlowLM chat model and evicted it. The embedder now follows the PC's profile (default_device): the FLM embedder on an NPU PC, the GGUF one elsewhere, the same for every chat. Memory keeps one set of vectors from one embedder, so a chat switched to the other device no longer clears them. Closes amd#6008
|
Verdict: Request changes On an NPU PC, chats now embed memory and documents with the NPU embedder instead of the GPU one, and the embedder no longer changes when a chat switches devices. That fixes the per-turn eviction for chat itself, and the one-embedder-per-PC reasoning holds up. One path was missed, though: uploading a document in the Agent UI still indexes it with the GPU embedder on an NPU PC. That brings back the problem this PR fixes, because loading the GPU embedder evicts the NPU chat model. It also gets worse than before: the chat now searches with the NPU embedder, so the upload's vectors don't match and the whole document is embedded a second time on the first question. Indexing for an upload should take the same per-PC embedder. Smaller point: issue #6008 also reports an empty chat reply on the NPU profile, which this PR doesn't touch. Use Real-world evidenceN/A: no evidence bundle was produced for this run, and the NPU behaviour needs a Ryzen AI PC. The verdict rests on static review alone. I couldn't run the PR's two tests here because pytest isn't installed on this runner. The test plan's on-hardware check (Lemonade's 🔍 Technical details🟡 Agent UI upload indexing still uses the GGUF embedder (
Before this PR, both paths used GGUF and at least agreed with each other. Found by grepping
🟢 Checked, no action needed
Strengths
|
The Agent UI and TUI run setup under the "gaia" profile on every PC, and that set default_device to gpu. Now that the device picks the embedder, running setup from the app on an NPU PC would switch it to the GPU embedder and clear every memory vector. The "gaia" profile now keeps an NPU PC on the NPU; only --profile npu or a GPU-only profile changes it. The Memory dashboard also falls back to the PC's embedder instead of the GGUF one, and its Rebuild uses the PC's embedder over a stamp left by an earlier profile. A config that can't be read now says how to fix it.
|
Verdict: Request changes 🟡 The earlier finding is still open. On an NPU PC, uploading a document in the Agent UI still indexes it with the GPU embedder. That load evicts the NPU chat model, which is the bug this PR fixes. Chat now searches with the NPU embedder, so the first question about the upload embeds the whole document a second time. Upload indexing should use the same per-PC embedder that chat uses. Add a test that sets the PC to the This push's setup change, which keeps an NPU PC on the NPU when the app runs setup, looks correct. 🔍 Technical details
The PR body still says |
On an NPU PC an upload still indexed with the GPU embedder, which loaded on Vulkan and evicted the FLM chat model, and its vectors sat under a different hash than chat's, so the first question re-embedded the whole document. Uploads now use the same per-PC embedder every chat uses.
|
Uploads in the Agent UI now index with the same per-PC embedder that chat uses, so an NPU PC no longer loads the GPU embedder (and evicts the NPU chat model) when a document is added. The first question about an upload also no longer embeds it a second time. This closes the open review finding. The body now says 🔍 Technical details
|
…ser move it Settings › Memory now names the embedding model and the device it runs on, and offers the other device. The PC's setup still picks it by default (the NPU after `gaia init --profile npu`); a choice the user makes is saved as embedding_device, outlasts setup, and reaches chats already open, since their agents and GAIA's agent are rebuilt with it. A device this PC can't use is listed but can't be chosen, and a change waits for a running reply.
…8' into kalin/embedder-follows-profile-6008
The Agent UI shows this error (upload, embedder setting), and app text must not send the user to a terminal. The wrapped config error already says to fix or delete the file.
# Conflicts: # docs/guides/agent-ui.mdx
…s on their next turn The Settings embedder choice offered the NPU on PCs without one: availability was looked up in the chat-model list, which has no embedders, so it never refused. It now asks the hardware the way setup does (an NPU present and set up, a backend for the recipe). A change no longer restarts GAIA's agent. That cut off replies from other apps and dropped memory still being saved. Each open chat rebuilds on its next turn instead, keeping its model, conversation and loaded skills, and the old agent finishes its memory work before it closes. Choosing the embedder already in use changes nothing. Dashboard writes, Telegram uploads and the error text follow the PC's embedder too. Refs amd#6008
…com/kovtcharov/gaia into kalin/embedder-follows-profile-6008 # Conflicts: # src/gaia/agents/registry.py
FastFlowLM's NPU embedder (embed-gemma-300m-FLM, FLM 1.0.7) ranks text no better than chance: a paraphrase scores below an unrelated sentence, and labelled retrieval finds the right passage about as often as picking at random. The GGUF build of the same model retrieves well, and its vectors don't match the NPU build's, so the two can't share an index. Every PC now embeds with the GPU embedder; `gaia init --profile npu` downloads it. The NPU option is shown in Settings › Memory but can't be chosen, and config refuses it, until a FastFlowLM release passes the new search check. A choice saved earlier still loads. Lemonade keeps one model per type, so the GPU embedder stays loaded beside the NPU chat model. The NPU hardware test only checked vector shape, which is how this passed. It now checks search quality: the GPU embedder must find each paraphrase, and the NPU check is a strict xfail, so a fixed release fails the run. Refs amd#6008
Moving the embedder rebuilt an open chat's agent with its conversation and loaded skills but dropped what the chat set on the old agent: an offline chat went back online, a private one started saving to memory, and folders granted mid-chat were lost. The rebuild now carries those over too, and the SPEC says what a rebuild keeps.
…ents The rebuild now also carries the attached and withheld document records, read-only grants and withheld paths, so a document taken off on the same turn as an embedder move loses its grant, and GAIA's own copy of a removed document stays refused. Same carry-over as the MCP settings rebuild.
Every chat on a PC now embeds documents and memory with one embedder, the GPU one, and Settings → Memory shows where it runs. Before, the embedder was chosen per session, so a chat on the other device could clear all of memory's vectors. And the NPU embedder that setup installed for NPU PCs (
embed-gemma-300m-FLM) turned out to rank text no better than chance on FastFlowLM 1.0.7: on a Strix Halo, a paraphrase of a question scored below an unrelated sentence, and labelled retrieval on GAIA's own docs found the right passage about as often as picking at random. The GGUF build of the same model retrieves well. So NPU PCs now embed on the GPU, and the NPU option is shown but can't be chosen until a FastFlowLM release passes the new search check.npuchoice still loads, so no existing config stops starting; nothing offers or accepts it any more.Not changed: the standalone email agent still picks the NPU embedder for its NPU triage model; it is being retired rather than fixed.
Refs #6008: fixes the embedder half of it; the empty NPU reply it also reports is a separate cause.
Test plan
python -m pytest tests/unit/test_memory_embedder_setting.py tests/unit/test_npu_flm_embedder.py tests/unit/test_memory_router.py tests/unit/chat/ui/test_upload_embedder_follows_profile.py tests/unit/test_init_command.py -qpython -m pytest hub/agents/gaia/python/tests/test_model_switch_follow.py hub/agents/gaia/python/tests/test_session_registry.py -q(an open chat follows a moved embedder and keeps its conversation and skills)cd src/gaia/apps/webui && npx vitest run && npm run test:rtl -- -g "settings memory pane"(screenshots read in English and Arabic)python -m pytest tests/test_npu_flm_embedder_hw.py -k SearchQuality -rxpasses the GPU check and xfails the NPU oneGAIA_REQUIRE_FLM=1 python -m pytest tests/test_npu_flm_embedder_hw.py -rxpasses, with the NPU search check xfailing; alternating NPU chat turns and GPU embeds reloaded neither modelRAG and memory quality on NPU PCs changes with this; the GPU eval baseline isn't affected. Categories to watch on the next RC:
rag_quality,gaia_rag,memory,gaia_memory.