Repository navigation
Profiles: named sets of settings per project, with analyze - #1003
Merged
Merged
Conversation
This was referenced Oct 3, 2026
The environment write guard allows writes only from a named list of functions that run at startup on the main thread. The blank LILBEE_DATA fix moved the data root export from top-level code into a function called once at import, so the guard flagged it. The function is now on the list.
The settings source refused LILBEE_LINKED_ROOTS, so every CLI command exited 1 with "is not a table of names and values", an error that named no fix. Serve, MCP and the TUI dropped the variable with no message. The settings source now skips the variable and adds one load warning, on the same path as the main_gpu warning. The warning tells the user to register a folder with `lilbee add` or to edit linked_roots in config.toml. The list of load warnings is built once at import, so a settings reload does not repeat it. Every other refused variable is still refused.
A profile apply or a setting change that touches the index asked for a rebuild even when the index was empty. There was nothing to rebuild, and the check opened the table, which created the library folder on a fresh data dir. The rebuild check now asks the store first whether it holds any chunks. The store answers from the data dir without opening a table and without creating the library folder. A data dir that has only older held-out record files still returns those records. HTTP, MCP, the CLI and the TUI all use this check, so none of them offers or reports a rebuild on an empty index, and all still do on a non-empty one. The test files that set a chunks mark on every test now set it only on the tests that need an index with chunks.
On Windows, pytest-timeout's thread method ends a worker with os._exit(1), and xdist then reports only "node down: Not properly terminated". Two CI runs looked like native crashes for this reason. A controller-side pytest_handlecrashitem hook in the hang watchdog now adds the configured pytest-timeout limit to the crash report, and the path of the hang dump the worker wrote for that test, or a plain line that no dump was written. The hook changes only the report text, never a test outcome, and it loads without xdist.
After a setting write in the fleet owner, a peer whose call was in flight still held the old value. Its one retry then failed to bind the owner's restarted fleet and started a full private fleet. The config follower now has a synchronous take that shares its lock with the per-request follow, and the fleet provider calls an injected refresh after it drops the stale binding and before it retries. Serve, the stdio MCP and the TUI wire the refresh to their follower; the one-off CLI has no follower and keeps its old behaviour. The refresh runs only on the retry, never on a call that succeeds, and a refresh that fails in serve or the MCP is logged and the retry runs as before. Locks on the retry path are taken in one order: the follower lock, then the settings write lock, then the fleet lock. A gap remains: the owner writes config.toml before it takes the build lock, so a peer that retries in that window can still overflow.
The key fleet test asserted that nothing was built, which also held on the old code. Its fakes now know the fleet directory, so on the old code the peer builds a private fleet and the test fails, and with the retry refresh it binds the owner's fleet. In the TUI, an error from the config take on the retry raised into the caller. It is now logged and the retry runs, as in serve and the stdio MCP.
The TUI profile apply read the library sources to count files for its rebuild text. On a fresh install that read created the library folder, although no rebuild was offered. The apply now reads the sources only when the index holds chunks and a rebuild is offered, so the text for a non-empty index is unchanged.
The empty index check in profile apply went through the full services, so `lilbee profile apply` without --reindex built a provider and, with eager start on, started a model warm-up only to ask whether the index held chunks. The check now asks the store directly: the bound container's store when there is one, else a short-lived store that opens no table, creates no library folder and is closed at once. No provider is built. The embed swap tests that exercise the post-write rebuild check now take an index with chunks, which that check needs since it skips an empty index.
On SIGTERM, serve waited about 10 s for the provider shutdown and then up to 5 s for a library open, so a slow open could take it past the launcher's 10 s grace. After the bound the open thread was still inside LanceDB at interpreter finalization, and on Linux 3.12 the process could abort with -6 (CPython gh-87135). The open wait now runs beside the provider shutdown, so the bound counts from the start of the stop. If an open is still running at the bound, serve tears down the container, flushes its logs and ends with os._exit through the existing exit helper, with the signal's status (143 for SIGTERM). A stop with no open running takes the normal exit. os._exit skips the background loop's task drain, which only liter_llm requests and ingest use. A Lance commit writes its manifest with put-if-not-exists, so a cut open leaves a library the next open can finish.
A reload of config.toml set the changed keys on the live settings one by one, so a request in flight could read a new chunk size with the old overlap. A refused change was then undone key by key. The reload and the settings update now apply the change to a copy, validate the copy whole, and publish the moved fields onto the live settings with one dict update. The settings stay the same object, so every module that bound them at import sees the change. A refused change raises before the live settings move, which deletes the key-by-key rollback. The commit publishes only the fields whose value differs from a base copy taken with the working copy. A write by another thread while the change runs therefore keeps its value, and data_root no longer moves on every commit. The one-step publish relies on the GIL. A sync or ingest job that reads two keys in two lookups can still see one from each side of a commit; a per-job snapshot is a follow-up.
Each index setting saved in the TUI Settings screen asked about a rebuild on its own, so a visit that changed three settings asked three times, and a visit that changed a value and set it back still asked. Settings now records the values that affect the index when the visit starts and checks them once when the user leaves. A changed value offers a rebuild under the same rule as a single save: a setting whose gate is off and an index with no chunks do not count. A looser chunk limit offers a retry of the files it held out, and when both apply one dialog shows both texts. A quit from Settings prints the text after the TUI exits. The save and reset in Settings no longer ask on their own, so the rebuild offer's follow-up callback and the helper that chained it to the wiki wipe offer are gone. A view change used switch_screen, which replaces only the top screen. A model catalog pushed over Settings therefore left Settings buried under the next view, so the leave check never ran, and each round trip left a stale view on the stack. A view change now pops every pushed screen above the current view first, so only one view is on the stack and no way out of Settings skips the check. Escape in a pushed catalog returns to the screen that opened it.
A reconnect retry dropped the fleet refs and then released this process's user holds before it found the fleet again. In that gap a second thread counted this process's own hold as a live user and built an overflow fleet, and a stream still in flight held no user lock, so a last-user stop by a peer could end it. The live-user check at the ladder now skips this process's own lock; the reaper and the last-user stop still count it. The retry no longer releases holds before it runs, so the gap does not exist. After the retry a sweep under the build lock drops each hold whose dir no group uses, unless a retired client still has a request in flight. A retired client calls an idle callback when its in-flight count reaches zero, outside its count lock. The callback starts a daemon thread that runs the sweep, so the release never runs on the request thread, which can hold the caller's locks. A count of open rediscovers keeps such a sweep out of another thread's retry.
A joiner on a small card whose memory a peer holds got a chat refusal that told it to use a smaller model or set num_ctx. The text never said that other processes hold the card's memory, which was the cause. The refusal reason now gets the device's total and free bytes and, when the memory others hold is more than the weights, adds "Other processes hold N GB of the card's M GB." It gets the devices only when the planner has a clean-box snapshot, the same test placement uses to charge against free memory. A live probe also counts a warm lilbee engine's own models, so without a snapshot no clause is shown.
With the machine slot free and a matching private fleet live under a peer, a new process built a second full fleet in the slot, so the card held the same models twice. The ladder's bind-only step is now one helper for the slot and a private dir. When the slot has no bindable engine and no live user, the ladder takes the private dir's build lock and tries that bind, and builds in the slot only if the bind fails. A bind takes this process's user lock in the private dir before it releases that dir's build lock, so the peer's last-user stop counts this process. The slot's build lock is always taken before a private dir's, and no path holds a private dir's lock while it waits on the slot's.
The sweep now checks for unused holds before it waits on the build lock, and a sweep thread that cannot start at interpreter exit is skipped. The open-rediscover count goes up inside the try, and a failed sweep is logged instead of replacing the retry's result. After a bind of a private fleet, a slot engine with no live user is stopped through the build path's reap-and-stop step, under the slot's build lock and after the private dir's lock is released. The small-card refusal says "does not fit" when the weights exceed free memory, "the cards'" for two or more cards, and uses GiB. The private-dir bind gets tests for a num_ctx mismatch, another engine version and a dead engine.
The catalog Escape test used the catalog as the test app's only screen. Escape now dismisses a catalog that is not an installed view, which raised ScreenStackError; the test installs the catalog as a view, as the app does. The small-card refusal tests made a 6 GiB model file with truncate. NTFS writes the zeros, so the tests ran past the 60 s limit on Windows. The file is now a few bytes, and a path subclass reports 6 GiB from stat() to both readers of the model size.
The [ and ] view keys are priority bindings, so Textual let them act through a modal dialog: pressing one while the Settings leave dialog was open closed it with no answer. They now do nothing while a modal screen is on top. The embedder pick asks the store whether the index holds chunks, so it no longer builds a provider when no services are bound. The chunk limit key lives in the settings map with the other setting names. The settings undo path gets a test for a file that changes under it, and comments and docstrings now say what the code does.
If the server run raised instead of returning, serve skipped the check that ends it when a library open outlives the exit bound. The stop now runs in one helper on both paths: a raise is logged with its traceback, serve exits with status 1, and a normal return or SIGTERM keeps the run's own status. A failed read of config.toml in `lilbee settings list` and `settings get` said "Could not write config.toml"; reads now get their own message. The shutdown gather logs every step that fails, the slow-open tests collect each worker thread's exception, and the timeout note and the fleet environ test say what they check.
Two concurrent provider shutdowns queue on the fleet build lock, and the second finds no engine left to stop and no hold left to release. A test now runs both at once and checks the engine stops once. The TUI opens the library on a normal worker thread, which Python waits for at exit, so a quit cannot cut an open short the way the serve abort does. A test now checks that the open thread is not a daemon thread.
A settings change committed the new embedding model first and set the matching dimension in a later step. Between the two, a reader could see the new model with the old dimension, and config.toml was written in that state. The dimension is now worked out on the working copy inside the same commit, so the model and its dimension publish together on a settings update, a reset, a profile apply and an outside config reload. If the dimension step fails, the live settings and config.toml stay as they were. The later step stays for the paths where an environment variable overrides the model.
The watchdog armed its teardown backstop at setup with the same window it used for the teardown. A stall of that length in setup or the call fired the dump with the test's own header, so the backstop tests failed now and then on a loaded runner. LILBEE_TEST_HANG_TEARDOWN_S now sets the teardown window and defaults to LILBEE_TEST_HANG_DUMP_S, so CI behaves as before. The backstop tests keep the long window for setup and the call and set a short teardown window, and a new case blocks setup past the teardown window to show the dump still names the teardown.
The startup-gate test asserted that the library opens on a thread whose daemon flag is false. A pool worker copies that flag from the thread that first starts the pool, so on Python 3.11 it was true in CI while Python still joins the worker at exit. The test now checks that the open runs on a thread the interpreter joins at exit, which is what keeps a TUI quit from cutting an open short.
The command palette and the chat autocomplete for /model and /remove listed the installed models on the event loop for every key typed. That listing probes each local server over HTTP. On Windows a refused localhost connect takes seconds instead of failing at once, so the TUI froze on each key when no local server was running. The palette now lists the models once per open, on a thread, and its search offers the static commands at once without waiting for the listing. Textual refreshes the hit list only when a hit arrives after a short delay or when every search ends, so a search that waits would hide every command. When the listing lands and the palette is still open, the search runs again only if a model row matches what is typed. The chat autocomplete lists the models once per model argument, on a thread. A key typed while the listing runs neither waits for it nor starts a second one; the rows appear when it ends. Each model argument has a session number that goes up when the input leaves it, as a submit does, so a result from an earlier argument is dropped and the new one starts its own listing. A late result is also dropped if the input changed or the app closed, and the list is cleared when the input no longer holds a model argument, so a removed model is not offered.
A plain profile Apply that asked about a rebuild was asked about again when the user left Settings. An answered question now moves the leave baseline for the keys it asked about: the rows the profile changed, or the one setting a generic offer named. Only a rebuild that starts takes the whole snapshot, so an index edit saved earlier in the visit still gets its offer on leave. An outside config change no longer holds the settings lock while the fleet stops. The TUI takes the new values under the lock and stops the fleet after it releases it, so a second take during the stop finds nothing to do.
A peer that took a new config value before the owner rebuilt the shared slot found no slot to bind, built a full private fleet, and then the owner built the new one too, so both stayed resident. This happened on a retry after a lost connection and on the per-request config check of a bound peer. The fleet state now records each group's plan origin: the config root and the values of the keys the owner rebuilds on after an outside config reload. The owner's check and the peer's view read one key set, defined in its own module so the fleet roles module never imports the config package (the engine reaper imports the providers first in a fresh interpreter). When a peer's bind fails while a live owner with the same root holds the slot under an older origin, the peer releases the build lock and polls for the owner to catch up, then runs the ladder once more. Its waits on the machine slot and its private dir share one 30 s deadline. A crashed owner ends the wait at the next poll, and an owner that never follows leaves the peer to overflow as before. A write to a key outside the set, such as top_k, does not change the origin. A reload that restarts nothing restamps the kept groups on a best-effort write: if it fails, as it can on Windows while a peer reads the file, the reload logs a warning and waiting peers fall back to their bound. Older processes read the new state file, and new ones read the old file.
The fleet started the engine reaper with its error output sent nowhere. A reaper that died at start left no trace, and the engine then outlived every user with nothing to say why. The reaper's error output now goes to logs/engine-reaper.log in the data dir, opened for append and closed in the parent right after the start. When the reaper does not report running, the warning names that log and quotes what the reaper wrote to it this time.
The TUI quit test asserted that quitting took under 4 s, and slept 0.2 s to let the sync park on its page wait. A loaded CI runner took 4.59 s and failed it. The 4 s bound stood for one fact: the quit path cancels the parked wait instead of letting it run out the 5 s exit budget. The test now records how the wait ended and asserts it was cancelled, and an event marks the sync as parked in place of the sleep.
Main gained the huggingface_hub 2.4.0 lock with httpx2, vision recovery and CLI fixes from profile testing, the llama.cpp engine bump, two random CI test fixes, the macOS flake check and the CUDA snap summary. The Profiles commits replay on top of it; this commit holds what both sides changed. The quit action keeps main's order: it cancels inference after the stream check, then prints the Settings exit text. The unused suggester widget is deleted, as on main, with its tests. Four tests needed the other side's names: two fleet tests patch the renamed vision_mmproj, a profile-tab test no longer imports the suggester, main's settings tests patch the store check that replaced the old helper, and main's eviction test expects the fleet re-plan where main dropped the fleet; it still requires one fleet-wide action and no role reload per load key. The saved-names property test keeps the Profiles address helpers and draws its label addresses from main's printable strategy, and main's refused --model test moves next to the Profiles override tests.
tobocop2
force-pushed
the
integration/profiles
branch
from
November 5, 2026 21:48
b8adf1b to
8251a55
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Profiles work was 32 open pull requests: a stack of 29 and three that target main alone. Main moved under them, so the stack no longer merged cleanly, and two of the standalone pull requests conflicted with it.
Solution
This branch lands all 32 as one commit on main, with the follow-up fixes from the profiles review. The table below lists the pull requests.
Profiles and analyze
A profile is a named set of ingest and retrieval settings. One resolver gives each setting its value and its source, and every surface shows that source.
lilbee analyzesamples a corpus and recommends a profile with fitted values. The CLI, the TUI, the REST API and MCP can apply it.Settings writes
A
config.tomlthat the user cannot write is refused with its path. The TUI shows a failed write as a toast instead of crashing. A change that another process writes is picked up before each chat question. A chunking change always reports that a rebuild is needed.Fixes from the end-to-end review
A drive of every profile surface found defects in the settings screen, the profile dialogs, the fleet re-plan, rebuilds and streaming. They are fixed here, with tests. Larger findings that predate this branch are tracked as their own pull requests.
Serve stops cleanly while the library opens
On Linux with Python 3.12, a SIGTERM that arrived while the library was still opening could abort
lilbee servewith -6 instead of exiting 143. The server now waits for the open, for at most 5 s, before it exits, which stays inside the CLI's and the plugin's stop grace.A stopped chat stops reading
When a model server sent response headers and then no body, stopping the chat left the reading thread blocked and the server never saw the client leave. Stopping a stream now shuts the connection's socket down, so the reader returns at once.
The warm stream agrees with health
After a swap from a local chat model to a remote one,
/api/healthsaid ready while/api/warm/streamkept sending the old "not installed" error. The stream now asks the same readiness question as health first.Syncs take turns
A second
lilbee addwhile the first was syncing ran its own sync over the first add's files, so it looked stuck. Syncs on one library now run one at a time on an exclusive lock, and a waiting sync says which command and process it waits for. Serve sends the same notice as async_waitstream event.Serve takes outside config writes
A setting write or profile apply from the CLI or TUI in another process did not reach a running serve until it restarted. Serve now checks config.toml before each request and applies a change through the same rules as a write through its own settings API. A bad file leaves the settings in memory as they were.
Advanced settings in the CLI
lilbee settings listnow leaves out the settings marked advanced, as the TUI does, and shows them with--advanced.Profile apply can rebuild over HTTP and MCP
The CLI could apply a profile and rebuild the index in one step, but HTTP and MCP could not. Both now take an optional
reindexflag, off by default, and decide whether to rebuild with the same check as the CLI.One rebuild rule in the TUI
The TUI Apply dialog offered a rebuild only when the profile's diff showed a reindex row, so it could stay quiet where the other surfaces rebuild. It now asks the shared check and shows its reason.
Chat path completions stay responsive
Path completion in the chat input listed large folders on the event loop, so the TUI froze during typing. The listing now runs on a worker thread, reads each folder once and keeps only the first matches. Tab, Ctrl+N and Ctrl+P use the same worker, and a result for text that changed in the meantime is dropped.
Profile discard can rebuild over HTTP and MCP
Discard now takes the same optional
reindexflag as apply over HTTP and MCP, and every surface decides the discard rebuild with the shared check. The check also skips a setting whose gate is off, such as the topic threshold while semantic chunking is off, so an apply of such a setting no longer rebuilds.A running stdio MCP takes outside config writes
A stdio
lilbee mcpnow checks config.toml before each tool call, with the same rules as serve. The MCP mount inside serve still checks once per request.A config write during startup is taken
Serve, the stdio MCP and the TUI now start following config.toml from the same read that loaded the settings, so a write that lands while they start is taken. One-off CLI flags stay in force, and a blank
LILBEE_DATAnow behaves like an unset one.A late stream abort leaves pooled connections alone
A response now leaves the abort list in its own close, before its connection goes back to the pool, so a late abort can no longer shut a socket that another request uses.
Serve waits for every library open at exit
The exit now waits, within the same 5 s bound, for every library open, also one that started after a reset.
Serve exits in time when a library open is slow
The open wait now runs beside the provider shutdown, and an open still running at the bound no longer keeps serve inside interpreter shutdown: serve tears down, flushes its logs and exits with the signal's status, 143 for SIGTERM, within the launcher's 10 s grace.
A waiting sync does not name a dead one
If the recorded sync holder was killed, a waiting sync shows the generic message instead of naming the dead process.
A config read during an editor write changes nothing
A settings PATCH that serve refuses because config.toml changed under it now writes nothing, and its undo restores only the settings in memory, so the editor's file stays as the editor wrote it.
No rebuild on an empty index
A profile apply or a setting change no longer offers or reports a rebuild when the index holds no chunks. The check opens no table and does not create the library folder on a fresh data dir.
Profile apply asks the store, not the services
The empty index check asks the store directly, so
lilbee profile applywithout--reindexbuilds no provider and starts no model warm-up. In the TUI, an apply on a fresh install no longer creates the library folder.A fleet peer rejoins a restarted fleet
Before its one retry, a fleet peer in serve, the stdio MCP or the TUI takes config.toml again, so a call in flight while the owner restarts its fleet binds the new fleet instead of starting a private one. A peer that retries in the moment between the owner's config write and its fleet build can still start one.
A fleet reconnect keeps its locks
A fleet reconnect released this process's locks before it found the fleet again. In that gap a second request could take the process's own lock for another user and start a second fleet, and a peer's stop could end a stream still in flight. The reconnect now keeps its locks, and a lock it no longer needs is released when the last request that used it ends.
A new process joins a live private fleet
With the shared slot free and a matching private fleet running under another process, a new process built a second fleet, so the card held the same models twice. It now joins the running fleet and builds in the slot only if that join fails.
A small-card refusal names the memory others hold
When chat cannot fit on a card because other processes hold its memory, the refusal now says how much they hold instead of only telling the user to pick a smaller model.
A test worker that times out says so
When an xdist worker dies, the crash report names the configured test timeout and points to the hang dump, instead of only "node down".
LILBEE_LINKED_ROOTS warns instead of refusing
A CLI command with
LILBEE_LINKED_ROOTSset used to exit 1 with an error that named no fix, and the other surfaces ignored the variable. Every surface now runs and shows one warning that says to uselilbee addor editlinked_rootsin config.toml.One rebuild offer when the TUI leaves Settings
The Settings screen asked about a rebuild after each index setting it saved, also for a value set back to where it was. It now compares the index values once, when the user leaves, and asks once; a looser chunk limit offers a retry of the held-out files in the same dialog. A model catalog opened from Settings no longer leaves Settings buried under the next view, so no way out skips the question, and Escape in that catalog goes back to Settings.
An outside config change lands in one step
Serve used to apply a config.toml change key by key, so a request in flight could read a new value next to an old one. A reload or a settings update now checks the change whole on a copy and publishes only the values it moved, in one step. A refused change leaves every setting as it was, and a write from another thread during the change keeps its value.
The embedding model and its dimension now move in that same step, so no reader sees a new model with the old dimension.
Leaving Settings asks once per change
A plain profile Apply that already asked about a rebuild was asked about again when the user left Settings. An answered question now covers only the values it named, so an index edit saved earlier in the visit still gets its offer. An outside config change no longer holds the settings lock while the fleet stops.
The TUI stays responsive on Windows
The command palette and the chat completions for
/modeland/removelisted the installed models on the event loop on every key, and that listing probes each local server. On Windows a refused localhost connect takes seconds, so the TUI froze when no local server ran. The listing now runs once, on a thread. The palette shows its commands at once while the listing runs, and a listing that ends after the input moved on, such as a/removebefore a new/model, offers nothing.A peer waits for the fleet owner to catch up
A peer that saw a new config value before the owner rebuilt the shared fleet built a private fleet of its own, and both stayed in memory. The peer now waits, at most 30 seconds, for an owner with the same config root to rebuild, then falls back as before. It waits only for keys the owner rebuilds on.
The test watchdog has its own teardown window
LILBEE_TEST_HANG_TEARDOWN_Ssets the teardown window and defaults toLILBEE_TEST_HANG_DUMP_S, so a stall before the teardown no longer fires the teardown dump.An engine reaper that dies leaves a log
The engine reaper's error output now goes to
logs/engine-reaper.login the data dir, and the warning for a reaper that does not report running names that log and quotes it.Merged here