Repository navigation
Conversation
semantic_chunking changed how documents split into chunks, but was missing from the reindex field set, so toggling it via the settings API reported no reindex needed. New files picked up the new chunker while the rest of the index kept the old boundaries, with nothing telling the user. topic_threshold has the same gap: it only affects output while semantic chunking is on, mirroring table_model and layout_detection, so it is inert-suppressed the same way when semantic chunking is off.
tobocop2
changed the title
tobocop2
marked this pull request as ready for review
September 28, 2026 04:25
Owner
Author
|
This change is now part of #1003, which collects every Profiles pull request on one branch. Review and merge happen there. The branch stays in place. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Turning
semantic_chunkingon or off changes how documents are split into chunks, but lilbee did not ask for a reindex. The change applied to new files only, and the index silently mixed two chunkings.topic_threshold, which only matters while semantic chunking is on, had the same gap.Solution
Both settings now ask for a reindex when they change, like
chunk_sizeandchunk_overlap. A change totopic_thresholdwhile semantic chunking is off asks for nothing, since it has no effect then.