Fix: hybrid lexical + semantic retrieval ordering - #348
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthroughThe changes update search documentation to describe hybrid lexical and cosine retrieval, lexical-only fallback when embeddings are disabled, and the rejection of BM25 because its score scale does not combine directly with cosine scores. ChangesHybrid search documentation
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to This change corrects hybrid-search documentation to describe the existing lexical and semantic retrieval behavior. It introduces no runtime behavior change or remaining merge-readiness risk. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🟡 Changes recommended
The updated docs introduce a broken JSDoc sentence and an internal inconsistency in the BM25 rationale wording that should be corrected before merge.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates repository documentation to accurately describe the current hybrid lexical + semantic retrieval strategy (sentinel-score UNION ALL of lexical LIKE/ILIKE rows plus semantic cosine-similarity rows) and clarifies why BM25 was evaluated but removed.
Changes:
- Updated README feature bullet to describe sentinel-score fusion behavior and rationale for dropping BM25.
- Corrected
src/embeddings/disable.tsJSDoc to refer to lexicalLIKE/ILIKEfallback (not “BM25 / ILIKE”).
File summaries
| File | Description |
|---|---|
| README.md | Updates the hybrid retrieval description and BM25 rationale in the feature list. |
| src/embeddings/disable.ts | Updates embeddings-disable JSDoc to reflect lexical fallback behavior and BM25 rationale. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - 📥 **Captures** every session's prompts, tool calls, and responses as structured traces in Deeplake | ||
| - 🧠 **Codifies** patterns into reusable `SKILL.md` files, available to every agent on your team | ||
| - 🔍 **Searches** traces and skills with hybrid lexical + semantic retrieval (BM25 fallback when embeddings off) | ||
| - 🔍 **Searches** traces and skills with hybrid lexical + semantic retrieval: a `UNION ALL` of `LIKE`/`ILIKE` substring rows (sentinel score 1.0, capped by `HIVEMIND_HYBRID_LEXICAL_LIMIT`) and cosine-similarity rows (real 0–1 score), ordered by score — so exact keyword matches always lead while semantic hits fill in below. When embeddings are off, falls back to lexical `LIKE`/`ILIKE` only. (BM25 was evaluated but dropped: its unbounded score scale (~1–3) overwhelmed cosine in a shared `ORDER BY`, requiring rank-based fusion (RRF) or score normalisation to use safely.) |
| * lexical `LIKE`/`ILIKE` matching on text columns (BM25 was evaluated but | ||
| * dropped — its score scale (~1..3) is incompatible with cosine in a single | ||
| * ORDER BY without RRF or normalisation; see grep-core.ts PR-NOTES F4c). | ||
| * readable. |
Summary
Fix stale documentation that described the hybrid retrieval strategy as using BM25 as a fallback. BM25 was evaluated and removed (PR-NOTES F4c) because its unbounded score scale (~1–3) overwhelms cosine similarity values (0–1) in a shared
ORDER BY, causing semantic hits to be pushed out of the top-K entirely. Using it correctly would require rank-based fusion (RRF) or score normalisation, which adds complexity without clear benefit for the "find any session mentioning X" use case.The actual hybrid strategy is a
UNION ALLof:LIKE/ILIKEsubstring rows, emitting a sentinel score of 1.0 (capped byHIVEMIND_HYBRID_LEXICAL_LIMIT, default 20)Results are ordered by score descending, so exact keyword matches always lead the ranking while semantic concept matches fill in below. When embeddings are off, retrieval falls back to lexical
LIKE/ILIKEonly.Files changed:
README.md— updated the feature bullet to describe the sentinel-score fusion and document why BM25 was droppedsrc/embeddings/disable.ts— corrected the JSDoc comment from"BM25 / ILIKE matching"to"lexical LIKE/ILIKE matching"with the same rationaleVersion Bump
This is a documentation-only change — no release needed. No version bump required.
Test plan
npm test)package.json, or no release needed for this change ✓ (docs only)Summary by CodeRabbit