Reciprocal Rank Fusion — why Gnosem's default search blends BM25 and cosine

Semantic search — embed the query, find the nearest neighbors in vector space — is astonishingly good at "find me the memory that means roughly this even though I used different words." It's also astonishingly bad at "find me the memory that mentions gh_a4c71a70" or "find me the memory dated 2026-06-04." Exact string matches are exactly what embeddings are worst at. The embedding of "gh_a4c71a70" gets learned as roughly-noise-shaped and hits every other roughly-noise-shaped identifier in the store.

Keyword search — old-school inverted index, BM25 scoring — has the mirror problem. It nails "the memory containing this exact ID" and misses "the memory about our database choice" if you happen to search "which SQL flavor did we pick."

Gnosem's default memory_search is a hybrid. It runs BOTH BM25 (via SQLite FTS5) and cosine (via Cloudflare Vectorize) on every query, then blends the two ranked lists into one using Reciprocal Rank Fusion. This post explains why, how, and what tripped us up building it on Cloudflare Workers.

The math

Reciprocal Rank Fusion is beautifully simple. Given N ranked lists of hits, the fused score for any candidate d is:

score(d) = Σ  1 / (k + rank_i(d))
           i

Where rank_i(d) is d's position (1-indexed) in list i, and k is a smoothing constant. Candidates that don't appear in list i contribute nothing to that sum. Top-K after fusion = the final result.

Two properties matter:

1. It doesn't require the two lists to share a score scale. BM25 scores are unbounded positive numbers; cosine similarity is bounded [-1, 1]. If we averaged raw scores, BM25's big numbers would dominate. RRF only uses ranks, so scale disagreements disappear.
2. The constant k=60 that everyone uses — the original 2009 paper settled on 60 empirically. It's a tuning knob that controls how much weight bottom-ranked hits get. Smaller k = more weight to the top few; larger k = flatter distribution. 60 is a sweet spot for TREC-style retrieval and turns out to be pretty universal.

Concretely, if a memory is #1 in the BM25 list and #3 in the cosine list, its fused score is 1/(60+1) + 1/(60+3) = 0.0164 + 0.0159 = 0.0323. A memory that's #1 in both lists gets 1/61 + 1/61 = 0.0328. A memory that's #1 in cosine but doesn't appear in BM25 at all gets 0.0164. Being in both lists (even at moderate rank) usually beats being #1 in only one.

The implementation on Cloudflare Workers

Gnosem's storage is D1 (SQLite at the edge) + Vectorize (vector DB) + Workers AI (embedding model). Adding hybrid search meant adding a keyword index alongside the existing embeddings.

SQLite has excellent full-text search via FTS5, which is available in D1. Our migration created a contentless FTS5 virtual table backed by triggers on the primary memories table:

CREATE VIRTUAL TABLE memories_fts USING fts5(
  content, content_optimized,
  content='memories', content_rowid='rowid'
);

-- keep it in sync
CREATE TRIGGER memories_ai AFTER INSERT ON memories BEGIN
  INSERT INTO memories_fts(rowid, content, content_optimized)
  VALUES (new.rowid, new.content, new.content_optimized);
END;
-- similar triggers for UPDATE and DELETE

On every memory_search:

1. Embed the query via Workers AI (BGE-base-en-v1.5, one call, ~10ms).
2. Fire two queries in parallel with Promise.all:
   • VECTORIZE.query(vec, { topK: 20, filter: { user_id } })
   • SELECT id FROM memories JOIN memories_fts ON memories_fts.rowid = memories.rowid WHERE memories_fts MATCH ? AND user_id = ? LIMIT 20
3. Blend both ranked lists via RRF.
4. Hydrate the top-K memory rows from D1 in one round-trip.

Total end-to-end: 50–150ms depending on region.

The mode arg

Not every caller wants hybrid. A tool that already knows the user wants exact-string retrieval (looking up an ID) can request pure keyword. A tool doing broad "what have I thought about X?" recall can prefer pure semantic. We added a mode argument:

• mode: "hybrid" (default) — RRF blend
• mode: "semantic" — Vectorize only (the old behavior)
• mode: "keyword" — FTS5 only

Most callers should leave the default. It's the strictly better choice for open-ended queries.

The gotchas we hit

FTS5 special characters break the query. FTS5 has its own mini-query-language: "phrase match", +required, NEAR/5, etc. If a user query contains any of those characters raw, the parser throws. We wrap every user query in double quotes and escape internal double quotes:

const ftsQuery = '"' + raw.replaceAll('"', '""') + '"';

This forces FTS5 to treat the whole query as a phrase / bag of tokens. Costs us some flexibility (users can't type FTS5 syntax on purpose) but eliminates a whole class of query-parse crashes.

FTS5 keys by rowid, our primary key is a UUID. We could have declared content='memories', content_rowid='rowid' — SQLite auto-generates rowid separately from our id column. Then joining back to memories was a JOIN ... ON memories_fts.rowid = memories.rowid. That's what we shipped, and it works, but it means the FTS index and the main table always share the same physical rowid. Backfill of existing memories on migration used a single INSERT INTO memories_fts SELECT rowid, content, content_optimized FROM memories.

Empty FTS results. If the query has zero keyword hits (e.g. a purely semantic query like "how does this feel?"), the FTS list is empty. RRF handles this naturally — every candidate's rank in an empty list is undefined, contributes nothing. The final ranking is effectively 100% semantic. No special case needed.

What we considered and skipped

Cross-encoder reranking. Take the top 20 from each side, feed all 40 into a cross-encoder that scores query-doc pairs jointly, take the top K. Cross-encoders are the state-of-the-art for retrieval quality but each pair costs an inference call. At 40 pairs per query on Workers AI, that's another 200–500ms of latency and 40× the compute cost. Skipped for now; may add as an opt-in mode: "rerank" later.

Weighted BM25 + cosine averaging with sigmoid normalization. Instead of RRF, normalize both scores into a [0,1] range and blend with a weight. This works but requires tuning the weight per corpus. RRF just works out of the box.

Query expansion (LLM-generates 3 synonymous queries, run all three, blend results). Genuine quality lift but triples the search cost. Skipped.

If you use Gnosem via MCP, the default hybrid search kicks in on every memory_search call unless you explicitly pass mode: "semantic" or mode: "keyword". If you want to see the difference, run the same query three times against the demo store with each mode and compare — the hybrid results are consistently the ones you'd actually pick.