Document Scoring Methods

When aggregating chunk-level search results into document-level scores, LocalVectorDB supports four scoring methods plus "auto", which is the default and picks between them based on search_type. Each has a different strength depending on your use case.

Note

Earlier releases exposed seven additional heuristic methods (worst, weighted_average, harmonic_mean, diminishing_returns, statistical, robust_mean, geometric_mean). They were removed in v0.1.0: measured on the NFCorpus retrieval benchmark none of them beat the methods below, they overlapped heavily, and several were non-monotonic (adding a low-scoring chunk could reorder documents). Passing a removed name now raises ValueError. percentile was pruned in the same pass but has since returned in a simplified single-knob form (see below) — its old two-percentile blend is gone for good.

Methods

"auto" (Default)

Picks the aggregator from search_type: "best" for search_type="vector", "frequency_boost" for "hybrid" and "keyword". Passing any explicit method pins it and disables this behaviour.

The two differ because of the scale the chunk scores arrive on, not the corpus. Hybrid and keyword min-max normalise each leg within the query’s own candidate pool, so the best chunk is 1.0 by construction and frequency_boost’s count multiplier acts on a bounded, query-relative scale. Vector search passes a raw bounded similarity through unchanged, where the same multiplier mostly rewards documents for owning more chunks. Measured on the vector leg, "best" beat "frequency_boost" by +0.0226 nDCG@10 on BEIR SciFact, +0.0150 on Natural Questions and +0.0084 on qasper.

A related result decided the section path: aggregation is a property of the unit being ranked, not of the corpus. Across 20 measured corpus/target/leg/pool cells a summing aggregator never lost on a document target and never won on a section target, so section-level roll-up keeps a plain maximum and exposes no knob.

"best"

Uses the highest-scoring chunk as the document score. Choose this when you want documents ranked by their single most relevant passage, regardless of overall document quality.

Parameters: None

"average"

Takes the arithmetic mean of all chunk scores. Good for documents where overall content quality matters more than peak relevance.

Parameters: None

"frequency_boost"

Boosts the best chunk score based on the number of quality chunks found, rewarding documents with multiple relevant passages. Ideal for comprehensive documents where breadth of coverage indicates relevance.

Parameters:

• frequency_bias (0.0-1.0, default=0.3): Controls how much to boost scores based on chunk frequency. Higher values favor documents with more matching chunks.

"percentile"

A soft maximum: the document score is the p-th order statistic of its chunk scores. At percentile=1.0 it is exactly "best"; lower values drift toward the mean as a document owns more chunks, so a document whose relevance is spread across several strong chunks can outrank one carried by a single outlier.

This is a document-target aggregator. Measured against "best" when rolling chunks up to documents it was worth up to ~+0.02 nDCG@10 on high-fanout corpora (many chunks per document), and it consistently lost at section targets — which is why "auto" never selects it. Pin it explicitly via document_scoring_method="percentile" if your workload matches.

Parameters:

• percentile (0.0-1.0, default=0.9): The order statistic to take, passed via document_scoring_options (0.9 means the 90th percentile; values outside [0, 1] raise ValueError). Interpolation matches numpy.percentile’s default linear rule, which is what the sweep behind this option measured.

Note

The percentile method that shipped before v0.1.0 was a two-percentile blend. That form lost to this clean order statistic in 19 of 20 measured cells, so its extra options do not come back with it: secondary_percentile and primary_weight in document_scoring_options are silently ignored.

Choosing a Method

• Not sure: leave the default "auto"

• Single best passage matters most: Use "best"

• Overall document quality important: Use "average"

• Want to reward multiple relevant sections: Use "frequency_boost"

• Document-level ranking on a high-fanout corpus, want a soft max: Use "percentile"

How Raw Scores Are Computed

Before document-level aggregation is applied, each chunk receives a raw similarity score between 0.0 and 1.0. The normalization depends on the search type.

Vector Search

FAISS returns raw distances which are converted to similarity scores by _distance_to_similarity:

• Inner Product (IP) index: similarity = (distance + 1) / 2, clamped to [0, 1]. This mapping assumes an inner product in [-1, 1], i.e. unit-norm vectors. The library guarantees that by L2-normalizing at the write and query boundary whenever the index metric is inner product, so IP scoring is correct regardless of whether the embedding provider’s own normalize option is set. (No normalization is applied to an L2 index, so its geometry – and the normalize option’s effect on it – is unchanged.)

• L2 index: similarity = 1 / (1 + distance). Larger distances map to lower similarity, approaching 0 for very distant vectors.

Keyword Search

Keyword search uses SQLite FTS5 with the BM25 ranking function. FTS5 BM25 scores are negative values where more negative means a better match. These are converted to similarity scores using an exponential mapping:

similarity = 1.0 - min(1.0, exp(rank))

This produces scores in [0, 1] where better BM25 matches yield higher similarity. Ranking is unaffected by the shape of this curve, because FTS5 orders by the raw BM25 score before the mapping is applied.

Note

This mapping saturates. Any reasonably good BM25 match lands within about 2e-05 of 1.0, and past a rank of roughly -36 it reaches exactly 1.0. Treat the absolute value of a keyword score as “matched”, not as a measure of how well. Hybrid fusion therefore normalizes the raw BM25 rank, never this number.

Hybrid Search

Hybrid search runs vector and keyword searches independently, then fuses them with relative-score fusion: each leg’s scores are min-max normalized within the current query’s candidate pool, and the normalized values are blended:

v = (vector_score - min_vector) / (max_vector - min_vector)
k = (-bm25 - min(-bm25)) / (max(-bm25) - min(-bm25))

final_score = vector_weight * v + (1 - vector_weight) * k

Normalizing first is what makes vector_weight (default 0.5) an actual blend. The two legs are otherwise on incompatible, corpus-dependent scales – a bounded similarity against raw BM25 – and summing them directly lets whichever leg happens to span the wider range decide the ranking. Chunks appearing in only one result set receive 0.0 for the missing component. The fused scores are then filtered by score_threshold and passed to document-level aggregation.

Warning

Hybrid scores are relative to the query’s own candidate pool. They are comparable within a single result set, but not across queries, and not across different values of k (which changes the pool size). A score_threshold on a hybrid query therefore selects by rank position within the pool rather than by absolute match quality. Two further consequences: the best chunk of a leg always normalizes to 1.0, and the worst normalizes to 0.0 – indistinguishable from a chunk that leg never retrieved at all.

Using Scoring Methods via the Server API

All search endpoints accept document_scoring_method and document_scoring_options in the request body. These parameters are forwarded directly to the local database’s query() method.

# Unified query endpoint
curl -X POST http://localhost:8000/api/v1/databases/my_db/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "machine learning",
    "search_type": "hybrid",
    "return_type": "documents",
    "k": 10,
    "score_threshold": 0.3,
    "vector_weight": 0.5,
    "document_scoring_method": "frequency_boost",
    "document_scoring_options": {
      "frequency_bias": 0.3
    }
  }'

The convenience endpoints (/search/vector, /search/keyword, /search/hybrid) also accept these parameters with the same schema.

Note

document_scoring_method and document_scoring_options only take effect when return_type is "documents". For chunk-level return types ("chunks", "context", "enriched"), raw chunk scores are returned directly.