localvectordb.reranking module

Reranking providers for LocalVectorDB.

This module provides cross-encoder and API-based reranking to improve search result quality by re-scoring candidates with more powerful models.

class localvectordb.reranking.Reranker(model: str, *, timeout: int = 90, max_retries: int = 3, **kwargs: Any)

Bases: ABC

Abstract base class for reranking providers.

__init__(model: str, *, timeout: int = 90, max_retries: int = 3, **kwargs: Any) → None
abstractmethod rerank(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results synchronously.

Parameters:

  • query (str) – The original query text.

  • results (List[QueryResult]) – Search results to rerank.

  • top_k (int, optional) – Maximum number of results to return. If None, returns all.

Returns:

Reranked results with updated scores, best first.

Return type:

List[QueryResult]

Notes

Every provider writes two metadata keys and leaves result.score holding an absolute [0, 1] relevance score – one that means the same thing regardless of the other candidates in the batch, so it survives score_threshold filtering and cross-query comparison:

  • metadata["original_score"] – the pre-rerank search score.

  • metadata["rerank_raw_score"] – the reranker model’s raw output before the provider’s [0, 1] mapping.

The mapping is provider-specific because the raw scores are:

  • Jina – the API returns a native [0, 1] relevance score; used as-is.

  • SentenceTransformers / HuggingFace – a cross-encoder logit, squashed with a logistic sigmoid. This is deliberately not a per-batch min-max: min-max is pool-relative (top always 1.0, bottom always 0.0) and would break score_threshold exactly as the pre-T1.1 hybrid fusion did.

  • OpenAI-compatible / OpenRouter – a self-hosted or routed server may return either, so the mapping is decided per score: values already in [0, 1] pass through and anything else is squashed. Override with score_transform when you know which your server produces.

  • Mock – word-overlap fraction, already in [0, 1].

async rerank_async(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results asynchronously. Default delegates to sync.

abstract property provider_name: str

Return the reranker provider name.

abstractmethod validate_model() → bool

Check if the model is available/valid.

class localvectordb.reranking.HTTPRerankerBase(model: str, *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, score_transform: str = 'auto', **kwargs: Any)

Bases: Reranker

Shared machinery for the /rerank wire format Jina popularised.

The same request shape – {model, query, documents, top_n} answered with per-document {index, score} pairs – is served by Jina, vLLM, text-embeddings-inference, Infinity and OpenRouter. Only the address, the credential and the exact spelling of the response differ, so subclasses override those and inherit the request, retry and scoring behaviour.

Response shapes accepted, because servers disagree on all three axes:

  • {"results": [...]} (Jina, vLLM, OpenRouter), {"data": [...]}, or a bare top-level list (text-embeddings-inference).

  • relevance_score or score for the value.

  • Scores already in [0, 1], or raw cross-encoder logits.

That last one matters more than it looks. The module contract is that result.score is an absolute [0, 1] relevance, because score_threshold filtering and cross-query comparison depend on it. A server returning logits would silently break both, so score_transform defaults to squashing anything that falls outside [0, 1].

DEFAULT_BASE_URL: str | None = None

Endpoint root used when the caller gives no base_url. None means required.

API_KEY_ENV: str | None = None

Environment variable consulted when no api_key is passed.

REQUIRES_API_KEY: bool = False

Whether construction fails without a credential.

DISPLAY_NAME: str = 'Reranker'

Human-readable name used in log and error messages.

MISSING_KEY_MESSAGE: str = 'An API key is required.'

Message raised when a required key is absent.

__init__(model: str, *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, score_transform: str = 'auto', **kwargs: Any) → None
validate_model() → bool

Check if the model is available/valid.

property endpoint: str
rerank(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results synchronously.

Parameters:

  • query (str) – The original query text.

  • results (List[QueryResult]) – Search results to rerank.

  • top_k (int, optional) – Maximum number of results to return. If None, returns all.

Returns:

Reranked results with updated scores, best first.

Return type:

List[QueryResult]

Notes

Every provider writes two metadata keys and leaves result.score holding an absolute [0, 1] relevance score – one that means the same thing regardless of the other candidates in the batch, so it survives score_threshold filtering and cross-query comparison:

  • metadata["original_score"] – the pre-rerank search score.

  • metadata["rerank_raw_score"] – the reranker model’s raw output before the provider’s [0, 1] mapping.

The mapping is provider-specific because the raw scores are:

  • Jina – the API returns a native [0, 1] relevance score; used as-is.

  • SentenceTransformers / HuggingFace – a cross-encoder logit, squashed with a logistic sigmoid. This is deliberately not a per-batch min-max: min-max is pool-relative (top always 1.0, bottom always 0.0) and would break score_threshold exactly as the pre-T1.1 hybrid fusion did.

  • OpenAI-compatible / OpenRouter – a self-hosted or routed server may return either, so the mapping is decided per score: values already in [0, 1] pass through and anything else is squashed. Override with score_transform when you know which your server produces.

  • Mock – word-overlap fraction, already in [0, 1].

async rerank_async(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results asynchronously. Default delegates to sync.

class localvectordb.reranking.JinaReranker(model: str = 'jina-reranker-v2-base-multilingual', **kwargs: Any)

Bases: HTTPRerankerBase

Jina AI reranker using the Jina Reranker API.

Parameters:

  • model (str) – The Jina reranker model. Default: “jina-reranker-v2-base-multilingual”

  • api_key (str, optional) – API key. Falls back to JINA_API_KEY env var.

  • base_url (str, optional) – Override the API root. Defaults to https://api.jina.ai/v1. Useful for a proxy; to reach a self-hosted reranker prefer the openai_compatible provider, which does not require a credential.

  • timeout (int) – Request timeout in seconds.

  • max_retries (int) – Number of retry attempts.

DEFAULT_BASE_URL: str | None = 'https://api.jina.ai/v1'

Endpoint root used when the caller gives no base_url. None means required.

API_KEY_ENV: str | None = 'JINA_API_KEY'

Environment variable consulted when no api_key is passed.

REQUIRES_API_KEY: bool = True

Whether construction fails without a credential.

DISPLAY_NAME: str = 'Jina'

Human-readable name used in log and error messages.

MISSING_KEY_MESSAGE: str = 'Jina API key is required. Set JINA_API_KEY environment variable. Get your key at: https://jina.ai/?sui=apikey'

Message raised when a required key is absent.

__init__(model: str = 'jina-reranker-v2-base-multilingual', **kwargs: Any) → None
property provider_name: str

Return the reranker provider name.

class localvectordb.reranking.OpenAICompatibleReranker(model: str, *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, score_transform: str = 'auto', **kwargs: Any)

Bases: HTTPRerankerBase

Any self-hosted server exposing a Jina/Cohere-shaped /rerank endpoint.

The reranking counterpart to OpenAICompatibleEmbeddings, so a fully local stack does not have to fall back to a hosted API for its second stage.

Server

Typical base_url

vLLM

http://localhost:8000/v1

text-embeddings-inference

http://localhost:8080

Infinity

http://localhost:7997
Parameters:

  • model (str) – Model name as the server reports it.

  • base_url (str) – Required. The rerank root; /rerank is appended. Note that text-embeddings-inference serves it at the root rather than under /v1.

  • api_key (str, optional) – Sent as a bearer token when present. Optional – most local servers need none. Falls back to RERANKER_API_KEY.

  • score_transform ({"auto", "none", "sigmoid"}, default "auto") – How to map raw scores into the absolute [0, 1] range the rest of the library assumes. "auto" passes through values already in range and squashes anything else, which is what makes a logit-returning cross-encoder safe to use with score_threshold.

API_KEY_ENV: str | None = 'RERANKER_API_KEY'

Environment variable consulted when no api_key is passed.

DISPLAY_NAME: str = 'OpenAI-compatible'

Human-readable name used in log and error messages.

property provider_name: str

Return the reranker provider name.

class localvectordb.reranking.OpenRouterReranker(model: str, *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, score_transform: str = 'auto', **kwargs: Any)

Bases: HTTPRerankerBase

OpenRouter reranker, routing to the rerank models it hosts.

Parameters:

  • model (str) – OpenRouter model slug for a rerank-capable model.

  • api_key (str, optional) – Falls back to OPENROUTER_API_KEY. Get a key at https://openrouter.ai/keys

  • base_url (str, optional) – Defaults to https://openrouter.ai/api/v1.

DEFAULT_BASE_URL: str | None = 'https://openrouter.ai/api/v1'

Endpoint root used when the caller gives no base_url. None means required.

API_KEY_ENV: str | None = 'OPENROUTER_API_KEY'

Environment variable consulted when no api_key is passed.

REQUIRES_API_KEY: bool = True

Whether construction fails without a credential.

DISPLAY_NAME: str = 'OpenRouter'

Human-readable name used in log and error messages.

MISSING_KEY_MESSAGE: str = 'OpenRouter API key is required. Set the OPENROUTER_API_KEY environment variable or pass api_key=. Get a key at https://openrouter.ai/keys'

Message raised when a required key is absent.

property provider_name: str

Return the reranker provider name.

class localvectordb.reranking.SentenceTransformersReranker(model: str = 'BAAI/bge-reranker-base', *, device: str | None = None, timeout: int = 90, max_retries: int = 3, **kwargs: Any)

Bases: Reranker

Cross-encoder reranker using sentence-transformers.

Parameters:

  • model (str) –

    The cross-encoder model name. Default: “BAAI/bge-reranker-base”.

    The default was cross-encoder/ms-marco-MiniLM-L-6-v2, which measured statistically indistinguishable from not reranking at all on two corpora (qasper −0.0022 n.s., NQ −0.0083) — and the failure is the MS MARCO training recipe, not model capacity: ms-marco-electra-base, on bge’s exact backbone geometry, is just as flat. bge-reranker-base measured +0.0297 (p<.05) on the same harness. Avoid any MS MARCO cross-encoder here, and do not truncate input below 512 tokens — bge loses two thirds of its gain at 256.

  • device (str, optional) – Device for inference (cpu/cuda/mps). Default: auto-detect.

__init__(model: str = 'BAAI/bge-reranker-base', *, device: str | None = None, timeout: int = 90, max_retries: int = 3, **kwargs: Any) → None
property provider_name: str

Return the reranker provider name.

validate_model() → bool

Check if the model is available/valid.

rerank(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results synchronously.

Parameters:

  • query (str) – The original query text.

  • results (List[QueryResult]) – Search results to rerank.

  • top_k (int, optional) – Maximum number of results to return. If None, returns all.

Returns:

Reranked results with updated scores, best first.

Return type:

List[QueryResult]

Notes

Every provider writes two metadata keys and leaves result.score holding an absolute [0, 1] relevance score – one that means the same thing regardless of the other candidates in the batch, so it survives score_threshold filtering and cross-query comparison:

  • metadata["original_score"] – the pre-rerank search score.

  • metadata["rerank_raw_score"] – the reranker model’s raw output before the provider’s [0, 1] mapping.

The mapping is provider-specific because the raw scores are:

  • Jina – the API returns a native [0, 1] relevance score; used as-is.

  • SentenceTransformers / HuggingFace – a cross-encoder logit, squashed with a logistic sigmoid. This is deliberately not a per-batch min-max: min-max is pool-relative (top always 1.0, bottom always 0.0) and would break score_threshold exactly as the pre-T1.1 hybrid fusion did.

  • OpenAI-compatible / OpenRouter – a self-hosted or routed server may return either, so the mapping is decided per score: values already in [0, 1] pass through and anything else is squashed. Override with score_transform when you know which your server produces.

  • Mock – word-overlap fraction, already in [0, 1].

class localvectordb.reranking.HuggingFaceReranker(model: str = 'BAAI/bge-reranker-v2-m3', *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, **kwargs: Any)

Bases: Reranker

HuggingFace Inference API reranker.

Parameters:

  • model (str) – HuggingFace model ID. Default: “BAAI/bge-reranker-v2-m3”

  • api_key (str, optional) – API key. Falls back to HF_TOKEN / HUGGINGFACE_TOKEN env vars.

  • base_url (str, optional) – API base URL. Default: https://api-inference.huggingface.co

__init__(model: str = 'BAAI/bge-reranker-v2-m3', *, api_key: str | None = None, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, **kwargs: Any) → None
property provider_name: str

Return the reranker provider name.

validate_model() → bool

Check if the model is available/valid.

rerank(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results synchronously.

Parameters:

  • query (str) – The original query text.

  • results (List[QueryResult]) – Search results to rerank.

  • top_k (int, optional) – Maximum number of results to return. If None, returns all.

Returns:

Reranked results with updated scores, best first.

Return type:

List[QueryResult]

Notes

Every provider writes two metadata keys and leaves result.score holding an absolute [0, 1] relevance score – one that means the same thing regardless of the other candidates in the batch, so it survives score_threshold filtering and cross-query comparison:

  • metadata["original_score"] – the pre-rerank search score.

  • metadata["rerank_raw_score"] – the reranker model’s raw output before the provider’s [0, 1] mapping.

The mapping is provider-specific because the raw scores are:

  • Jina – the API returns a native [0, 1] relevance score; used as-is.

  • SentenceTransformers / HuggingFace – a cross-encoder logit, squashed with a logistic sigmoid. This is deliberately not a per-batch min-max: min-max is pool-relative (top always 1.0, bottom always 0.0) and would break score_threshold exactly as the pre-T1.1 hybrid fusion did.

  • OpenAI-compatible / OpenRouter – a self-hosted or routed server may return either, so the mapping is decided per score: values already in [0, 1] pass through and anything else is squashed. Override with score_transform when you know which your server produces.

  • Mock – word-overlap fraction, already in [0, 1].

class localvectordb.reranking.MockReranker(model: str = 'mock-reranker', *, timeout: int = 90, max_retries: int = 3, **kwargs: Any)

Bases: Reranker

Mock reranker for testing. Uses word-overlap scoring.

__init__(model: str = 'mock-reranker', *, timeout: int = 90, max_retries: int = 3, **kwargs: Any) → None
property provider_name: str

Return the reranker provider name.

validate_model() → bool

Check if the model is available/valid.

rerank(query: str, results: List[QueryResult], top_k: int | None = None) → List[QueryResult]

Rerank search results synchronously.

Parameters:

  • query (str) – The original query text.

  • results (List[QueryResult]) – Search results to rerank.

  • top_k (int, optional) – Maximum number of results to return. If None, returns all.

Returns:

Reranked results with updated scores, best first.

Return type:

List[QueryResult]

Notes

Every provider writes two metadata keys and leaves result.score holding an absolute [0, 1] relevance score – one that means the same thing regardless of the other candidates in the batch, so it survives score_threshold filtering and cross-query comparison:

  • metadata["original_score"] – the pre-rerank search score.

  • metadata["rerank_raw_score"] – the reranker model’s raw output before the provider’s [0, 1] mapping.

The mapping is provider-specific because the raw scores are:

  • Jina – the API returns a native [0, 1] relevance score; used as-is.

  • SentenceTransformers / HuggingFace – a cross-encoder logit, squashed with a logistic sigmoid. This is deliberately not a per-batch min-max: min-max is pool-relative (top always 1.0, bottom always 0.0) and would break score_threshold exactly as the pre-T1.1 hybrid fusion did.

  • OpenAI-compatible / OpenRouter – a self-hosted or routed server may return either, so the mapping is decided per score: values already in [0, 1] pass through and anything else is squashed. Override with score_transform when you know which your server produces.

  • Mock – word-overlap fraction, already in [0, 1].

class localvectordb.reranking.RerankerRegistry

Bases: object

Registry for reranker providers with plugin discovery.

classmethod register(name: str, provider_class: Type[Reranker]) → None

Register a new reranker provider.

classmethod get(name: str) → Type[Reranker]

Get a reranker provider by name.

classmethod create_reranker(provider_name: str, model: str | None = None, **kwargs: Any) → Reranker

Create a reranker instance.

classmethod list() → List[str]

List all registered reranker providers.

classmethod refresh_plugins() → None

Force re-discovery of plugins (useful for testing).

localvectordb.reranking.create_reranker(provider: str, model: str | None = None, **kwargs: Any) → Reranker

Create a reranker instance.

localvectordb.reranking.list_rerankers() → List[str]

List available reranker providers.