localvectordb.embeddings module

Plugin-based embedding providers for LocalVectorDB v1.0

This module provides a flexible embedding system with support for multiple providers through a registry pattern.

class localvectordb.embeddings.ModelPrefixes(document: str = '', query: str = '')

Bases: NamedTuple

The instruction prefixes an asymmetric embedding model expects.

document is prepended to text being embedded for storage; query is prepended to text being embedded to search with. Either may be empty for a model that only instructs one side (most BERT-era retrievers only prefix the query) or for a symmetric model that wants no prefix at all.

Create new instance of ModelPrefixes(document, query)

document: str

Alias for field number 0

query: str

Alias for field number 1

localvectordb.embeddings.resolve_model_prefixes(model: str) → ModelPrefixes

Look up the retrieval prefixes a model was trained with.

Returns empty prefixes for any model not in the registry – an unknown model is assumed symmetric rather than guessed at.

Parameters:

model (str) – Model identifier, with or without a registry path or version tag.

Returns:

The document-side and query-side prefixes for the model.

Return type:

ModelPrefixes

class localvectordb.embeddings.EmbeddingProvider(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, max_batch_tokens: int | None = None, document_prefix: str | None = None, query_prefix: str | None = None, auto_prefix: bool = True, **kwargs: Any)

Bases: ABC

Abstract base class for embedding providers.

Parameters:

  • model (str) – Model identifier passed to the provider.

  • max_batch_tokens (int, optional) – Ceiling on the estimated token volume of a single request, applied on top of max_batch_size. Lower it when requests time out; pass 0 to disable the token cap and batch purely by count. Defaults to the provider’s own value.

  • document_prefix (str, optional) – Instruction prefix prepended to text embedded for storage. When None (the default) it is taken from the model’s known training prefix; pass "" to force no prefix.

  • query_prefix (str, optional) – Instruction prefix prepended to text embedded as a search query. Same None/"" semantics as document_prefix.

  • auto_prefix (bool, default True) – Whether to look prefixes up by model name when neither prefix is given. Set False to opt out of the registry entirely.

__init__(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, max_batch_tokens: int | None = None, document_prefix: str | None = None, query_prefix: str | None = None, auto_prefix: bool = True, **kwargs: Any) → None
document_prefix: str
query_prefix: str
property async_supported: bool
property max_batch_tokens: int | None

Estimated token ceiling for one request, or None for no token cap.

Applied alongside max_batch_size, not instead of it: whichever binds first ends the batch. A provider that charges or rate-limits per request, rather than per second of compute, can return None here and keep batching by count.

property uses_prefixes: bool

Whether either side of this provider prepends an instruction prefix.

prefix_for(task: Literal['document', 'query']) → str

Return the instruction prefix for task (‘document’ or ‘query’).

apply_prefix(texts: List[str], task: Literal['document', 'query']) → List[str]

Prepend this provider’s task prefix to each text.

Returns texts unchanged when the prefix is empty, so a symmetric model pays nothing for this path.

Idempotent: a text that already opens with either of this model’s known prefixes is left alone. Checking both sides matters, because the damaging case is cross-task rather than same-task – a caller that has already applied the query prefix and then embeds through a document path would otherwise produce "title: none | text: task: search result | query: ...", putting a query in document space, which is the exact asymmetry the prefixes exist to create. Same-task doubling is milder but equally pointless. The skip is warned about once per provider rather than done silently, since it means the caller is also managing prefixes and one of the two layers is redundant.

A document that genuinely opens with a prefix string is left unprefixed. That is rare, and far cheaper than the alternative failure.

async embed_batch(texts: List[str], batch_size: int | None = None, progress_callback: Callable | None = None, *, task: Literal['document', 'query'] = 'document') → ndarray

Generate embeddings with automatic retry handling.

task selects which instruction prefix is applied and defaults to "document"; search paths must pass task="query" for an asymmetric model to rank correctly. For a batch of queries prefer embed_queries_async() / embed_queries(), which cannot be reached with the wrong task by accident.

async embed_async(texts: List[str], batch_size: int | None = None, *, task: Literal['document', 'query'] = 'document') → ndarray

Generate embeddings for a list of texts.

async embed_query_async(query: str) → ndarray

Embed a single search query, applying the query-side prefix.

Returns a 1-D (dimension,) vector, not a batch of one.

async embed_queries_async(queries: List[str], batch_size: int | None = None) → ndarray

Embed many search queries, applying the query-side prefix to each.

The batch entry points (embed_batch(), embed_async(), embed_sync()) default to task="document", so bulk-embedding queries through them silently applies the DOCUMENT prefix and collapses the query/document asymmetry – with no error and no warning, just quietly worse ranking. Use this instead of passing task="query" by hand.

abstractmethod get_dimension() → int

Get the embedding dimension for this model

abstractmethod validate_model() → bool

Check if the model is available/valid

abstract property provider_name: str

Return the provider name

abstract property max_batch_size: int

Maximum batch size for this provider

property context_tokens: int | None

The encoder’s effective context window, in tokens, or None if unknown.

Deliberately separate from max_input_tokens, which is a client-side truncation policy: a provider that returns a value here is describing the encoder, not asking us to pre-truncate. Conflating the two would give Ollama a truncation pass driven by tiktoken’s cl100k_base, which is not the tokenizer any Ollama model uses.

Read by localvectordb.database._span_embed to size the windows a long section is split into. When every provider returned None that code fell back to a fixed 24,000-char (~6,860-token) window, which overflows a 2,048-token encoder by ~3.3x – so each window was silently truncated, defeating the windowing it exists to perform.

“Effective” is load-bearing: report the smallest binding ceiling, not the architectural maximum. Under-reporting only makes windows smaller than necessary; over-reporting reintroduces the silent truncation.

embed_sync(texts: List[str], batch_size: int | None = None, *, task: Literal['document', 'query'] = 'document') → ndarray

Synchronous wrapper for embed_batch with proper event loop handling.

embed_query(query: str) → ndarray

Embed a single search query synchronously, applying the query-side prefix.

Returns a 1-D (dimension,) vector, not a batch of one.

embed_queries(queries: List[str], batch_size: int | None = None) → ndarray

Embed many search queries synchronously, applying the query-side prefix.

Synchronous counterpart to embed_queries_async(). Prefer this over embed_sync(queries), which defaults to task="document" and would prefix a query as though it were a passage.

class localvectordb.embeddings.HTTPEmbeddingProvider(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, base_url: str | None = None, **kwargs: Any)

Bases: EmbeddingProvider, ABC

Embedding Providers which utilize HTTP requests to get embeddings.

Subclasses need to implement _embed_single_batch(self, texts: list[str], client: httpx.AsyncClient) which provides an async httpx client to use to make the http request.

property max_input_tokens: int | None

Maximum input tokens per text. Override in subclasses.

Returns None if no limit is enforced at the provider level.

__init__(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, base_url: str | None = None, **kwargs: Any)
base_url: str | None
class localvectordb.embeddings.OllamaEmbeddings(model: str, *, timeout: int = 300, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 3, base_url: str | None = None, requested_dimensions: int | None = None, normalize: bool = False, num_ctx: int | None = None, num_batch: int | None = None, truncate: bool = True, **kwargs: Any)

Bases: HTTPEmbeddingProvider

Ollama embedding provider.

Parameters:

  • model (str) – The OpenAI model to use for embeddding

  • base_url (str) – The base url for the ollama server (default http://127.0.0.1:11434, matching Ollama’s default bind address). Alternatively, you can set the OLLAMA_URL environment variable.

  • timeout (int, default = 90) – Timeout in seconds for the http request

  • max_retries (int, default = 3) – How many times to retry on a failed request.

  • retry_delay (float, default = 1.0) – How long to delay after a failed request (the backoff is exponential)

  • max_concurrent_requests (int, default = 3) – How many requests to make concurrently to the ollama server.

  • num_ctx (int, optional) – Context window (options.num_ctx) to request from Ollama. Ollama loads each model at a default num_ctx (often 2048) regardless of the model’s nominal maximum, and silently truncates longer inputs; set this to embed longer inputs in full (e.g. 8192 for bge-m3). When None, Ollama’s per-model default applies.

  • num_batch (int, optional) – Batch size (options.num_batch -> llama.cpp n_batch). This is the real ceiling for embeddings: an encoder (non-causal) model embeds the whole input in a single batch, so an input longer than n_batch (default 2048) is silently truncated regardless of num_ctx. Raising num_ctx alone therefore does nothing for embeddings past 2048 – n_batch (undocumented for /api/embed) must move too. When None but num_ctx is set, this defaults to num_ctx so a raised context actually takes effect; pass an explicit value to decouple them.

  • truncate (bool, default = True) – Whether Ollama truncates an input that exceeds the batch/context window. The default (True) matches Ollama’s own default and avoids a hard error on an over-long input; set False to make an over-long input fail loudly.

__init__(model: str, *, timeout: int = 300, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 3, base_url: str | None = None, requested_dimensions: int | None = None, normalize: bool = False, num_ctx: int | None = None, num_batch: int | None = None, truncate: bool = True, **kwargs: Any) → None
num_batch: int | None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

property context_tokens: int | None

Effective per-input token ceiling, derived from the server. No policy.

Four ceilings can bind, and the smallest one wins:

  • the architecture’s <family>.context_length from model_info;

  • the model card’s own num_ctx / num_batch PARAMETER lines;

  • whatever the caller passed for num_ctx / num_batch;

  • _DEFAULT_NUM_BATCH when nothing else pins the batch.

Taking the minimum is not defensive coding, it is required. nomic-embed-text ships num_ctx 8192 against an architectural context of 2,048 – trusting the model card alone would overstate its window 4x and truncate three quarters of every long window. And num_batch is a real ceiling for embeddings, not a throughput knob: an encoder model embeds its whole input in one batch, so an input longer than n_batch is truncated regardless of num_ctx.

Checks out against the one value we measured directly: embeddinggemma:300m reports 2,048 on all three sources, and bisecting Ollama’s real truncation boundary for that model also landed on 2,048.

validate_model() → bool

Check if the model is available in Ollama

get_dimension() → int

Get embedding dimension by making a test call

class localvectordb.embeddings.OpenAIEmbeddings(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, base_url: str | None = None, requested_dimensions: int | None = None, normalize: bool = False, **kwargs: Any)

Bases: HTTPEmbeddingProvider

OpenAI embedding provider.

Parameters:

  • model (str) – The OpenAI model to use for embedding

  • api_key (str, optional) – Optionally provide the api key as a str. If not provided, tries to use “OPENAI_API_KEY” environment variable. You can specify a custom environment variable to use by prefixing with a “$”, for example using: apikey=”$CUSTOM_ENV_VAR” would try to load the api key from the CUSTOM_ENV_VAR environment variable.

  • timeout (int, default = 90) – Timeout in seconds for the http request

  • max_retries (int, default = 3) – How many times to retry on a failed request.

  • retry_delay (float, default = 1.0) – How long to delay after a failed request (the backoff is exponential)

  • max_concurrent_requests (int, default = 5) – How many requests to make concurrently to the OpenAI server.

DEFAULT_BASE_URL = 'https://api.openai.com/v1'
__init__(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, base_url: str | None = None, requested_dimensions: int | None = None, normalize: bool = False, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

property max_input_tokens: int

Maximum input tokens per text for this model.

validate_model() → bool

Check if the model exists

get_dimension() → int

Get embedding dimension

class localvectordb.embeddings.OpenRouterEmbeddings(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, base_url: str | None = None, dimension: int | None = None, requested_dimensions: int | None = None, normalize: bool = False, site_url: str | None = None, app_name: str | None = None, **kwargs: Any)

Bases: HTTPEmbeddingProvider

OpenRouter embedding provider (OpenAI-compatible).

OpenRouter exposes a single OpenAI-compatible endpoint at https://openrouter.ai/api/v1 that routes embedding requests to many upstream providers (OpenAI, Google, Mistral, Nvidia, …). Any embedding model listed at https://openrouter.ai/models (filter by the embedding output modality) can be used by passing its slug as model – for example "openai/text-embedding-3-small" or a free model such as "nvidia/nv-embed-v2".

Because the set of models is large and changes over time, dimensions are not hard-coded. Resolution order for the index dimension (first match wins): requested_dimensions (also asks the API to truncate to that size) -> dimension (a plain declaration of the model’s native size, no payload effect) -> a one-off probe request the first time the dimension is needed. Provide dimension (or requested_dimensions) to avoid the probe entirely – useful for offline setup or to keep database creation from making a network call.

Parameters:

  • model (str) – OpenRouter model slug (e.g. "openai/text-embedding-3-small").

  • api_key (str, optional) – API key. If omitted, read from the OPENROUTER_API_KEY environment variable. A "$OTHER_VAR" value reads from that environment variable instead. Get a key at https://openrouter.ai/keys

  • base_url (str, optional) – Override the API base URL. Defaults to https://openrouter.ai/api/v1.

  • dimension (int, optional) – Declare the model’s native embedding dimension. Used as the index dimension and skips the probe; unlike requested_dimensions it does not alter the request payload. Cannot disagree with requested_dimensions if both are given.

  • requested_dimensions (int, optional) – Request this output dimension (only honored by models that support Matryoshka truncation) and use it as the index dimension. When omitted, the native dimension is probed unless dimension is given.

  • normalize (bool, default False) – Apply L2 normalization to returned embeddings.

  • site_url (str, optional) – Sent as the HTTP-Referer header for OpenRouter attribution (optional).

  • app_name (str, optional) – Sent as the X-Title header for OpenRouter attribution (optional).

  • timeout (int, default = 90) – HTTP timeout (seconds).

  • max_retries (int, default = 3) – Automatic retry attempts.

  • retry_delay (float, default = 1.0) – Base delay for exponential backoff.

  • max_concurrent_requests (int, default = 5) – Concurrent requests to the OpenRouter API.

DEFAULT_BASE_URL = 'https://openrouter.ai/api/v1'
__init__(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, base_url: str | None = None, dimension: int | None = None, requested_dimensions: int | None = None, normalize: bool = False, site_url: str | None = None, app_name: str | None = None, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Return the embedding dimension, probing the API once if unknown.

class localvectordb.embeddings.OpenAICompatibleEmbeddings(model: str, *, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, dimension: int | None = None, requested_dimensions: int | None = None, normalize: bool = False, max_batch_size: int = 64, **kwargs: Any)

Bases: HTTPEmbeddingProvider

Any server exposing an OpenAI-compatible /v1/embeddings endpoint.

This is one provider rather than one per runtime, because llama.cpp’s server, LM Studio, vLLM, text-embeddings-inference, LocalAI and Jan all speak the same wire format – only the address differs. Point base_url at the server:

Server

Typical base_url

llama.cpp (--embedding)

http://localhost:8080/v1

LM Studio

http://localhost:1234/v1

vLLM

http://localhost:8000/v1

text-embeddings-inference

http://localhost:8080/v1

Ollama is supported natively by OllamaEmbeddings, which is preferable – it can enumerate installed models – but Ollama’s own /v1 shim works here too.

Because the served model set is open-ended, the model name is trusted and dimensions are not hard-coded. Resolution order for the index dimension (first match wins): requested_dimensions (also asks the server to truncate) -> dimension (declares the native size, no payload effect) -> a one-off probe request the first time the dimension is needed. Declare dimension to keep database creation from making a network call.

Parameters:

  • model (str) – Model name as the server reports it. For llama.cpp this is often ignored (it serves whatever was loaded), but it is still recorded as part of the database’s embedding identity, so use the real name.

  • base_url (str) – Required. The OpenAI-compatible root, including the /v1 suffix if the server uses one. There is no default: guessing an endpoint would silently embed against the wrong server.

  • api_key (str, optional) – Sent as a bearer token when provided. Most local servers need no key, so this is optional – unlike OpenAIEmbeddings. If omitted, read from OPENAI_COMPATIBLE_API_KEY; a "$OTHER_VAR" value reads that variable instead. When there is no key, no Authorization header is sent at all, since some servers reject a malformed one.

  • dimension (int, optional) – Declare the model’s native embedding dimension and skip the probe.

  • requested_dimensions (int, optional) – Ask for this output dimension (only honored by models supporting Matryoshka truncation) and use it as the index dimension.

  • normalize (bool, default False) – Apply L2 normalization to returned embeddings. Useful when a server returns unnormalized vectors and the index uses inner product.

  • max_batch_size (int, default 64) – Texts per request. Conservative because local servers are configured with much smaller batch ceilings than hosted APIs; raise it if your server allows more.

  • timeout (int, default = 90) – HTTP timeout (seconds). Local CPU inference can be slow – raise this if large batches time out.

  • max_retries (int, default = 3) – Automatic retry attempts.

  • retry_delay (float, default = 1.0) – Base delay for exponential backoff.

  • max_concurrent_requests (int, default = 5) – Concurrent requests. Most local servers are single-GPU or CPU-bound and gain nothing above 1-2.

__init__(model: str, *, base_url: str | None = None, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, dimension: int | None = None, requested_dimensions: int | None = None, normalize: bool = False, max_batch_size: int = 64, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Return the embedding dimension, probing the server once if unknown.

class localvectordb.embeddings.GoogleEmbeddings(model: str = 'gemini-embedding-001', *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, task_type: Literal['semantic_similarity', 'classification', 'clustering', 'retrieval_document', 'retrieval_query', 'code_retrieval_query', 'question_answering', 'fact_verification'] | None = None, document_task_type: str | None = None, query_task_type: str | None = None, requested_dimensions: int | None = None, normalize: bool = True, base_url: str | None = None, **kwargs: Any)

Bases: HTTPEmbeddingProvider

Google AI (Gemini) embedding provider using the Generative Language API.

Parameters:

  • model (str, default "gemini-embedding-001") – The Google AI embedding model, e.g.: - “gemini-embedding-001” (stable) - “gemini-embedding-exp-03-07” (experimental)

  • api_key (str, optional) – API key string or an env var reference (e.g., “$GEMINI_API_KEY”). If not provided, tries env vars (in order): GEMINI_API_KEY, GOOGLE_API_KEY.

  • task_type (Literal, optional) – One of: {“semantic_similarity”, “classification”, “clustering”, “retrieval_document”, “retrieval_query”, “code_retrieval_query”, “question_answering”, “fact_verification”} See: https://ai.google.dev/gemini-api/docs/embeddings#supported-task-types Setting this forces the same task type on both sides, which turns off the asymmetric retrieval default. Use it for non-retrieval workloads (clustering, classification); for search, leave it unset.

  • document_task_type (str, optional) – Task type to send when embedding for storage. Google’s API takes an explicit task type instead of a text prefix, so this is the provider-native form of document_prefix. Defaults to "retrieval_document", or to task_type if that was given.

  • query_task_type (str, optional) – Task type to send when embedding a search query. Defaults to "retrieval_query", or to task_type if that was given.

  • requested_dimensions (int, optional) – MRL-controlled output size (128–3072). Defaults to 3072 if not set by API. If provided, get_dimension() returns this value without a test call.

  • normalize (bool, default False) – If True, L2-normalize returned vectors (recommended for non-3072 outputs). For 3072 output, vectors are already normalized by the API.

  • base_url (str, optional) – Override the base API URL. Defaults to the public Google endpoint.

  • timeout (int, default 90) – Request timeout in seconds.

  • max_retries (int, default 3) – Retry attempts on transient errors.

  • retry_delay (float, default 1.0) – Base delay between retries (exponential backoff).

  • max_concurrent_requests (int, default 5) – Concurrency for batch processing.

__init__(model: str = 'gemini-embedding-001', *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, task_type: Literal['semantic_similarity', 'classification', 'clustering', 'retrieval_document', 'retrieval_query', 'code_retrieval_query', 'question_answering', 'fact_verification'] | None = None, document_task_type: str | None = None, query_task_type: str | None = None, requested_dimensions: int | None = None, normalize: bool = True, base_url: str | None = None, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Validate the model by querying the models endpoint.

get_dimension() → int

Return embedding dimension, using API probe if needed.

class localvectordb.embeddings.JinaEmbeddings(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, task: str | None = 'auto', document_task: str | None = None, query_task: str | None = None, truncate: bool = False, late_chunking: bool = False, requested_dimensions: int | None = None, **kwargs: Any)

Bases: HTTPEmbeddingProvider

Jina AI embedding provider.

Parameters:

  • model (str) –

    The Jina model to use for embedding. Examples:

    • ”jina-embeddings-v4” (multimodal/multilingual, 2048 dims)

    • ”jina-embeddings-v3” (1024 dims)

    • ”jina-clip-v2” (1024 dims)

    • ”jina-code-embeddings-0.5b”

    • ”jina-code-embeddings-1.5b”

  • api_key (str, optional) – Optionally provide the API key as a str. If it starts with “$” and the rest is uppercase, the key will be read from that environment variable. Otherwise, defaults to JINA_API_KEY env var. Get your Jina AI API key for free: https://jina.ai/?sui=apikey

  • timeout (int, default = 90) – HTTP timeout (seconds)

  • max_retries (int, default = 3) – Automatic retry attempts

  • retry_delay (float, default = 1.0) – Base delay for exponential backoff

  • max_concurrent_requests (int, default = 5) – Concurrent requests to Jina API

  • body (Additional keyword arguments are passed through to the Jina Embeddings API request) –

    • embedding_type: str, default “float” (other options: “base64”, “binary”, “ubinary”)

    • task: str, e.g., for v4: “retrieval.query” | “retrieval.passage” |

      ”text-matching” | “code.query” | “code.passage”; for code models: “nl2code.query” | “nl2code.passage” | “code2code.query” | “code2code.passage” | “code2nl.query” | “code2nl.passage” | “code2completion.query” | “code2completion.passage” | “qa.query” | “qa.passage” Used for both sides unless document_task/query_task override it.

    • document_task: str, task sent when embedding for storage (e.g. “retrieval.passage”)

    • query_task: str, task sent when embedding a search query (e.g. “retrieval.query”)

    • dimensions: int, to truncate output embeddings to this size

    • truncate: bool

    • late_chunking: bool (v4)

    • return_multivector: bool (v4; not supported by this provider, will raise if True)

    • normalized: bool (v3)

  • example (for) –

    • embedding_type: str, default “float” (other options: “base64”, “binary”, “ubinary”)

    • task: str, e.g., for v4: “retrieval.query” | “retrieval.passage” |

      ”text-matching” | “code.query” | “code.passage”; for code models: “nl2code.query” | “nl2code.passage” | “code2code.query” | “code2code.passage” | “code2nl.query” | “code2nl.passage” | “code2completion.query” | “code2completion.passage” | “qa.query” | “qa.passage” Used for both sides unless document_task/query_task override it.

    • document_task: str, task sent when embedding for storage (e.g. “retrieval.passage”)

    • query_task: str, task sent when embedding a search query (e.g. “retrieval.query”)

    • dimensions: int, to truncate output embeddings to this size

    • truncate: bool

    • late_chunking: bool (v4)

    • return_multivector: bool (v4; not supported by this provider, will raise if True)

    • normalized: bool (v3)

  • Behavior

  • --------

  • default (- By)

  • vectors. (embeddings are returned as float)

  • dimensions (- If you provide) –

    1. tell the API to output that dimension; and

    2. use that value to pre-allocate output arrays.

  • both (this provider will) –

    1. tell the API to output that dimension; and

    2. use that value to pre-allocate output arrays.

  • provided (- If no dimensions is) – Otherwise, dimension is determined via a one-off probe request.

  • (v4=2048 (well-known models use known sizes) – Otherwise, dimension is determined via a one-off probe request.

  • v3=1024 – Otherwise, dimension is determined via a one-off probe request.

  • clip-v2=1024). – Otherwise, dimension is determined via a one-off probe request.

__init__(model: str, *, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, api_key: str | None = None, task: str | None = 'auto', document_task: str | None = None, query_task: str | None = None, truncate: bool = False, late_chunking: bool = False, requested_dimensions: int | None = None, **kwargs: Any) → None
requested_dimensions: int | None
truncate: bool
task: str | None
document_task: str | None
query_task: str | None
late_chunking: bool
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Try a lightweight probe to confirm the model is usable.

get_dimension() → int

Return embedding dimension. Uses known sizes, user-requested dimensions, or probes via API.

class localvectordb.embeddings.SentenceTransformerEmbeddings(model: str, *, device: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, trust_remote_code: bool = False, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any)

Bases: EmbeddingProvider

SentenceTransformers embedding provider for local inference.

Parameters:

  • model (str) – The SentenceTransformer model name (e.g., “all-MiniLM-L6-v2”).

  • device (str, optional) – Device for inference (cpu/cuda/mps/auto). Default: auto-detect.

  • normalize (bool) – Whether to L2-normalize embeddings. Default: True.

  • requested_dimensions (int, optional) – Truncate embeddings to this dimension (Matryoshka support).

  • trust_remote_code (bool) – Whether to trust remote code when loading models. Default: False.

__init__(model: str, *, device: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, trust_remote_code: bool = False, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_input_tokens: int | None

The model’s context window, from its own max_seq_length.

Read by localvectordb.database._span_embed to size the windows a long section is split into before mean-pooling. Without it that code falls back to a fixed 24,000-char (~6,857-token) window, and every window is then silently truncated by the model: for all-MiniLM-L6-v2 (max_seq_length 256) a 24.5k-char section had 7.3% of its text encoded and the rest discarded.

This class does not inherit HTTPEmbeddingProvider’s _validate_and_truncate_texts, so the value only sizes span windows – it does not add a truncation pass to ordinary chunk embedding, which sentence-transformers already handles internally.

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Get the embedding dimension for this model

class localvectordb.embeddings.HuggingFaceInferenceEmbeddings(model: str, *, api_key: str | None = None, base_url: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, **kwargs: Any)

Bases: HTTPEmbeddingProvider

HuggingFace Inference API embedding provider.

Parameters:

  • model (str) – HuggingFace model ID (e.g., “BAAI/bge-small-en-v1.5”).

  • api_key (str, optional) – API key. Falls back to HF_TOKEN / HUGGINGFACE_TOKEN env vars.

  • base_url (str, optional) – Base URL. Default: https://api-inference.huggingface.co

  • normalize (bool) – Whether to L2-normalize embeddings. Default: True.

  • requested_dimensions (int, optional) – Truncate embeddings to this dimension.

__init__(model: str, *, api_key: str | None = None, base_url: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, max_concurrent_requests: int = 5, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Get the embedding dimension for this model

class localvectordb.embeddings.HuggingFaceLocalEmbeddings(model: str, *, device: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, trust_remote_code: bool = False, pooling_strategy: str = 'mean', timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any)

Bases: EmbeddingProvider

Local HuggingFace transformers embedding provider.

Parameters:

  • model (str) – HuggingFace model ID (e.g., “BAAI/bge-small-en-v1.5”).

  • device (str, optional) – Device for inference (cpu/cuda/mps). Default: auto-detect.

  • normalize (bool) – Whether to L2-normalize embeddings. Default: True.

  • requested_dimensions (int, optional) – Truncate embeddings to this dimension.

  • trust_remote_code (bool) – Whether to trust remote code. Default: False.

  • pooling_strategy (str) – Pooling strategy: “mean”, “cls”, or “max”. Default: “mean”.

__init__(model: str, *, device: str | None = None, normalize: bool = True, requested_dimensions: int | None = None, trust_remote_code: bool = False, pooling_strategy: str = 'mean', timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_input_tokens: int | None

The model’s context window, from the tokenizer’s model_max_length.

Same purpose as the sentence-transformers property of this name: it sizes the windows localvectordb.database._span_embed splits a long section into, so the section is pooled in full rather than truncated to a fixed 24,000-char guess.

HuggingFace uses a sentinel model_max_length (a very large int) to mean “unspecified”; treat anything implausible as unknown so the caller falls back to its default rather than trusting a bogus window.

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Get the embedding dimension for this model

class localvectordb.embeddings.MockEmbeddings(model: str, dimension: int = 384, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any)

Bases: EmbeddingProvider

Mock embedding provider for testing.

Prefix auto-detection is off by default here. Mock vectors are seeded from a hash of the text, so an auto-detected prefix would make the query vector for “foo” unrelated to the document vector for “foo” purely because the model name happened to match the registry – turning a symmetric test double asymmetric. Pass document_prefix/query_prefix explicitly (or auto_prefix=True) to exercise prefix behaviour deliberately.

__init__(model: str, dimension: int = 384, timeout: int = 90, max_retries: int = 3, retry_delay: float = 1.0, **kwargs: Any) → None
property provider_name: str

Return the provider name

property max_batch_size: int

Maximum batch size for this provider

validate_model() → bool

Check if the model is available/valid

get_dimension() → int

Get the embedding dimension for this model

class localvectordb.embeddings.EmbeddingRegistry

Bases: object

Registry for embedding providers with plugin discovery

classmethod register(name: str, provider_class: Type[EmbeddingProvider]) → None

Register a new embedding provider

classmethod get(name: str) → Type[EmbeddingProvider]

Get an embedding provider by name

classmethod create_provider(provider_name: str, model: str, **kwargs: Any) → EmbeddingProvider

Create an embedding provider instance

classmethod list() → List[str]

List all registered providers

classmethod refresh_plugins() → None

Force re-discovery of plugins (useful for testing)

localvectordb.embeddings.create_embedding_provider(provider: str, model: str, **kwargs: Any) → EmbeddingProvider

Create an embedding provider instance

localvectordb.embeddings.list_providers() → List[str]

List available embedding providers

async localvectordb.embeddings.embed_texts(texts: List[str], provider: str, model: str, batch_size: int | None = None, *, task: Literal['document', 'query'] = 'document', **provider_kwargs: Any) → ndarray

Convenience function to embed texts

localvectordb.embeddings.embed_texts_sync(texts: List[str], provider: str, model: str, batch_size: int | None = None, *, task: Literal['document', 'query'] = 'document', **provider_kwargs: Any) → ndarray

Synchronous convenience function to embed texts