localvectordb.validation package

The fact-checking (“reverse RAG”) module — see Fact-Checking (Reverse RAG) for the guide.

Fact-checking / validation module for LocalVectorDB.

Provides “reverse RAG” – verify LLM-generated text against documents stored in one or more LocalVectorDB instances.

Quick start:

from localvectordb.validation import FactChecker

checker = FactChecker(databases=[db], llm=anthropic_client)
result = checker.check("The policy allows 10 days PTO per year.")

print(result.overall_score)
print(result.annotated_text)

class localvectordb.validation.FactChecker(databases: LocalVectorDB | list[LocalVectorDB], llm: LLMProvider | Any, model: str | None = None, similarity_threshold: float = 0.3, min_grounding_score: float = 0.7, search_type: str = 'hybrid', top_k: int = 5, max_concurrent: int = 5)

Bases: object

Fact-check LLM-generated text against one or more LocalVectorDB instances.

Parameters:

databases – One or more LocalVectorDB instances to search for evidence.
llm – An Anthropic, OpenAI, or Google GenAI client, or any object implementing the LLMProvider protocol.
model – Model name passed to the LLM provider for claim extraction and polarity classification. Defaults are provider-specific (Haiku for Anthropic, gpt-4o-mini for OpenAI, gemini-2.0-flash for Gemini).
similarity_threshold – Minimum similarity score for a retrieved chunk to be considered relevant.
min_grounding_score – Minimum polarity confidence for a claim to count as grounded.
search_type – Search mode used when querying the databases ("vector", "keyword", or "hybrid").
top_k – Number of chunks to retrieve per claim per database.
max_concurrent – Maximum number of claims to process concurrently.

__init__(databases: LocalVectorDB | list[LocalVectorDB], llm: LLMProvider | Any, model: str | None = None, similarity_threshold: float = 0.3, min_grounding_score: float = 0.7, search_type: str = 'hybrid', top_k: int = 5, max_concurrent: int = 5) → None

async check_async(text: str, sources: list[str] | None = None) → FactCheckResult

Fact-check text asynchronously.

Parameters:

text – The LLM-generated text to verify.
sources – Optional list of document IDs that were used to generate text. When provided, these are searched first; the full database is only queried when no supporting evidence is found or a contradiction is detected.

check(text: str, sources: list[str] | None = None) → FactCheckResult: Synchronous wrapper around check_async().

class localvectordb.validation.FactCheckResult(claims: list[ClaimResult] = <factory>, overall_score: float = 0.0, has_contradictions: bool = False, citation_text: str = '', annotated_text: str | None = None, error: str | None = None)

Bases: object

Aggregate result of fact-checking a text against one or more databases.

error is set only when the check could not be performed at all (e.g. the LLM provider failed during claim extraction). It is distinct from a genuine zero score: callers should treat a non-None error as “unverified”, never as “verified” or “refuted”.

claims: list[ClaimResult]

overall_score: float = 0.0

has_contradictions: bool = False

citation_text: str = ''

annotated_text: str | None = None

error: str | None = None

__init__(claims: list[ClaimResult] = <factory>, overall_score: float = 0.0, has_contradictions: bool = False, citation_text: str = '', annotated_text: str | None = None, error: str | None = None) → None

class localvectordb.validation.ClaimResult(claim: str, grounded: bool, confidence: float, source_id: str | None = None, source_excerpt: str | None = None, contradiction: bool = False, polarity: Polarity | None = None, similarity: float | None = None, original_sentence: str | None = None, database_name: str | None = None)

Bases: object

Result of checking a single factual claim against sources.

claim: str

grounded: bool

confidence: float

source_id: str | None = None

source_excerpt: str | None = None

contradiction: bool = False

polarity: Polarity | None = None

similarity: float | None = None

original_sentence: str | None = None

database_name: str | None = None

__init__(claim: str, grounded: bool, confidence: float, source_id: str | None = None, source_excerpt: str | None = None, contradiction: bool = False, polarity: Polarity | None = None, similarity: float | None = None, original_sentence: str | None = None, database_name: str | None = None) → None