localvectordb.validation package

The fact-checking (“reverse RAG”) module — see Fact-Checking (Reverse RAG) for the guide.

Fact-checking / validation module for LocalVectorDB.

Provides “reverse RAG” – verify LLM-generated text against documents stored in one or more LocalVectorDB instances.

Quick start:

from localvectordb.validation import FactChecker

checker = FactChecker(databases=[db], llm=anthropic_client)
result = checker.check("The policy allows 10 days PTO per year.")

print(result.overall_score)
print(result.annotated_text)

class localvectordb.validation.FactChecker(databases: LocalVectorDB | list[LocalVectorDB], llm: LLMProvider | Any, model: str | None = None, similarity_threshold: float = 0.3, min_grounding_score: float = 0.7, search_type: str = 'hybrid', top_k: int = 5, max_concurrent: int = 5)

Bases: object

Fact-check LLM-generated text against one or more LocalVectorDB instances.

Parameters:

databases – One or more LocalVectorDB instances to search for evidence.
llm – An Anthropic, OpenAI, or Google GenAI client, or any object implementing the LLMProvider protocol.
model – Model name passed to the LLM provider for claim extraction and polarity classification. Defaults are provider-specific (Haiku for Anthropic, gpt-4o-mini for OpenAI, gemini-2.0-flash for Gemini).
similarity_threshold – Minimum similarity score for a retrieved chunk to be considered relevant.
min_grounding_score – Minimum polarity confidence for a claim to count as grounded.
search_type – Search mode used when querying the databases ("vector", "keyword", or "hybrid").
top_k – Number of chunks to retrieve per claim per database.
max_concurrent – Maximum number of claims to process concurrently.

__init__(databases: LocalVectorDB | list[LocalVectorDB], llm: LLMProvider | Any, model: str | None = None, similarity_threshold: float = 0.3, min_grounding_score: float = 0.7, search_type: str = 'hybrid', top_k: int = 5, max_concurrent: int = 5) → None

async check_async(text: str, sources: list[str] | None = None) → FactCheckResult

Fact-check text asynchronously.

Parameters:

text – The LLM-generated text to verify.
sources – Optional list of document IDs that were used to generate text. When provided, these are searched first; the full database is only queried when no supporting evidence is found or a contradiction is detected.

check(text: str, sources: list[str] | None = None) → FactCheckResult: Synchronous wrapper around check_async().

class localvectordb.validation.FactCheckResult(claims: list[ClaimResult] = <factory>, overall_score: float = 0.0, has_contradictions: bool = False, citation_text: str = '', annotated_text: str | None = None)

Bases: object

Aggregate result of fact-checking a text against one or more databases.

claims: list[ClaimResult]

overall_score: float = 0.0

has_contradictions: bool = False

citation_text: str = ''

annotated_text: str | None = None

__init__(claims: list[ClaimResult] = <factory>, overall_score: float = 0.0, has_contradictions: bool = False, citation_text: str = '', annotated_text: str | None = None) → None

class localvectordb.validation.ClaimResult(claim: str, grounded: bool, confidence: float, source_id: str | None = None, source_excerpt: str | None = None, contradiction: bool = False, polarity: Polarity | None = None, similarity: float | None = None, original_sentence: str | None = None, database_name: str | None = None)

Bases: object

Result of checking a single factual claim against sources.

claim: str

grounded: bool

confidence: float

source_id: str | None = None

source_excerpt: str | None = None

contradiction: bool = False

polarity: Polarity | None = None

similarity: float | None = None

original_sentence: str | None = None

database_name: str | None = None

__init__(claim: str, grounded: bool, confidence: float, source_id: str | None = None, source_excerpt: str | None = None, contradiction: bool = False, polarity: Polarity | None = None, similarity: float | None = None, original_sentence: str | None = None, database_name: str | None = None) → None