vibe.review.classifier

Requirement classification using LLM evaluation.

Implements two-step classification: 1. Relevance check: Is the document part relevant to the requirement? 2. Compliance check: Does the relevant part satisfy the requirement?

The classifier uses few-shot examples from similar previously-reviewed documents to improve consistency and accuracy.

PartClassification

Classification result for a single document part.

total_tokens 

total_tokens: int

Sum of prompt and completion tokens for this part.

RequirementClassificationResult

Complete classification result for a requirement.

total_tokens 

total_tokens: int

Sum of prompt and completion tokens across all parts.

RequirementClassifier

Classifies document parts against requirements.

Uses a two-step process: 1. Relevance: Filter parts that are relevant to the requirement 2. Compliance: Classify relevant parts as Y/N/P

Usage

classifier = RequirementClassifier(session, llm_client) result = classifier.classify( requirement=req, parts=search_results, examples=few_shot_examples, )

__init__ 

__init__(session: Session, llm_client: BaseLLMClient | None = None, example_retriever: ExampleRetriever | None = None, skip_relevance_check: bool = False, orchestration: Literal['batched', 'per_part'] = 'batched') -> None

Initialize the classifier.

Parameters:
  • session (Session) –

    SQLAlchemy session.

  • llm_client (BaseLLMClient | None, default: None ) –

    LLM client for classification. If None, creates mock.

  • example_retriever (ExampleRetriever | None, default: None ) –

    Retriever for few-shot examples. If None, creates one.

  • skip_relevance_check (bool, default: False ) –

    If True, skip relevance and go straight to compliance.

  • orchestration (Literal['batched', 'per_part'], default: 'batched' ) –

    Batch strategy for requirement evaluation.

classify 

classify(requirement: Requirement, parts: list[Any], context: PromptContext | None = None, max_parts_to_classify: int = 5) -> RequirementClassificationResult

Classify document parts against a requirement.

Parameters:
  • requirement (Requirement) –

    The requirement to evaluate.

  • parts (list[Any]) –

    Ranked document parts (best matches first).

  • context (PromptContext | None, default: None ) –

    Pre-built prompt context, or None to build from requirement.

  • max_parts_to_classify (int, default: 5 ) –

    Maximum parts to run through compliance check.
Returns:

close 

close() -> None

Close LLM client and example retriever if owned by this classifier.

classify_requirement 

classify_requirement(requirement: Requirement, parts: list[Any], session: Session | None = None) -> RequirementClassificationResult

Perform a one-off classification.

Create classifier, classify, and clean up. For multiple classifications, use RequirementClassifier directly.