vibe.review.assessment

Unified AI assessment for questions and requirements.

This module provides a single entry point for AI-assisted document analysis, handling both requirement compliance checks and question answering with the same search → rerank → classify pipeline.

The key insight is that questions and requirements follow the same pattern: 1. Search for relevant document parts 2. Rerank to get the best candidates 3. Ask LLM to analyze (classify compliance OR answer question) 4. Return structured result

By unifying these, questions get the same reranking quality as requirements, and the codebase has less duplication.

AssessmentItem

Unified representation of something to assess (question or requirement).

This provides a common interface for the assessment pipeline regardless of whether we're checking requirement compliance or answering a question.

from_requirement 

from_requirement(requirement: Requirement) -> AssessmentItem

Create AssessmentItem from a Requirement.

from_question 

from_question(question_id: str, question_def: dict[str, Any]) -> AssessmentItem

Create AssessmentItem from a question definition.

get_search_query 

get_search_query() -> str

Build search query from label and description.

AssessmentResult

Unified result from AI assessment.

For requirements: classification is set (YES/NO/PARTIAL/NOT_APPLICABLE) For questions: answer is set (the suggested value)

Both share confidence, reasoning, and matched_part_ids.

confidence_float 

confidence_float: float

Convert confidence letter to float (0.0-1.0).

from_requirement_result 

from_requirement_result(item_id: str, result: RequirementClassificationResult) -> AssessmentResult

Create from RequirementClassificationResult.

from_question_result 

from_question_result(item_id: str, result: QuestionAnswerResult) -> AssessmentResult

Create from QuestionAnswerResult.

AssessmentProgress

Progress information for streaming assessment.

to_sse_data 

to_sse_data() -> str

Format as SSE data payload.

AssessmentClassifier

Unified classifier for both questions and requirements.

Provides a single interface for AI assessment regardless of item type. Both questions and requirements go through the same pipeline:

  1. Search for relevant document parts (HybridSearcher)
  2. Rerank to prioritize best candidates (RerankProvider)
  3. Classify/Answer using LLM (RequirementClassifier or QuestionAnswerer)

This ensures questions get the same quality of document retrieval as requirements, including reranking for better context.

Usage

classifier = AssessmentClassifier(db_session, llm_client, ...) result = classifier.assess(item, document_parts)

__init__ 

__init__(db_session: Session, llm_client: BaseLLMClient | None = None, embedding_provider: EmbeddingProvider | None = None, rerank_provider: RerankProvider | None = None, example_retriever: ExampleRetriever | None = None, language: str = 'sv', skip_relevance_check: bool = False, max_parts: int = 10) -> None

Initialize the unified classifier.

Parameters:
  • db_session (Session) –

    SQLAlchemy session.

  • llm_client (BaseLLMClient | None, default: None ) –

    LLM client for classification. If None, creates default.

  • embedding_provider (EmbeddingProvider | None, default: None ) –

    Provider for embeddings. If None, auto-selects.

  • rerank_provider (RerankProvider | None, default: None ) –

    Provider for reranking. If None, auto-selects.

  • example_retriever (ExampleRetriever | None, default: None ) –

    Retriever for few-shot examples.

  • language (str, default: 'sv' ) –

    Language code for prompts ("sv" or "en").

  • skip_relevance_check (bool, default: False ) –

    If True, skip relevance step for requirements.

  • max_parts (int, default: 10 ) –

    Maximum document parts to include in context.

assess 

assess(item: AssessmentItem, parts: list[Any]) -> AssessmentResult

Assess an item (question or requirement) against document parts.

The parts should already be searched and reranked. This method handles the LLM classification based on item type.

Parameters:
  • item (AssessmentItem) –

    The assessment item (question or requirement).

  • parts (list[Any]) –

    Ranked document parts (best matches first).
Returns:

close 

close() -> None

Close owned resources.

assess_item 

assess_item(item: AssessmentItem, parts: list[Any], db_session: Session | None = None, llm_client: BaseLLMClient | None = None, language: str = 'sv') -> AssessmentResult

Perform a one-off assessment.

Create classifier, assess, and clean up. For multiple assessments, use AssessmentClassifier directly.