vibe.review.llm¶
LLM client for document classification.
Provides both mock and real LLM clients for requirement classification. Mock client is used for testing and development; real clients integrate with Berget API or other OpenAI-compatible endpoints.
Design Note: Separate Implementation from vibe.llm_providers¶
This module intentionally provides its own LLM clients rather than reusing the providers from vibe.llm_providers (used by Assistant). The rationale:
-
Module Separation: Per the implementation plan (Section 1.7), Review is designed as a separate package (vibe-review) that depends on Core but avoids deep coupling. Core's llm_providers are tightly integrated with Assistant's conversation model (streaming, tool calling, multi-turn).
-
Different Use Cases:
- Assistant: Interactive conversation with streaming responses, tool calling, multi-turn context, complex message history management
-
Review: Single-shot classification with structured JSON output, no streaming, no tools, no conversation history
-
Structured Output: Review requires strict JSON schema enforcement via the
response_formatparameter for deterministic classification results. This is a specific feature that Review uses extensively but Assistant doesn't need in the same way. -
Simpler Interface: Review's
classify()method is purpose-built for classification tasks: system prompt + user prompt → structured response. No need for the complexity of conversation management. -
Dependency Footprint: This implementation uses httpx directly rather than the openai SDK, keeping Review's dependencies minimal.
If Review's needs grow closer to Assistant's (e.g., multi-turn classification discussions), consider refactoring to share infrastructure. For now, the ~100 lines of client code is simpler than adapting Core's providers.
See Also: - vibe.llm_providers.openai.OpenAIProvider (Core's provider for Assistant) - vibe.embedding_providers (shared embedding abstraction) - vibe.rerank_providers (shared reranking abstraction)
RelevanceResult ¶
Result of relevance classification.
ComplianceResult ¶
Result of compliance classification.
ClassificationResponse ¶
Structured response from LLM classification.
StructuredClassificationResponse ¶
Structured JSON response from LLM with metadata.
LLMClientConfig ¶
Configuration for LLM client.
BaseLLMClient ¶
Abstract base class for LLM clients.
classify ¶
classify(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> ClassificationResponse
Send classification request to LLM.
| Parameters: |
|
|---|
| Returns: |
|
|---|
classify_structured ¶
classify_structured(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> StructuredClassificationResponse
Send a structured output request to the LLM and return the parsed JSON.
Subclasses that support arbitrary schemas should override this.
MockLLMClient ¶
Mock LLM client for testing and development.
Features: - Deterministic responses based on input hash - Configurable default responses - Call tracking for test assertions - Specific response injection for tests - Question-specific response injection for question answering tests
set_question_response ¶
set_question_response(question_id: str, answer: str | bool | list[str] | None, confidence: str = 'M', reasoning: str = 'Mock reasoning', supporting_part_ids: list[str] | None = None, needs_user_input: bool = False) -> None
Set a specific response for a question ID in question answering.
Used for testing AI-suggested question answers.
| Parameters: |
|
|---|
classify ¶
classify(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> ClassificationResponse
Generate mock classification response.
classify_structured ¶
classify_structured(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> StructuredClassificationResponse
Generate mock structured response for arbitrary schemas.
set_response ¶
set_response(system_prompt: str, user_prompt: str, response: ClassificationResponse) -> None
Set specific response for a prompt combination.
set_response_for_text ¶
set_response_for_text(text_contains: str, response: ClassificationResponse) -> None
Set response for any prompt containing specific text.
OpenAICompatibleClient ¶
LLM client for OpenAI-compatible APIs (Berget, vLLM, etc.).
Requires structured output support from the model.
classify ¶
classify(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> ClassificationResponse
Send classification request to OpenAI-compatible API.
classify_structured ¶
classify_structured(system_prompt: str, user_prompt: str, response_schema: dict[str, Any]) -> StructuredClassificationResponse
Send structured output request to OpenAI-compatible API.
enable_prompt_logging ¶
enable_prompt_logging(log_dir: str | Path | None = None) -> Path
Enable logging of LLM prompts to files.
| Parameters: |
|
|---|
| Returns: |
|
|---|
configure_mock_llm ¶
configure_mock_llm(deterministic: bool = True, default_classification: str = 'Y', default_confidence: str = 'M', default_reasoning: str = 'Mock classification reasoning.', responses: dict[str, dict[str, Any]] | None = None) -> None
Configure the mock LLM client behavior.
| Parameters: |
|
|---|
set_mock_response ¶
set_mock_response(prompt_hash: str, response: ClassificationResponse) -> None
Set a specific response for a prompt hash.
create_llm_client ¶
create_llm_client(provider: str = 'mock', **kwargs: object) -> BaseLLMClient
Create an LLM client.
| Parameters: |
|
|---|
| Returns: |
|
|---|