LLM Provider Architecture¶

Architectural reference for VIBE's LLM provider abstraction layer. Optimized for LLM consumption.

See also:

• assistant.md - AI-assisted interview system (uses providers)
• review.md - Document review (uses separate LLM client)

1. OVERVIEW¶

The provider system decouples assistant logic from specific LLM APIs via a common streaming interface. All providers implement the same base class and message conversion pattern.

Key Insight: The assistant module and review module use separate LLM abstractions by design:

• Assistant: Streaming + tool-calling via LLMProvider (this doc)
• Review: Structured JSON output + single-shot classification via vibe/review/llm.py::BaseLLMClient

2. BASE PROVIDER¶

Location: vibe/providers/llm/base.py

LLMProvider abstract class defines:

• stream_generate() -- Returns Generator[StreamChunk]
• get_capabilities() -- Returns ProviderCapabilities (frozen dataclass: structured_output, streaming, tools, streaming_tools, chat)
• Message converter (via MessageConverter subclass)
• Session recording/playback (see Section 5)

ProviderConfig dataclass:

• model, temperature, max_tokens, timeout, api_key, base_url
• tools_config: bool | None -- Override provider's default tool support
• get_effective_tools_enabled() resolves: explicit config > provider capability default

ProviderWithConfig dataclass (returned by ProviderFactory.create()):

• provider: LLMProvider, endpoint_config: dict, endpoint_name: str

3. AVAILABLE PROVIDERS¶

SystemProxyProvider: Composition pattern -- on sequence 1 checks for pending system questions and emits as ask_question tool calls. On sequence 2+ delegates to real provider. Chunks marked proxy_generated=True.

4. MESSAGE CONVERSION¶

Location: vibe/providers/llm/message_converter.py

Each provider has a different message format. The MessageConverter base class uses @singledispatchmethod for type-based dispatch.

Converters:

• InternalFormatConverter -- Uses message_to_dict() for internal format
• IdentityConverter -- Returns Message objects unchanged (MockProvider)
• Provider-specific: OpenAIChatConverter, AnthropicMessageConverter, etc. (in respective modules)

Principle: Message classes remain pure data containers. Each provider defines its own converter without touching Message classes.

5. CONFIGURATION & REPLAY¶

Endpoints defined in config.yml:

llm_endpoints:
  gpt4:
    provider: "vibe.providers.llm.openai.OpenAIProvider"
    config:
      model: "gpt-4-turbo"
      api_key: "${OPENAI_API_KEY}"
      tools: true

Dev Mode Features:

• Endpoint switching via ?endpoint=...
• Tools toggle via ?tools=0
• Recording via ?record=name (saves JSONL to data/logs/assistant/)
• Playback via ?playback=name (no API calls)

Replay System: Records JSONL with request/response entries. Playback config:

config:
  playback_from_file: "data/logs/assistant/llm_20241201.jsonl"
  playback_session_id: "abc123"     # Optional filter
  playback_sequence: 2              # Optional specific turn
  playback_delay_ms: 50             # Simulate streaming delay

Each provider overrides _recorded_payload_to_native() to convert recorded JSON back to SDK-specific objects.

6. FILE LOCATION INDEX¶

Document Version: 1.0 Last Updated: 2026-02-02 Notes: Split from assistant.md Section 6