nr-llm
Architecture Decision Records
Every significant design decision in nr-llm is recorded as an ADR — the context, the decision, and its consequences. Browse the full set below.
89 decisions.
Provider & Resilience 10
- ADR 001 Provider Abstraction Layer Accepted Introduces a ProviderInterface contract plus optional capability interfaces (vision, streaming, tools, documents), an AbstractProvider base class, and LlmServiceManager as the unified entry point, so multiple LLM providers share one consistent API.
- ADR 007 Multi-Provider Strategy Accepted Registers providers as a tagged service collection ordered by priority so the manager can select and iterate providers by rank.
- ADR 008 Error Handling Strategy Accepted Defines a hierarchical exception system rooted at ProviderException (connection, response, configuration, unsupported-feature, fallback-exhausted) plus extension-level configuration and argument exceptions.
- ADR 018 Multi-Provider Model Discovery Accepted Abstracts model discovery behind ModelDiscoveryInterface (testConnection + discover) with per-adapter dispatch so each provider's model list and connection test share one contract.
- ADR 021 Provider Fallback Chain Accepted A configuration's ordered fallback_chain of configuration identifiers is walked by FallbackMiddleware on narrowly-defined retryable failures (connection errors, HTTP 429), returning the first success or throwing FallbackChainExhaustedException.
- ADR 022 Attribute-Based Provider Registration Accepted Adds a #[AsLlmProvider(priority)] attribute scanned by ProviderCompilerPass to auto-tag providers at compile time, keeping providers private while an explicit YAML tag still wins when both are present.
- ADR 026 Provider Middleware Pipeline Accepted Introduces a PSR-15-inspired provider middleware pipeline (ProviderMiddlewareInterface with an immutable ProviderCallContext and callable $next), replacing the earlier PSR-14 event system.
- ADR 034 Remove the ExtensionConfiguration Default-Provider Fallback Accepted Removes the defaultProvider concept from LlmServiceManager (state, seed, accessors) making the database the single source of truth for provider selection; a breaking change where getProvider(null) now throws.
- ADR 063 Provider Resilience — Circuit Breaker, Health, Idempotency Accepted Adds a per-provider CircuitBreakerMiddleware (opens after consecutive tripping failures, half-open probe after cooldown) placed innermost, tripping only on the same connection/429 failures FallbackMiddleware treats as retryable.
- ADR 080 Typed Provider HTTP Exceptions (Authentication 401, Rate-Limit 429) Accepted Adds ProviderAuthenticationException (401) and ProviderRateLimitException (429) as final subclasses of a de-finalized ProviderResponseException, chosen by a shared AbstractProvider factory so buffered and streaming 4xx branches never drift, preserving backward compatibility via inheritance.
Core Services & Architecture 16
- ADR 002 Feature Services Architecture Accepted Adds dedicated high-level feature services (Completion, Embedding, Vision, Translation) that use LlmServiceManager internally, expose domain-specific methods, handle caching, and return typed response objects.
- ADR 004 PSR-14 Event System Superseded Originally dispatched BeforeRequest/AfterResponse PSR-14 events from LlmServiceManager to let consumers modify requests and process responses; superseded by the provider middleware pipeline (ADR-026).
- ADR 005 TYPO3 Caching Framework Integration Accepted Integrates with TYPO3's caching framework using the nrllm_responses cache with no hardcoded backend, keying entries on provider+model+input hash, so deterministic calls (embeddings, temperature-0 completions) can be cached while streaming and tool calls are not.
- ADR 016 Thinking/Reasoning Block Extraction Accepted Extracts model reasoning via a two-tier strategy — native structured thinking blocks (Anthropic) plus a regex <think>-tag fallback — surfaced as an optional thinking property on CompletionResponse.
- ADR 017 Safe Type Casting via SafeCastTrait Accepted Provides a reusable SafeCastTrait with static toStr/toInt/toFloat helpers that coerce mixed input to sensible defaults without PHPStan suppressions.
- ADR 028 Public Services Policy Accepted Documents the deliberate public-service set in Services.yaml as a policy of load-bearing categories, enforced by a unit test that keeps the count honest; its count/tail rationale is later superseded by ADR-065.
- ADR 032 Specialized Usage Tracking and Pricing Catalog Accepted Makes specialized services report real usage units (images, characters, audio seconds, tokens) and adds a static OpenAI price catalog with a DB override so image/speech/transcription calls get accurate cost tracking.
- ADR 053 One Marker Interface for All Thrown Exceptions Accepted Adds NrLlmExceptionInterface (extending Throwable) implemented by every exception the extension throws, with value-object normalization errors now throwing the nr_llm InvalidArgumentException and a reflection test enforcing the marker.
- ADR 058 Telemetry Middleware Accepted Adds a TelemetryMiddleware at the outermost priority and a UI-less tx_nrllm_telemetry table writing exactly one immutable row per pipeline run (latency, cache-hit, fallback attempts, success/error) on both success and failure.
- ADR 059 Decompose LlmServiceManager into Focused Collaborators Accepted Stage-1 decomposition extracting self-contained groups into private autowired collaborators (KeyedProviderRegistry, ConfigurationResolver, ...) while the manager stays final and keeps its unchanged interface via thin delegation.
- ADR 065 Reduce the Public Service Surface (ADR-028 Follow-up) Accepted Keeps public: true only where genuinely required (documented API contract, interface alias, concrete-only surface, standalone consumer API, or makeInstance reuse) and privatises everything public solely for test resolution.
- ADR 076 Document Understanding — Native-First, Rasterize Fallback Accepted Adds a stateless DocumentAnalysisService that sends a whole PDF in one chat call as a Base64 document block when the provider supports it, falling back to page-by-page rasterization for providers without document support.
- ADR 081 Agent run persistence and a durable event stream Accepted Persists each tool-loop agent run and its steps in two UI-less log tables, so a run can be resumed after a restart, queued, paused for approval, audited or replayed — instead of living only for a single HTTP request.
- ADR 082 Schema-validated structured outputs with one repair round-trip Accepted completeJson() now validates results against a business JSON-Schema and runs one repair round-trip on a malformed response (reusing the evaluation subsystem's structural matcher), so AI output is reliable to program against.
- ADR 083 Conversation sessions and memory Accepted Adds an explicit, persisted session model — two UI-less log tables read via a raw-SQL repository — so multi-turn conversations keep their history server-side instead of each consumer re-assembling and re-sending it.
- ADR 090 One extension until 1.0, with documented split seams Accepted Keeps nr-llm as a single extension through the 1.0 line rather than splitting its subsystems (tools/RAG, skills, guardrails) into separate packages now, while documenting the seams along which a future split could occur so the option stays open.
Domain Model & Types 5
- ADR 003 Typed Response Objects Accepted Provider and service responses are returned as immutable readonly value objects (e.g. CompletionResponse) instead of loosely-typed arrays.
- ADR 006 Option Objects vs Arrays Superseded Introduced typed Option objects with fluent builders and factory presets (initially alongside array backward-compatibility) to replace ad-hoc option arrays; superseded by the object-only API (ADR-011).
- ADR 011 Object-Only Options API Accepted Removes array option support entirely; all methods take nullable typed Option objects (ChatOptions, EmbeddingOptions, VisionOptions) with factory presets.
- ADR 015 Type-Safe Domain Models via PHP 8.1+ Enums & Value Objects Accepted Uses PHP 8.1+ string-backed enums for all domain constants, each providing values()/isValid()/tryFromString() helpers plus domain methods like label() and getIconIdentifier().
- ADR 054 Typed Tool Turns on ChatMessage Instead of Wire Arrays Accepted Extends ChatMessage with two validated optional tail fields (toolCalls on assistant, toolCallId on tool) and named constructors, keeping it the single value object for all four roles instead of per-shape subclasses.
Configuration 9
- ADR 013 Three-Level Configuration Architecture (Provider-Model-Configuration) Accepted Separates concerns into three hierarchical levels — use-case Configuration references a Model, which belongs to a Provider — rather than embedding models in providers.
- ADR 031 Tagged Prompt Snippet Library Accepted Adds a lightweight PromptSnippet entity (identifier/name/description/fragment with free-form CSV tags) queried by case-insensitive exact tag tokens, separate from the heavier PromptTemplate.
- ADR 033 Specialized Models in the Model Registry Accepted Adds IMAGE/TEXT_TO_SPEECH/TRANSCRIPTION capabilities so image, TTS and transcription models are regular registry records, with fail-soft capability-based default-model resolution in the specialized services.
- ADR 055 Embeddings Join the Configuration Path; Dimensions Metadata Accepted Adds embedForConfiguration() mirroring the chat configuration path (adapter resolution, middleware, budget metadata, feature guard) with per-call options overriding stored defaults and cache keys keyed on the effective model.
- ADR 056 Configuration Presets — Consumer-Declared, Admin-Imported Records Accepted Consuming extensions declare needed configurations as presets via a DI tag (requirements/criteria only, never a provider/model/key); nr_llm lists undeclared-but-unimported presets as pending for one-click admin import.
- ADR 066 Criteria-Mode Configurations Resolve in the Service Layer Accepted getAdapterFromConfiguration() resolves the model via ModelSelectionService (direct model for fixed mode, criteria-selected for criteria mode) without writing the resolved model back onto the Extbase entity to avoid silently converting it to fixed-mode.
- ADR 069 Remove the Unusable PromptTemplate Stack Accepted Removes the never-usable PromptTemplate stack in full (entity, repository, service+interface, exception, table, tests/fixtures) since it had no TCA mapping and zero consumers, leaving Task's prompt_template string field and PromptSnippet untouched.
- ADR 070 User-less Configuration Resolution by Identifier Accepted Adds ConfigurationResolver::getActiveByIdentifier() throwing typed exceptions (NotFound/Inactive/AccessDenied) instead of returning null, and refuses beGroups-restricted records in user-less contexts.
- ADR 077 Plain Completion Joins the Named-Configuration Path Accepted Adds completeForConfiguration(prompt, configuration, options) mirroring chat's default-configuration branch (builds ChatMessages, injects skills, threads budget/idempotency metadata) taking typed ChatOptions rather than low-level arrays.
Backend & UI 7
- ADR 014 AI-Powered Wizard System Accepted Adds three backend wizards (Setup, Configuration, Task) that turn natural-language descriptions into structured configurations/task chains via WizardGeneratorService, with graceful fallback when no LLM is available.
- ADR 019 Internationalization Strategy Accepted Follows TYPO3 XLIFF conventions (one label file per backend module, with German translations) for static UI strings and adds locale-aware placeholder substitution for dynamic LLM interactions.
- ADR 020 Backend Output Format Rendering Accepted Stores raw LLM output and renders format (HTML/Markdown/JSON/text) entirely client-side, with an ephemeral, non-persisted toggle operating on the cached raw content.
- ADR 024 Dashboard Widgets Accepted Ships two dashboard widgets (monthly AI cost, requests-by-provider over 7 days) reusing TYPO3's built-in widget classes, registered conditionally only when the Dashboard system extension is present.
- ADR 027 Split TaskController Accepted Splits the monolithic TaskController into four focused per-pathway controllers (list, wizard, execution, ...) plus service extraction and uniform typed responses.
- ADR 029 Usage Analytics Dashboard Accepted Ships a read-only usage-analytics module backed by a richer usage table (adding model and per-side token columns) and real cost computation derived from model pricing in UsageMiddleware.
- ADR 040 Playground Run Trace and Tool-Path Prompt Augmentation Accepted Adds an opt-in RunTrace recorder to ToolLoopService capturing per-round messages, tokens, timing and thinking for the admin playground, plus one-time prompt assembly, leaving production callers unaffected.
Security, Privacy & Budgets 9
- ADR 012 API Key Encryption at Application Level Superseded Encrypted stored API keys at application level using sodium_crypto_secretbox with a key derived from TYPO3's encryptionKey; later superseded (keys now live in nr-vault as UUIDs).
- ADR 023 Native Backend Capability Permissions Accepted Registers every ModelCapability enum value as a native TYPO3 BE group custom permission, with CapabilityPermissionService resolving checks (CLI/admin allowed, otherwise delegating to the backend user's permission).
- ADR 025 Per-User AI Budgets Accepted Adds a tx_nrllm_user_budget table of per-user ceilings (requests/tokens/cost × daily/monthly), enforced as a pure pre-flight BudgetService::check() that aggregates usage on demand from the existing usage table without a second write.
- ADR 030 Specialized Services Authenticate Through nr-vault Accepted Migrates keyed specialized services onto the nr-vault secure HTTP client, storing an API-key identifier (vault UUID) instead of the key and using placement hooks instead of manual auth-header building.
- ADR 037 Backend AJAX Admin Guard Accepted Adds one shared RequiresBackendAdminTrait::denyNonAdmin() returning a 403 JSON response for non-admins, applied at the very top of every AJAX-routed backend action.
- ADR 052 Usage Attribution Honours the Caller-Supplied beUserUid Accepted Adds an optional trailing beUserUid to trackUsage() and has UsageMiddleware read the same pipeline-context key the budget gate uses, so usage attribution and budget enforcement can no longer disagree.
- ADR 057 Speech and Image Services Carry Attribution in Options Accepted Extends options-based attribution (ADR-052) to specialized services via BudgetAwareOptionsInterface (optional beUserUid/plannedCost kept out of toArray to avoid leaking to remote APIs) — attribution only, no enforcement.
- ADR 064 Central Privacy Model Accepted Introduces one central PrivacyLevel enum (none/metadata/redacted/full, default metadata) read from extension configuration and applied at every content sink before a write, combining with strictest-wins.
- ADR 078 Budget Pre-Flight for the Specialized Image/Speech Services Accepted Adds a protected enforceBudget() to AbstractSpecializedService mirroring BudgetMiddleware, called after option resolution and before any HTTP dispatch in the image, TTS and transcription services so limits are enforced pre-flight.
Guardrails & Safety 6
- ADR 084 Human-in-the-loop tool approval Accepted Adds a human approval gate for side-effecting tools: a tool opts in via the empty RequiresApprovalInterface marker, and the agent loop suspends on such a call by throwing a control-flow signal that carries the serialised run state, persists it (status WAITING_FOR_APPROVAL), and resumes on approval or denial — the 41 read-only tools stay unchanged.
- ADR 085 Guardrail pipeline for provider responses Accepted Introduces a content-policy layer over provider responses: a GuardrailInterface returns an ALLOW/REDACT/RETRY/REQUIRE_APPROVAL/DENY verdict, auto-collected via the nr_llm.guardrail DI tag and run by GuardrailMiddleware at priority 115 (outermost). Ships secret-redaction and provider content-filter guardrails.
- ADR 086 Guardrail enforcement gaps Accepted Closes two gaps in the guardrail pipeline: the tool playground settles guardrail verdicts as policy outcomes (HTTP 200 guardrail_blocked / guardrail_approval_required) instead of a 500, and the streaming dispatcher screens the assembled completion at end-of-stream as a bounded, fail-soft audit (streamed chunks cannot be retracted).
- ADR 087 Input-side guardrails Accepted Extends guardrails to the outgoing prompt: an InputGuardrailInterface (tag nr_llm.input_guardrail) runs on the send path in LlmServiceManager, where a REDACT verdict rewrites the messages in place and DENY / REQUIRE_APPROVAL throw the same exceptions as the output side. Ships a secret-redaction input guardrail.
- ADR 088 Live streaming redaction Accepted Adds live REDACT for streamed output: each chunk re-redacts the whole raw buffer and emits only the stable prefix beyond a 128-byte holdback, so a secret — even one split across chunk boundaries — collapses to its marker before it is yielded. Opt-in via StreamRedactableInterface, bounded and multibyte-safe; DENY remains the end-of-stream audit's job.
- ADR 089 Guardrail boundary completeness — reasoning, system prompt, vision Accepted Extends secret redaction and the safety guardrails to boundaries an adversarial audit found unguarded — the model's reasoning/thinking output, the system prompt, and vision text inputs — so a secret cannot leak through a path the earlier output-side (ADR-085) and input-side (ADR-087) guards did not cover.
Streaming 3
- ADR 009 Streaming Implementation Accepted Uses PHP Generators to stream chat responses, yielding parsed SSE chunks so consumers can iterate output incrementally.
- ADR 041 Playground Live Run Streaming Accepted Streams the playground run as newline-delimited JSON (one event per RunStep via an opt-in onRecord closure) over the existing admin AJAX route, keeping the batch JSON response as the no-JS fallback.
- ADR 062 Streaming Request Lifecycle Accepted Introduces a dedicated StreamingDispatcher (a wrapping generator preserving the public Generator contract) owning a four-stage lifecycle whose budget pre-flight is eager so an over-budget caller is rejected at call time.
Tools & RAG 17
- ADR 010 Tool/Function Calling Design Accepted Adopts the OpenAI-compatible tool/function schema (type/function/name/description/parameters) as the common tool-definition format across providers.
- ADR 038 Tool Runtime (function-calling agent loop) Accepted Introduces a DI-tagged ToolInterface/ToolRegistry (getSpec/execute/isEnabledByDefault/requiresAdmin) as the authoritative allow-set, so an extension adds a tool just by tagging a class, driving a bounded function-calling agent loop.
- ADR 039 Global Per-Tool Availability State Accepted Adds a UI-less tx_nrllm_tool_state override table and ToolAvailabilityService computing each tool's effective enabled state (admin override, else its default) sitting between per-tool defaults and per-run allow-lists.
- ADR 042 Content and Configuration Read Tools for the Agent Accepted Adds five native read-only tools (search_records, get_page_content, read_records, get_typoscript, ...) implemented in-house without depending on mcp_server or ai_mate, using bound named-parameter equality filters (never raw SQL).
- ADR 043 Tool Groups with a Fail-Closed Enable Cascade Accepted Every tool declares a group (breaking getGroup() addition) and enablement cascades fail-closed across central group state, per-tool state and per-run allow-list, so a disabled group also covers later same-group tools.
- ADR 044 Error-Analysis Tools with Fail-Closed Guards Accepted Adds admin-only code-group tools (get_last_exception with inlined stack context, read_source, search_code) that inspect TYPO3 logs and project source via pure-PHP walks under hard budgets, sharing two guards.
- ADR 045 Schema and Resolution Tools Accepted Adds four read-only structure-group tools (get_full_tca index, get_table_schema with relation view, get_flexform_schema, ...) using a navigate-don't-dump approach that never serializes the whole TCA at once.
- ADR 046 History, URL and Validation Tools Accepted Adds four read-only tools (get_record_history, resolve_url via real SiteMatcher/PageRouter, validate_tca structural checks, ...) for auditing changes, routing-only URL mapping and live TCA validation.
- ADR 047 FAL Tools Accepted Adds five read-only tools in a new files group (list_fal_storages, browse_fal_folder, search_fal_files, ...) resolved exclusively through the FAL storage API, never exposing server-side base paths.
- ADR 048 Diagnostics Tools Accepted Adds six read-only admin-only tools (list_extensions, get_site_config with credential redaction, list_scheduler_tasks, ...) that enumerate instance configuration and attack surface without egressing package paths or raw YAML.
- ADR 049 RAG Site-Search Tools over Installed Search Indexes Accepted Adds a Service/Retrieval layer with a SearchBackendInterface and multiple tagged backends (Solr via HTTP select, ke_search, ...) exposing site_rag_query/site_fetch_source tools that read indexes others own, with public-access filtering.
- ADR 050 Retrieval and Embedding Scope — the Boundary with nr_ai_search Accepted Draws the scope boundary: nr_llm may read from indexes others maintain (or transient embed-and-compare) but must never own a persistent index it has to keep synchronised; cross-encoder reranking placement is later amended by ADR-075.
- ADR 051 Tool-Calling Feature Service — Narrow Consumer Interface Accepted Adds a ToolCallingService/Interface feature-service pair delegating to the manager's two tool-calling methods and adding beUserUid auto-population so per-user budget enforcement works without caller wiring.
- ADR 067 Solr Per-Language Core — No Language Filter Query Accepted Drops the fq=language filter from Solr search/fetchSource since language is already selected by the per-language read core, keeping only the public-access filter; shared-core language separation is a documented unsupported boundary.
- ADR 071 Public Keyword-Search Facade over the Retrieval Cascade Accepted Exposes a deliberately narrow KeywordSearchInterface (search + isAvailable) returning a stable KeywordHit DTO, so retrieval-cascade internals can evolve without touching the public contract.
- ADR 074 Reciprocal Rank Fusion as a Hosted Utility Accepted Hosts a pure final ReciprocalRankFusion class with a signature identical to the nr_ai_search original so consumers migrate by swapping the import; newable (not a DI service), so the audited public-service count is unchanged.
- ADR 075 Neutral Cross-Encoder Reranker Protocol Accepted Defines a neutral RerankerInterface (rerank(query, candidates)) with no consumer DTO crossing the boundary, plus an HttpReranker sidecar implementation that splits over-cap pools score-neutrally and throws typed errors — nr_llm never decides degradation.
Skills 3
- ADR 035 Skill Ingest (GitHub-hosted SKILL.md sources) Accepted Adds dedicated SkillSource and Skill Extbase entities to fetch, parse and review GitHub-hosted SKILL.md files, with an app-level GitHub host allowlist on top of the nr-vault SSRF guard.
- ADR 036 Skill Injection (attach + compose into prompts) Accepted Composes attached skill prose in a shared SkillInjectionService and prepends it to the user prompt (never the system role), applied only to text-generation paths (completion/translation/task) and never to embeddings, vision or speech.
- ADR 061 Skill Trust Levels, Signed Manifests, Injection Scanning Accepted Adds a SkillTrustLevel provenance scale gating injection/allowed-tools against a configurable minimum, plus manifest fingerprints and prompt-injection scanning, separate from support status.
Testing & Quality 4
- ADR 060 Quality Evaluation — Golden Sets, Grading and Regression Detection Accepted Adds an opt-in evaluation layer (DI-tagged golden prompt sets, pluggable graders, run/aggregate service, result persistence with regression detection, CLI) that never touches the request pipeline.
- ADR 072 Retrieval-Quality Evaluation — Golden Questions and Top-k Hit Rates Accepted Adds the retrieval counterpart of ADR-060 — DI-tagged golden question sets, a pluggable retriever contract, document-level top-1/top-3 hit-rate scoring and a CLI — reusing ADR-060's persistence/regression machinery, shipping methodology not questions.
- ADR 073 First-Party Test Doubles for Consumer-Facing Interfaces Accepted Ships maintained fakes (FakeToolCallingService, FakeEmbeddingService) in a runtime-autoloaded Testing namespace so consumers can use them without nr_llm's dev dependencies, with PHPStan enforcing they track the real interfaces.
- ADR 079 First-Party Fakes for Completion, Vision and Budget Services Accepted Extends ADR-073 with FakeCompletionService, FakeVisionService and FakeBudgetService (FIFO/canned returns plus call recording) as plain final classes implementing the real interfaces for consumer tests.