# nr-llm — Shared AI/LLM Foundation for TYPO3

> nr-llm is shared infrastructure for AI in TYPO3 — like the caching framework, but for language models. Administrators configure LLM providers once in a central backend module; every AI-powered extension on the site uses them automatically. It provides a unified provider abstraction (OpenAI, Anthropic Claude, Google Gemini, Ollama, OpenRouter, Mistral, Groq, Azure OpenAI, and any OpenAI-compatible endpoint), encrypted API-key storage via nr-vault, and ready-to-use services for chat, translation, vision, and embeddings — with streaming, tool/function calling, response caching, and typed error handling. License GPL-2.0-or-later. Vendor: Netresearch DTT GmbH.

## Key facts

- **Package:** `netresearch/nr-llm` (Composer), extension key `nr_llm`
- **Requires:** PHP 8.2+, TYPO3 v13.4 LTS or v14.3 LTS, a PSR-18 HTTP client
- **License:** GPL-2.0-or-later
- **Repository:** https://github.com/netresearch/t3x-nr-llm
- **Install:** `composer require netresearch/nr-llm`
- **Latest version:** 0.41.1 (beta)
- **Providers:** 7 first-class adapters (OpenAI, Anthropic Claude, Google Gemini, Ollama, OpenRouter, Mistral, Groq) plus Azure OpenAI and any OpenAI-compatible endpoint
- **Built-in tools:** 62 function-calling tools in 10 toggleable groups — 45 read-only, plus seventeen writing tools that ship disabled and pause for human approval; includes cited site-content RAG
- **Architecture decisions:** 227 documented ADRs (see /adr/)
- **Supply chain:** SLSA Level 3 provenance, Cosign signatures, and SBOMs (SPDX + CycloneDX) on every release
- **Architecture:** three-tier — Provider (API connections) → Model (per-provider models) → Configuration (use-case presets)

## Landing pages

- [Project home](https://netresearch.github.io/t3x-nr-llm/): Overview of nr-llm, the problem it solves, and who it is for.
- [English landing page](https://netresearch.github.io/t3x-nr-llm/en/): Full English overview for developers, administrators, and agencies.
- [German landing page](https://netresearch.github.io/t3x-nr-llm/de/): Deutschsprachige Übersicht der Extension.
- [Governance & Security](https://netresearch.github.io/t3x-nr-llm/en/security/): Encrypted keys, guardrails, human-in-the-loop approval, per-user budgets, data residency, and signed releases.
- [llms-full.txt](https://netresearch.github.io/t3x-nr-llm/llms-full.txt): Full flattened content (this overview + every feature + every ADR summary) for one-fetch ingestion.

## For developers

- [Developer Guide](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Developer/Index.rst): Full API reference, custom provider registration, typed options, and response objects.
- [Integration Guide](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Developer/IntegrationGuide.rst): Step-by-step tutorial for building a TYPO3 extension on nr-llm.
- Inject `Netresearch\NrLlm\Service\LlmServiceManagerInterface` and call `chat()`, `complete()`, `streamChat()`, or `chatWithTools()`; dedicated services cover translation, vision (alt-text), and embeddings.

## For AI coding agents

A coding agent integrating nr-llm needs only these facts:

- **Install:** `composer require netresearch/nr-llm`
- **Inject** (constructor promotion): `Netresearch\NrLlm\Service\LlmServiceManagerInterface`
- **Core calls:** `chat(array $messages, ?ChatOptions): CompletionResponse`, `complete(string $prompt, ?ChatOptions): CompletionResponse`, `streamChat(array $messages, ?ChatOptions): Generator`, `chatWithTools(array $messages, array $tools, ?ToolOptions): CompletionResponse`, `embed(string|array $input, ?EmbeddingOptions): EmbeddingResponse`
- **Build messages** with `Netresearch\NrLlm\Domain\ValueObject\ChatMessage::system()` / `::user()`; read the answer from `$response->content`.
- **Feature services** are injected the same way: `TranslationServiceInterface::translate()` returns a `TranslationResult` (`->getText()`); `VisionServiceInterface::generateAltText()`; `CompletionServiceInterface::completeJson()` returns a decoded array.
- **Custom provider:** implement `Netresearch\NrLlm\Provider\Contract\ProviderInterface`, add the `#[AsLlmProvider]` attribute (auto-registers), return an identifier from `getIdentifier()`.
- **Errors are typed** exceptions under `Netresearch\NrLlm\Provider\Exception\` (`ProviderRateLimitException`, `ProviderConnectionException`, `FallbackChainExhaustedException`, `ProviderResponseException`) and `Netresearch\NrLlm\Exception\` (`BudgetExceededException`, `GuardrailViolationException`, `GuardrailApprovalRequiredException`).
- **Fetchable raw sources** (plain RST/Markdown, no HTML):
  - Developer guide: https://raw.githubusercontent.com/netresearch/t3x-nr-llm/main/Documentation/Developer/Index.rst
  - Integration guide: https://raw.githubusercontent.com/netresearch/t3x-nr-llm/main/Documentation/Developer/IntegrationGuide.rst
  - API reference: https://raw.githubusercontent.com/netresearch/t3x-nr-llm/main/Documentation/Api/Index.rst
  - Repo agent rules (AGENTS.md): https://raw.githubusercontent.com/netresearch/t3x-nr-llm/main/AGENTS.md

## For administrators

- [Admin: Analytics](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Administration/Analytics.rst): Estimated cost and usage trends per provider, model, and service.
- [Admin: User Budgets](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Administration/UserBudgets.rst): Cap per-backend-user spending (requests / tokens / cost, daily / monthly).
- [Admin: Skills](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Administration/Skills.rst): Ingest SHA-pinned SKILL.md instruction files from GitHub and attach them to tasks.
- [Admin: Tools](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Administration/Tools.rst): 62 built-in function-calling tools in 10 toggleable groups — 45 read-only plus seventeen approval-gated writing tools; includes cited site-content RAG.
- The **Admin Tools > LLM** backend module manages providers, models, configurations, tasks, and a two-minute Setup Wizard that auto-detects the provider from the endpoint URL.

## Architecture decisions

- [Architecture Decision Records](https://netresearch.github.io/t3x-nr-llm/adr/): Index of the project's ADRs documenting design rationale (three-tier provider architecture, vault-based API-key storage, provider fallback chains, per-capability permissions, dashboard widgets, and more).
- [Architecture overview](https://github.com/netresearch/t3x-nr-llm/blob/main/Documentation/Architecture/Index.rst): Design decisions and how the tiers fit together.

## Supported providers

OpenAI, Anthropic Claude, Google Gemini, Ollama (local, no API key), OpenRouter, Mistral, Groq, Azure OpenAI, and any OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM). Switching providers is an admin setting change — no code changes in consuming extensions.

## Provider capabilities

Capabilities below reflect what each shipped adapter supports; actual availability also depends on the configured model. Embeddings are a per-model capability rather than a per-adapter one.

| Provider | Chat | Streaming | Tools | Vision | Documents | API key |
|----------|------|-----------|-------|--------|-----------|---------|
| OpenAI | yes | yes | yes | yes | no | required |
| Anthropic Claude | yes | yes | yes | yes | yes | required |
| Google Gemini | yes | yes | yes | yes | yes | required |
| Ollama | yes | yes | yes | no | no | none (local) |
| OpenRouter | yes | yes | yes | yes | no | required |
| Mistral | yes | yes | yes | no | no | required |
| Groq | yes | yes | yes | no | no | required |

Azure OpenAI and any OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM) use the OpenAI adapter path.

## Glossary

- **Provider** — a connection to an LLM API (endpoint, encrypted key, adapter type), e.g. an OpenAI or Ollama connection.
- **Model** — a backend entity referencing a Provider that defines a model id, capabilities, and pricing.
- **Configuration** — a use-case preset referencing a Model, adding system prompt, temperature, and token limits; can list fallback configurations.
- **Task** — a named unit of work that binds a Configuration to optional Skills and Tools.
- **Skill** — a SHA-pinned SKILL.md instruction file ingested from GitHub and attached to a Task.
- **Tool** — a built-in function the model may call (62 available, in 10 toggleable groups; 45 read-only), e.g. cited site-content RAG.
- **Guardrail** — a policy pipeline that screens and can redact input and output.
- **HITL (human-in-the-loop)** — optional manual approval before a sensitive action runs.

## Security

- API keys are stored as nr-vault UUID identifiers using envelope encryption; nr-llm never stores or logs raw keys.
- The backend module is restricted to administrators.
- Every release ships SLSA Level 3 provenance, Cosign signatures, and SBOMs (SPDX + CycloneDX).
- Guardrail pipeline (screens input, output, reasoning, system prompt, vision, and streamed chunks), human-in-the-loop approval for sensitive actions, per-user budgets, and per-capability backend-group permissions.
- Full governance & security overview: https://netresearch.github.io/t3x-nr-llm/en/security/

## FAQ

**Where are API keys stored?** As nr-vault UUID identifiers using envelope encryption. nr-llm never stores or logs raw API keys in plain text, and the backend module is restricted to administrators. nr-vault is a required dependency.

**How do I add AI to my own TYPO3 extension?** Run `composer require netresearch/nr-llm`, inject `LlmServiceManagerInterface` (or a specific feature service), and call `chat()`, `complete()`, `streamChat()`, `chatWithTools()`, or the translation/vision/embedding services. Custom providers register with the `#[AsLlmProvider]` attribute.

**Can I switch providers without changing code?** Yes. Every provider implements a common interface, so switching from OpenAI to Anthropic or a local Ollama model is a backend configuration change, not a code edit. Configurations can list fallback configurations retried on connection errors, HTTP 5xx, or rate limits.

**Can I run it without sending data to a third-party provider?** Yes. Point a configuration at a self-hosted Ollama or OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM) and the request never leaves your network. Provider choice is per configuration.

**Why use nr-llm instead of calling a provider SDK directly?** Building your own means re-implementing key encryption, provider switching, caching, streaming, tool calling, cost tracking, budgets, guardrails, and error handling in every extension — and hard-coding one vendor. nr-llm centralizes all of that once, so administrators manage providers and keys in one backend module and any extension reuses them with no vendor lock-in.

**What governance and safety controls does it provide?** Per-user budgets (requests/tokens/cost, daily/monthly), usage analytics per provider/model/service/user, admin-only configuration, per-capability backend-group permissions, a guardrail pipeline that redacts secrets across input/output/streaming, and optional human-in-the-loop approval. 45 of the 62 built-in tools are read-only; the seventeen that write ship disabled and suspend the run for human approval.

**How do I verify a release hasn't been tampered with?** Each release ships a Cosign signature, SHA-256 checksums, SPDX and CycloneDX SBOMs, and SLSA Level 3 provenance you can verify with `gh attestation verify`.