One provider integration instead of one per extension
Every extension that wants AI calls the same service. Adding the second AI feature no longer means building the second provider integration, the second key store and the second error handler.
TYPO3 Extension · nr-llm
The AI control plane for TYPO3.
Connect a provider once, steer access centrally, cap budgets, and make AI features available to every extension on the site — under control.
Supported providers
+ Groq, OpenRouter, Azure OpenAI
+ Groq, OpenRouter, Azure OpenAI
- Maturity
- Beta
- Latest release
- v0.32.0
- Main branch
- 0.32.0
- Documentation describes
- 0.32.0
- Page last reviewed
- Requires
- TYPO3 13.4 / 14.3 · PHP 8.2+
Four different values. A difference between them is not a contradiction — the release is what you install, main is what is being built, and the review date is when a person last checked this page's wording.
What changes for the organisation
The technical description is further down. These are the effects that decide whether it is worth adopting.
Switching provider is a setting, not a project
Configurations name a provider and a model. Moving from a commercial API to a self-hosted endpoint changes a backend record, not the code in the consuming extensions.
Keys stop multiplying
API keys live in nr-vault as identifiers. They are not in the database in plaintext, not in a configuration file, and not in three extensions at once.
Cost becomes visible before the invoice
Usage is recorded per provider, model, service and backend user, with an estimated cost. Budgets cap it per user and per period.
AI features become reusable
Translation, alt text, embeddings and retrieval are services other extensions inject rather than reimplement.
Local and self-hosted stay on the table
Ollama, vLLM and LocalAI are configured the same way as a commercial API, so a data-residency requirement does not force a different architecture.
Governance and control
What an administrator can actually decide, and where the software enforces it.
Who may use what
Per-capability permissions are granted to backend groups. The configuration module itself is administrator-only.
How much each person may spend
Budgets cap requests, tokens or estimated cost per backend user, daily or monthly. A run that would exceed the budget fails with a typed exception rather than silently continuing.
Which tools a model may call
Built-in tools are grouped and each group is toggled on or off. Read-only tools are the default; the writing tools ship disabled.
What needs a human
An approval-gated tool suspends the run and waits for a person to decide. There is no timeout that approves by default.
What is recorded
Provider, model, service, user, token counts and estimated cost per request. Prompt content is not part of the usage record.
What happens when a provider fails
A configuration can name fallbacks, retried on connection errors, HTTP 5xx and rate limits, in the order declared. An exhausted chain raises a typed exception rather than returning an empty answer.
Where the data goes
One request, end to end.
- An extension calls the service It passes messages and options. It does not know which provider is configured and never sees a key.
- nr-llm resolves the configuration Configuration → model → provider. The key is fetched from nr-vault by identifier and used for this request only.
- Guardrails screen the payload Input, output, reasoning and streamed chunks pass a policy pipeline that can redact. Secret patterns are denied on the way out.
- The provider receives the request Whatever the configuration names: a commercial API, an EU-hosted provider, your own Azure deployment, or an endpoint inside your network.
- The answer comes back and is recorded The response goes to the calling extension. The usage record keeps provider, model, service, user, tokens and estimated cost — not the prompt.
- What is transferred: the prompt the extension built, plus any site content a retrieval tool included.
- Where keys are stored: in nr-vault, envelope-encrypted, referenced by identifier. nr-llm never stores or logs a raw key.
- What can stay local: everything, if the configuration points at a self-hosted endpoint. Provider choice is per configuration, not per site.
Provider and operating models
All configured the same way. The choice is an administrative decision, not an architectural one.
| Model | Where data is processed | Fits when | What it costs you |
|---|---|---|---|
| Commercial API (OpenAI, Anthropic, Google, Mistral, Groq) | At the provider | Model quality and choice decide | A contractual and data-protection review per provider |
| EU-hosted provider | At the provider, inside the EU | The processing location is prescribed | A smaller model selection |
| Azure OpenAI or an OpenAI-compatible endpoint | In your own cloud account | A framework agreement already exists | Commitment to the platform |
| Self-hosted (Ollama, vLLM, LocalAI) | In your infrastructure | Data must not leave the network | Your own operations, hardware and model maintenance |
| Mixed | Different per configuration | Sensitive and uncritical cases differ | More configuration and more to explain |
Cost control
What the extension actually does about cost. No savings figure is claimed here — that depends on volumes we do not know.
Budget types
Requests, tokens or estimated cost. Each can be capped daily or monthly, per backend user.
Cost per model and service
Usage analytics break estimated cost down by provider, model, service and user, over a rolling window.
What happens at the limit
The request is refused with a typed BudgetExceededException. The calling extension decides how to present that.
Falling back to something cheaper
A configuration can name fallbacks — a smaller model, or a self-hosted endpoint with no per-token cost.
How the estimate is made
From the per-model pricing recorded in the backend, applied to the token counts the provider returns. It is an estimate, not the provider's invoice.
Three ways to evaluate it
Each is small enough to finish and specific enough to judge.
Centrally steered translation
Configure one provider, grant the translation capability to the editorial group, set a monthly budget, translate a page tree. Afterwards you can see who translated what, at which model, at what estimated cost.
Alt text for images
Point a configuration at a vision-capable model and generate alt text for a media folder. Compare a commercial model against a self-hosted one on the same images.
Editorial research with cited sources
Enable the retrieval tool group and ask questions about your own site content. Check that every answer names the pages it came from.
AI capability card
The same fields on every Netresearch AI product, so they can be compared rather than admired.
- Intended purpose
- Shared, administrator-controlled access to language models for TYPO3 extensions: one provider configuration, one set of permissions and budgets, reused by every extension on the site.
- Excluded uses
- Unattended decisions about people — hiring, credit, benefits, access or any other consequential outcome.
- Autonomous content changes without a human approval.
- A source of facts. Model output is untrusted content and is treated as such.
- A compliance control in its own right. It supports controls; it does not make an organisation conformant.
- Maturity
- Beta
- Models and providers
- OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Mistral, Groq, Azure OpenAI, any OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM)
- Data processed
- The prompt an extension sends, plus any site content a retrieval tool includes.
- Usage metadata: provider, model, service, backend user, token counts and estimated cost.
- No API keys — those stay in the vault as identifiers.
- Processing location
- Your own server, An external provider
- Human oversight
- Writing tools ship disabled and, once enabled, suspend the run for an explicit approval. Everything else is read-only by default.
- Permissions
- Per-capability backend-group permissions; the backend module is restricted to administrators.
- Logging
- Requests are recorded with provider, model, service, user, token counts and estimated cost. Prompt content is not part of the usage record.
- Retention
- Usage records live in the site's database and follow the site's own retention rules. nr-llm sets no expiry of its own.
- Cost control
- Per-user budgets on requests, tokens or estimated cost, daily or monthly, with fallback to cheaper or local models.
- Security controls
- API keys as vault identifiers, never plaintext
- Guardrail pipeline across input, output, reasoning and streamed chunks
- Secret-pattern denial on the egress path
- Approval-gated writing tools
- Signed releases with SBOMs and SLSA provenance
- Known limitations
- Models produce wrong answers with confidence. Nothing here prevents that; the controls bound what a model may do, not what it may say.
- Prompt injection through retrieved site content is possible. Tool permissions and the approval gate are the mitigation, not immunity.
- Cost figures are estimates derived from the model's published pricing, not from the provider's invoice.
- Streaming responses bypass parts of the output pipeline; the trade-off is documented in the ADRs.
- Provider availability, model behaviour and pricing are outside this extension's control.
- Last reviewed
- · Netresearch DTT GmbH — nr-llm maintainers
The problem
Every TYPO3 extension that wants AI capabilities today has to solve the same infrastructure problems on its own. When a site runs three AI extensions, that means three separate API key configurations, three places to check when something breaks, and no way to switch providers globally.
- Build its own provider integration — HTTP calls, authentication, error handling, streaming
- Store API keys in its own way, often as plaintext in extension settings
- Create its own backend configuration UI
- Leave administrators with no central overview of AI usage or costs
The solution
nr-llm provides the missing shared layer between your extensions and the LLM providers. Extension developers add AI in a few lines of dependency injection; administrators manage every connection, key, and budget from one backend module.
Extensions inject a single service interface and call methods for chat, completion, translation, vision, embeddings, streaming, and tool calling. Provider selection, API keys, caching, and error handling are all managed by nr-llm.
Underneath, a provider abstraction layer maps a common interface onto OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Mistral, Groq, Azure OpenAI, and any OpenAI-compatible endpoint. Switching providers is an admin setting, not a code change.
The Admin Tools > LLM backend module holds encrypted keys, usage and cost tracking, per-user budgets, and a setup wizard — restricted to administrators.
Your extensions
Cowriter · SEO Assistant · …
nr-llm service layer
Chat · Translation · Vision · Embeddings · Streaming · Tools · Caching
Provider abstraction
OpenAI · Anthropic · Gemini · Ollama · Mistral · Groq · …
Admin Tools > LLM
Encrypted keys · Usage & cost · Setup wizard
Core concepts
nr-llm is organized around a small set of building blocks. Each one solves a problem you would otherwise re-implement in every AI extension.
Provider abstraction
All providers implement one common interface. OpenAI, Anthropic Claude, Google Gemini, Ollama, OpenRouter, Mistral, Groq, Azure OpenAI, and any OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM) are reachable through the same service calls. Switch provider with a single configuration change — no code edits, no vendor lock-in.
Encrypted API keys via nr-vault
Every API key is stored as a vault identifier (UUID) using nr-vault envelope encryption. nr-llm never stores or logs raw keys in plain text. Error messages are sanitized to strip secret-bearing query parameters before anything is logged.
Three-tier configuration
A Provider holds an endpoint, encrypted key, and adapter type. A Model references a provider and defines its model id, capabilities, and pricing. A Configuration references a model and adds use-case settings — system prompt, temperature, token limits. This lets you keep multiple keys per provider (prod/dev/backup) and reuse model definitions across use cases.
Feature services
High-level services cover common tasks: CompletionService for text generation with format and creativity control, TranslationService with formality and glossary support, VisionService for alt-text and image analysis, and EmbeddingService for text-to-vector conversion with similarity calculations.
Streaming and tool calling
Stream responses chunk by chunk with a single foreach over streamChat() for real-time UIs. Tool/function calling lets the model request functions your code executes, via chatWithTools() — the response reports which tools were called so you can process them.
RAG site-search tools
48 built-in function-calling tools in 9 toggleable groups give the model grounded access to the TYPO3 instance — content search, TCA/FlexForm schema, TypoScript, source and exception reads, FAL files, diagnostics, and backend accounts. 41 of them only read. The rag group returns cited site-content evidence from the installed search index (EXT:solr, ke_search, indexed_search, or a database fallback), and the editing group holds the seven writing tools, which ship disabled and pause the run for human approval before they act.
Per-user budgets and usage tracking
Cap per-backend-user spending across every preset by requests, tokens, or estimated cost, on a daily or monthly basis. The Analytics view shows cost and usage trends with breakdowns per provider, model, and service, plus per-user consumption against monthly budgets.
TYPO3 caching-framework integration
Responses are cached automatically through TYPO3's caching framework — using whatever backend the instance configures (Redis, Valkey, Memcached, or the default). Embedding results cache deterministically with a 24-hour default lifetime, and cache lifetimes are configurable per operation type.
Who it is for
nr-llm serves three audiences with the same shared foundation.
Extension developers
Add AI capabilities without building provider integrations, handling API keys, or implementing caching and streaming. Inject one service interface and call it. Register custom providers when you need them.
TYPO3 administrators
Manage every AI connection, encrypted key, and provider configuration from a single backend module. Switch from OpenAI to Anthropic without touching extension code. Set per-user budgets and watch cost and usage in one dashboard.
Agencies and solution architects
Reduce integration effort across client projects with a consistent AI architecture and no vendor lock-in. Encrypted keys, admin-only access, and SBOM plus SLSA provenance on every release support compliance. Ollama gives a local-first option for data-sensitive environments.
Developers
Developer kickstart
Add AI to your TYPO3 extension in a few minutes — no API key handling, no HTTP client code, no provider-specific logic.
Require the package
Install via Composer. Then activate in Admin Tools > Extensions and run Admin Tools > LLM > Setup Wizard.
bash
composer require netresearch/nr-llmInject the service you need
Inject LlmServiceManagerInterface via constructor promotion and call it. Provider selection, API keys, caching, and error handling are all managed by nr-llm.
php
use Netresearch\NrLlm\Service\LlmServiceManagerInterface;
class MyController
{
public function __construct(
private readonly LlmServiceManagerInterface $llm,
) {}
public function summarizeAction(string $text): string
{
return $this->llm->complete("Summarize: {$text}")->content;
}
}Use the services you need
Chat, completion, streaming, embeddings and tool calling are methods on the injected LlmServiceManagerInterface. Translation and vision alt-text are dedicated feature services — inject them the same way. Tool calling is a request/execute/reply loop; the Streaming & Tool Calling deep dive has the complete example.
php
use Netresearch\NrLlm\Domain\ValueObject\ChatMessage;
$messages = [
ChatMessage::system('You are a helpful TYPO3 assistant.'),
ChatMessage::user('Explain TYPO3 content elements in one paragraph.'),
];
// Chat & completion (LlmServiceManagerInterface)
$answer = $this->llm->chat($messages)->content;
$answer = $this->llm->complete('Summarize the TYPO3 release cycle.')->content;
// Streaming — yields string chunks
foreach ($this->llm->streamChat($messages) as $chunk) {
echo $chunk;
}
// Embeddings — EmbeddingResponse carries the vector
$embedding = $this->llm->embed('semantic search query');
// Translation — dedicated service, returns a TranslationResult
$german = $this->translationService->translate('Hello world', 'de')->getText();
// Vision alt-text — dedicated service
$altText = $this->visionService->generateAltText($imageUrl);Handle failures with typed exceptions
Every provider error is a typed exception. Catch the ones you care about and show a friendly message; the fallback chain and retries have already run before these surface.
php
use Netresearch\NrLlm\Exception\BudgetExceededException;
use Netresearch\NrLlm\Provider\Exception\ProviderRateLimitException;
use Netresearch\NrLlm\Provider\Exception\ProviderConnectionException;
use Netresearch\NrLlm\Provider\Exception\FallbackChainExhaustedException;
use Netresearch\NrLlm\Provider\Exception\ProviderResponseException;
try {
return $this->llm->complete("Summarize: {$text}")->content;
} catch (BudgetExceededException) {
return 'The AI budget for this account is exhausted.';
} catch (ProviderRateLimitException) {
return 'The AI provider is rate-limiting requests. Please retry shortly.';
} catch (FallbackChainExhaustedException | ProviderConnectionException) {
return 'Could not reach any AI provider right now.';
} catch (ProviderResponseException $e) {
$this->logger->warning('LLM provider error', ['status' => $e->httpStatus]);
return 'The AI service returned an error.';
}Tune output and get structured JSON back
ChatOptions ships tuned presets — factual, creative, balanced, json, code — plus fluent overrides. CompletionService::completeJson() returns a decoded array, so you extract fields directly.
php
use Netresearch\NrLlm\Service\Option\ChatOptions;
// Deterministic output, capped length
$options = ChatOptions::factual()->withMaxTokens(200);
$summary = $this->llm->complete('Summarize the changelog.', $options)->content;
// Decoded JSON straight from the model
$data = $this->completionService->completeJson(
'Return {"title": ..., "tags": [...]} for this article: ' . $article,
);
$title = $data['title'];For admins
For administrators
The Admin Tools > LLM backend module gives administrators full control over AI on the site — providers, models, configurations, budgets, and analytics in one place.
Providers, Models, Configurations
Register API connections (OpenAI, Anthropic, Gemini, Ollama, and more), define which models are available and their capabilities, and create use-case presets with temperature, system prompts, and token limits.
Setup wizard
The Setup Wizard auto-detects your provider type from the endpoint URL, discovers available models, and generates a ready-to-use configuration in five guided steps. Paste your API key and go.
AI-powered wizards
The Task Wizard and Configuration Wizard generate complete tasks and configurations — system prompt, parameters, and model recommendation — from a plain-language description. A Fetch Models button auto-fills capabilities and pricing from the provider API.
User budgets and analytics
Cap per-backend-user spending by requests, tokens, or cost on a daily or monthly basis across every preset. The Analytics view shows estimated cost and usage trends with breakdowns per provider, model, and service, plus per-user consumption against budgets.
Tools and RAG
48 function-calling tools in 9 toggleable groups let models inspect content, schema, configuration, code, files, system diagnostics, and accounts — with the rag group returning cited evidence from the installed search index. 41 only read; the six writers ship disabled and pause for approval.
Tool Playground
The admin-only Playground runs the bounded agent loop against any configuration and streams the whole dialog live — every request, response, and tool execution — plus a dry-run mode that shows the exact prompt without calling the model.
Resilience and security
Configurations can list fallback configurations to retry against on connection errors, HTTP 5xx, or rate limits. Per-capability permissions map to native TYPO3 backend-group options. Keys are stored encrypted via nr-vault and the module is restricted to administrators.
On-device AI
Ask nr-llm
Ask a question about nr-llm and get an answer generated entirely in your browser by Chrome's built-in AI (Gemini Nano). Answers are grounded in this site's content.
Runs entirely in your browser. Nothing is sent to any server.
Architecture
Architecture
nr-llm uses a three-tier configuration hierarchy that separates concerns cleanly. A Configuration (use-case settings such as system prompt, temperature, and max tokens) references a Model (model id, capabilities, pricing), which references a Provider (endpoint, encrypted API key, adapter type). This lets you keep multiple API keys per provider type, point at custom endpoints such as Azure OpenAI or a local Ollama or vLLM instance, and reuse model definitions across configurations. Requests flow through a middleware pipeline that enforces fallback chains and records usage after each successful call. The extension targets PHP 8.2+ and TYPO3 v13.4 LTS or v14.3 LTS, with a PSR-18 HTTP client.
Frequently asked questions
Which TYPO3 and PHP versions are supported?
TYPO3 v13.4 LTS or v14.3 LTS, and PHP 8.2 or higher. A PSR-18 compatible HTTP client (such as guzzlehttp/guzzle) is also required. The extension is currently in beta (version 0.32.0).
Which AI providers can I use?
OpenAI, Anthropic Claude, Google Gemini, Ollama, OpenRouter, Mistral, Groq, Azure OpenAI, and any OpenAI-compatible endpoint (vLLM, LocalAI, LiteLLM). Capabilities vary by provider — for example OpenAI, Gemini, and OpenRouter support chat, embeddings, vision, streaming, and tools, while Groq focuses on fast chat and streaming.
Where are API keys stored?
Keys are stored as vault identifiers (UUIDs) via nr-vault envelope encryption. nr-llm never stores or logs raw keys in plain text, and the backend module is restricted to administrators. nr-vault is a required dependency.
Is it free and open source?
Yes. nr-llm is licensed under GPL-2.0-or-later and developed by Netresearch DTT GmbH. The source is on GitHub and the package is on Packagist.
Does it work offline with Ollama?
Yes. Ollama runs models locally and needs no API key, so AI features can work without sending data to external APIs — a local-first option for data-sensitive environments. Ollama supports chat, embeddings, and streaming.
How do I add AI to my own extension?
Require netresearch/nr-llm via Composer, inject LlmServiceManagerInterface (or a specific feature service), and call its methods for chat, completion, translation, vision, embeddings, streaming, or tool calling. You can also register custom providers. See the Developer and Integration guides.
How do I control cost?
Set per-backend-user budgets that cap spending by requests, tokens, or estimated cost on a daily or monthly basis across every preset. Response caching through the TYPO3 caching framework reduces repeat calls, and the Analytics view tracks estimated cost and usage per provider, model, service, and user.
What about data privacy?
You choose the provider, including a local Ollama instance that keeps data on your own infrastructure. Keys are encrypted at rest, access is admin-only, and error messages are sanitized to strip secrets. Treat LLM responses as untrusted content and sanitize user input before sending it, as with any AI integration.
Can I switch providers without changing code?
Yes. All providers implement a common interface, so switching from OpenAI to Anthropic or a local model is a configuration change in the backend, not a code edit. Configurations can also list fallback configurations to retry against on connection errors, HTTP 5xx, or rate limits.
Why use nr-llm instead of calling a provider SDK directly?
Building your own means re-implementing key encryption, provider switching, caching, streaming, tool calling, cost tracking, budgets, guardrails, and error handling in every extension — and hard-coding one vendor. nr-llm centralizes all of that once, so administrators manage providers and keys in one backend module and any extension on the site reuses them with no vendor lock-in.
Is nr-llm GDPR-friendly, and can I keep data in the EU?
You choose the provider per configuration. Running Ollama or another OpenAI-compatible endpoint locally keeps all prompt data on your own infrastructure with no external API calls; for hosted providers you can select EU-region endpoints such as Azure OpenAI. API keys are encrypted at rest via nr-vault, access is admin-only, and error messages are sanitized to strip secrets.
How is nr-llm different from other TYPO3 AI extensions?
nr-llm is not an end-user AI feature — it is shared infrastructure, like the TYPO3 caching framework, that other extensions build on. It provides one provider abstraction across seven-plus providers, encrypted key storage, and typed services, rather than a single bundled use case.
What governance and security controls does nr-llm provide?
Encrypted API keys via nr-vault, an admin-only backend module with per-capability backend-group permissions, a guardrail pipeline that redacts secrets across input, output, and streaming, optional human-in-the-loop approval, and per-user budgets with usage analytics. 41 of the 48 built-in tools are read-only; the six that write ship disabled and suspend the run for human approval before they act. The Governance & Security page covers each control in detail.
Evaluating AI in TYPO3?
We can work through which capabilities you need, which operating model fits your constraints, and what a first controlled step looks like.