Governance & Security

Governance and security, built in — not bolted on

nr-llm centralizes the parts of AI that a security review actually asks about: where secrets live, who can do what, what the model may send and receive, how spend is capped, and where data goes. Every control here is documented in an Architecture Decision Record.

AI in a CMS raises the same questions every time: are our API keys safe, can an editor run up the bill, could the model leak a secret, and does our data leave the building? Because nr-llm is shared infrastructure rather than a single feature, it answers those questions once, in one place, for every extension on the site.

Controls at a glance

  • API keys stored as encrypted nr-vault identifiers — never plaintext, never logged
  • Admin-only backend module, plus per-capability TYPO3 backend-group permissions
  • Guardrail pipeline that screens and redacts input, output, reasoning, and streamed chunks
  • Human-in-the-loop approval gate for actions with side effects
  • 71 built-in tools in fail-closed groups — 48 read-only, twenty-three writing and approval-gated — with fail-closed network egress
  • Per-user budgets (requests, tokens, cost — daily and monthly) with usage analytics
  • Connect self-hosted models through Ollama or a compatible API
  • Signed releases: SLSA Level 3 provenance, Cosign signatures, SPDX + CycloneDX SBOMs

Secrets & key management

nr-llm never stores an LLM API key in plaintext. Every provider and specialized-service credential is held as an nr-vault identifier — a UUID — while the secret itself lives in the netresearch/nr-vault extension, which provides envelope encryption, audit logging, and access control.

At call time the vault's secure HTTP client resolves the secret, injects it into the outbound request, records the access, and scrubs it from memory immediately after. The plaintext key never appears in a database dump or a log line, and chat, completion and other inference calls never hand it to nr-llm's code. Fetching a provider's model list, detecting model limits and verifying model capabilities in the backend are the exception: they read the key from the vault to query the provider. On top of that, error messages are sanitized: secret-bearing query parameters such as ?key= and ?token= are stripped before anything is logged or shown.

Decisions: ADR 012ADR 030

Access control

The entire nr-llm backend module is admin-only. Every backend endpoint — provider, model, and configuration changes, the test calls that decrypt vault keys, task execution, the tool playground — is additionally guarded, so a non-admin receives a uniform 403 before any state is changed or any key is decrypted.

Beyond the admin gate, each of the eleven AI capabilities — chat, completion, embeddings, vision, streaming, tools, JSON mode, audio, image, text-to-speech, and transcription — is a native TYPO3 backend-group permission, so an administrator can grant or revoke a single capability site-wide from the standard group-edit screen. Named configurations carry their own access-group ACL as well, and a call must clear both the configuration ACL and the capability permission.

Decisions: ADR 023ADR 037

Guardrails: input, output & streaming redaction

Model output and user prompts are treated as untrusted content and run through a guardrail pipeline that returns a real verdict — allow, redact, retry, require-approval, or deny — not a boolean. On the response side, screening runs inside the persistence layer, so a secret a model echoes is masked, or the response blocked, before it is ever cached or replayed. On the input side, the outgoing prompt is screened and can be redacted before it leaves the extension.

Streaming, historically a blind spot, is covered live: a holdback buffer masks a secret even when it is split across two chunks, and the emit boundary is UTF-8-safe. Screening spans every boundary the same secret can cross — the answer, the model's reasoning, the user turn, the system prompt, and vision text. Secret redaction and provider content-filter enforcement ship active, and new guardrails become active simply by existing.

Decisions: ADR 085ADR 087ADR 088ADR 089

Human-in-the-loop approval

A tool that writes or has side effects can require a human gate before it runs. The tool opts in with a single marker; the agent loop then pauses the whole turn, persists a serialized transcript to the run in a waiting-for-approval state, and surfaces it for an operator to approve or deny.

On approval the run resumes exactly where it paused; on denial the model is told and continues. 48 of the 71 tools that ship today are read-only, so this path never fires for them and their behavior is unchanged. The twenty-three writing tools declare a write effect, and a declared write effect is what triggers the pause — none can act without a human decision first. A guardrail's require-approval verdict routes into the same review seam.

Decisions: ADR 084

Tool safety

48 of the 71 built-in tools are read-only, and each follows one security contract: model-chosen arguments are validated and scoped, output volumes are capped, and any secret-bearing output is redacted or gated behind a separate raw variant that ships disabled. Most tools require admin; the read-only structure, content, and file tools offered to non-admins self-enforce that user's own TYPO3 permissions. Twenty-two of the twenty-three writing tools are narrow by design — each performs one named act on one record or on one cache group — and the twenty-third, create_record_draft, serves only the tables no narrow writer covers, under a reviewed list of exclusions; content any of them creates is created hidden, while a redirect applies as soon as it is approved. The four site tools among them — three for redirects and clear_cache — are administrator-only, and a full cache flush is named as one on the approval card. All twenty-three ship disabled, act as the acting backend user, and pause for human approval first.

Tools are organized into groups with a three-level, fail-closed enable cascade: a disabled group cannot be re-enabled by a per-tool override, and a tool a skill did not grant can never be reached through prompt injection. Network egress is governed per group and is fail-closed — a group with no declaration makes no outbound request, and there is no "any host" scope. Host-level tools resolve through a path guard that makes settings.php and key material structurally unreadable even for admins, and the URL probe is contained to the instance's own site. Each run is a bounded agent loop.

Decisions: ADR 038ADR 043ADR 044ADR 047

Cost governance

Set daily and monthly limits per backend user for requests, tokens and estimated cost. The check uses recorded usage and any known planned cost. It does not reserve funds. Concurrent requests and usage recorded after dispatch can exceed the limit.

When the preflight check denies a call, it reports the reason. Analytics shows recorded requests and estimated costs by provider, model, service and user. Daily limits for a particular configuration are checked separately.

Decisions: ADR 025ADR 052

Data residency & privacy

nr-llm supports Ollama and OpenAI-compatible services such as vLLM and LocalAI. For an internal workflow, check the model endpoint, every fallback provider, and each tool and additional model service it calls.

Because tool output and every model response go to the chosen provider, the extension treats both as untrusted: tool results are volume-capped and strip personally identifying fields, responses are rendered escaped, and any HTML is shown only inside a sandboxed preview.

Decisions: ADR 001

Supply-chain integrity

Every release is signed and verifiable. Release artifacts carry Cosign keyless signatures (Sigstore), SHA-256 checksums, a Software Bill of Materials in both SPDX and CycloneDX formats, and SLSA Level 3 provenance attestations you can check with gh attestation verify. Commits are cryptographically signed and DCO signed-off.

The CI pipeline runs CodeQL semantic analysis, Gitleaks secret scanning, OpenSSF Scorecard, dependency review with CVE and licence gating, and weekly dependency audits; third-party GitHub Actions are pinned to full commit SHAs, and release orchestration runs through a shared, Scorecard-monitored reusable workflow.

Licensing & openness

nr-llm is free and open source under GPL-2.0-or-later, published by Netresearch DTT GmbH. Its design is documented in the open: 302 Architecture Decision Records set out every security and architecture choice — including the ones on this page — together with the alternatives considered and why they were rejected.

Security vulnerabilities are handled through GitHub's private vulnerability reporting. Transparency is itself part of the posture: nothing on this page is a claim you have to take on trust, because the reasoning behind each control is published as an ADR.

Developers

For developers: the governance contract

Guardrails, approval, and budgets are enforced by the middleware pipeline — you do not call them, you catch what they throw. Every governance failure is a typed exception, so a consuming extension degrades gracefully instead of surfacing a stack trace.

php
use Netresearch\NrLlm\Exception\GuardrailViolationException;
use Netresearch\NrLlm\Exception\GuardrailApprovalRequiredException;
use Netresearch\NrLlm\Exception\BudgetExceededException;

try {
    $response = $this->llm->chat($messages);
} catch (GuardrailViolationException $e) {
    // Input or output blocked by policy — show a safe, generic message.
    return 'This request was blocked by the content policy.';
} catch (GuardrailApprovalRequiredException $e) {
    // Paused: a backend user must approve before the action runs.
    return 'Submitted for approval. It will run once an administrator signs off.';
} catch (BudgetExceededException $e) {
    // This account's per-user AI budget is exhausted.
    return 'The AI budget for your account is used up for now.';
}

Governance & security FAQ

Are my API keys ever exposed in plaintext?

No. Keys are stored as nr-vault identifiers (UUIDs), not values. The plaintext secret is resolved inside the vault's audited HTTP client at call time and scrubbed from memory immediately after; it never appears in nr-llm's database, logs, error messages, or backend output.

Can I run this without sending data to a third-party AI provider?

Yes. For an internal workflow, check the model endpoint, every fallback provider, and each tool and additional model service it calls.

What stops the AI agent from reading secrets or changing data?

48 of the 71 built-in tools are read-only; secret-bearing output is redacted or kept behind disabled raw variants; files such as settings.php are structurally unreadable; tool groups and network egress are fail-closed; and the twenty-three write-capable tools ship disabled and suspend the run for human approval first.

How do I stop one editor from running up the bill?

Set a per-user budget for the editor and follow their recorded usage in Analytics. The preflight check can deny further calls. Concurrent requests and costs recorded after dispatch can still exceed the budget.

Could a model leak a secret it was given?

The guardrail pipeline screens the prompt, the answer, the model's reasoning, the system prompt, vision text, and streamed chunks, masking secret-shaped strings before they are cached, streamed, or displayed — including a secret split across two streaming chunks.

How do I verify a release hasn't been tampered with?

Each release ships a Cosign signature, SHA-256 checksums, SPDX and CycloneDX SBOMs, and SLSA Level 3 provenance you can verify with gh attestation verify.