Agent engineering at Netresearch

Real friction becomes durable engineering knowledge.

Q: What role does AGENTS.md play?

AGENTS.md is a compact index pointing to the authoritative project truth. Architecture, commands, dependencies and details stay in the actual sources such as docs, composer.json, Makefile, DDEV configuration and scripts. The harness verifies that references and documented commands are correct.

Skills are not the system. They are one possible output of a closed learning loop that classifies experience from real agent sessions, materializes it, verifies it, enforces it technically and carries it into the next projects as a versioned dependency.

Understand the system

Problem-drivenLearnings emerge retrospectively from real work.

Authority firstEvery truth moves to its owner; verifiable rules become gates.

Source of truthProject knowledge stays in the project and is referenced.

Closed loopMaterialize, verify, distribute, learn again.

The big picture

A learning and enforcement system, not a prompt collection.

The decisive change of perspective: domain skills are the visible results. The real architecture lies in the process that decides what an experience should become, where it permanently belongs and how its effect is verified.

Short answer: Real work produces friction and learnings. Retro routes them into the right form of persistence. Skill-repo standardizes skill artifacts. Assessment verifies requirements. The harness anchors enforcement in the project. Distribution carries the results into further projects as versioned dependencies.

Real agent work

A coding agent works in a real repository, with real constraints, tools, CI and failure modes.

Friction & Learning

Mistakes, unnecessary loops, good new patterns or missing rules become visible as a learning signal.

Retro classifies

Not every learning belongs in a skill. Authority, enforceability and reach are determined – in that order.

Materialize

Skill, checkpoint, project rule, harness artifact, upstream patch or personal rule.

Verify

Validators, evals, assessment, CI, hooks and review make the effect visible.

Reconcile & prune

Once a fact lands with its canonical owner, the local copy shrinks to a reference. A system that can only add degrades.

Version & distribute

Skills and rules become reproducibly available as engineering dependencies.

Next session

The next agent already starts out with the preserved experience.

Feedback loop: New work produces new evidence – the system keeps learning.

Memory

Skills are our consolidated experience.

A coding agent has no memory across sessions. The question is therefore not how a session gets stored, but what form an experience has to take so that the next session already brings it along. A skill does not store the session – it stores the reusable change in behavior distilled from sessions.

          raw experience · episodic, bound to one session

            ↓

          retro = consolidation

            ↓

          skill  ·  rule  ·  gate

            ↓

          consolidated experiential and procedural knowledge

Session transcript

Raw experience: complete, unsorted and normally transient. Raw material, not result.

Skill

Consolidated, reusable experience. Workflow, decisions and failure patterns, distilled from real sessions.

Checkpoint / harness

Executable and enforceable experience. The lesson no longer sits in prose but in a check that fails without an agent involved.

AGENTS.md

Project instruction and index. What holds in this repository, and where the authoritative sources are.

Docs / code / config

Canonical facts. They belong to the project or to upstream and are referenced, not copied.

personal-rule

A user-specific durable instruction – a persistent directive, not a recollection of past sessions.

consolidation

Consolidating includes deleting.

A system that can only add gets worse over time. That is why reconcile & prune, outcome grading and audit are part of the loop: an experience that is superseded, refuted, or has arrived at its canonical owner is reduced to a reference or removed. That experience reused unfiltered carries errors forward, and that even correct-looking runs can be misleading as experience, is empirically documented.

no memory store

Not everything is called memory.

There is no vector database, no similarity search over old sessions and no archive of complete past solutions kept “just in case”. Each form is named after what it does – instruction, check, fact – instead of putting everything under one shared storage metaphor.

Retrieval is part of the problem: a perfectly preserved experience that is not activated at the right moment is worthless. So the measurement happens twice – whether a loaded skill helps, and whether the agent works out for itself that it needs it. See verification.

Origin

Retro is the learning router.

New skills are not created because a topic looks big or interesting. They are created when real work exposes a reusable gap. And even then a skill is only one of several possible destinations.

personal-rule

Personal rule

A stable preference or way of working that applies across projects, but only to one person. A durable instruction – which is why it is no longer called user-memory; that name survives only as an alias.

project-rule

Project rule

A convention or a command that belongs to exactly this repository and is anchored in its agent index.

skill-update

Skill update

An existing, reusable skill has a genuine gap, wrong guidance or a missing failure pattern.

new-skill

New skill

The friction reveals a new, recurring capability category that no existing skill sensibly owns.

checkpoint

Verifiable invariant

A rule can be checked against a project mechanically or as a defined LLM review.

harness-artefact

Technical enforcement

The right answer is a hook, CI gate, linter, ruleset, template or another self-sustaining project artifact.

canonical-source

Canonical source

The fact belongs to an artifact outside the agent system – official documentation, code or a schema. The fix goes there as a patch; the skill keeps only a reference plus the agent-specific delta.

1. Authority

Who owns this truth? A fact about the world moves to its canonical owner – often outside the agent system. A skill owns only agent behavior and its own procedure.

2. Enforceability

Can the rule be enforced technically as a gate, check or artifact? That is stronger than a reminder written in prose.

3. Reach

Only then is it decided whether the learning is reusable personally, project-locally or organization-wide.

Paired materialization: When a rule is technically enforceable in the current repository and the same gate is needed in sibling projects, a learning can be materialized on two tracks: the concrete gate in the project plus a skill change carrying the installation recipe for further projects. The same pairing also works upstream: the patch to the canonical owner plus the skill cleanup that reduces the local copy to a reference once the patch is accepted.

Project context

The project describes itself.

AGENTS.md is a compact index, not a second set of documentation. From there the agent is directed to the authoritative sources. Knowledge stays where it is maintained anyway – and stays independent of a particular agent client or central knowledge service.

docs/ARCHITECTURE.md

Architecture, boundaries, important flows and long-term structural decisions.

composer.json / composer.lock

Declared commands, packages, constraints and the reproducible dependency state.

Makefile / scripts/

Project operations that are actually executable, instead of copied command lists.

.ddev/config.yaml / Docker

Local development environment, services and technical runtime configuration.

CI / Hooks / Rulesets

The verifiable part of the project constitution: what really blocks and what really has to pass.

Important: Project context does not have to be copied into every skill. The skill knows where to look; the repository owns the truth. Among other things, the harness verifies that references resolve and that documented commands match the real targets.

Skill quality

Skill content has to earn its context price.

A skill is not a catch-all for everything known about a topic. It carries exactly the information whose presence at the right moment measurably improves the agent’s behavior.

Organization- and project-specific knowledge

Conventions, tooling and decisions that a general model cannot derive from public knowledge.

Version or ecosystem facts that models reliably get wrong

Above all new, niche or moving facts – as long as no better dynamic owner exists.

Retro-born Failure Patterns

Symptom → cause → required behavior → verification. A real map of the minefield instead of generic best practice.

Executable validators

Deterministic checks belong in scripts or checkpoints rather than in prose that is reinterpreted on every run.

Inference Suppression

Rules such as “read source X, never guess Y”, where exactly that guessing has led to errors or loops in real sessions.

Anti-Rationalization Guards

Clear barriers against known shortcuts: no claim of success without an executed check, no skipping of a gate.

canonical ownership

One fact, one owner.

Facts and trigger phrases should have a canonical home – often outside the agent system, in official documentation or code. Other skills reference the owner instead of copying the same fact and drifting apart later.

eval evidence

“The model forgets it” is a testable claim.

If skill content is justified only by a model knowing something but not activating it reliably, an A/B eval should make the benefit visible. The measured delta then holds for one skill version under one model, one harness and one eval set – a stronger actor may derive the same thing on its own, and then only the context cost remains.

          KEEP  workflow · failure pattern · org policy · decision rule · verification

          REFERENCE  long examples · API tables · templates · detailed checklists

          DO NOT DUPLICATE  facts with another canonical owner · project state · generic textbook material

Verification

Guidance becomes a verifiable specification.

Checkpoint-enabled skills define requirements that an assessment can check systematically against a repository. Quality therefore does not start with manual bug hunting but with a structured gap report.

01Discover skills

Matching skills and their preconditions determine the relevant scope of the check.

02Check mechanically

Evaluate files, contents, regex, JSON/YAML, commands and platform state deterministically.

03LLM Reviews

Only where genuine semantic judgment is needed; grouped by domain.

04Gap Report

Errors and warnings become a prioritized task list instead of iterative discovery.

05Fix & Re-Verify

Fixes can be routed to the responsible skill and then verified again.

Mechanical checkpoint

For stable, deterministically measurable invariants. Fast, reproducible, free of interpretation.

LLM review

For architecture, documentation or quality judgments that need an understanding of context.

Harness gate

For rules that have to block before an action – a hook, CI, branch protection or linter.

Two layers of evals

per-skill A/B eval

Does the skill help once it is loaded?

The same prompt with and without the skill in context. What it measures is a delta against a no-skill baseline – bound to a skill version, a model, a harness and an eval set, not a universal property of the skill.

system eval

Does the agent realize it needs the skill?

A realistic, deliberately underspecified request – naming no skill, no tool and no method. Discovery and correct activation are part of the test rather than a precondition of it.

Why both: an eval that hands the agent the method measures prompt compliance. Only the Open Forward Review in agent-system-evals asks whether the agent frames the job correctly by itself – scored across several dimensions rather than as one number, because a run that investigates well and reports nothing useful and a run that guesses correctly are both failures.

Enforcement

The harness makes agent readiness self-sustaining.

The harness does not merely install hints for the agent. It creates artifacts that keep working afterwards through CI, hooks, branch protection and repository conventions, independently of the skill run.

LEVEL 1

Basic

AGENTS.md exists as a compact index and documents the central project commands.

LEVEL 2

Verified

CI verifies harness integrity; references resolve; documented commands match the targets; architecture is discoverable.

LEVEL 3

Enforced

Required checks, hooks, PR templates and drift detection make important rules technically hard to bypass.

Integration through artifacts: The harness verifies the outputs of specialized skills, not their internal implementation. A correctly generated AGENTS.md, a fitting hook or a valid CI rule is verifiable regardless of which agent or skill produced it.

Distribution

Skills are versioned engineering dependencies.

A skill is not just a Markdown file in a home directory. Repository structure, versioning, trust, discovery and project indexing form a reproducible supply chain.

Source Repository

SKILL.md, references, scripts, checkpoints, evals, manifest and validation live versioned at one canonical source.

Package / Direct Source

A Composer package, a skill inside an ordinary package, or a locked GitHub or project source. Trust is managed explicitly.

Project discovery

The project knows its skills, can index them and loads detailed knowledge progressively, only when it is needed.

git-workflowdocker-developmentsecurity-auditphp-modernizationgo-developmentcontext7data-toolsgithub-projectenterprise-readinessconcourse-cijiraTYPO3 domain skills

Why it learns

The next session should not have to pay for the same lesson twice.

The value is not in owning as many skills as possible. The value is in turning the cost of real mistakes into permanently better starting conditions.

          historical problem

            ↓

          retro detects friction + learning

            ↓

          classified by authority, enforceability and reach

            ↓

          skill / checkpoint / harness / project rule / upstream patch

            ↓

          Eval + Assessment + CI/Hook

            ↓

          the next real session starts out better

The goal: Experience is not merely documented, it is converted into the strongest sensible form. What can be verified is verified. What can be enforced is enforced. What needs judgment and context is preserved as precise agent guidance.

Local Q&A

Ask this page – locally in your browser.

Where the browser provides it, the assistant uses the built-in Chrome Prompt API via LanguageModel. The page itself is the grounding source. There is no dedicated Q&A backend endpoint and no simulated model answers.

On-device Q&A with a real fallback

If the local language model is available, it answers questions solely from this page. If the API is unavailable, a deterministic page search takes over and shows the most relevant sourced sections.

Checking local AI …

FAQ

Answers to the core questions.

What is the Netresearch Agent Engineering System?

A closed learning and enforcement loop for coding agents. Real work produces friction and learnings; retro classifies them; skills, checkpoints, project rules, harness artifacts or upstream patches to the canonical source materialize them; assessment and CI verify the effect; the next session benefits from it.

How do new skills come about?

Not from speculative topic lists, but retrospectively from real problems, recurring friction and reusable learnings. A new skill is only one possible destination; facts owned elsewhere go upstream to their canonical source, and rules that can be checked mechanically are preferably materialized as a checkpoint or as enforcement.

Where is the memory in this system?

In the form an experience takes after consolidation. The session transcript is raw experience and transient; retro consolidates it into reusable experience as a skill, executable or enforceable experience as a checkpoint or harness artifact, project instruction in AGENTS.md, canonical facts in docs, code and config, and a durable personal instruction as a personal-rule. There is deliberately no vector database and no similarity search over old sessions: consolidating includes deleting, because experience reused unfiltered carries errors forward.

What role does AGENTS.md play?

AGENTS.md is a compact index pointing to the authoritative project truth. Architecture, commands, dependencies and details stay in the actual sources such as docs/, composer.json, Makefile, DDEV configuration and scripts. The harness verifies that references and documented commands are correct.

What is the difference between a skill and a checkpoint?

A skill carries workflow, decisions, failure patterns and non-mechanical guidance. A checkpoint encodes a verifiable invariant that can be evaluated against a project mechanically or as a defined LLM review.

How is it prevented that skills grow into large knowledge dumps?

Skill content has to earn its context price. Preferred content is organization-specific knowledge, real failure patterns, volatile facts that models reliably get wrong, executable validators, inference suppression and anti-rationalization guards. Facts and triggers have a canonical owner – often outside the agent system; details are referenced on demand.

How are skills and rules verified?

Automated assessment discovers matching checkpoint-enabled skills, checks preconditions, runs mechanical checks and grouped LLM reviews, produces a gap report and can route fixes to the responsible skill. After that, verification runs again.

How are agent rules enforced technically?

When a rule is enforceable, it is not merely preserved as prose. Depending on the case it is materialized as a checkpoint, pre-commit hook, CI check, linter rule, branch protection or another harness artifact. The principle is: authority first – who owns this truth? – then enforceability, then reach.

How are skills distributed?

Skills are versioned engineering dependencies. They can reach projects via Composer packages, skills inside ordinary packages, directly locked GitHub sources and further distribution paths. Discovery, trust and AGENTS.md indexing form one shared pipeline.

Primary sources

The architecture is documented in executable form.

This page summarizes the publicly versioned Netresearch artifacts. For implementation details, the respective repository remains the canonical place.

retro-skillFriction detection, learning routing and materialization. agent-harness-skillAgent readiness, verification, bootstrap and enforcement. skill-repo-skillSkill structure, quality rules, release and repository standards. automated-assessment-skillCheckpoint runtime, gap reports, autofix and re-verification. agent-system-evalsOpen Forward Reviews: measures on an underspecified request whether the agent frames the job correctly by itself. composer-agent-skill-pluginDiscovery, trust, locking and Composer-based distribution. Netresearch on GitHubDomain skills and generic engineering skills as the visible outputs of the system.