ADR 138 · Accepted

ADR-138: Criteria-mode selection matches the operation, not only the criteria

Context

A criteria-mode LlmConfiguration carries no model relation. Its model is chosen at call time from stored criteria — capabilities, adapter types, a context-length floor, a cost ceiling. Until now that was the whole input. ModelSelectionService::resolveModel() never learned which call it was resolving for.

So a configuration whose criteria say {"adapterTypes": ["ollama"]} could serve a tool call with a model whose own record states it cannot do tools. The selection succeeded, the adapter was built, and the request failed at the provider — as a transport-shaped error, several layers away from the configuration that caused it.

Two related facts shaped the fix.

The same call resolves twice. embedForConfiguration() resolves once outside the pipeline to build the embedding cache key and once inside the terminal to pick the adapter. Two resolutions that can disagree would let cache entries stored under model A serve a call that ran against model B. The eager streaming-capability check had the same shape.

The capability column never reached the entity. Extbase resolves property types through Symfony's PropertyInfo, whose ReflectionExtractor infers a collection from an adder/remover pair. Model::addCapability() / removeCapability() inflect to $capabilities, so the property resolved as array; the DataMapper has no array mapping and dropped the column. Every repository-loaded model came back with an EMPTY capability set. The pre-existing capabilities criterion therefore matched nothing in production either — the defect this ADR fixes was one of two, and the second one hid the first.

Decision

Thread the operation into the resolution. ModelSelectionServiceInterface::resolveModel() and ConfigurationCallPlanner::resolveModel() take a ?ProviderOperation. It has no default: every resolution belonging to a concrete call must name that call, and the one caller that genuinely has none — the bare adapterFor() lookup behind getAdapterFromConfiguration() — says null out loud. In criteria mode the capability the operation requires is merged into the criteria under its own key before findMatchingModel() runs. Fixed mode is untouched: the operator named that model, so nothing is being chosen and there is nothing to constrain.

Both resolutions of one call pass the same operation. The embedding cache-key site and the embedding terminal both pass Embedding; the eager streaming check calls the planner directly with Stream rather than routing through the operation-less public entry point. A unit test asserts that the two embedding resolutions receive the same operation and return the same model.

Restore the capability mapping. An explicit @var string on Model::$capabilities puts PhpDocExtractor — which runs first — back in charge of the property type. Without it this ADR would ship decoration: every model would read as undeclared and the new check would never fire.

An empty capability CSV means undeclared, not "cannot". The field is optional and many installations never filled it. The operation-derived check is therefore skipped for such a model, in both switch positions. This is a separate criteria key from capabilities precisely so the two can differ: what an operator explicitly asked for is still matched strictly, and a model that declares nothing still fails that.

Enforcement is a fail-closed switch, following ADR-113. routing.operationCapabilityEnforcement defaults to enforce. Only a literal observe observes — a missing value, a malformed routing section, an unreadable extension configuration and a typo all enforce, so a broken setting cannot silently disable the axis. Fail-closed governs the SWITCH, not the empty CSV: reading an absent statement as a denial would break working installations for a fact nobody ever stated.

The map is narrower than the vocabulary. Only chat, vision and tools are enforced. chat is the only token every producer writes; the other two are written unevenly, and this decision rests on that being understood rather than glossed over.

Since #671 each discoverer seeds only what its provider's API substantiates. Mistral, OpenRouter and Ollama report tools and vision per model, so a record from them lacking one is a statement. Anthropic and the curated OpenAI / Gemini entries write both from curated knowledge. Groq reports nothing — its listing carries no capability field, so its models are seeded chat alone and a missing tools there is a gap, not a statement.

Enforcing vision / tools against a Groq-discovered corpus therefore refuses working models until an operator completes the checkboxes. That is the reason the enforcement switch exists, and the reason observe is the safer default for an upgraded installation.

No discoverer writes completion or embeddings at all, and streaming is written only by Gemini (derived from supportedGenerationMethods) and the curated OpenAI / Anthropic / Gemini entries — requiring those would refuse models that work, on the strength of a field their own discoverer never filled. A requirement no producer satisfies is not a check, it is an outage.

A misconfiguration is named, not disguised. When enforcement is on and the criteria match models but none that can serve the operation, resolution throws UnsupportedFeatureException naming the configuration, the capability and the operation. Criteria that match nothing at all still return null — that is the pre-existing "has no model assigned" condition and it keeps its behaviour.

UnsupportedFeatureException stays UNKNOWN in the failure classifier, and therefore not retryable. This is deliberate and must not be "fixed": the exception now reports the installation's own misconfiguration. A retry cannot repair it, and a fallback that silently answered from another configuration would hide the very defect this ADR exists to surface. The operator would see a working system quietly running on the wrong model.

Boundary: the generic path keeps its adapter checks

chat(), complete() and streamChat() prefer the default DB configuration and reach resolveModel() through it — those calls get the check. Their ad-hoc branch does not, and neither do embed(), vision() or chatWithTools(), which always run ad-hoc: they synthesize a transient configuration carrying only an identifier — no model, no provider, no criteria — and resolve a provider by key instead. There is no model record to match against, so there is nothing for this ADR to check.

Those paths keep the adapter-level guards they already have (ToolCapableInterface, VisionCapableInterface, supportsFeature('embeddings')). This is stated as a boundary rather than papered over: making the generic path capability-aware means giving it a model record, which is a different change with a different blast radius.

Consequences

Fixed-mode configurations — the majority — are unaffected.

Criteria-mode configurations become stricter for chat, vision and tool calls, and the capability criterion starts working at all now that the column reaches the entity. An installation whose model records understate what their models can do will see a resolution refused where it previously succeeded and failed later at the provider. The escape hatch is one setting, and the real fix is one checkbox on the model record.

Restoring the capability mapping is visible beyond selection: anything reading getCapabilities() off a repository-loaded model saw an empty string before and now sees the persisted value.

Revisit when

The discoverers write the capabilities they currently omit. streaming is the nearest, but "closing the gap" now means something different than it did before #671: a discoverer may only write a token its provider's API substantiates, so Stream becomes enforceable when the remaining providers report streaming, not when someone fills the field in. Where an API stays silent — Groq's listing has no capability field at all — the gap closes through the operator's own checkboxes, which is a different mechanism and a slower one.

embeddings and completion need a producer before they mean anything at all.

Also revisit if the generic path ever gains a model record. The boundary above exists because it has none, not because operation matching is unwanted there.

← All ADRs