Routing every call

View as Markdown

Agents from the library ask the router before each call. The tools your engineers already use do not: Claude Code, Cline, Cursor, or a script written against an OpenAI or Anthropic SDK call a model provider directly, where none of your caps, pins or residency rules apply. In LightHouse™, you point them at your own gateway by changing their base URL, and each call asks the router exactly as a library agent’s call does.

Everything that holds for a library agent holds here: your pins and exclusions, your budgets and the kill switch, your residency rule and the injection screen. The answer names the model that ran, and the call is recorded and priced at that model, so the caller sees and pays for the model that answered.

Routing every call adds attributes to routing decisions, so LightHouse must accept the contract revision that carries them before the deployment sends them: upgrade LightHouse first, then raise AAL_CONTROL_PLANE_REVISION on the deployment. The minimum revision and engine are in the compatibility reference.

Switch it on

Switch the router on in LightHouse, under Agents → Routing (Control). While it is off, auto means your baseline_model and every other call keeps the model it names.

Tell the gateway what auto means. In your gateway’s own policy, add a routing.compat section. It lives in your policy and not in LightHouse, because it decides what a person’s own tool is served:

routing:
  compat:
    baseline_model: eu.anthropic.claude-sonnet-4-5   # what "auto" means without the router
    default_class: drafting     # the class for a call that says none (optional)
    named_model: pin            # pin (default) or default
    auto_model: auto            # the model id a caller sends to ask the router

Without baseline_model, a request for auto is refused. Without default_class, a call whose class cannot be found keeps the model it named.

Issue each person a key for each tool. A key belongs to one person of one client for one tool. It carries the tier and data class the call runs under, and a request cannot change them:
Terminal window
```
python -m aal_engine.gateway.keys --store var/gateway.sqlite issue \
  --client <client> --person <person id> --tool claude-code \
  --tier S1 --data-class internal --days 90 --by <your name>
```
The command prints the key once and does not store it, so store it with other secrets. --person is a stable id or a hash, not an email address. A key lives at most 366 days. --models narrows the models the key may use; without it, the policy’s list applies. revoke and list are the other two subcommands.

Point the tool at the gateway. The gateway listens on port 8090 unless AAL_GATEWAY_PORT says otherwise; GATEWAY below is its address and KEY the key from step 3.

Tool	Setting
Claude Code	`ANTHROPIC_BASE_URL=GATEWAY`, `ANTHROPIC_AUTH_TOKEN=KEY`, `CLAUDE_CODE_MAX_RETRIES=0`
Cline	API provider Anthropic, base URL `GATEWAY`, key `KEY` (or OpenAI-compatible, base URL `GATEWAY/v1`)
Cursor	OpenAI API key `KEY`, Override OpenAI base URL `GATEWAY/v1`
OpenAI SDK	`OpenAI(base_url="GATEWAY/v1", api_key=KEY)`
Anthropic SDK	`Anthropic(base_url="GATEWAY", api_key=KEY)`

The gateway takes the key as Authorization: Bearer aal-… or x-api-key: aal-…. Claude Code needs ANTHROPIC_AUTH_TOKEN: with ANTHROPIC_API_KEY alone it reports “Not logged in”. CLAUDE_CODE_MAX_RETRIES=0 stops it retrying a refused call ten times over about three minutes.

The gateway speaks two dialects: Anthropic’s Messages API (POST /v1/messages) and OpenAI’s chat completions (POST /v1/chat/completions). A request in the backend’s own dialect passes through unchanged, tools included. A request in the other dialect is converted when it is plain text, and refused when it carries tools, images or documents, because the two tool protocols differ in ways a conversion would get wrong silently.

Which model a request gets

In order:

The injection screen reads the person’s latest message first. A request it refuses is refused before anything is classified, routed or spent.
The model the request names. auto (or whatever auto_model is set to) asks the router, starting from baseline_model. Any other model is the caller’s pin: the caller is served the model it named, and the router decides only in shadow. To treat a named model as a starting point the router may improve on, set named_model: default.
The router, as for any call: pins and exclusions, residency, the allow list, evidence and the bar for the task class.
The person’s key. If the router chose a model the key does not allow, the call keeps the model it named, and the decision’s reason is key_not_allowed.
Admission, on the model that will run. The allow list, the budget and the kill switch are checked against the chosen model, so the router can never widen what a call may use or spend.

Every answer says what happened in four headers. These are from the engine’s own test of this path, with the router on, baseline_model set to Opus 4.6 and default_class: summarising, against a fake backend and invented text:

# A request for "auto", with X-AAL-Task-Class: summarising
x-aal-model: eu.anthropic.claude-haiku-4-5
x-aal-requested-model: auto
x-aal-routing: route
x-aal-task-class: summarising

# A request that names eu.anthropic.claude-opus-4-6
x-aal-model: eu.anthropic.claude-opus-4-6
x-aal-requested-model: eu.anthropic.claude-opus-4-6
x-aal-routing: off
x-aal-task-class: summarising

x-aal-routing is the decision’s mode: route (chosen on evidence), off (the call kept the model it named, with the router’s choice recorded in shadow) or fallback. The model also appears in the body’s own model field.

A named model is treated as a pin unless the policy says otherwise, so that a tool requesting one model is not served another without notice.

Where the task class comes from

The router needs the kind of work a call is. A library agent’s card says so. A call from your own tool does not, so the gateway reads it from the first of these that gives one:

`class_source`	Where the class came from
`header`	The caller’s `X-AAL-Task-Class` header, when it names one of the eight classes: `extraction`, `classification`, `retrieval_qa`, `summarising`, `drafting`, `coding`, `tool_use`, `long_context`
`classifier`	The classifier model, at or above its confidence threshold
`rules`	The classifier’s deterministic rules, at or above the threshold
`default`	`routing.compat.default_class`, because the classifier was off or below its threshold
`none`	No class. The router has no evidence, and the call keeps the model it named

The gateway does not guess a class. A call with no class is not routed.

The classifier

The classifier is off by default. It is switched on in the gateway’s policy and cannot be switched on from LightHouse:

routing:
  classifier:
    mode: model                 # off (default) | rules | model
    model: eu.anthropic.claude-haiku-4-5   # required for "model"
    threshold: 0.7              # below it, the default class
    cap_usd_per_day: 1.0        # its own spend, per client per UTC day
    max_chars: 4000             # how much of the latest message it reads

rules reads the request’s shape and words with fixed rules and calls no model.
model asks a small model through your own gateway, so the call it classifies never leaves your account to be classified. The classification is budgeted, screened and killable like any other call.
It is capped per day, checked before each classification on its projected cost. A classification that would pass the cap is not made, and the rules or the default decide instead.
It reads the latest message only, and only the label and its confidence are recorded, never what it read. A classification failure falls back to the default and does not refuse the call.

See what it did

On Agents → Routing, the section Every call, including agents you did not build with us shows the compatible-endpoint calls, the estimated saving on them, the calls that named their own model (decided in shadow, with the saving they would have had, estimated and not counted), and the calls where the key did not allow the router’s choice. A second section, The task classifier, shows its state, its daily cap, how many calls it classified and how many fell below the threshold, and its own cost, shown separately from every saving and not deducted from one.

Each decision in the list also says where it came from (surface: compat), where its class came from and what classifying it cost.

By the API, as any viewer:

curl "https://$AAL_TENANT.atlanticagenticlabs.com/api/v1/routing/compat?since=2026-11-01&until=2026-11-30" \
  -H "Authorization: Bearer $AAL_KEY" \
  -H "CF-Access-Client-Id: $CF_ACCESS_CLIENT_ID" \
  -H "CF-Access-Client-Secret: $CF_ACCESS_CLIENT_SECRET"

import os

import httpx

headers = {
    "Authorization": f"Bearer {os.environ['AAL_KEY']}",
    "CF-Access-Client-Id": os.environ["CF_ACCESS_CLIENT_ID"],
    "CF-Access-Client-Secret": os.environ["CF_ACCESS_CLIENT_SECRET"],
}
doc = httpx.get(
    f"https://{os.environ['AAL_TENANT']}.atlanticagenticlabs.com/api/v1/routing/compat",
    params={"since": "2026-11-01", "until": "2026-11-30"},
    headers=headers,
).json()
for row in doc["by_surface"]:
    print(row["name"], row["decisions"], "decisions, estimated saving", row["saving_usd"])
print("named their own model:", doc["caller_pinned"])
print("classifier:", doc["classifier"]["name"])

const q = new URLSearchParams({ since: "2026-11-01", until: "2026-11-30" });
const doc = await (
  await fetch(`https://${process.env.AAL_TENANT}.atlanticagenticlabs.com/api/v1/routing/compat?${q}`, {
    headers: {
      Authorization: `Bearer ${process.env.AAL_KEY}`,
      "CF-Access-Client-Id": process.env.CF_ACCESS_CLIENT_ID!,
      "CF-Access-Client-Secret": process.env.CF_ACCESS_CLIENT_SECRET!,
    },
  })
).json();
for (const row of doc.by_surface) console.log(row.name, row.decisions, row.saving_usd);
console.log("named their own model:", doc.caller_pinned, "classifier:", doc.classifier.name);

Through MCP, the read-only tool is routing_compat. since, until and deployment narrow the answer. An abridged answer, from the control plane’s own tests with invented decisions:

{
  "by_surface": [
    {
      "key": "compat",
      "name": "Agents pointed at the gateway (compatible endpoints)",
      "decisions": 4, "routed": 3, "off": 1, "fallback": 0, "escalated": 0,
      "saving_usd": 0.35, "off_saving_usd": 0.4,
      "quality": 0.91, "bar": 0.9, "estimated": true
    },
    { "key": "library", "name": "Agents built on the engine", "decisions": 5, "…": "…" }
  ],
  "class_sources": [
    { "key": "header", "name": "The caller's X-AAL-Task-Class header", "decisions": 2 },
    { "key": "classifier", "name": "The classifier model", "decisions": 2, "confidence": 0.655 }
  ],
  "caller_pinned": 1,
  "key_not_allowed": 0,
  "classifier": {
    "state": "model", "name": "On, a small model through your own gateway",
    "cap_usd_per_day": 2.0, "classified": 2, "below_threshold": 1,
    "cost_usd": 0.00011, "calls": 1, "estimated": true
  },
  "label": "estimated"
}

saving_usd is the estimated saving on calls the router changed. off_saving_usd is what calls decided in shadow would have saved: reported and not counted. The classifier’s model calls are ordinary model_call records in the pack routing, agent classifier, so Cost shows them too.

Did the cheaper model answer as well?

A saving is only a saving if the answers hold up. The router chooses on measured evidence from a public suite; the sampler checks the choice on your own traffic. It samples the calls of agents built on the engine. It does not sample calls that came through the compatible endpoints, whose prompts are not kept as candidates. It is off by default, capped, and switched on in your gateway’s policy:

routing:
  sampling:
    enabled: true
    judge_model: eu.anthropic.claude-sonnet-4-5   # required when enabled
    every_hours: 24
    sample: 10                  # calls per run
    cap_usd: 2.0                # per run, judge included; checked before each call
    tolerance: 0.05
    capture_rate: 0.1           # share of eligible calls kept as candidates
    pool_per_day: 50            # at most this many candidates kept a day

How it works, in order:

A share of the calls of library agents that the router changed (or, with it off, would have changed) is kept as a candidate: the masked prompt, sealed in your own gateway store, for as long as your policy’s retention.prompts: days:N already says. With prompt retention off there are no candidates, so the sampler can only reread what you already chose to keep.
On your connector schedule, the gateway reruns each sampled prompt on both the model the router chose and the model the call would have used without routing, through your own gateway, under its own envelope, so its spend is on its own line and the kill switch stops it.
A judge model you name scores both answers blind, against the same fixed criteria for the task class.
An answer is as good when its score is within the tolerance you set of the baseline’s. Both costs are priced on your one price card.
One eval record per task class and model pair (suite routing.quality) leaves your account, with counts, mean scores and costs. The prompt and both answers stay in memory for the run and are never stored or sent.

LightHouse shows it as Did the cheaper model answer as well? on Agents → Routing, and in Insights. By the API, GET /api/v1/routing/quality (since, until, task_class), or the MCP tool routing_quality. An abridged answer, from the control plane’s own tests with invented samples:

{
  "suite": "routing.quality",
  "totals": { "sampled": 10, "as_good": 9, "as_good_share": 0.9, "cost_delta_usd": 0.198, "spent_usd": 0.31, "estimated": true },
  "pairs": [
    {
      "task_class": "retrieval_qa",
      "model": "eu.anthropic.claude-haiku-4-5",
      "baseline_model": "eu.anthropic.claude-opus-4-6",
      "routing_mode": "route",
      "sampled": 10, "as_good": 9, "worse": 1, "better": 2,
      "quality_routed": 0.88, "quality_baseline": 0.9, "quality_delta": -0.02,
      "cost_delta_usd": 0.198, "spent_usd": 0.31,
      "judge_models": ["eu.anthropic.claude-sonnet-4-5"],
      "sentence": "Answered as well as eu.anthropic.claude-opus-4-6 on 9 of 10 sampled retrieval q&a calls: eu.anthropic.claude-haiku-4-5 against the model the calls would have used without routing.",
      "estimated": true
    }
  ],
  "label": "estimated"
}

It reports “9 of 10 sampled calls” and is not a rate for all calls; quality_delta can be negative. With the router off, a pair is a shadow pair (routing_mode: off) and reads “Would have answered…”. These evaluations measure the router, not an agent, so they do not appear on any agent’s page. Every cost in them is estimated, at your deployment’s price card.

What it does not do

It does not keep what was said. The gateway keeps counts, costs, reason labels and scopes, and the tests verify this on every build by passing canary text through every route. The one exception is a candidate prompt for the sampler, which exists only if you set prompt retention yourself.
It does not change your policy. Everything on this page that spends money or decides what a person’s own tool is served (routing.compat, routing.classifier, routing.sampling) is in your gateway’s own policy. LightHouse cannot change it.
It does not read grounding into a chat. Grounding checks an answer against fenced material, and a chat has none, so it is off on these endpoints.
It does not sample these calls for quality. The sampler’s candidates come from library agents’ calls only (above).
It does not route what it cannot place. No class and no default means no router decision, and the call keeps its model.

See also Control for pins, exclusions and the explanation of every choice, A decision, worked for the arithmetic, and The gateway.