The router for every profile

Each Octy™ answer that needs a model uses the cheapest model that meets the profile’s quality bar, and every routing decision is recorded for review in LightHouse™.

An Octy profile has no fixed model. Every answer that needs a model goes through the model router, which picks the cheapest model that meets the profile’s quality bar for the answer. Questions answered from retrieval alone, and questions the knowledge does not cover, do not reach the router.

1. Candidates

   : the models allowed where the profile runs, restricted to the profile's own list if it has one

2. Evidence

   : how often each model's answers have met this profile's bar; a model below the bar is dropped, and one not yet measured is tried only after the measured ones

3. Order

   : cheapest first, by the provider's price for a typical answer

4. Answer and check

   : the chosen model writes the answer from the passages; the answer is checked against the profile's bar

   Otherwise: Escalate

   missed the bar: try the next candidate, while model steps remain

5. Record

   : each decision and escalation, with the model, the reason and the cost

The router keeps the candidates the profile allows, orders them by evidence and price, and tries the cheapest first. A model's answer that misses the profile's bar is escalated to the next candidate, up to the profile's model steps. Every rung is recorded as a routing decision.

The quality bar

Each profile sets one of three bars. Every answer that used a model is checked against the bar before it is sent.

Bar	What an answer must do
Standard	Not empty, and cites only the passages it was given
High	As standard; cites more than one source where more than one was found; meets a minimum length
Strict	As high; every sentence carries a citation

A stricter bar increases the estimated cost per answer. Cheaper models miss a stricter bar more often, and more answers are escalated. The exact definitions are in the client documentation.

Candidates

By default the router chooses from every model allowed where the profile runs:

• In a client deployment, the models the gateway’s policy allows and prices, in the deployment’s region. See Inputs.
• Atlantic-hosted, the models Atlantic runs for Octy in its own account.
• Air-gapped, only the models served on the site’s own hardware. A profile that names any other model is refused by the deployment. See Where Octy runs.

A profile can list the models the router may choose from. The list restricts the candidates. The router determines their order by evidence and price.

Escalation

When a cheaper model’s answer misses the bar, the router tries the next candidate. The profile’s model steps setting caps the number of models tried for one answer, which bounds the cost of a single answer. If no answer meets the bar within the allowed steps, Octy returns the retrieved passages with their sources, and the answer that missed the bar is not sent. Octy responds the same way when the monthly cap is reached. Escalation for agents is described in Escalation.

Each escalation is recorded with the reason the cheaper answer missed the bar, such as an empty answer, a missing citation, a citation of a passage it was not given, a single source where several were needed, or a sentence without a citation.

Where the router runs

Profile runs on	The router	Its evidence
Client deployment	The engine’s router, in the deployment, as for the client’s agents	Measured in the deployment, on its answers
Atlantic-hosted	LightHouse’s router for Octy, with the same rules	The profile’s own answers

Test answers are recorded and marked as tests. They are not counted as evidence.

Reading the decisions

The profile’s Costs and questions tab shows the router’s decisions for the period: the model chosen first, how often answers were escalated and to which model, why answers missed the bar, and what each model cost. A deployment’s decisions also appear with its other routing decisions under Agents → Routing, and from GET /api/v1/routing. See Cost analysis and Control.