Model routing

Definition

Model routing is the set of provider-side decisions that map an API request to a serving path: which model version, which serving tier, which region, and whether the request is served directly, queued, cached or redirected to a different backend. It happens inside the provider’s trust domain.

The problem

The consumer addresses a model name and observes a response. Everything between the two is invisible unless the provider chooses to disclose it in response metadata, so a change of underlying version, tier or region is not detectable at the API boundary in advance.

Why it matters

Routing is where behavioural and reliability differences originate. An application pinned to a model name can experience a change it did not request and cannot see, and an incident attributed to "the model" may in fact be a routing change.

Practical example

Latency and error-rate shifts that correlate with nothing the consumer changed are the signature of a routing or serving-tier change. Detecting them after the fact requires capturing response metadata, not just status codes.

How RELIASTRA approaches it

RELIASTRA records what it can observe - endpoint, region, status, latency, timestamp - and states explicitly what its record does not cover. Model-level endpoints and per-route API latency are not part of the public observatory today, and the hub says so in its own sections rather than in a footnote.