Provider routing

Later

Later. Account-wide defaults for which provider serves a model. A request can override them with a provider object. Not applied yet.

Order and sort

Sort providers by

Try first, in order

Fall back to other providers when the first is downOnly use providers that support every parameter in the request

OpenAIAnthropicGoogleMistralGroqTogetherFireworksAWS BedrockAzure OpenAIVertex AI

Skip providers that may store or train on promptsZero data retention providers only

Max input $ / M

Max output $ / M

Measured at

Preferred min throughput (tokens/s)

Preferred max latency (s)