Add a credential
provider:
An optional
label names the credential for your own reference. The credential is
verified live before being stored — a bad key or a typo’d Azure resource name is
rejected immediately with the provider’s own error, nothing is persisted. For azure
and azure-foundry, every entry in modelDeployments gets its own live check (not
just the first) — the credential is stored as long as at least one mapping works, and
the response’s model_availability reports which of your mappings are actually good
so you can fix the rest.
Azure only: deployment names are whatever you named them in your own Azure
resource, so the gateway can’t infer which deployment serves which model. Map it
explicitly with A request for
modelDeployments, keyed by the gateway’s model id:openai/gpt-5.4 then routes to my-gpt5-deployment on your Azure
resource. A model with no entry in the map falls back to the gateway’s own
(shared-key) route for that model, if one is configured.Azure AI Foundry only: Two cases still need an explicit
endpoint accepts any of the four shapes Azure hands you —
paste it exactly as shown in the portal:- Serverless model catalog (Llama, Mistral, Phi, etc.):
https://<resource>.<region>.inference.ai.azure.com - Unified project/hub endpoint:
https://<resource>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview - Shared regional Cognitive Services endpoint (same Model Inference API as the one
above, on the operator’s own regional host):
https://<region>.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview - Classic/modern Azure OpenAI resource (Responses API):
https://<resource>.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview— this one speaks the OpenAI Responses API, a different wire format from the other three (which are all Chat Completions); the gateway handles the distinction internally.
api-version query parameter; a missing one is rejected at submission
time.Unlike Azure OpenAI, modelDeployments is optional here — Foundry deployments keep
their catalog model name unless you rename them, so the gateway tries every one of its
registered Foundry-native models (Llama-3.3-70B-Instruct, Mistral-Large-2411,
Phi-4, deepseek-v4-flash, deepseek-v4-pro, deepseek-v3-2, deepseek-v3-2-speciale)
against your endpoint and keeps whichever respond:modelDeployments entry (same syntax as Azure OpenAI
above): a deployment you renamed, and models that already have a native route through
another provider (e.g. openai/gpt-5.4). An explicit mapping always wins over
auto-detection, and a model with no mapping at all falls back to the gateway’s own
(shared-key) route, if one is configured.Bedrock only:
apiKey is a Bedrock API key
(from the Bedrock console’s API keys page) — a bearer token, not an IAM access
key ID/secret pair. region must be the same AWS region the key was generated for
(e.g. eu-west-3), and must be in the us-*, eu-*, or ap-* region groups —
any other region is rejected when you add the credential. Requests route to the
same geography your key lives in.Even in a supported region, the specific model still has to be enabled for your AWS
account (Bedrock’s Model access page) — an unenabled or provider-retired model
returns an AWS error at request time (e.g. “Legacy” or “reached end of life”), not
a gateway error.azure/azure-foundry, the response also includes model_availability, keyed by
gateway model id — for azure-foundry this includes any auto-detected catalog matches
alongside whatever you explicitly mapped:
provider replaces the existing credential in place.
List and remove
GET returns the same redacted shape as above for every credential you’ve added
(any status). DELETE revokes one — matching requests fall back to the gateway’s
shared key (or 503 provider_not_configured if none is set up for that provider).
Routing precedence
A single gateway model id can have more than one possible backend — e.g.openai/gpt-5.4
can be served by your own Azure Foundry deployment (if you’ve mapped it), your own native
OpenAI key, or the operator’s shared key. This is the exact order the gateway tries them:
- Your BYOK credential for the model’s re-hosting route(s) (e.g. Bedrock or Azure
Foundry), in the order those routes are listed for that model — only if you’ve mapped
this specific model id (
modelDeployments, or Bedrock’s own model naming). Unmapped routes are skipped, never guessed. - The operator’s shared key for that same route, only if you have no BYOK credential for it. BYOK and the shared key are never blended for the same route — whichever one is configured wins that slot outright.
- Your BYOK credential for the model’s native provider (e.g. your own OpenAI key) — tried last, since re-hosting routes (step 1) are preferred first when available.
- The operator’s shared key for the native provider — the final fallback, used only if none of the above resolved.
openai/gpt-5.4: if you’ve added an Azure Foundry (or Azure OpenAI)
credential mapping openai/gpt-5.4, that’s tried first. If you haven’t, but you’ve added a
native OpenAI credential, that’s tried next. If you’ve added neither, it falls through to
the operator’s shared OpenAI key — billed normally.
A route with neither a BYOK credential nor the operator’s shared key configured is
skipped silently, not an error — you only get 503 provider_not_configured if literally
nothing resolves for that model.
This is a preference/failover order, not an exclusive first-and-only pick: if your top
preference’s actual call fails (before any content is returned — a failure partway through
a streamed response is surfaced directly, never silently retried elsewhere), the gateway
transparently falls through to the next one down the list. Billing reflects whichever
backend actually ended up serving the request — normally your top preference, but it can
shift if that one fails over.
Billing
Requests fully served by your own credential deduct 0 credits — they still appear inGET /v1/usage (so you can see the traffic) but never touch your prepaid
balance. If a model has both a BYOK-configured route and a shared-key fallback (e.g.
you’ve added a Bedrock credential but not a native Anthropic one), a request that
fails over to the shared route is billed normally for that portion.
Because BYOK traffic doesn’t require gateway credit, accounts with a BYOK
credential configured skip the 402 insufficient_quota balance check entirely for
requests that resolve to their own credential — a zero balance doesn’t block you
from using a model you’re paying the provider for directly.
