Custom Configurations

A custom configuration is one set of LLM routing and provider settings: which AI providers and models are available, and optionally API keys. An account can have multiple custom configurations (e.g. “Production”, “Staging”). When the API is called (e.g. for chat completions), the system uses the config tied to the request. Every API key is bound to a config—there is no account default config. If the request does not specify a provider or model, the system chooses one from the config’s providers and models.

Multiple configs per account

• Each account can have multiple custom configurations, each with a unique config ID.
• API keys are always bound to a config (config_id is required when creating a key). Requests with that key use that config.
• Accounts can list their configs, get one by config ID, create, update, or delete only their own configs.

In other words: each custom configuration is a named set of providers and models (and optional keys). The config used per request is determined by the API key’s binding. The system decides which provider and model to use when the request does not suggest one.

What can be set

In a custom configuration, the following are defined:

1. Providers and models — For each provider:
   • Which models are allowed (e.g. gpt-4o, gpt-4o-mini).
   • Optionally an API key for that provider (otherwise the product uses its own keys).
   • For custom OpenAI-compatible endpoints: the base URL (api_base).

When a request does not specify a provider or model, the system picks from the config’s providers and models (e.g. first provider, first model in that provider’s list).

So the configuration controls which providers, which models, and whether account keys or the service’s keys are used.

Two ways configs are used

Default config (no account or shared config)

• When there is no account or no custom configuration has been created, the API uses the default config (the shared one with a special “zero” user ID).
• Requests then use whatever providers and models the product has set up there.

Account custom configuration (API key bound)

• When the caller uses an API key, that key is bound to a config_id. The API uses that config’s list of providers and models and any API key overrides.
• If the request does not specify a model (or provider), the system chooses one from the config.
• A custom configuration can inherit from the global default or another of the account’s configs: only the desired providers need to be specified (with optional overrides); the rest (e.g. model list) can come from the parent.

So in practice: either the shared default is used, or the config bound to the API key (optionally built on the default via inheritance).

Inheritance in plain terms

• A custom configuration can inherit from another config—either the default or another config owned by the same account.
• Only the desired additions or overrides need to be specified (e.g. “only OpenAI and Gemini,” or “this API key for OpenAI”).
• The parent must be the default config or another config belonging to the same account; the system prevents loops and limits depth (e.g. max three levels).

So inheritance means basing the configuration on the default (or another of the account’s configs) and only changing what is needed.

How this affects requests

• With a custom configuration — The API uses that config’s providers and allowed models; if API keys are set in the config, those are used. If the request does not specify a provider or model, the system picks one from the config.

• Without a custom configuration (e.g. no auth or no config created) — The API falls back to the default config and uses the product’s shared providers, models, and keys.

• When a model is specified in the request — The product may still use the account’s custom configuration to decide which provider that model belongs to and which key to use; when there is no account (e.g. some keyless usage), it can fall back to environment or product defaults.

In short: custom configurations define the allowed providers and models and optional API keys; the system uses the config bound to the API key when the caller uses a key, otherwise the default config. The system chooses provider and model when the request does not suggest one.

Managing custom configuration via the API

Custom configurations are managed through the config API: POST /config to create (returns a config_id), GET /config to list summaries, GET /config/{config_id} to get one, PUT or PATCH /config/{config_id} to update, DELETE /config/{config_id} to remove. Optional name lets you label configs. When creating an API key, config_id is required. For full endpoint details, request/response shapes, and authentication, see Routing & Provider Config.

Related

• Models and Providers — How providers and models are chosen for workloads.
• Routing & Provider Config — Config API reference.
• API key management — Link API keys to an account and config.