LLM Providers

LLM Providers is the top-level page where you define reusable, tenant-scoped LLM configurations. Each config bundles a provider key, a model, and the generation knobs (temperature, top-p, max tokens, reasoning mode, …) into a single named record that any agent step can reference by ID.

Configs replace the older inline per-step parameter editor. Instead of every step carrying its own temperature: 0.2, you publish one "Claude 4 Sonnet — analytical" config at the tenant level and point steps at it. Update the config once; every step that references it picks up the change on the next deploy.

When to use a config vs inline

In short: almost always a config.

Configs also make audits trivial — you can answer "what model is in production right now?" by looking at the LLM Providers page rather than spelunking through agent revisions.

Define a config

LLM Providers → Add Config.

A config has:

• Name — what step authors see in the picker. Make it about intent: summarizer-fast, router-cheap, analyst-careful.
• Provider — the underlying account (OpenAI, Anthropic, Google, Bedrock, …). Provider credentials are configured once and referenced here. See AI Providers for the credential model.
• Model — the specific model ID.
• Generation parameters — temperature, top-p, max tokens, frequency / presence penalties, reasoning mode where applicable.
• Description — optional, surfaced in the picker.

Test before you publish

Hit Test to fire a minimal call against the provider with the configured parameters. This catches expired keys, region/access mismatches, and parameter shapes the model rejects — before any agent step depends on the config.

Draft, publish, rollback

Configs are versioned. Every save creates a draft; deploying a config marks that draft as published and pins it as the value any step picker resolves to.

• Draft — work in progress. Visible only to authors; not used by any agent step.
• Published — the live version. Step bindings resolve here.
• Rollback — promote any prior published version back to current. Useful when a model rev or parameter tweak regresses production.

The Revision History view shows every published version with who deployed it, when, and the diff against the previous version.

Use a config in an agent step

1. Open the agent in Agent Studio.
2. Select an Agent Step.
3. In the step's config panel, open the LLM Config picker.
4. Pick a published config. The step records a reference — not a snapshot.

When you next deploy a config, every step that references it executes with the new parameters on the next run. No agent re-deploy needed.

Usage

The Usage tab on each config shows:

• Total requests and tokens consumed by that config in the selected window.
• Which agents and steps reference it.
• Cost broken down per agent.

This is the right place to identify configs that are unused (candidates for retirement) or that drive disproportionate cost.

API surface

The scope on API keys is llm_providers:read / llm_providers:write / llm_providers:test. See API Keys.

Best practices

• Name by intent, not by model. summarizer-fast survives a model upgrade; gpt-4o-mini-temp-0.2 doesn't.
• Keep the config catalog small. A handful of high-quality configs is easier to reason about than dozens of near-duplicates.
• Test then publish. The :test call is fast and catches the vast majority of provider issues before they hit production.
• Roll back, don't hot-patch. If a deployed config regresses, roll back to the previous published version and investigate from a green state.

Next steps

• AI Providers — provider credentials referenced by configs
• Agent Studio — wire configs into steps
• Cost & Budgets — track spend per config and per agent
• API Keys — scopes that govern config management