Testing

Test chat in Studio is your inner loop. Make a change, run it, read the trace, fix, repeat. This page covers how to use it well — and how it fits with the heavier Evaluations workflow for systematic quality.

Test vs Evaluate

You'll spend most of your build time in the Test panel and most of your shipping time in Evaluations.

Running a test

1. Open the agent in Studio. Test chat is open on the right; if it isn't, click its icon on the rail.
2. Pick a mode: Conversation (chat with the agent, offered when its only input is user_query), Single run (fill in each parameter and run once), or Trigger (fire one of the agent's triggers, with an optional sample event).
3. Send a message, or fill in the parameters and run.

The test runs your draft — the form exactly as you see it, before you deploy. You don't need to save a version to test. To run a published version instead, pick it under Run settings → Runs.

Reading the trace

Every test run produces a complete trace, just like a production run:

• Each step of the run — every model call with its resolved prompt and response, and every tool call.
• Tokens, latency, cost — for each step and in total.
• Errors — the failing step highlighted, full message and upstream state preserved.
• Tool calls — request/response pairs for every tool invocation, expandable inline.

Click any step to drill in. Click any tool call to see exactly what was sent and what came back.

What to test for

The same agent fails for different reasons in different cases. Cover at least:

• The happy path. A typical, well-formed input.
• The empty case. Empty string, empty array, missing optional field.
• The malformed case. Invalid JSON, wrong type, way-too-long input.
• The adversarial case. Prompt injection ("ignore previous instructions"), unusual unicode, jailbreak attempts.
• The boring case. Inputs that look identical to ones the agent should answer differently. Distinguishes a working classifier from a lucky one.
• The expensive case. Inputs that trigger many tool calls or long responses — useful for spotting cost outliers and runaway loops.

A pattern that works: keep a small file of "test cases I always run" and paste them in as you iterate.

Multi-turn testing

For agents with sessions enabled, use Conversation mode — submit follow-up turns and the conversation accumulates. Use this to check:

• Whether the agent remembers what was said earlier.
• Whether session overflow strategies (drop / summarize) behave the way you expect at the boundary.
• Whether the session_search tool actually fires when needed.

Sessions →

Testing tool integrations

Tools execute against your real integrations during a test run. That means:

• A test that calls Salesforce will create a real Salesforce record.
• A test that runs a SQL query against PostgreSQL will hit your actual database.
• A test that posts to Slack will post a real message.

For destructive integrations, use sandbox mode (Sandbox Mode) or a dedicated dev organization with separate credentials. Don't iterate against production.

Iterating fast

A few habits that compound:

• Deploy in small steps. Every deploy saves a version, so your history is preserved.
• Diff versions. When something stops working, open the version picker and Compare with the last good version — you'll often spot the change immediately.
• Tag interesting traces. Hit the tag button on runs you want to come back to. Tag candidates for the evaluation dataset as you find them.
• Test the failing case first. When debugging, start by reproducing the bad output, then tweak. Don't tweak in the dark.
• Keep the draft clean. A half-configured tool or an empty prompt can throw confusing errors. Remove what you aren't using yet.

When the test panel runs but production doesn't

Test runs hit your draft (work you haven't deployed). Production hits the deployed revision. Common gotchas:

• Tools, secrets, or knowledge attached during testing aren't deployed yet.
• The model you tested with isn't the one configured on the deployed revision.
• The deployed revision is older than you remember.

Always look at the deployed revision's diff before assuming a production bug is reproducible from your draft.

Promoting a test case to a regression test

Found a bug, fixed it, and want to make sure it stays fixed? Promote the failing input to your evaluation dataset:

1. From the trace, click Add to Dataset.
2. Pick the dataset, set expected output (if applicable), tag it.
3. Future evaluation runs include this case automatically.

This is how the evaluation dataset grows organically — every bug becomes a test.

Evaluations →

Test panel keyboard shortcuts

Next steps

• Evaluations — datasets, judges, deploy gating
• Observability — production traces
• Troubleshooting — common failure modes
• Versioning — revisions, comparing, rollback