Observability

Every agent run produces a complete record of what happened — its source channel, inputs, rendered prompts, every LLM call and tool call, reasoning, tokens, latency, cost, approvals, the final result, and any error. Fruxon keeps that record so you can answer two questions at any time: what did this specific run do? and how is this agent behaving in aggregate?

This page covers both — the per-run trace and the per-agent and organization-wide metrics that roll up from it — plus tagging, alerts, retention, and how to pipe runs from external agent frameworks into the same view over OpenTelemetry.

Where to look

Every per-agent surface has a Production / Test toggle. Production runs come from deployed revisions serving real traffic; test runs come from the studio (:test / :streamTest) while you're building. They're tracked, costed, and budgeted independently so your experiments never pollute production metrics.

Execution History

The Execution History tab lists every run of the agent, newest first:

Status

A run is always in exactly one of these states:

Source

The Source chip tells you how the run entered, mapping the underlying trigger to operator-meaningful vocabulary:

Click a participant or trigger source to jump straight to it in Agent Network.

Filters

Narrow the list to what you care about:

• Status — successes vs failures; the usual triage starting point.
• Version — isolate one revision to compare behavior across versions.
• Time range — recent runs, a longer window, or a custom range.
• Production / Test — the origin toggle described above.

The trace

Click any run to open its trace — the full record. The header carries the version, status, start/finish times, total duration, and a copyable execution ID (the shareable handle for the whole run). Below it is the run's step hierarchy, with a timeline of durations so you can see where the time went. A summary line keeps you oriented: N steps · N LLM · N tool · N errors.

The entry node

Pinned to the top of the tree is the entry node — the channel the run came in through. Open it to see the source, the channel/provider, the named participant or trigger (a deep link), and the inbound message that started the run.

Steps

Every node in the workflow is a typed step:

Select any step to open its detail pane. What you see depends on the step type:

LLM steps

• Provider & model — exactly which model served this step.
• Tokens — input, cached, output, and thinking (extended-reasoning) tokens, plus the total.
• Cost — per-step input/output/cached/thinking cost, summed at the run level.
• Output — the model's full response.
• Prompt messages — expand View prompt messages to read the complete conversation sent to the model for this step.
• Reasoning — for models that emit extended thinking, the reasoning blocks are shown inline. Provider-redacted or ingest-stripped blocks are surfaced as such rather than hidden, so "the model reasoned here but we can't show it" is never silently dropped.
• Server-side tools — built-in provider tools the model ran on its own (e.g. web_search) appear with their invocation summary.

Tool steps

• Input — the parameters the tool was called with.
• Output — the tool's response.
• Cost — for metered tools, the per-call cost.

HITL steps

When a step is gated by a human approval, the gate carries:

• Decision — Approved, Rejected, or still Pending.
• Wait — how long the run was paused.
• By — who responded.
• Sent to — the route the approval request was delivered through.
• The outbound message shown to the approver, and the rejection reason if there was one.

If the run is still WAITING_FOR_HUMAN, you can respond to the pending approval directly from the trace.

Sub-agent traces

A step that calls a sub-agent nests that sub-agent's steps inline. Drill into the child's LLM and tool calls without losing your place in the parent, and the totals (steps, cost, tokens) account for the whole tree.

Errors

Failed runs surface the failing step inline. Its detail pane shows the error class and message; the surrounding steps show the upstream state — what the inputs and previous outputs were at the moment of failure — so you can reconstruct why without re-running.

Delivery status

Delivery is tracked separately from execution. A finished run carries a delivery outcome per destination:

When a COMPLETED run failed to deliver, the trace shows a callout listing each failed destination — its connector, provider, error class, and message — so a successful agent run that never reached the user doesn't slip by unnoticed.

Final result

The run's final output is rendered at the bottom of the trace, formatted for reading.

Metrics

The Overview tab rolls the agent's runs up into live cards. Like everything per-agent, it respects the Production / Test toggle.

Cost over time

A timeseries chart of spend, with selectable windows (24h, 7d, 30d, 90d, all time). Each bucket breaks down into total / input / output cost, execution count, and token volume — so a spike is immediately attributable to more runs vs more expensive runs.

Model usage

A per-model table for the revision: provider, model, input / output / cached / thinking tokens, and cost. Toggle between a cost view and a token view. This is where you catch an agent quietly leaning on an expensive model, or confirm prompt caching is actually landing (watch the cached-tokens column).

Budget

Each origin shows its monthly budget card — the cap, current usage as a percentage, the configured alert thresholds, and whether hard enforcement is on. Full configuration lives in Cost & Budgets; the short version:

• Set a monthly cap per agent, per origin (Production and Test are independent).
• Add threshold rules — fire an alert at, say, 80% and 100% of cap.
• Optionally enforce the limit — block further executions once the cap is hit, rather than just alerting.

Tool usage

The Overview's Tool usage card breaks the agent's runs down by which tools it actually called. Each row is one tool, with:

Rows are sorted by call volume, with a share bar so the heaviest tools jump out. A time-range selector (24h / 7d / 30d / all time) scopes the card, and it respects the Production / Test toggle like the rest of the Overview. This is where you catch a tool that's erroring half the time, a slow integration dragging latency, or a metered tool quietly dominating cost.

Observability by environment

When one agent serves many end-customers, tag each run with an environment slug and Fruxon splits every cost and health metric per customer. Setting the tag is covered in Environments — pass environmentSlug on the call, or pin an inbound channel to an environment. This section is what you get back once runs are tagged.

The fleet table

The agent Overview (Production) shows an environments fleet table — one row per environment, sortable, searchable, and pinnable:

It's the fastest way to spot that one customer is erroring, slowing down, or costing more than the rest. Pin your highest-value customers to the top so they're always in view.

Comparing environments

Select a handful of rows to open a compare view:

• A quick drawer with totals — executions, cost, each environment's cost share, and last-active.
• A full compare view with a cost-over-time chart overlaying each environment on one set of axes (bucketed automatically — hourly, daily, or weekly depending on the window), plus a side-by-side metrics table: executions, error rate, total cost, p95 latency, and last active. A time-range picker rescales everything together.

This answers "is Acme's traffic getting more expensive than Globex's, and since when?" in one screen.

Scoping the rest of observability to one environment

The environment tag also narrows the other surfaces:

• Execution History filters to a single environment, so you can read the actual traces behind one customer's error rate.
• Cost and model-usage breakdowns accept an environment, so the per-model table and cost timeseries can be scoped to one customer instead of the whole agent.

What is not per-environment

Managing environments

Directory → Environments lists every environment in the organization. Slugs are immutable, but you can give each a friendly display name, a description, and a logo so the fleet table and compare views read in human terms rather than raw slugs. See Environments for contacts and agent enrollment.

External agents & OpenTelemetry

Agents that don't run on Fruxon's native engine — LangGraph, LangSmith, CrewAI, and other frameworks — get the same trace and metrics view by pushing their telemetry to Fruxon over OpenTelemetry (OTLP).

In the agent's Observability Setup panel, Fruxon gives you the environment variables to point your tracer at its ingest endpoint:

OTEL_EXPORTER_OTLP_ENDPOINT=<your tenant's OTLP ingest URL>
OTEL_EXPORTER_OTLP_HEADERS=X-API-KEY=<your token>
LANGSMITH_OTEL_ONLY=true   # LangSmith/LangGraph only

The panel includes:

• Ingest health — a live pill showing spans received in the last 5 minutes, and how many were attributed vs dropped, so you know wiring worked before you go hunting for a missing trace.
• Test connection probe — fire a synthetic span and confirm it lands.
• Content redaction — a toggle to strip prompt and completion content at ingest, for teams that must keep raw model I/O out of the platform. It applies to future runs and is honored before anything is stored.

Spans land in the same Execution History and trace UI as native runs. Each span's detail view adds a Raw OTel tab showing the verbatim OTLP attributes — useful when your framework emits attributes Fruxon doesn't map into the projected step. (Native agents build their trace directly and have no raw OTLP layer, so the tab is hidden for them.)

Tagging runs into datasets

Production traffic is the best source of evaluation cases. From any completed run, use the Eval control to save it into one or more evaluation datasets, optionally with tags (e.g. category:billing, difficulty:hard, reviewed) so you can slice the dataset later. The run's inputs and output become a case you can score future revisions against.

This is the bridge from observing an interesting or broken run to preventing the regression — tag it today, and it's in your golden dataset tomorrow.

Alerts

Fruxon raises three kinds of alert, delivered by email to an agent's collaborators:

Alerts are throttled — a given alert type for a given agent won't re-fire within its cooldown window — so a failure storm sends one email, not a thousand. Throttling is coordinated across all server replicas, so duplicates don't slip through during high load.

Integration credential health

A deployed agent is only as reliable as the integration credentials it runs on. When an OAuth credential a published agent depends on stops working — a refresh token revoked, an account that now demands interactive re-auth — every run that needs it fails until someone reconnects. Credential health surfaces that before you find it in the failure log.

Health is tracked from real OAuth token-refresh outcomes — there's no separate liveness probe. When Fruxon refreshes an application-level credential and the provider rejects it, the credential is marked unhealthy with the reason:

Where it shows up

• Agent overview banner. When an agent's currently-deployed revision references a credential that's gone unhealthy, a banner reads "N integration credentials need attention", naming each one with a Reconnect (or Fix) link straight to the integration config page. The rollup only counts credentials the deployed revision actually uses, so the banner reflects live exposure, not historical configs.
• Integration config page. The credential's status swaps from a green "Connected" to an amber state with the reason, and the Reconnect button is emphasized — re-running the OAuth flow restores the credential in place.
• Email alert. The first time a credential flips from healthy to unhealthy, Fruxon emails the admin collaborators (who've opted into email alerts) of every agent whose deployed revision uses it — a "credential needs attention" email naming the affected integration, with a Review credential link. It fires on the transition, not on every failed refresh, and is throttled per agent-and-credential (~60 minutes) so a flapping credential can't spam the inbox.

Organization-wide

Two organization-level pages in Settings aggregate across every agent:

Settings → Usage

• Monthly executions — runs this billing period against your plan's limit, with a per-agent breakdown and the reset time (1st of the month, 00:00 UTC). Respects the Production / Test toggle.
• Tool Usage — the same per-tool breakdown as the agent card, but aggregated across every agent in the organization for the current month. One row per tool with its call count, error badge, and cost, plus a footer total. This is the org-wide answer to "which tools are we leaning on, and which are erroring?"
• Standing counts for agents and team members.

Settings → Billing

• Your plan tier, with created / expires / last-modified dates and an upgrade path.
• Plan limits — every license rule and its value, including trace retention days (see below) and support level.

For token-spend reporting, see also Tokens & Usage.

Retention

Trace and execution data is queryable for a window set by your plan — the trace retention value shown on the Billing page (Enterprise plans are typically unlimited). Once a run ages past that window it's no longer returned by the history and trace views.

The API

Everything above is backed by tenant-scoped REST endpoints under v1/{tenant}/agents/{agent}, so you can build your own dashboards or wire observability into your own tooling:

See the API Reference for full request and response shapes.

Patterns

• Triage failures every morning. Filter to FAILED in the last 24h, scan the error classes, fix the top one. Five minutes a day keeps small breakages from becoming outages.
• Tag interesting runs as you see them. The run that surprised you today is the regression test you'll wish you had next month. The Eval control makes it one click.
• Trust p95 over the average. A healthy average with an ugly p95 means a subset of users are suffering — usually one slow tool or one heavy environment.
• Investigate cost outliers. A run that costs 10× the median is almost always a tool-retry storm, a runaway loop, or an unexpectedly large context. The trace's per-step token and cost columns point straight at the culprit.
• Watch ingest health after wiring an external agent. If the health pill says spans are dropping, fix attribution before you trust the metrics.
• Don't let a COMPLETED-but-undelivered run fool you. Check delivery status when a user says "the agent never replied" — the run may have succeeded and the channel rejected the post.

Next steps

• Conversations — replay end-user threads across channels
• Cost & Budgets — set caps, thresholds, and hard enforcement
• Evaluations — turn tagged production runs into a scored test suite
• Environments — attribute cost and health per end-customer
• Troubleshooting — common failure modes and how to debug them