FruxonDocs
ApiEvaluation runs

Start a golden evaluation run against a dataset

Enqueues an evaluation run and returns immediately with the run ID. Poll GET evaluation-runs/{evaluationRun} to track status and retrieve results once completed.

POST
/v1/tenants/{tenant}/agents/{agent}:evaluate

Authorization

Bearer
AuthorizationBearer <token>

JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token.

In: header

Path Parameters

agent*string

The unique identifier of the agent

tenant*string

The tenant identifier

Request Body

The evaluation configuration and dataset reference

datasetId?string

One or more dataset IDs to evaluate against. If empty → use all datasets for the published revision.

Formatuuid
candidateRevision?integer|null

The saved revision to score, for the deploy flow (:evaluate). The server loads this revision authoritatively — no draft is required and no flow is trusted from the request body. Null for the draft flow (draft:evaluate identifies its candidate by pinned snapshot) and for external agents (which resolve from their stored config).

Formatint64
candidateExternalConfig?

Candidate-side override of the agent's stored Fruxon.Model.Agents.Agent.ExternalConfig. Used only when the agent is non-native — mirrors how the native candidate reference overrides the deployed flow on native A/B. The baseline always uses the agent's stored config.

When this is null and the agent is non-native, the eval runs in single-target mode: only the candidate (= the agent's stored config) is invoked, and the judge scores it standalone.

[key: string]?never

Response Body

curl -X POST "https://api.fruxon.com/v1/tenants/string/agents/string:evaluate" \  -H "Content-Type: application/json" \  -d '{}'
{
  "id": "00000000-0000-0000-0000-000000000000",
  "agentId": "string",
  "datasetId": "00000000-0000-0000-0000-000000000000",
  "candidateRevision": 0,
  "candidateBaseRevision": 0,
  "candidateSnapshotSeqNo": 0,
  "deployedRevision": 0,
  "status": "PENDING",
  "score": 0,
  "deploymentRecommendation": "string",
  "summary": {
    "totalSamples": 0,
    "successfulEvaluations": 0,
    "failedEvaluations": 0,
    "averageLlmScore": 0,
    "scoreDistribution": {},
    "assessmentDistribution": {},
    "topConcerns": [
      "string"
    ],
    "keyImprovements": [
      "string"
    ],
    "runtime": {
      "totalCandidateMs": 0,
      "totalBaseMs": 0,
      "averageCandidateMs": 0,
      "averageBaseMs": 0,
      "stdevCandidateMs": 0
    },
    "cost": {
      "totalCandidate": 0,
      "totalBase": 0,
      "averageCandidate": 0,
      "averageBase": 0
    }
  },
  "errorMessage": "string",
  "createdAt": 0,
  "modifiedAt": 0,
  "startedAt": 0,
  "completedAt": 0,
  "createdBy": "string"
}
{
  "type": "string",
  "title": "string",
  "status": 0,
  "detail": "string",
  "instance": "string",
  "property1": null,
  "property2": null
}
Empty
Empty
{
  "type": "string",
  "title": "string",
  "status": 0,
  "detail": "string",
  "instance": "string",
  "property1": null,
  "property2": null
}