FruxonDocs
ApiEvaluation runs

Start a golden evaluation run against a dataset

Enqueues an evaluation run and returns immediately with the run ID. Poll GET evaluation-runs/{evaluationRun} to track status and retrieve results once completed.

POST
/v1/tenants/{tenant}/agents/{agent}:evaluate

Authorization

Bearer
AuthorizationBearer <token>

JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token.

In: header

Path Parameters

agent*string

The unique identifier of the agent

tenant*string

The tenant identifier

Request Body

The evaluation configuration and dataset reference

datasetId?string

One or more dataset IDs to evaluate against. If empty → use all datasets for the published revision.

Formatuuid
candidateRevision?integer|null

The saved revision to score, for the deploy flow (:evaluate). The server loads this revision authoritatively — no draft is required and no flow is trusted from the request body. Null for the draft flow (draft:evaluate identifies its candidate by pinned snapshot) and for external agents (which resolve from their stored config).

Formatint64
candidateExternalConfig?

Configuration for an externally-hosted agent. Persisted as JSONB on Fruxon.Model.Agents.Agent.ExternalConfig; null when Fruxon.Model.Agents.Agent.Origin is Fruxon.Model.Agents.External.AgentOrigin.Native.

Connection details are a free-form System.Collections.Generic.Dictionary2 whose schema is declared per-platform by the matching IAgentPlatformDescriptor.ConnectionParameters. This mirrors how connectors and integrations work: persistent state is a flat dict, descriptors carry the schema, secrets are auto-encrypted by walking the metadata. Adding a new platform with novel auth (OAuth, signed JWT, mTLS) needs no shared-code changes — just new ParameterMetadata` entries on the descriptor.

Conventional keys (most platforms use most of these):

  • endpoint: the HTTP base URL
  • assistant_id: platform-side agent identifier (LangGraph assistant_id, OpenAI assistant id)
  • auth_header_name / auth_header_value: generic header-based auth
  • api_key: when the platform expects a fixed header
  • timeout_seconds: per-invocation timeout
[key: string]?never

Response Body

curl -X POST "https://api.fruxon.com/v1/tenants/string/agents/string:evaluate" \  -H "Content-Type: application/json" \  -d '{}'
{
  "id": "00000000-0000-0000-0000-000000000000",
  "agentId": "string",
  "datasetId": "00000000-0000-0000-0000-000000000000",
  "candidateRevision": 0,
  "candidateBaseRevision": 0,
  "candidateSnapshotSeqNo": 0,
  "deployedRevision": 0,
  "status": "PENDING",
  "score": 0,
  "deploymentRecommendation": "string",
  "summary": {
    "totalSamples": 0,
    "successfulEvaluations": 0,
    "failedEvaluations": 0,
    "averageLlmScore": 0,
    "scoreDistribution": {},
    "assessmentDistribution": {},
    "topConcerns": [
      "string"
    ],
    "keyImprovements": [
      "string"
    ],
    "runtime": {
      "totalCandidateMs": 0,
      "totalBaseMs": 0,
      "averageCandidateMs": 0,
      "averageBaseMs": 0,
      "stdevCandidateMs": 0
    },
    "cost": {
      "totalCandidate": 0,
      "totalBase": 0,
      "averageCandidate": 0,
      "averageBase": 0
    }
  },
  "errorMessage": "string",
  "createdAt": 0,
  "modifiedAt": 0,
  "startedAt": 0,
  "completedAt": 0,
  "createdBy": "string"
}
{
  "type": "string",
  "title": "string",
  "status": 0,
  "detail": "string",
  "instance": "string",
  "property1": null,
  "property2": null
}
Empty
Empty
{
  "type": "string",
  "title": "string",
  "status": 0,
  "detail": "string",
  "instance": "string",
  "property1": null,
  "property2": null
}