Your agent already told you what to evaluate

Most conversational-agent teams already have more behavioral evidence than they use.

It sits in production traces: the questions users ask, the details they leave out, the tools the agent calls, the paths that repeat, and the places where conversations break down.

But traces are difficult to turn into an evaluation plan. They are verbose, sensitive, and shaped by whatever logging format the product happened to produce.

Teams usually sample conversations manually or write evals from imagined examples. Both are useful. Neither gives you a systematic view of what users actually ask for and how the agent repeatedly behaves.

That is the problem Triana Preview solves.

From traces to evidence

Triana Preview turns authorized conversation traces into a private, evidence-linked Behavior Map.

The map proposes groups of related user demand, recurring agent behaviors, emerging observations, and gaps in the available evidence.

The point is not another generated summary. Every published observation remains connected to the conversations and tool events supporting it, so an engineer can inspect the evidence before deciding what to do.

A Behavior Map might show that an important class of requests is missing from the eval set, that the agent repeatedly handles a situation in the same way, or that a suspected behavior does not yet have enough evidence.

Triana does not declare the agent correct. It gives the team a better basis for deciding what should become an eval, a simulation, a product issue, or accepted behavior.

What comes out

The useful output is not a score. It is a reviewable bridge between production conversations and the engineering work that follows.

A result from ABCD

In one ABCD run, Triana processed 200 conversations while conserving all 3,659 source events.

Every published observation remained linked to supporting evidence.

These numbers are not a correctness score or proof of complete coverage. They show that Triana can preserve the source record while turning a substantial trace set into inspectable behavioral evidence.

We have also exercised the workflow on TAU² and design-partner data.

Blitz is beginning to use Triana Preview on its own production conversations to create grounded evaluation cases. This is early design-partner use, but it represents the loop we built the product for: start with actual behavior, inspect the evidence, then decide what deserves a durable test.

Local by default

Conversation traces can contain customer language, tool arguments, internal identifiers, and other sensitive context. Raw traces should not need to enter another vendor’s platform just to become useful.

Triana Preview runs in the user’s environment and sends nothing to Triana. Raw traces remain local.

When semantic analysis is enabled, authorized redacted excerpts may go directly to the model provider selected by the user, under the user’s credentials. That provider’s retention policy still applies, and redaction is not anonymity.

The authority split is deliberate:

• Models propose semantic labels and explanations.
• Deterministic code owns evidence coordinates, accounting, lineage, progress, failures, and report integrity.
• Humans decide what the evidence means for the product.

Built for unfamiliar trace exports

Self-serve software has to work outside the repository where it was built. That changed the product.

doctor checks the runtime, trace structure, private output path, plugin version, and optional provider configuration without making a provider call.

Declarative adapters map unfamiliar JSON exports without executing customer Python. Progress receipts show which phase completed. Typed failures preserve where and why analysis stopped instead of collapsing everything into a vague error.

These are not secondary features. They are what makes the analysis usable by an engineer who has a trace export and no Triana engineer beside them.

Try it

Triana Preview is available now as a source-available Python package under the PolyForm Free Trial license.

uvx --python 3.13 \
  --from 'triana-preview==0.1.0a2' \
  triana-preview --help

The workflow is straightforward:

1. Run doctor against an authorized trace file and a fresh private output directory.
2. Use PreviewTrace directly or generate a declarative adapter for your export.
3. Analyze the traces and inspect the local JSON, Markdown, or HTML Behavior Map.

Triana Skills provides the same onboarding workflow for Codex and Claude Code under Apache-2.0. The engine remains in Triana Preview; the skill guides the local process.

Triana Preview is the first narrow product in a broader release-infrastructure direction for conversational agents.

Production traces tell us what deserves attention. Behavioral evidence turns that signal into inspectable cases. From there, teams can build better evals today and, over time, run grounded multi-turn simulations before a change reaches users.

Start with the conversations your agent already had.

Install Triana Preview · View the source · Use Triana Skills