Release infrastructure for production AI agents

Evals pass.Production still surprises you.

Triana helps teams understand how agents behave, test changes against real usage, and build evidence for better release decisions.

Trace to Evidence available now Grounded simulation in design-partner preview Release decisions remain human-owned

Preserve eval-to-decision follow-through #181

✓ OPENDev B wants to merge 3 commits into main from eval-decide-followthrougha4f1c9d → 7b2e08f

READY TO MERGE

FILES CHANGED · 2agent/prompts/candidate_screening.md · agent/decide.py+60 −16

CHECKS3 SUCCESSFUL

✓lintruff · 0 findings18sPASS

✓unit-tests412 passed2m 04sPASS

✓buildimage published1m 41sPASS

○triana / release-checkwaiting for the merge queueQUEUED

Triana will replay 18 approved production checks.

Available now · Triana Preview

See what your agent is doing in production.

Triana turns authorized conversations into a Behavior Map of recurring user needs and agent behaviors. Every observation links back to the conversations that support it.

Try it yourself

Production behavior

Observed requests, frequency, outcomes, and supporting traces.

91% of traffic explained

185 sessions11 user needs3 confirmed regressions

Observed user needSessionsTypical handlingSupporting trace

P01Evaluate an applicant and send outreach77 41.6%70% answeredhr_0099 View trace →

P04Rank applicants for a role19 10.3%68% answeredhr_0108 View trace →

P06Explain the rubric and scoring method18 9.7%89% answeredhr_0121 View trace →

Setuplocal

01 · Install Preview

uvx --python 3.13 --from 'triana-preview==0.1.0a2' triana-preview --help

02 · Add the Codex skill

codex plugin marketplace add Triana-AI/triana-skills codex plugin add triana@triana-skills

Codexonboard

03 · Run the workflow

Use the Triana onboard skill to analyze these conversational traces end to end. Trace path: /path/to/traces.jsonl Agent: A customer-support agent handling orders, delivery issues, refunds, and escalations. Ask before reading the traces or sending anything to my model provider.

Deployment simulation

Test situations your static evals did not anticipate.

Triana turns behavior observed in production into new test situations, then runs adaptive multi-turn conversations against your existing agent. Each run returns inspectable transcripts, coverage, cost, and uncertainty.

See how simulation works

Support agentSIMULATION · DESIGN-PARTNER PREVIEW

SIMULATION RESULTDelivered status, missing packageSIM-024

ADAPTIVE MULTI-TURN TRANSCRIPT4 turns

SIMULATED USER

Tracking says delivered, but the package is not here.

AGENT

I can check the delivery scan. What is the order number?

TOOL ACTIVITYget_delivery_status("ORDER-2048") → delivered

AGENT

The carrier marked it delivered. The delivery photo still needs review.

Coverage: 2 of 3 branchesAddress check open
Cost: $0.075 model calls
Fidelity: 4 sources linkedSituation grounded
Uncertainty: MediumPhoto not verified

Release decisions

Keep humans in charge of what ships.

Triana organizes behavioral evidence and uncertainty for review. Your team still decides what matters, what is acceptable, and whether a release moves forward.

Support agentRELEASE DECISIONS · PRODUCT DIRECTION

EVIDENCE

What is known

Conversations, outcomes, gaps, cost, and lineage.

POLICY

What matters

Team-owned requirements, limits, and approval rules.

DECISION

What happens next

Approve, hold, investigate, or accept an exception.

Keep your stack. Add a release process.

Triana does not replace your agent runtime, evaluator, or deployment system. It connects the behavioral evidence those systems produce into a process your team can inspect and improve.

YOUR EXISTING STACK

AgentConversational and tool-using runtime

ToolsCalls, results, and terminal states

Conversation tracesAuthorized PreviewTrace JSONL

Release contextCandidate change · review scope

TrianaEVIDENCE LAYERBEHAVIOR IN · EVIDENCE OUT

TRIANA

Deployment SimulationGrounded situations · adaptive conversations

Release ControlCI · review · deployment decision

Triage MemoryConversations · outcomes · typed gaps

Available now · Triana Preview

Run Triana Preview on your traces.

Install Preview locally, add the Triana skill to Codex, and point it at an authorized conversational trace file.

Run Previewuvx --python 3.13 --from 'triana-preview==0.1.0a2' triana-preview --helpAdd the Codex skillcodex plugin add triana@triana-skills

Open Preview quickstart

Design-partner deployment simulation

Try deployment simulation before your next rollout.

Apply to turn observed production behavior into grounded test situations and run adaptive conversations against your existing agent.