Arize AI
Articles about Arize AI
Together AI integration: Evaluate open models and optimize evaluation costs in Arize AX
Evaluate open models with Together AI in Arize AX. Compare model performance, run LLM-as-a-Judge evaluations, and explore lower-cost alternatives.
How to test agent tool descriptions with Harbor and Phoenix
A tool-description change cut Codex’s extra skill loads from 51 to zero and lowered Harbor benchmark cost from $11.85 to $8.34, without changing correctness.
Alignment is a human-AI workflow: how Custom Views make agent review possible
Automated evals score every session, but they don’t reflect what your team considers a good outcome. Custom Views make human review fast enough to align them.
Cost Agent: find where your LLM and agent spend goes with your traces
Cost Agent analyzes your Arize AX traces to find recurring sources of LLM and agent spend, rank cost issues, estimate savings, and suggest fixes.
Decision model benchmark: Jev, Kev, Liquid d1, and more
We benchmark eight decision models for hallucination detection, LLM evaluation, and agent routing, comparing accuracy, latency, cost, and prompt sensitivity.
How we benchmark AI agents and tools with Harbor and Arize Phoenix
See how Arize uses Harbor and Phoenix to benchmark AI agents, MCP servers, skills, and CLIs in reproducible sandboxes, then inspect traces and annotations.
Prompt caching benchmark: high cache reuse doesn’t always mean lower cost
We benchmark prompt caching across DeepSeek, GLM, GPT, and Claude on the same multi-turn agent, comparing cache reuse, estimated cost, and latency.
Introducing Arize AX MCP: When to use MCP, CLI, or skills
MCP, the CLI, and skills are three ways to extend an agent with Arize AX. Here is when to use each, now that context cost no longer decides it.
Claude’s hillclimb loop for AI agents: start with production traces
Anthropic’s Claude hillclimb loop designs evals and improves agents one change at a time. Here’s how to run that loop against production traces in Arize AX.
How we built long-term memory for Alyx: why we chose a file over a knowledge graph
How we built long-term memory for Alyx: why we chose one 8,000-character file over retrieval and knowledge graphs, and how we tested it.
Alyx now remembers your work across sessions with long-term memory
Alyx now remembers project goals, conventions, and decisions across sessions in Arize AX, helping you continue work on traces, evals, and experiments.
NVIDIA proposes an AI agent kill switch in silicon after a year of sandbox escapes
Agents escaped sandboxes at OpenAI, Anthropic, and Google. NVIDIA’s kill switch targets time-to-detect and time-to-kill: 12 minutes to seven months.
Building production-ready AI agents with Atlas Agent Engine and Arize AX
Export OpenTelemetry traces from MongoDB Atlas Agent Engine to Arize AX to inspect agent runs, evaluate outcomes and paths, and test fixes before redeploying.
What everyone was talking about at WeAreDevelopers World Congress North America
Lessons from WeAreDevelopers World Congress North America on AI code review, agent evals, sandboxing, context engineering, and developer roles.
Are agent harnesses dying? What harness distillation changes
Learn what harness distillation means for AI agent harnesses, which behaviors models can absorb, and what developers should keep in the harness as models improve.
Anthropic says it fixed Claude’s writing. I ran the evals to check
Opus 5.5 dropped em dashes from 12.9 per 1,000 words to two in 57,000 words, and cut the rest of the Claudisms in half. Better, but not fixed.
Evaluate production traces with Jev-as-a-Judge directly in Arize AX
Arize AX now integrates directly with TypeSafe AI, bringing native Jev-as-a-Judge evaluations into the Arize evaluation workflow.
What Jev’s probabilities reveal that repeated LLM judgments miss
Jev’s probabilities flag its own errors better than rerunning an LLM judge ten times. Results from 36,190 judgments across 10 Arize Phoenix evaluators.
How we built UI Code Mode into Arize Phoenix
We replaced the 58 tools PXI used to drive the Phoenix UI with two tools and a JavaScript sandbox that runs in your browser tab. This post explains why, how it works, and what it cost.
How to build a remote evaluator in Arize AX with Jev
Evaluate agent responses with Jev and Arize AX. Build a FastAPI remote evaluator, return labels and scores, and choose a threshold for your data.
Jev vs. LLM-as-a-Judge: Accuracy and cost benchmarks
We benchmarked Jev against Claude Opus 5 and GPT-5.6 Terra on accuracy, cost, and latency. Learn how threshold tuning changes hallucination detection.
Real-time LLM guardrails with Jev: comparing latency and cost
Compare Jev and GPT-5.4 nano for real-time LLM guardrails, with demo results on latency, cost, and checks on agent inputs, replies, and tool calls.
What changes when AI agents use your software
Daytona CEO Ivan Burazin on building tools for AI agents, with practical lessons on delegated access, safe retries, and evaluating whether work gets done.
New in Arize AX: first-class sessions, Agent-as-a-Judge, and vision evals
Arize AX product update for August 6–September 18, 2026: session-level annotations and queues, Alyx + Signal, Agent-as-a-Judge on every plan, new models, vision judges, and faster datasets.
TypeSafe’s Jev: Can decision models replace LLM judges?
TypeSafe’s Jev classifies, scores, and routes without generating text — up to hundreds of times cheaper than an LLM judge. What that changes for evals, confidence routing, and application architecture.
How to find and debug agent failures your evals are missing
Evals measure failures you know how to name. Arize Signal continuously reviews production traces to find recurring trajectory failures you do not, then turns them into evidence for tests and fixes.
The future of AI operations teams
Your agents produce more traces and eval results than your team can read. SallyAnn DeLucia on how engineering teams are restructuring around managed agents, and which work stays human.
AI agent regression testing with Agent Experiments in Arize AX
Learn how to run AI agent regression tests in Arize AX: compare baseline and candidate experiments, score action safety vs task completion, and inspect traces before you ship.