The Challenge: Testing What You Can't Predict
When building traditional software, testing is relatively straightforward. Input A always produces Output B. Your calculator app will always tell you that 2+2=4. This deterministic nature makes testing conventional software a matter of verifying expected outcomes against actual results. AI systems, particularly those built on large language models (LLMs), operate differently. Tamir, Founding Engineer at Wordware, went deep on this recently and shared some axioms that have become clearer as time has progressed.
With AI agents, however, you can ask the same question 100 times and potentially get 100 different answers. This non-deterministic nature creates a fundamental challenge: how do you evaluate whether an AI system is working correctly when "correct" isn't a single fixed outcome?
Why AI Evaluation Matters
The stakes are high. Consider these real-world scenarios:
A healthcare provider using AI to respond to patient reviews must ensure HIPAA compliance
A financial service using AI to monitor crypto markets and execute trades needs reliability
A customer service system routing urgent issues needs both speed and accuracy
In each case, failure isn't just an inconvenience—it could mean regulatory violations, financial losses, or damaged customer relationships. As AI systems become more integrated into critical workflows, evaluation becomes not just a technical challenge but a business imperative.
The Wordware Approach to AI Evaluation
1. LLM as Judge: Using AI to Evaluate AI
One powerful technique is using what's called "LLM as judge"—essentially employing another AI system to evaluate the outputs of your primary system:
"What you do is you set up another LLM... And that LLM will then look at the output that was produced by the system... and it has a set of rules that it will then evaluate that output against."
This approach allows you to systematically assess outputs against criteria like:
Accuracy: Does the output contain factually correct information?
Compliance: Does it adhere to specific rules (like HIPAA requirements)?
Quality: Is it concise, relevant, and useful?
Speed: Was it generated quickly enough for the use case?
2. Balancing Different Evaluation Metrics
A common mistake is optimizing for what's easily measurable rather than what truly matters. As Tamir notes:
"Certainly a mistake I've fallen into in the past is optimizing for performance too soon. Because that is a very easy thing we can see... But if your system is one that can take minutes to run and from an end user's perspective, it doesn't matter if it takes one minute or three minutes or 10 minutes, spending a lot of time trying to evaluate on performance is a waste of time."
The key is understanding what metrics actually matter for your specific use case:
For emergency response systems, speed might be critical
For content generation, quality and relevance might matter more than speed
For financial systems, accuracy might outweigh all other considerations
3. Self-Reporting for Real-World Evaluation
One of the most challenging aspects of evaluating AI agents is determining whether they successfully performed actions in the real world.
"The solution that we're experimenting with is a self-reporting system. So the AI agent would go out and do a bunch of things in the world. And the way that AI agents work is that they create a trace, like a log... the AI's diary."
This approach allows the agent to document what it did, what responses it received, and what outcomes it achieved. A meta-evaluation can then assess whether the agent accomplished its goals based on this trace information.
Building Better AI Through Better Evaluation
The process of creating evaluations has benefits beyond just testing:
"Writing evals does make you think very deeply about the system that you're building... If you think about that from the perspective of 'I need to evaluate this message,' and you then think about what are the qualities of that message that's gonna be really good... you become a better prompter."
This insight highlights how the discipline of evaluation improves the entire AI development process—from initial design through implementation and into production.
The Future of AI Evaluation
As AI becomes more integrated into our digital infrastructure, evaluation approaches will continue to evolve. Some emerging trends include:
Digital twins: Creating simulated environments where AI can be tested safely
LLMs.txt: Standardized ways for websites to communicate with AI systems
Agent-to-agent architectures: Frameworks for how AI systems communicate with each other
These developments point to a future where the internet itself becomes more AI-friendly, making evaluation both more sophisticated and more standardized.