Agentic reliability and evaluations: Fewer enterprises plan to let AI agents ship production changes on automated checks alone — 56% allow it or plan to, down from 75% in July

What happened

VentureBeat Intelligence conducted a survey of enterprises with 100+ employees to assess how they evaluate and deploy AI agents. The findings reveal a decline in the percentage of companies allowing automated deployment without human review, increased reports of AI failures despite passing evaluations, and a shift towards using OpenAI's native tools for evaluation.

Why it matters

The survey highlights the challenges enterprises face in balancing automation with reliability when deploying AI agents. Understanding these trends is crucial for AI developers and businesses to ensure that automated systems are both efficient and safe, minimizing the risk of production failures that could impact customers.

• The trend of allowing AI agents to deploy changes without human review is decreasing, indicating a cautious approach to automation.
• Despite automated evaluations, many AI systems still fail in production, highlighting the need for more comprehensive testing methods.
• Enterprises are increasingly adopting integrated evaluation tools like OpenAI's native platform, which may shape future AI development and deployment strategies.

The bigger picture

The findings suggest a growing awareness of the risks associated with fully automated AI deployment, leading to increased human oversight and investment in evaluation tools. This shift could influence the development of more robust AI systems and the adoption of better monitoring practices in the industry.

The survey underscores the ongoing challenges in ensuring the reliability of AI agents in production. While automation offers efficiency, the risks of unreviewed deployments and evaluation failures necessitate a balanced approach that includes human oversight and advanced monitoring tools.

Technical details

• Automated evaluations: Tests that score an AI agent's code or system change before release, allowing changes to be deployed without human review.
• Production monitoring: Systems that track AI agent performance, with 29% using real-time quality checks and 53% relying on trace logging or gateway tracking.
• Evaluation platforms: Tools like OpenAI's native evaluations, Confident AI, and Braintrust that help assess AI agent reliability and performance.
• Pre-deployment evaluations: Tests conducted before releasing an AI agent or LLM feature, which can sometimes fail to catch issues that arise in real-world use.

FAQ