5 reasons AI agents aren’t delivering (yet)
Lucidworks’ 2025 AI Benchmark Report reveals 5 hard truths about AI agents, and what enterprises need to do differently to see real value.
AI hype isn’t new. We’ve lived through ERP, CRM, and cloud booms. But generative AI has been different, right? It exploded faster than most enterprises can process. And while flashy demos and billion-dollar valuations make headlines, the real challenge for business leaders is figuring out what to actually do with AI today.
As Lucidworks CEO Mike Sinoway put it:
“The initiative they launched six months ago is out of date. Orchestration is no longer relevant. Now it’s AI agents. And the AI agents they started with are no longer relevant because the security requirements are moving so fast.”
So how should enterprises cut through the noise? Here are 5 lessons from the 2025 Lucidworks AI Benchmark Report, and from Mike’s perspective leading companies through hype cycles.
1. AI agents are here — but they’re mostly basic
Everyone is talking about AI agents, but most are still at the entry level.
In fact, Lucidworks’ Benchmark Report found:
- Most deployed agents are analytical (pulling data) or logical (feeding data back into systems).
- Only about 6% of organizations are using agents with true transactional or physical capabilities.
Translation: despite the headlines, we’re still in early days. There’s enormous runway for businesses to experiment and scale, but expectations need to match reality.
2. Don’t skip the basics
One of the clearest themes from the AI Benchmark Report: companies are rushing to adopt AI while ignoring fundamentals.
Sinoway explains:
“We found companies rushing to put in natural language processing, and they haven’t even put in Spanish language processing on their websites. Or they’re investing in advanced supply chain planning AI, but don’t even show inventory availability online.”
The takeaway? AI isn’t a shortcut past foundational customer experience. Sometimes the smartest AI strategy is to shore up the basics first.
3. Small wins beat big orchestrations
Many enterprises started their AI journey by building massive, multi-model systems. But the market shifted. Customers don’t want a giant platform solving everything. They want specific solutions that solve something.
“Instead of selling an AI tool to do all of your go-to-market planning,” Sinoway notes, “you had to have an agent that did interactive Q&A with online shoppers, or an agent that did technical spec interrogation.”
Smaller, targeted AI tools are often faster to deploy, easier to prove ROI on, and more likely to get buy-in from boards and executive teams.
4. Security moves faster than you do
One of the Benchmark Report’s experiments was letting an AI agent loose across 1,100 e-commerce sites. What happened was eye-opening:
- It sailed past CAPTCHA tests.
- It tricked live human agents into thinking they were talking to a person.
- It even applied for credit cards and attempted to register as a business to complete B2B transactions.
Without tight guardrails and governance, enterprises risk deploying “little monsters” into production. Security can’t be an afterthought. Security actually needs to move at the same speed as innovation.
5. The ROI question is coming
Right now, many AI projects are still funded on hype. But that won’t last.
“I’ve seen the ERP hype, the internet hype, the CRM hype… and they all go through the same cycle. Eventually someone says, what’s the payback on this stuff?”
That shift is already happening with model selection. Leaders are starting to ask: is an open-source model good enough? Do we need a high-cost commercial LLM? What’s the ROI difference between a chatbot answering toothbrush questions vs. refrigerator questions?
If your AI roadmap doesn’t have a business case discipline built in, it will soon.
Final word: Keep the hype in check
Generative AI and AI agents are changing the way enterprises think about information access, customer experience, and operational efficiency. But the pace of hype doesn’t match the pace of reality.
Mike summed it up best:
“Never a dull moment when it comes to AI and the evolution of it and how fast it’s changing. The trick is to keep moving forward — even if it feels like you’re paddling like a duck under the water.”
For enterprise leaders, the takeaway is clear:
- Start with basics.
- Focus on small, valuable wins.
- Build for ROI and security from the start.
- AI hype may grab headlines, but execution wins markets.
Want to see how Lucidworks helps enterprises build real-world AI agents? Explore Agent Studio.