AWS Executive to Enterprise Leaders: Ditch AI Metrics and Watch Your Business Numbers Instead

Quick Facts

• AWS SVP Colleen Aubrey says companies should stop creating new AI-specific metrics and focus on existing business measures like revenue per planner, forecast accuracy, and time to resolution.
• Amazon Connect, AWS's contact center platform, reached a $1 billion annualized revenue run rate after optimizing over 12 billion minutes of customer interactions in the past year.
• AWS is expanding the Connect product line into four agentic AI solutions covering supply chain, hiring, customer experience, and health care.

Colleen Aubrey, Senior Vice President of Applied AI Solutions at AWS, has a direct message for enterprise leaders chasing AI returns: stop inventing new ways to measure the technology and use the numbers you already track. Speaking at a recent event, Aubrey said the fixation on AI-specific metrics is a distraction that keeps companies from measuring what actually matters.

"Companies are stuck on what's the ROI of AI, and they're looking at return on tokens and this sort of stuff," she said. "The question is, can you move your business metrics by working in a different way?"

The Wrong Question

Aubrey, a 20-plus year Amazon veteran and member of its S-team, argues that AI's value must show up in metrics companies have always tracked. She named revenue per planner, time to resolution, forecast accuracy, inventory turns, and working capital as examples.

"Stay focused on the business metrics you've always measured," she said, "and go deeper. Because the real opportunity is to reshape how the business operates. AI needs to deliver your ROI in the form of changing your business metrics."

Her prescription for technology leaders is concrete. Before any project gets funded, it must be tied to an existing business metric with a clear baseline. If that metric does not move within a defined window, the project should not scale regardless of how the demo performed.

Four Rules for Deployment

Aubrey outlined a set of principles for companies deploying AI. First, she said to find a narrow, painful problem rather than trying to replicate an entire workflow. Projects that stall tend to be too broad. The ones that succeed identify a specific decision point and make it substantially better.

Second, budget for what comes after. She said AI rarely removes a bottleneck outright. It relocates it. Amazon's gains in shipping speed came from fixing test coverage and pipelines after a coding assistant surfaced the constraint. Companies should earmark resources for the next problem before deployment, not six months after discovering it.

Third, treat usability as a purchasing requirement, not an afterthought. "Shipping good AI with a bad user experience will fail, in my opinion," she said. Aubrey told buyers to weigh setup, configuration, analytics, and day-to-day experience as heavily as model capability when evaluating vendors.

Connect as Proof

Aubrey grounded her argument in Amazon Connect, AWS's contact center platform. The product reached a $1 billion annualized revenue run rate, making it the only usage-based customer experience solution at that scale, with AI handling more than 12 billion minutes of customer interactions in the past year.

She said the Connect team succeeded because it started narrow. Rather than chasing feature parity with established players, the team led with superior voice quality on the AWS backbone. "They understood they weren't going to achieve feature parity," she said. That discipline paid off. AWS was named a Leader in the 2026 Gartner Magic Quadrant for Contact Center as a Service for the fourth consecutive year.

Expanding the Product Line

AWS is now extending the Connect brand into four agentic AI products. Connect Customer handles customer experience. Connect Decisions targets supply chain teams, drawing on what AWS describes as 30 years of Amazon operational science and more than 25 specialized tools. Connect Talent focuses on high-volume hiring in manufacturing, logistics, retail, and hospitality, using AI agents to conduct voice interviews candidates can complete at any time. Connect Health targets the health care sector.

Companies including Wells Vehicle Electronics and TVS Motors are already using Connect Decisions. Aubrey described one early customer that has started running what-if scenarios in real-time business meetings using the product, with its procurement team requesting access beyond the original supply chain use case.

Aubrey described the expansion as a "day zero" moment, reached after roughly two years building out AWS's applications team and deciding where to focus. For the executives she is advising, the starting point is not a new dashboard. It is a business metric that already matters.