Why AI’s Productivity Boost Is Still Hard To Prove
CEO of Acumatica, a fast-growing cloud ERP company, John Case has nearly 30 years of industry leadership in cloud services.
gettyThere’s plenty of enthusiasm around AI. For small and midsized businesses in particular, its potential is significant. AI can automate repetitive work, expand the capacity of smaller teams and help people make faster and better-informed decisions.
That said, the more complex question I hear from businesses embarking on their AI journeys is this: How can I quantify AI’s impact on my business?
This question matters because there’s still a gap between AI’s promise and its bottom-line impact.
Research from MIT found that 95% of organizations studied saw no measurable profit-and-loss (P&L) impact from their generative AI investments six months after a pilot. Upwork research found that while more than two-thirds of SMB leaders reported productivity gains, most gains were below 25%. Even though a 10% or 20% gain is still meaningful, expectations often fall short of the dramatic transformation often associated with AI.
My advice to businesses is, before introducing AI, to be specific about the problem they want it to solve. This way, they can apply AI where it can deliver results and offer something concrete to measure afterward.
One of the biggest challenges with measuring AI is when companies launch pilots without establishing how the process performed initially.
Let’s take a finance team that introduces AI to help close the books faster. Six months later, everyone agrees that the process feels smoother. But how many employee hours were saved? Did error rates change? If the company never measured those metrics before implementing AI, proving the value is much harder.
Businesses can also mistake activity for outcomes. Adoption rates and hours of AI usage can indicate engagement, but they reveal far less about whether the technology is improving the business.
Disconnected data creates another challenge. If customer, financial, inventory and operational information is stored across different systems, organizations may struggle to establish a consistent view of performance or spend hours reconciling data before they can evaluate outcomes.
Time is another important factor. Early gains may appear through shorter cycle times, greater employee capacity or better access to information, but leaders also need to see whether those improvements are repeated over time.
Some of our customers have told me their employees reverted to old processes because the AI was too kludgy or poorly integrated into their workflows. That’s a perfect illustration of why sustainable ROI depends on whether the technology continues to deliver value as it becomes part of regular business operations.
Getting a clearer view of AI’s impact comes down to discipline and structure.
Before deciding on AI metrics, ask whether your systems can provide a reliable baseline.
Can your teams pull up current, consistent operational and financial information and connect what’s happening across the business to the outcomes it’s driving?
Clean, connected data gives leaders a foundation and makes it easier to track changes after introducing AI. For SMBs, that’s especially important because most don’t have the resources to build an entirely new measurement infrastructure for every experiment. Their existing business systems have to carry much of that load.
Before deploying AI, you should be able to answer two questions: What are we trying to improve, and how are we measuring it today?
Start with a specific business outcome, then establish the basis for how that process currently performs. You might want to reduce the time employees spend processing invoices, improve inventory forecasting, respond to customer requests more quickly or give employees faster access to operational data.
Consider starting with a manageable workflow, such as customer service, administrative work, analytics or inventory, where the relationship between technology and the desired result is relatively clear.
Once you know the starting point, define what success will look like. I find that most useful AI outcomes fall into four broad categories:
• Cost reduction, like lower outside spend, fewer manual processing costs or fewer corrections.
• Productivity and capacity, with the same team completing more work, or seeing shorter cycle times or fewer repetitive tasks.
• Revenue attributed to improved conversion, faster sales processes, improved retention or greater capacity to serve customers.
• Customer experience, defined by quicker responses, fewer errors or higher satisfaction.
The more specific the outcome, the easier it is to measure. If invoice processing currently requires a certain number of employee hours each month, for example, organizations have a confirmed baseline to compare against after introducing AI.
Choose the outcomes that matter for the use case, then check them against the baseline over time.
Evaluate whether the target metric improves, and if you can reasonably connect the improvement to the technology. Consider whether the new workflow introduces unexpected costs or challenges, and if employees continue to use it. Most importantly, determine if the improvement is consistent enough to justify continued investment.
Even small efficiency gains matter when it’s scalable, giving businesses evidence to inform their next steps.
AI adoption should be iterative. Identify a problem, establish a benchmark, test a solution, measure the results, and scale successful applications where the results justify further investment.
SMBs are already adopting this approach in areas such as data analytics, content generation and inventory management, with some businesses moving beyond experimentation to broader adoption.
Over time, individual efficiencies add up. A few hours saved in one process may seem small, but when you apply that improvement consistently across teams and workflows, the business can begin to create additional capacity.
AI’s potential to help SMBs operate more efficiently, expand capacity and make better decisions will only be realized when business leaders can clearly see where it saves time, improves customer outcomes or supports growth with greater confidence.
To achieve that level of confidence, organizations should be able to answer one question with evidence: Is AI making our business better?
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
