Stop measuring AI usage. Start building AI capability.

Across industries, organizations are increasingly tracking AI usage through dashboards, token utilization and platform engagement metrics. Some of the most visible companies in the world have stood up leaderboards ranking employee AI use; others have tied AI adoption directly to raises and promotions.

The intent is reasonable - leaders want a signal that the investment is landing.

But we’ve reached a point where managers, executives and boards carry a false assumption that AI usage means a more AI-ready workforce. Usage and capability are not the same thing.

Recent research, based on a survey of 2,000 workers across the U.S. and U.K., suggests many organizations are measuring AI adoption faster than employees are learning to use it effectively. While 46% of employees report using AI tools at work, nearly half have received no formal AI training and 56% have no clear path for developing AI-related skills.

Perhaps most concerning, 17% admit they are pretending to use AI at work. If organizations mistake AI usage for capability and readiness, they risk building strategies and processes for a workforce that doesn’t actually have the skills to execute them.

The gap isn't a talent problem; it's a systems problem between technology adoption and workforce development. Solving it means CIOs and HR leaders must move beyond coordination and take joint accountability for translating AI usage into true workforce capability.

The measurement trap

As AI becomes embedded into everyday work, leaders are looking for ways to track progress. Dashboards, usage reports, prompt counts and engagement metrics seem to offer an obvious way to demonstrate momentum. But activity is not the same as capability.

An employee generating 10 prompts daily may appear highly engaged. That doesn’t mean they know how to provide effective inputs, evaluate outputs, recognize hallucinations, or apply AI responsibly in ways that meaningfully improve performance.

When organizations treat activity as a proxy for competency, leaders develop a false sense of confidence about workforce readiness while critical capability gaps remain hidden beneath the surface.

Don't just agentify the mess

There’s a parallel trap on the technology side, where there’s a race to “agentify” everything, wrapping an agent around every existing process and SKU.

But automating a broken workflow simply produces a faster broken workflow. The point isn’t to agentify the mess. It’s to rethink the work first, then apply AI to what matters.

The same discipline applies to how we measure return. Automation and the productivity gains are real, but treating efficiency as the finish line badly undersells the opportunity.

The larger prize is transformation: moving the topline and the bottom line, not just shaving cost per task. Organizations that aim only at incremental productivity will capture a fraction of what AI can actually deliver.

The visibility gap nobody is talking about

This creates a new challenge for CIOs and HR leaders. Most organizations can now see who is using AI tools. Far fewer can see whether employees are using them effectively.

The next phase of AI transformation will not be determined by access to tools, which most organizations have already solved. It will be determined by whether employees possess the judgment, confidence and skills necessary to use those tools productively and impactfully.

Without that visibility, organizations risk optimizing for adoption metrics while underinvesting in the development that generates long-term business value. Technology procurement lives with one team. Learning and skills data lives in another. Performance data often lives somewhere else entirely.

Consequently, organizations struggle to connect AI usage with business outcomes.

This is a CIO problem as much as an HR one.

Joint accountability, not coordination

The conversation I’m having with peers is about moving from coordination to joint accountability. Coordination means IT and HR talk to each other. Joint accountability means they own the same outcome together; specifically, whether the workforce can execute the organization’s AI strategy.

Forget using AI. Are employees using it effectively enough to have a measurable impact on the business?

That reframe changes where decisions get made and who makes them. HR leaders understand what capabilities the business will need and where the development gaps are widening. CIOs understand how AI tools are deployed, where agents sit in the workflow, and where technical infrastructure can support learning at the point of work.

Neither function can solve the problem alone.

The skills that endure

Through all this churn - new models, new tools, new agents every quarter - one thing stays durable: domain expertise expressed as work. The specific, task-level skills that make someone effective at their core job don’t depreciate the way a given tool does. As AI transforms how work gets done, those domain-grounded skills are what compound and carry forward.

When organizations examine why AI adoption often stalls or remains shallow, the same issue tends to surface: AI is deployed without being meaningfully anchored to the skills and tasks of the workforce. Adoption becomes activity - visible, but not compounding.

Addressing this requires a shift in focus. AI needs to be connected directly to how work is actually performed and improved. When adoption is tied to real tasks and outcomes, it becomes a mechanism for continuously strengthening underlying skills, rather than just increasing tool usage.

What CIOs need to own

The AI-readiness conversation has largely focused on technology deployment. The harder question is whether organizations are building the workforce capabilities necessary to translate adoption into results.

For CIOs, that means taking ownership of something that extends beyond technology infrastructure. It’s creating the systems, partnerships and feedback loops that allow their organizations to build capability at the speed AI is evolving, with visibility into the AI skills that their people are developing.

The most successful organisations will have CIO and HR leaders jointly turning AI usage into sustained workforce capability and measurable business value. They give employees not just the tools, but the support to use them effectively. That is what the AI-empowered workforce of the future looks like.

We've featured the best AI website builder.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit

Like this article? For more stories like this, follow us on MSN by clicking the +Follow button at the top of this page.