Across B2B tech, AI pilots are launching with strong models, real budget, and executive sponsorship, and then quietly stalling.
This readiness gap commonly derails AI pilots in tech service organizations, with the hardest problems to solve having nothing to do with the AI itself.
Hideki Hashimura , CRM/CX Strategist at redk, argues that most pilots stall because they’re built too small to ever deliver the transformation leaders say they want, which does not justify the investment or strategic attention:
“You don’t spend the money on AI to get a 5% improvement. You want something transformational.”
The Isolation Trap – Why Strong Models Still Fail
AI pilots often stall because they are deployed in isolation from the systems they are meant to enhance.
When leaders assume it stalls because the underlying models are weak, many begin by asking which tool or model to adopt; however, this framing misses a more fundamental issue.
If the organization itself is not structured to absorb and operationalize the capability, even high-performing models can remain disconnected from real outcomes.
“The AI tool, the AI assistant – it’s only the tip of the iceberg,” explained Hashimura.
“For it to deliver full value, it needs to be well integrated, have actionable capabilities, [and] have all the knowledge of procedures, processes, and policies that the organization works with, just like a human would.”
This creates an ambition gap when leaders treat pilots as contained experiments rather than embedded business initiatives, constraining their impact and often resulting in marginal improvements.
When AI investments are made with the expectation of meaningful change in performance, efficiency, and customer experience, deployments limited in scope cannot deliver justifiable outcomes.
Without careful design and integration into end-to-end processes and systems, the pilot remains confined with its impact far below its true potential.
The Four Foundations Leaders Consistently Underestimate
When AI pilots stall, the cause is often a gap in one of four foundations:
- Data
- Cross-functional architecture
- Knowledge management
- People and skills
If a pilot is underperforming, the question is how the gaps are interacting. When leaders identify these gaps, they are able to gain a clearer understanding into unlocking value.
Data
If the underlying data is incomplete, the model’s performance is capped regardless of its sophistication.
“Data is a constraint,” Hashimura explained.
“If you haven’t figured out the data you need to make this happen, the project is just not going to deliver the value it needs.”
This limitation affects a model’s ability to act, integrate into workflows, and produce outputs that can be trusted and operationalized.
Cross-functional architecture
Many pilots are scoped within a single team, preventing them from addressing the full lifecycle of a customer or operational issue.
When resolution rarely lives in one function, it requires coordination across service, operations, finance, and product, meaning without this design, AI can handle fragments but cannot complete outcomes, as Hashimura explained:
“What does the end-to-end of resolving an issue look like? How much interaction do you need with back-office operations, finance, [and] product to deliver that value back to the end user?”
Knowledge management
This is perhaps the least understood of all foundational pieces. This is how an organization captures and maintains what it knows, how and why it operates the way it does, serving as the information layer that people and AI assistants will rely on to respond accurately.
Being the least visible of the foundations, LLMs depend entirely on structured, up to date accessible information to generate useful responses.
If this knowledge is inconsistent across teams, AI systems will likely produce unreliable answers or fall back to generic responses.
“Knowledge management is a foundational piece to leverage the LLMs, potential” explained Hashimura.
“An LLM works on natural language and uses natural language processing to deliver its value. If you don’t have content, it cannot be leveraged.”
People and skills
While AI can execute tasks, it does not replace the need for human oversight in orchestrating processes and ensuring that outputs translate into real value.
As a result, teams must be able to configure systems, interpret results, and continuously refine workflows so systems can scale and adapt. This is what we call AI service architects, roles who understand the nuances of the business so they can orchestrate AI activity.
“The process orchestration – how those tasks deliver that end-to-end resolution and value – that’s what humans need to do,” explained Hashimura.
“The role of the human is changing in the service team, but they still play a very important part.”
Building to Scale – Governance, OKRs, and the 30-Day Question
Preventing an AI pilot from remaining an isolated experiment requires cross-functional ownership that connects operations, customer experience, and technology around shared outcomes.
This means leaders need a clear view of what success looks like across short and long-term horizons, ensuring that accountability sits beyond a single function.
“You need a delivery framework that is geared towards delivering objectives and key results. The objective is to build the AI capability that can deliver the full value of LLM technology,” Hashimura said.
“AI projects are business projects where the technology is an enabler. You can’t just put the technology in there and expect the business results.”
Treating AI as a standalone technical deployment leads to fragmented ownership and weak accountability for outcomes, meaning when leaders position it as a business initiative, the focus expands to become a transformative initiative that includes process redesign, workflow integration, this is the only way you can deliver significant change that can truly impact CX.
As a result, the delivery framework then becomes a mechanism for translating capability into operational value, with metrics that reflect how the organization performs.
Instead of focusing narrowly on whether the model is performing, leaders should interrogate whether the conditions for scale are being established.
In practice, many leadership teams default to tracking early output metrics within the first 30 days, meaning those that reframe the initial checkpoint toward foundations signal a higher level of organizational maturity and increases a pilot’s likelihood to evolve into something that can scale beyond its initial scope. Many times, the most significant output is actually “being able to measure” things that you couldn’t measure before, or new things that are now better indicators of performance.
