AI pilots can be a nightmare
When founders admit their pilots aren't great - real solutions emerge
Here's what nobody talks about when running an AI startup: We're all secretly running the same failed experiment. Over and over. And calling it "customer validation".
Yesterday I discovered I wasn't alone in this particular flavor of founder hell. Neither were the other 15 people staring back at me through Zoom, each nursing their own pilot horror stories like war wounds.
The question that started everything
"Quick show of hands," I asked, already knowing the answer. "Who has a pilot that's been running for more than 30 days?"
The silence hit first. Then, slowly, hands started creeping up. We weren’t judging, but regonizing.
When "Interesting" becomes a four letter word
“Interesting is that red flag at least for us and talking about the mechanics that's besides the point. Let's course correct and identify the business problem and why that is needed to be solved now rather than later.”
(This rings familiar from The Mom Test)
Martin from Signify had just finished a paid pilot - actually satisfied every success criteria in the contract. The business team loved it. Then IT showed up with "different opinions," and suddenly the work became a very expensive case study.
Chris from Quarq jumped in with a metaphor - It's like customers wanting to get in shape before going to the gym. They're preparing to prepare. Meanwhile, we're supposed to be the personal trainer saying 'No, you come as you are. That's the whole point’.
The $200K solution that we don’t know how to price
Others had a different dilemma: Replacing analysts who cost $200K a year. But when the price is named, they look at me like I’m crazy.
This seems obvious until you realize what's actually happening. We're not selling software anymore. We're selling organizational transformation, and nobody (not us, and not them) knows what that's worth.
Krishna added "I tell prospects they'll make $1,000 more per month. They look at me like 'okay, but why would I want that?' The value is so obvious we literally can't explain it."
The counterintuitive truth is that making the ROI too clear actually makes it less believable.
The Two Words That Changed Everything
"As soon as we said the word AI, because we're in a highly regulated business, we deal with business with banks and large financial institutions. Then I got down this track of, we don't even have an AI governance strategy yet in place for something like that. Right. So I kind of got off the rails because I was calling an AI agent. I literally changed it to a digital worker. Didn't even talk about AI."
Wait, that's it? Just terminology?
Jim continued: "They had 15 people doing an OFAC check. So that's checking people... They had 15 people in a room doing this every day. Right. And with our agent, they can do it with three people."
The Uncomfortable Mirror
Marcus had been observing from a different angle -he helps enterprises evaluate AI solutions. His perspective was interesting:
"You know what your buyers are really thinking? They have no idea what their organization looks like in 12 months if they adopt your solution. You're not selling them software. You're selling them a future they can't visualize. And futures are expensive."
We've all been there. Painting visions of transformation while our prospects are just trying to make it through Q4.
Workflows beat solutions
"Really trying to demonstrate how you own the workflow, not the solution. Like the solution, the solutions are a dime a dozen out there".
This resonated with me as this is something we’ve spoken about before -
My final takeaway
Let me tell you what I personally learned hosting this beautiful mess:
Every founder thinks their pilot problems are unique. They're not. We're all fighting the same battle - proving value for something that's never existed before while speaking a language our buyers don't understand.
Sure, you can build better tech - but you need to learn to translate that to value, from CTO to CFO. From features to headcount. From "AI transformation" to "digital workers".
Marcus said it best as we were wrapping up:
"Stop selling them vision. Sell them headcount reduction. The vision comes after they see the results."
I leave you with this final clip - AI and agentic pilots are probably just normal pilots.
As people started dropping off, the chat had some of the commitments:
"Tell our customer which person should be the POC owner"
"Qualify the potential pilot better by looking at questions and asking better questions"
"Qualify the Jokers"
"Longer view/plan how to operationalize it throughout the years instead of (problem, solution)"
Nobody tells you about building AI companies: The technology is the easy part. The hard part is admitting our go-to-market is broken and finding others brave enough to fix it with you.
The pilot trap is real. But apparently, so is the escape route. You just need 15 other founders to help you find it!
Our next Paid roundtable session: Forward Deployed Engineers is in three weeks on August 27th