Question 1

Why do so many AI pilots fail to show measurable return?

Accepted Answer

Usually not because the model underperforms — it's because the project never had a measured baseline or a defined success threshold to begin with, so there's nothing concrete to compare the result against, even if it worked.

Question 2

We're still exploring — do we need all of this locked down yet?

Accepted Answer

For open-ended exploration, no. But the moment you're asking for budget beyond a small experiment, you need at least a rough baseline and success threshold, or you won't be able to tell leadership whether the spend was worth it.

Question 3

What's the single highest-leverage fix if we score low?

Accepted Answer

Ownership. A named owner with real authority tends to force the other gaps closed — they'll push for a baseline, a success metric, and a production plan because their name is attached to the outcome.

Question 4

Why does cost-at-scale matter if the pilot itself is cheap?

Accepted Answer

Pilot costs and production costs at real volume are often an order of magnitude apart. Projects that look affordable in a small test can become budget problems once usage scales, and teams that never modeled that get blindsided.

Question 5

Does a low score mean we should kill the project?

Accepted Answer

Not necessarily — it means the project isn't ready for more investment yet. Often the fix is narrower scope: define one measurable problem, assign one owner, and re-run the assessment before asking for the next round of budget.

AI ROI Reality Check

Is the problem this project solves clearly measurable?

About the Assessment