A CEO of a 400-person services company once told me: "I want to invest $250,000 in AI. What do you recommend?"
I asked one question back: "What's the problem you're trying to solve?"
He paused. Then said: "I don't know yet. I just feel like we should be doing something."
That sentence is the most expensive sentence in corporate AI today. Companies are spending hundreds of thousands of dollars on AI initiatives without first earning the right to spend a single dollar. The checklist below is the gate. If a proposed AI initiative can't pass it, the answer is not "implement carefully" — the answer is no investment yet.
This is the Brown Belt of AI Black Belt operationalized into a pre-investment instrument. 47 questions across six dimensions. If you can't answer most of them with specifics — not aspirations — you are about to join the 95%.
Dimension 1 — Problem clarity (the gate before the gate)
Before vendor, before data, before team. If the problem is fuzzy, every downstream decision multiplies the fuzziness.
- What specific business outcome will this AI initiative produce? Not "improve efficiency." A number, a metric, a delta.
- What's the baseline today? If you can't measure where you are, you can't measure improvement.
- What's the dollar value of solving this problem manually with the team you have today? If it's under $200K/year, AI is probably not the right tool — process improvement is.
- What's the dollar value of NOT solving this problem? Cost of inaction frames urgency.
- Who feels this problem most acutely day-to-day? Their voice matters more than the CEO's hunch.
- Has anyone tried to solve this with non-AI tools? What happened? Why didn't it work?
- Is the problem stable or is it changing month-to-month? AI on a moving-target problem produces moving-target results.
- Can you write the success criteria on one page that a non-technical board member would accept? If not, the problem isn't clear yet.
Most failed AI projects fail here, not in implementation. The vendor demo is impressive. The pilot looks promising. Six months later, leadership realizes the underlying problem was never well-defined — so "success" was never measurable. The 95% statistic is largely a problem-clarity statistic dressed up as a technology statistic.
Dimension 2 — Data readiness
AI without good data is theater. These eight questions surface whether your data foundation can carry the initiative.
- Where does the data live today? One system, ten systems, scattered spreadsheets?
- Is the data structured, semi-structured, or unstructured? Each requires a different approach and a different vendor profile.
- How clean is the data? Honest answer, not the answer you tell the board.
- Who owns the data internally? Without a named owner, data initiatives stall in week three.
- What's the data retention policy and the compliance perimeter (GDPR, HIPAA, SOC 2, industry-specific)? AI processing changes the perimeter.
- Can you legally share this data with the AI vendor? Many companies discover the answer is no after signing the contract.
- How much historical data exists? Less than 12 months usually means insufficient training signal.
- Is there any single source of truth or are there contradictions across systems? AI trained on contradictory data produces contradictory outputs at scale.
Dimension 3 — Team and operational capacity
AI doesn't run itself. The 47-question executive AI implementation checklist accounts for the human system that will operate, monitor, and improve the AI day after day.
- Who on the team will own the AI system after rollout? A named human with allocated time, not "IT."
- What's their AI fluency baseline today? If it's zero, budget 3-6 months of training before judging the system.
- Does the team currently have bandwidth, or are you stacking AI on top of a saturated team? Stacking on saturation guarantees failure.
- What's the change-management plan for the team whose work the AI touches? Communication, training, role redefinition.
- Who are the early skeptics on the team? Bring them in early — they'll find the failure modes before launch.
- Is there a clear escalation path when the AI is uncertain or wrong? Without an escalation path, the system silently makes bad decisions.
- What's the training budget for the team in year 1? Industry minimum: 2% of total team cost.
- Is leadership visibly using the AI themselves, or is it "for the team"? Leadership-not-using is the strongest predictor of project failure I've seen.
Dimension 4 — Vendor evaluation
The vendor market is loud. These eight questions cut through the demo theater.
- Can the vendor show three customer references in your industry, your size, your geography? "We can connect you" ≠ "Yes, here they are."
- What's the vendor's average customer tenure? Under 2 years suggests churn problems.
- What happens to your data when you cancel? Read the deletion clause out loud.
- What's the vendor's model behind the product? Built in-house, fine-tuned, or pure API reseller?
- What's the vendor's roadmap for the next 12 months? Specifics, not "we'll keep innovating."
- What's the price 12 months and 36 months from now? Most vendors raise prices aggressively after lock-in.
- What's the SLA and what's the actual measured uptime in the last 6 months? Demand the report.
- How does the vendor handle security incidents? Get the runbook, not the marketing.
The vendor that demos beautifully and answers vaguely is the vendor whose contract will hurt you in year two.
Dimension 5 — Governance and risk
This is where the Brown Belt meets the Red Belt. The 95% failure rate is overwhelmingly a governance failure, not a model failure.
- Who is the human accountable when the AI produces a bad outcome? A name, not a department.
- What decisions can the AI make autonomously, and what must escalate? Written perimeter.
- What's the audit frequency and audit dimensions? Quality, bias, security, alignment with company values, cost.
- Is there an ethics-operations committee for AI decisions affecting customers or employees? If not, you're flying blind on the most reputationally sensitive surface.
- What's the kill-switch? How do you stop the AI within minutes if it goes wrong?
- What's the disclosure policy to customers when AI touches their interaction? Trust-relevant.
- How are the AI's outputs documented for legal review? Discoverable in litigation.
- What's the model-update policy? When the vendor pushes a new model, do you re-test or trust silently?
For more on the structural governance behind these questions, the deeper piece is AI Governance Leadership.
Dimension 6 — ROI and exit
Money in, money out, money back. Without these seven questions, the AI initiative becomes a black-box expense line.
- What's the all-in 12-month cost? License + integration + team time + training + audit. Most executives undercount by 40-60%.
- What's the expected 12-month return? Quantified.
- What's the payback period? Under 18 months is healthy; over 36 months requires strategic justification.
- What's the year-2 and year-3 economics? Most AI projects lose money in year 1 by design — the question is when they turn.
- What's the kill-criteria? At what point of underperformance do you cancel? Define this BEFORE signing, not after.
- What's the success-replication plan? If it works in Department A, how does it land in B, C, D?
- What's the cost of ripping it out? Knowing the exit cost upfront prevents lock-in surprises.
How to use the checklist
Three modes:
Mode 1: Gate before approval. A proposed AI initiative must score ≥40/47 with documented answers before budget approval. Below 40 = the project needs more pre-work, not more budget.
Mode 2: Vendor RFP. Hand the relevant subset (dimensions 2, 3, 4, 5) to vendors. The vendor that answers crisply with specifics is the vendor you can trust. The vendor that hedges, deflects, or answers with marketing copy is the vendor whose contract will hurt you.
Mode 3: Internal post-mortem on a stalled initiative. Score the existing initiative against the 47. The questions scored under 3/5 are the actual failure points — not the model, not the budget, not "the team didn't adopt." Apply discipline to those points specifically.
Frequently asked questions
Score the proposed initiative against the 47 questions in this checklist. Minimum threshold to approve: 40/47 with documented answers (not aspirations). Below 40, the gap is rarely "we need more budget" — it's "we need more pre-work on problem clarity, data ownership, or governance design." Spending money on an under-40 initiative reliably produces the 95% failure pattern MIT documents. Spending pre-work to get above 40 reliably puts you in the 5% that succeed.
Three tests. (1) The problem can be stated in one page that a non-technical board member would accept as a real business problem. (2) The baseline metric exists today and can be measured monthly. (3) The dollar value of solving the problem manually with the current team for one year is at least $200K — below that, AI is rarely the right tool and process improvement usually wins. Pitches that fail any of these three are buzzword pitches dressed as strategy.
Four questions are heavily predictive. (1) Can the vendor show three customer references in your industry, size, and geography? Vague "we can connect you" answers signal weak adoption. (2) What's the vendor's average customer tenure? Under 2 years suggests churn problems. (3) What happens to your data when you cancel? Read the deletion clause out loud. (4) What's the price 12 and 36 months from now? Most vendors raise prices aggressively after lock-in. Vendors who answer all four crisply are dramatically more likely to deliver.
Three rules. (1) Pilot scope: one well-defined process with a measurable baseline, not five processes with vague baselines. (2) Pilot budget: enough to test with discipline, not so much that the org commits before learning — typically 10-20% of the full implementation budget. (3) Pilot duration: 90 days with a hard kill-criteria defined before launch. The most common pilot mistake is scope creep: starting with one process, expanding to three "while we're at it," and ending without clear signal on any.
Ready for the next step?
If your initiative scores above 40 and you want a second opinion on the gaps — or if it scores below 40 and you want to close the gaps before spending — that's exactly what the AI consulting engagement is designed for. For executive boards needing the framework explained to their leadership team, the keynote speaking format delivers this checklist in 60 minutes with case studies.
For the structural piece on why corporate AI fails at scale, read Corporate AI Implementation Failure. For the leadership angle on what to do instead, read AI Leadership Failure.
Want the complete method — the seven belts, the named frameworks (AMARTE, Hwa·Won·Ryu, Tumanov Filter, Green Matrix, PAF, PMP Triangle, Master Map of AI Systematization), and integrated case studies? Read AI Black Belt: Fundamentals Before the Prompt. Available now on Amazon in Spanish; English edition in final author review.
Go deeper
Want to bring your team to the next belt?
Book a discovery call or explore the full book.