The decision you never revisit
Somewhere in the last year, you made a call: build a custom tool, buy something off the shelf, or defer and wait it out.
Most businesses make that decision once and never check it again. The tool either works well enough that nobody complains, or it doesn't and everyone just grumbles and routes around it. Neither reaction tells you whether you actually made the right call.
This is a Check-stage problem. You don't need a new decision — you need a scorecard for the one you already made.
Build, buy, and defer decisions usually get made under time pressure, with a gut-level sense of "this seems like the obvious path." That's fine — it's often the right instinct. But nobody schedules a follow-up to see if the instinct held.
The result: a custom build nobody remembers was supposed to pay for itself in eight months. A subscription nobody remembers was supposed to replace three spreadsheets. A deferred decision that was supposed to get revisited at 50% order growth, except nobody wrote down the 50%.
If you've read about sorting automation decisions into build, buy, or defer, this is the other half of that process — the part where you find out if the test actually passed.
The four numbers that matter
Forget whether people "like" the tool. Like is not a metric. Track these four instead.
| Metric | What it tells you | How to get it |
|---|---|---|
| Total cost vs. projected | Whether the real bill matches the plan | Add license fees, dev hours, contractor invoices, and maintenance time; compare to your original estimate |
| Time-to-value vs. projected | Whether you got useful output on schedule | Date of first real use vs. the date you expected it |
| Maintenance burden | The hidden cost nobody budgets for | Hours per month spent fixing, updating, or babysitting the tool |
| Fit score | How well it actually matches your workflow | Count of manual workarounds still needed to make it function |
None of these require fancy tracking. A spreadsheet with four columns and one row per decision is enough.
What "good" looks like, by path
If you bought: cost should track close to the subscription price, with maybe a small bump for onboarding time. Time-to-value should be fast — days, not months. Maintenance burden should be near zero; that's the whole point of buying. If you're spending real hours each week fighting the tool, the "buy" call may have been wrong, or you picked the wrong vendor. This is exactly the gap the subscription audit is built to catch — run that alongside this one if cost is the question mark.
If you built: expect a longer time-to-value and a real, nonzero maintenance line. The red flag isn't that maintenance exists — it's maintenance that keeps growing instead of leveling off. A build that still eats five hours a week a year in isn't done; it's a second job. If your maintenance hours are climbing while your original use case hasn't changed, you're accumulating automation debt, and it's worth checking that post's criteria too.
If you deferred: the metric isn't cost, it's discipline. Did you hit your trigger — the order volume, the headcount, the pain threshold you set — and actually revisit the decision? Or did the trigger pass quietly while you kept using the workaround? A deferred decision with no revisit date is not a decision. It's an accident waiting to look like one.
A before/after worth running
Picture a small logistics company that deferred buying a route-planning tool, betting the free spreadsheet-and-phone-calls system would hold until they hit 15 trucks. They set that number as the trigger.
The trigger was 15 trucks. By the time anyone looked up, they were at 19, and the dispatcher's manual routing time had tripled. The defer decision wasn't wrong when it was made — it just needed a hard calendar check, not a vague "we'll know when it's time" feeling.
Running the audit without fooling yourself
A few traps to watch for when you run this:
- Sunk cost blindness. If you built something and it's underperforming, the instinct is to ke