The Adoption Metric Nobody Wants to Look At

This one's speculative — a thought experiment about design principles, not a report on something I've built.

A usage count is not an adoption signal

A dashboard showing that a new AI feature was used a thousand times looks like success. It's also consistent with a hundred people trying it once out of curiosity and never touching it again, which is a very different outcome than the same number produced by a smaller group using it repeatedly because it actually became part of how they work.

Repeat usage is the signal that matters

The metric that actually distinguishes real adoption from novelty interest is return usage over time by the same person or account, not total volume. A feature with modest total usage but high repeat rates among the people who tried it is a healthier signal than a much larger total usage number dominated by one-time tries that never happened again.

The uncomfortable version of this question

Asking whether a feature has real repeat usage risks an answer nobody wants: a shiny AI feature that was fun to build and demo well but that the actual user base tried once and quietly abandoned. That's an uncomfortable finding, but it's a far more useful one than a vanity total-usage number that looks good in a slide and tells you nothing about whether the feature earned a permanent place in anyone's workflow.

Build the honest metric in from day one

Retention and repeat-usage tracking for a new AI feature needs to be instrumented at launch, not added later once someone asks the harder question — because by the time someone asks, the early, most diagnostic usage data is already gone, and all that's left to look at is the vanity number that was easy to track from the start.

I'm Jesse Myers — Marine veteran, 32 years in enterprise IT, now building production AI systems. This site is where I write about what I've actually built, and occasionally about ideas I haven't built yet but think are worth taking seriously.