naomi. Native AI Operational Maturity Index

Native AI Operational Maturity Index

NAOMI

A single-page index for working out how far an engineering organisation has actually moved — from a few people experimenting with AI, to an operating model that assumes it.

Read across a dimension to find the description that is honestly true today. The gap to the next column is the work.

Draft. The dimensions and level descriptors below are placeholders while the index is being written. Treat the shape as real and the wording as provisional.

The index

Dimension 0 Ad hoc Individuals experiment. Nothing is shared, measured, or repeatable. 1 Exploring Pockets of deliberate use. Early guardrails, no operating model. 2 Repeatable Agreed practices and tooling. Outcomes are inspected, not yet trusted. 3 Integrated AI work is part of the delivery system, with its own controls and metrics. 4 Native The operating model assumes AI. Removing it would change how the org works.
Strategy & governance Who decides where AI is used, and who is accountable when it goes wrong. No stated position Stated intent, no owner Named owner, written policy Policy enforced in the delivery path Governance adapts as capability moves
Data & context What the systems are allowed to see, and how good that context is. Whatever is pasted in Ad hoc access to some sources Curated context for key workflows Context is a maintained product Context freshness is an SLO
Tooling & platform The shared substrate teams build on rather than each assembling their own. Personal accounts A sanctioned tool or two Shared platform, self-serve Platform is on the paved road Capabilities ship as platform primitives
Engineering practice How AI shows up in day-to-day design, review, and delivery. Occasional autocomplete Assisted authoring Agreed review expectations Agents run inside CI and on-call Work is designed for agents from the start
Evaluation & assurance How the organisation knows the output is good enough to ship. Vibes Spot checks Repeatable manual evaluation Automated evals gate release Evals evolve with production signal
People & skills Whether capability is concentrated in enthusiasts or held by the team. A few enthusiasts Informal sharing Documented practice, onboarding covers it Expected of every engineer Practice is taught, measured, and refreshed

How to use it

Score each dimension independently. Organisations are rarely level across the board, and the uneven ones are the interesting ones — a team running automated evals on top of context nobody maintains has a different problem from one with excellent context and no way to tell whether output is good.

A level is only claimed when the description holds without a named individual holding it up. "One person could do this" is the previous level.

The index is descriptive, not a target. Level 4 is not automatically the right place to be; it is what the right place looks like when it is where you need to be.

Download

The printable index PDF · not published yet