AI Coding Agent Metrics: What Engineering Teams Should Actually Track

Adoption metrics answer who has started

Adoption metrics include licensed users, activated users, weekly or monthly active users, teams with at least one user, and feature enablement. They answer rollout questions: did the intended population receive access, and are people trying the tool?

Adoption is a leading indicator of exposure, not proof of benefit. A high activation rate can coexist with low sustained use, shallow use, or extra review burden. Use adoption to manage enablement and access, then move to the next layer.

Usage metrics answer how the tool is being consumed

Usage metrics include sessions, requests, agent turns, model selection, tool calls, files referenced, accepted suggestions, generated lines, tokens, credits, and spend. They help with capacity planning, procurement, budget monitoring, and understanding which capabilities are being exercised.

Usage is especially useful when normalized by a meaningful unit. Spend per session can reveal cost distribution; tokens per comparable task can reveal variance; model mix can reveal routing choices. Without task context, usage is descriptive rather than evaluative.

Workflow metrics answer how work unfolded

Workflow metrics are often the missing middle. They include time spent exploring, clarification loops, repeated requests, failed attempts, alternatives considered, context changes, model changes, test and verification steps, and post-agent correction. They describe the path between request and result.

A workflow metric should not be interpreted as a virtue by itself. Exploration can be necessary. A retry can be valuable when it adds diagnostic evidence. A short session can be risky if verification was skipped. The goal is to understand the pattern in the task’s context.

Cost metrics answer what the organization consumed

Cost metrics include subscription seats, API spend, credits, tokens by model, cost per session, cost per task, and cost over time. They are essential for financial control and vendor decisions. They also help identify where deeper engineering review may be worthwhile.

Cost is not waste by default. A difficult migration may legitimately consume more than a routine change. The useful comparison is cost alongside task complexity, workflow, quality, and outcome evidence. AI coding cost versus engineering output explores this distinction in more detail.

Quality metrics answer what survived review

Quality metrics include automated test results, review findings, defects, security findings, rollback or revert signals, maintainability indicators, and rework after an agent reports completion. They can be captured at the change, session, repository, or service level, depending on the organization’s systems.

Quality signals must be interpreted carefully. A defect may have many causes, and a review comment is not necessarily a failure. The important question is whether the distribution of quality and rework changes for comparable work after a workflow or tool intervention.

Outcome metrics answer whether the work mattered

Outcome metrics include delivery performance, successful releases, incident recovery, customer-facing behavior, resolved support demand, or another result that the work was meant to produce. DORA’s delivery metrics are useful at the service or team level, while product and operational outcomes depend on the organization.

Outcome metrics are farther from the tool and therefore harder to attribute. That is a strength, not a weakness: they discourage the organization from treating tool activity as the result. Combine them with workflow evidence and state the limits of attribution.

Build a question-to-metric map

For a rollout question, use adoption and sustained usage. For a budget question, use spend, credits, model mix, and cost per comparable episode. For a workflow question, use retries, context changes, verification, and rework. For a quality question, use review, tests, defects, and production signals. For an outcome question, use delivery or product measures and treat agent telemetry as explanatory context.

This map prevents dashboard sprawl. It also makes it easier to explain why a metric is present and what it must not be used to conclude. That discipline is more valuable than adding another chart.

Sources and further reading

GitHub’s official metrics documentation is a useful example of separating adoption, usage, and code-generation activity. DORA and SPACE provide complementary reminders that engineering performance is multidimensional and should not be reduced to one activity measure.

Metric design questions to write beside the chart

For every metric, write four short notes: the question it answers, the work included, the decision it may inform, and the conclusion it cannot support. For example, “tokens per comparable task” can identify unusual consumption and support a workflow review; it cannot prove that the developer was inefficient.

This practice turns a dashboard into a measurement contract. New fields can be added when they answer a real question, and unused fields can be removed without losing the purpose of the system.

Start with a small metric portfolio

A first release might contain one adoption measure, one usage measure, one workflow measure, one quality measure, one outcome measure, and one cost measure. Add detail only when a finding cannot be explained. More telemetry is not automatically more insight.

The portfolio should also show missingness. If outcome evidence exists for only a subset of sessions, say so. A clean-looking chart based on incomplete coverage creates more risk than an honest blank.

An example metric catalogue

Adoption: active users and active teams. Usage: sessions, turns, tool calls, model mix, and tokens. Workflow: clarification loops, repeated attempts, context additions, alternatives, and verification. Cost: spend by model, session, task type, and period. Quality: review findings, tests, defects, security findings, and reverts. Outcome: delivery, reliability, customer, or operational results tied to the work.

The catalogue is deliberately layered. A leader investigating a budget change can move from cost to usage and then to workflow. A quality review can move from a defect to the change, the session, and the verification evidence. Keeping the layers separate makes it possible to use the same data for different questions without collapsing them into one score.