Measuring whether any of this is working
Vendor dashboards mostly count usage. What each number really counts, why you can't A/B your own team, and four measures your repo and forge already hold.
Somebody is going to ask, and the dashboard answers a different question
What's up? Someone must ask at some point if the tool is worth it, either the person who approves the licence or you looking back on the previous quarter.
The vendor dashboard usually has lines of code (LOC), acceptance rate, daily active users (DAU) and sessions; a leaderboard in Claude Code's case or Usage Leaderboard in Cursor's. These all show how much the tool was used and how often its output is kept — it's useful information, but not what you want to know.
Just look at the vendors themselves: Anthropic are so reserved about the scope of their contribution metrics they say in the docs those are "deliberately conservative and represent an underestimate of Claude Code's actual impact"; GitHub describe acceptance rate as a metric that "helps you understand whether suggestions are relevant and trusted" — modest and correct.
What you want to know is whether the software is being shipped faster, and remains robust. The figures on the dashboard aren't that, this is about the rest.
Read the definition before you put the number in a slide
But before we get there we need to point out you should look at how the vendors calculate things, because they use heuristics to estimate these — not a flaw, it's what you can do given the task, and both Anthropic and Cursor document their methods.
For example, take Claude Code's PR attribution from its docs: they go over the diff of every PR and count all the new lines there, then look at sessions between 21 days before and 2 days after the merge date. Then they do some normalisation — trim leading and trailing whitespaces, replace multiple spaces with a single space, standardise quotes, convert to lowercase — and only then keep lines that are longer than three characters after that (so they ignore empty lines and lines consisting of brackets), which they call "effective lines". They omit lock files, generated code, anything in dist/, build/, node_modules/, target/ directories, test fixtures, and any line longer than a thousand characters. Anything the devs substantially rewrote — changed by more than 20% — is not attributed either.
The 20% figure is about how much of the code was changed, not about its quality; it's the threshold from which they're not attributing anything, with no judgement on whether the rewrite improved the code.
It means that a team that reviews the agent's output and makes alterations might have lower numbers than a team that creates the first version and merges it as is, so it's a good metric to measure adoption but a bad target because aiming for it will motivate keeping the agent's output untouched.
It's the same with acceptance rate — Anthropic define it as the "percentage of times users accept Claude Code's code editing suggestions, including Edit, Write, and NotebookEdit tool usage". It captures how often people accept changes, not whether accepting them is sensible.
Cursor calculate differently — they keep a signature log, which we'll get back to — and also uses an estimate, with the 20% figure being Anthropic's thing. But even if it was a universal standard, the A/B test doesn't work in the real world.
The experiment you want doesn't survive contact with your team
The ideal experiment would be a half of a team using the tool, and a half without it, for the same period of time, comparing the outcomes — but they've already done that, and it didn't go well.
METR's randomised controlled trial from 2025 used 16 experienced open-source developers and 246 real issues in repositories they were familiar with. It turned out the tasks took 19% longer when they could use AI tools; after the experiment those same devs thought that with the help of AI tools they were 20% faster.
METR are careful with their conclusions though — they say it's just a snapshot of the state of AI in early 2025 in one particular context, and explicitly don't generalise that to "AI is not good for most developers".
But what matters is what happened when they tried it again.
They published a redesigned experiment in February 2026 because the original design didn't work any more — the devs didn't want to participate in the non-AI half even for $50/hour, and 30%–50% of the participants said they didn't do some of the tasks during the experiment because they didn't want to do them without AI. METR's own interpretation is that devs are probably faster now than they were in early 2025, but given that it's a self-selection experiment the data they collected is only very weak evidence for how much faster.
So the specialists lost their control arm and are rebuilding around it — higher compensation, observational data, randomisation per dev rather than per task. You can't keep a control group during a sprint with six people doing tasks that were not assigned randomly; don't try.
What's left is a before-and-after in an environment full of confounders — the company has reorganised, you've hired someone, last quarter was a bit messier than usual — so pick measures that can't be made worse by these confounders.
Pair every speed number with a stability number
Every speed figure needs a stability figure next to it. DORA did their 2025 report with almost 5,000 tech professionals participating in it; 90% use AI at work, 80%+ think it makes them more productive. But what's important is that the report showed increased adoption of AI correlates with both higher delivery throughput and higher delivery instability.
So just the speed doesn't mean anything — you're faster both when things are better and worse — so it's about the speed being higher while stability is the same, which is why we always show these two together. DORA say that AI is an amplifier: it magnifies both strengths of strong organisations and weaknesses of weak ones; it turns out there's a lot of weak points in organisations, like sparse test coverage.
Oh, and btw 30% of the participants report little or no trust in the code written by AI. Just saying.
Four numbers you already have
But we have four figures already, from the forge and git history. They're the same for the whole team no matter what anyone uses, and you need to run just two commands in the terminal — you probably have the forge CLI already anyway — but the crucial thing is that they have a baseline before the adoption, while surveys and vendor dashboards collect data after you flip the switch.
Lead time and PR size are from a single forge query; here's an example:
gh pr list --repo <owner>/<repo> --state merged --limit 50 \
--json number,createdAt,mergedAt,additions,deletions,changedFilesIt lists recent merged PRs with their creation times, merge times, number of new lines (additions) and number of deleted lines (deletions), and the number of files that have changed. The median gap from createdAt to mergedAt is lead time, the median of additions plus deletions is the size of what a reviewer needs to read.
When we ran it on a public repo we got 19.3 hours for lead time and 20 lines for PR size — your numbers will be different but these are yours, one before and one after. It's from the forge, not git, because squash-merge repos throw away the branch history so a recipe built on git log --merges has nothing to read — we ran it on a repository with a single merge commit across its entire history versus 474 commits in the last six months.
Add reviews to that query and take the earliest one for each PR, that's the review latency — where everything else manifests so we're collecting it before we need it.
Rework is a cheap replacement for change failure rate; it's the number of reverts and hotfixes divided by the number of commits in the last 180 days:
git log --since='180 days ago' --oneline -i --grep='^Revert' --grep='hotfix' | wc -l
git log --since='180 days ago' --oneline | wc -lAnd then you divide it by the total count, and that's the rework. It's rough, it only works if your team consistently names reverts and hotfixes as such, but for a stability signal it's as close as you can get with a free tool — it's also what gives lead time context.
These four figures, two commands, history going back to when you created the repository.
The number that goes up first means the least
But the first thing that changes is the least useful.
It's the quantity — more PRs, commits, LOC — and people notice it first; that's why it gets onto the chart. It says nothing about where the work went though; it was never the hard part — it's the reviewing, comprehending, maintaining.
So if you're faster but your review latency increased, it means the bottleneck got queued up rather than eliminated, which is good, but not shipping things more often. If you track the quantity on its own you can't tell the difference between a success and a failure for three months.
Ask people, but not whether they're faster
So ask people about it, but not about speed — pick another axis. The thing is, developers think they're 20% faster while in fact they take 19% longer, as reported by the very people who do the work.
Let's ask them instead what was hard this week, where they lost faith and took over, what took three tries.
These are all different questions — they rely on recollection rather than estimation — and they point to things that can be improved: a line missing in the instructions file, a tool used wrongly, a part of the codebase not ready for an agent yet.
The measurement that turns into a target
The thing is, if you have a metric, it becomes a target. There are leaderboards in both Claude Code's and Cursor's dashboards — top contributors by Claude Code usage, or Usage Leaderboard — which are okay when looked at in aggregate but become targets if you display them to someone who writes performance reviews.
And the worst thing is the attribution — in a leaderboard of AI-written lines people get paid for leaving the agent's output untouched. If you really need an individual metric (someone is stuck, or has a workflow worth sharing), fetch it, use it, and keep it to yourself.
In Claude Code
In terms of Claude Code, there are two different dashboards depending on the billing plan you have.
If you have a Teams / Enterprise plan, you can access the dashboard from claude.ai/analytics/claude-code. It's restricted to the Admin and Owner roles and shows a few numbers about usage — accepted lines of code, suggestion acceptance rate, DAU (Daily Active Users), sessions. There are also contribution-related numbers there. The last part is a table with 10 heaviest users, but it seems that if you export it via the Export button, you'll get a CSV file with all the users rather than just 10 of them.
If you have the UsageView permission for the Console, you can access the Console dashboard from platform.claude.com/claude-code. It shows accepted lines, acceptance rate, activity, and spend, as well as the team-insights table split by user. What it doesn't have is the GitHub-integration contribution metrics — the docs say plainly those aren't available for API customers, so the Console dashboard is about consumption and cost only and doesn't tell you anything about the work shipped.
If you want to set up the contribution part, you need to enable it. The GitHub admin installs Claude GitHub app, then the Claude Owner enables Claude Code analytics and then the GitHub analytics toggle. After that, the Claude Owner does the GitHub authentication and selects which organisations to analyse; the first data appears after around 24h and is updated once a day. It's in public beta for now, but what's more important is that if you use Zero Data Retention for any of your orgs, you don't get any contribution metrics — only usage ones.
Whichever dashboard you choose though, whenever you merge a PR with some lines being attributed to Claude Code, it gets a claude-code-assisted label in GitHub and the docs suggest querying this label as a kind of a dashboard replacement if you can't access it for some reason. That way you'll be able to see how your lead time looks like or examine revert metrics for example by correlating how often you used Claude Code's assistance with these figures. You could for instance list all merged PRs that have this label using the CLI and print their IDs, creation and merge times as well as additions and deletions.
gh pr list --state merged --label claude-code-assisted --limit 100 \
--json number,createdAt,mergedAt,additions,deletionsIf you're on the Enterprise plan, you can access the Claude Enterprise Analytics API with the Primary Owner using the read:analytics-scoped key to get per-user engagement, usage and cost figures across Claude products (including Claude Code), but it's not available for the Teams plan. For the Console, the docs point to the Claude Code Analytics API with an Admin API key which is supposed to return the same per-user daily figures programmatically.
Setting CLAUDE_CODE_ENABLE_TELEMETRY=1 and configuring an exporter will send OpenTelemetry metrics to a backend you own: claude_code.session.count, claude_code.lines_of_code.count, claude_code.pull_request.count, claude_code.commit.count, claude_code.cost.usage, claude_code.token.usage, claude_code.code_edit_tool.decision, claude_code.active_time.total. The most valuable of these are the third and fourth — counters of PRs and commits which are a reflection of the work you delivered and can be tagged with your own dimensions to visualise together with the rest of your monitoring.
But before integrating anything, we'd recommend exporting this data to a console exporter first just to have a look at it:
descriptor.name: "claude_code.cost.usage" COUNTER, unit "USD", value 0.000581
attributes:
user.id, session.id, organization.id, user.email, user.account_uuid,
user.account_id, terminal.type: "non-interactive",
model: "claude-haiku-4-5-20251001", query_source: "auxiliary"Here's an example of the cost metric from our run — it's a USD counter with the value of 0.000581 and a user, session, org, email, account UUID and ID, terminal type, model, query source dimensions attached to it; as you can see, it contains an email address by default, and none of the five documented cardinality environment variables (session ID, version, account UUID, entrypoint, resource attributes) has any say over that one. That's worth deciding on purpose rather than discovering in a dashboard — this is the telemetry instrumented for answering questions about your team at a high level and it arrives carrying individual identities.
Another thing you can see is that even though we asked it to just Reply with exactly: ok, the cost part of the metric comes from an auxiliary Haiku call, so it tracks every call you make beyond the model you chose.
There's also a claude gateway subcommand taking a --config flag that points at YAML, described in its own help as running the enterprise auth/telemetry gateway — the first-party route if you'd rather the telemetry left the machine through infrastructure you run.
In Codex CLI
Org-level. Available telemetry surfaces: the Codex CLI has several of them, there are three of them that are accessible when you have an enterprise ChatGPT workspace with Codex enabled, which are all described in the admin rollout guide:
Workspace analytics — an interactive, explorative view
Analytics API — a programmatic way to access aggregated reports
Compliance API — an auditing and investigation surface
The guide links to the official authenticated API references, make sure to take a look at them to see the most up-to-date access requirements, schemas, fields, retention etc — they're where the actual endpoint URLs, necessary scopes and field lists live. We don't have an enterprise workspace so we couldn't open them ourselves, which means you should treat any endpoint you find quoted on a blog as unverified until you've read it in those references yourself.
The admin rollout guide says they are all bound to the enterprise workspace administration. From the perspective of the public docs, that means that if you're a personal plan user, you can access only the rest of this.
Your own pipeline. If you want to set up your own thing, the Codex is emitting OpenTelemetry, which you can configure in the config.toml under [otel] section:
environment — the environment it operates in
exporter — the type of an exporter used (possible values are none, statsig, otlp-http and otlp-grpc)
trace_exporter— a configuration of an OpenTelemetry exporter for tracesmetrics_exporter— a configuration of an OpenTelemetry exporter for metricslog_user_prompt— a flag to configure if the prompt should be loggedspan_attributes— a way to customise attributes in the spanstracestate— a configuration to customise thetracestatein the emitted spans
We read those last two off the key names in the binary rather than from a docs page, so check them before you rely on them.
The shape of the exporter key is the thing that trips you up. Writing it as a string — exporter = "otlp-http" — won't do, because it's a table keyed by transport: [otel.exporter.otlp-http]. Put both in and you get a parse error rather than a silent fallback, as it's not possible to provide the string and table versions of the same key — at least it's transparent.
$ codex debug prompt-input
Error: config.toml:5:7: cannot extend value of type string with a dotted key
Caused by:
TOML parse error at line 5, column 7
|
5 | [otel.exporter.otlp-http]
| ^^^^^^^^
cannot extend value of type string with a dotted keyThat error came from a string line sitting above the table. Drop it, leave the table form as it is, and it validates:
[otel]
environment = "prod"
[otel.exporter.otlp-http]
endpoint = "http://127.0.0.1:4318/v1/logs"
protocol = "json"You can test your configuration easily by using the codex debug prompt-input command — it logs the prompt the model sees as JSON (no API call made), so you can be sure you won't incur any costs while exploring the configuration.
The names of events that are being emitted in the binary files are:
codex.user_promptcodex.tool_decisioncodex.process.start
They have properties like:
conversation.idmodel
auth_modeoriginatoruser.account_iduser.emailterminal.type
You can also see [REDACTED] next to the prompt-related fields, with a log_user_prompt key to go with it — we think that's consistent with the fact this is an opt-in thing so by default it's not enabled (which we haven't checked ourselves) and no collector has been attached to verify it.
The gotcha to check before you build on it. There's a gap known though, there's an issue for codex-cli 0.105.0 from February 2026 titled "codex exec emits no OTel metrics; codex mcp-server emits no OTel telemetry at all", saying that the interactive CLI fully honours the OpenTelemetry config while the two other entry points don't, status is closed.
It's good to check if the measurement setup you want to have in place requires headless CI runs before starting the project — didn't verify this on 0.146.0.
In GitHub Copilot CLI
The Copilot CLI is a perfect example of how useful it is that the team's utilisation data and the shipping data are kept together. It's more valuable than any of these numbers in isolation. That being said, the Copilot metrics API allows you to examine both of them for organisations and enterprises, broken down by repository or user. The way GitHub describes it, it's based on a variety of telemetry from multiple surfaces of Copilot, including IDE and CLI — so yeah, terminal usage is included too.
There are two caveats though:
It takes up to 2 days after the end of a day for the data to be processed and published — not very useful in the middle of an incident but great for retrospectives every week or month
To use it you need to have access to do an API request which requires enterprise or organisation-level admin permissions
The most popular metric is the acceptance rate which as GitHub says represents how relevant and how much people trust Copilot's suggestions are. That's a correct but very narrow angle, so make sure you don't think about this number as anything more than that — it's the acceptance rate of inline completions, so it's different if your terminal agent has altered 9 files and if you've hit Tab to accept a single greyed suggestion.
What's more important is that in Copilot you also have the "delivery side" of the same picture on the same machine — lead time, PR size or review duration are all available thanks to the CLI itself which you already use anyway, so for example you can run this:
gh pr list --state merged --limit 100 \
--json number,createdAt,mergedAt,additions,deletions,reviewsThe list of the 100 most recently merged PRs with number, creation and merge time, additions and deletions and reviews. For that you need a read access to the repository while to use the metrics API you need an organisation or enterprise admin permissions — but in both cases you don't need to build anything nor run any infrastructure.
So give it a try before you spend a week implementing a metrics pipeline.
In Cursor
The dashboard. Cursor's team analytics reports a fair few things, which are the following:
AI Share of Committed Code
Agent Edits
Tab Completions
Messages Sent
Active Users
Daily Usage
Usage Leaderboard
Repository Insights
Conversation Insights
Client Versions
Cloud Agent stats — number of agents created, PRs opened, PRs merged and lines written
Some of these are behind the paywall (Usage Analytics — for Team and Enterprise tiers, Conversation Insights — default to enabled only on Enterprise). The AI Share of Committed Code is the thing which is worth to be mentioned. Cursor keeps signatures under every line of code generated by the AI (through Tab or Agent) and then matches these signatures with further git commits, so it can tell what percentage of the code was actually committed thanks to the AI. This happens locally (as per the docs, all the detection is done on device and never leaves the user's computer). There's no threshold documented here the way Anthropic document theirs, but given that it's based on matching signatures, whenever sth gets changed the line stops matching — so it likely points the same way. So keep this in mind and check against your numbers before you decide if you trust this metric.
The API. There's also an Admin API (for Enterprise only) which works on the basis of Basic auth, using the API key as a username. There are multiple endpoints there that might be useful:
/teams/daily-usage-data— number of lines added and removed, acceptance rates, Tab completions, usage of models and number of requests; limited to 20 reqs per minute so it'd be good not to call it more often than once per hour/teams/filtered-usage-events— the most granular information about every request, in particular number of input tokens for the request (inputTokens), number of output tokens returned by the model (outputTokens) and cost in cents (chargedCents); per conversation identifier (conversationId)/teams/spend— spend per user in cents (spendCents) and total spend in cents (overallSpendCents) for the current billing cycle/teams/audit-logs— team audit log events, filterable
But it's worth to know that as per the docs the daily-usage-data endpoint does not include billable request units so this is not a good place to look for a cost information. It's only filtered-usage-events and spend endpoints which carry the information about money here.
Also, it's important to remember that these are separate things — activity (which you can see when calling the /teams/daily-usage-data endpoint) and cost; if you try to get a cost information by calling this endpoint you'll get a sensible figure but it will be not correct, there won't be any error.
If you have a Pro seat for an individual the above doesn't apply at all, so for such case the best place to look is the git and forge metrics from the main body.
In Antigravity CLI
No built-in way to export usage data. agy --help lists these subcommands and not one of them is about usage, telemetry or analytics:
agent
agents
changelog
help
install
models
plugin
plugins
update
The --log-file flag is not what we're looking for, it just points where the CLI logs to itself, doesn't produce anything useful as far as tracking metrics is concerned.
The enableTelemetry option in the config menu is a different thing. As described in docs, it's about sending anonymous usage stats and crash reports so that Google can improve the product, but it's all data that gets sent to the vendor; enabling this option doesn't return any data to the user.
The exception, if you have it. Unless you have an enterprise licence — in such case, the entire landscape changes, as explained in the docs:
Your customer telemetry and model interactions are logged directly to the Google Cloud project corresponding to the license you select.
That's actually the best data access of all the harnesses in this lesson — not a prebuilt dashboard with metrics that are defined by the vendor, but the raw events in your own project on Google Cloud, which you can query using tools you're already using. Also, worth noting, the docs describe no admin-level analytics dashboard that comes with it as part of this solution — if you want to create one on top of that, it's up to you.
Everyone else. If you don't have an enterprise licence, you can always instrument the repository instead of the tool. The four metrics in the main body were picked for exactly this case, and they cost nothing to start tracking today. And as far as the per-run numbers are concerned, you can always instrument your own wrapper around headless invocations and log from there during the run, instead of waiting for some hypothetical reporting feature that the docs don't even suggest is in the cards.
In Kimi Code CLI
We don't have a reporting tool for the team/organisation and no way to export data for multiple people at once. Analysing the output of kimi --help, we can see it lists only export, provider, acp, web, server (deprecated), login, doctor and vis, migrate, upgrade. It doesn't include anything that would show how many times it's used for instance by an individual or an admin.
What we have is local-only and bound to a single machine:
Sessions data is stored in
~/.kimi-code/sessions/and indexed in~/.kimi-code/session_index.jsonlfile. Each record there hassessionDir,sessionId,workDirfields —workDiris the one that's important here as it allows counting sessions per repo without inspecting sessions themselvesWe can use
kimi export [sessionId]to create a ZIP for a single session (the latest one by default) which includes~/.kimi-code/logs/kimi-code.logfile unless you use--no-include-global-logflag. It's designed to be used when reporting bugs, not as a way to collect numbers~/.kimi-code/telemetry/is just a directory that's empty on the machine we have checked so no contents there to report
That means there's no version of this where you compare six people's numbers, because the numbers never leave the six laptops. But rather than treat that as a gap to work around, take it as the case the main body was written for — we can examine repository-visible signals like:
lead time
PR size
review latency
reverts
And all these are agnostic to what tool someone uses, also the history for them goes as far as the history of the repo itself.
If you really want it to be per-person, asking about friction rather than speed is the most sensible way to achieve it.
What a real answer sounds like
Okay, so let's say we're talking with our stakeholders and they ask if the tool is worth it, we could say "yes, we're 30% faster" but we'd go for something like "we create a third more PRs per week, median PR size is 40% bigger, time to first review increased by two hours, revert rate didn't change".
It shows the output has increased, nothing broke so far and the burden shifted to reviewers. We can share it in a single sentence with whoever asked, we know what to tackle next, and we could give sources for all of these figures.
And if you really need to keep it cheap, just watch the rework number for the six weeks after any significant change in how the team works. Speed announces itself, instability doesn't work like that.