Twice the tokens, just 10% more commits: what JetBrains data reveals

What's going on

In 2026, token usage has become a stand-in currency for hard work. Tokens are the chunks of text AI providers bill by. At Meta, an internal leaderboard called "Claudeonomics" ranked tens of thousands of employees by their usage. Salesforce set minimum spending levels, according to the newsletter The Pragmatic Engineer. At GTC in March, Nvidia CEO Jensen Huang suggested giving engineers tokens worth about half their salary on top of their pay "so they can be amplified tenfold." The practice now has its own name: tokenmaxxing, burning as much AI as possible to look good on the leaderboards.

Until now, the pushback consisted mostly of blog posts and anecdotes. Now, for the first time, there's company data. In the preprint "The AI Era's Lines of Code?", researchers from Carnegie Mellon University, UC Irvine and JetBrains linked around 3.1 million coding agent calls to the commits of 404 developers. Commits are saved code changes. The title nods to lines of code, which were once abused as a productivity metric. The question this article asks: what does the token count actually measure, and what doesn't it?

The data: five weeks, two agents, no targets

The study covers five weeks, from June 29 to August 2, 2026. JetBrains has around 2,600 employees. Nearly all tokens ran through two agents the paper doesn't name: "Agent A" with 69 percent and "Agent B" with 31 percent. The gateway, the central checkpoint all requests pass through, logged the tokens and the internal credits charged for each call. On top of that came an anonymous survey with 412 participants, 15.8 percent of the workforce.

One detail matters for the debate: according to the paper, usage was voluntary, with "no target and no mandate." So JetBrains wasn't a tokenmaxxing shop. It still wasn't cheap. According to InfoQ, JetBrains' AI development costs rose roughly tenfold within six months. That's why the company introduced a central billing layer.

The key numbers and what they really mean

Some of the figures making the rounds tend to get oversimplified. The table below shows what's behind each one:

An elasticity tells you by what percentage one quantity changes when another changes by one percent. A percentile is a rank: the 90th percentile is higher than 90 percent of all values.

Usage depends mainly on who the person is. According to the paper, that explains 79 percent of the variance. Heavy users are heavy users every week. Commits and changed lines explain only 38 percent of the differences between people and only 13 percent of week-to-week fluctuations. Role, team and management position actually predicted new people's usage worse than a simple average. And high-usage weeks didn't come with bigger commits.

One event measurably drives usage up: a new model. In week 3, a new frontier model, i.e. a top-tier model, was released and accounted for 45 percent of spending within two weeks. Weeks using the new model had about 45 percent more tokens for the same number of commits.

400 to 1: where the tokens really go

Perhaps the most important finding is tucked away in the paper. A typical agent call (median) sends about 400 times as many tokens to the model as it gets back. Generated text makes up less than one percent of all tokens in both normal and expensive weeks, 0.6 and 0.9 percent respectively. Almost everything is context: code, files, logs and the conversation history that the agent resends at every step, a bit like someone reading the whole case file aloud again before every follow-up question.

That's where caveman comes in, a GitHub project that now has around 120,000 stars. Its slogan: "why use many token when few token do trick." The well-known "65 percent fewer tokens" isn't precisely defined on the project page. The values measured there refer to the responses: across ten developer questions, output tokens fell by about 61 percent in the most aggressive mode compared with no instruction at all. Compared with a simple "Answer concisely," the reduction was only 3 to 35 percent. The author himself admits that the rules cost about 1,160 additional input tokens.

JetBrains tested the skill independently in July. According to InfoWorld and Botmonster, the test ran over 86 paired tasks in Claude Code. Output tokens fell by only about 8.5 percent, with quality and runtime unchanged. An initial run with ten tasks had shown around 30 percent. Overall, the benchmark was actually more expensive with caveman, because a single task crossed the threshold for the pricier long-context tier.

Our math, using current list prices for Claude Opus 5.5 ($4 per million input tokens, $20 output, $0.20 for cache hits, $5 for cache writes) and the 400-to-1 ratio:

This is a simplification. The 400:1 ratio is a per-call median, not an overall figure. And responses feed back in as input in later turns, so short answers have a small knock-on effect. Still, the order of magnitude is clear: caveman-speak trims the part that barely matters. To be fair, caveman can do more now. A proxy module called "Read less" compresses logs, JSON and test output before the model reads them. By the project's own measurements, that saves 33.2 percent of input tokens across full sessions, though this hasn't been independently verified. This part targets exactly where the paper would.

What the vendors themselves recommend

Anthropic's cost guide points the same way. The first step, it says, is to turn on prompt caching, i.e. temporarily storing unchanging parts of the prompt so they cost only a fraction when resent. Without caching, Anthropic says, costs grow roughly with the square of the number of conversation turns. With Opus 5.5, cache hits cost just 0.05 times the normal input price. If your hit rate is below about 80 percent, something is breaking the cache, such as a timestamp ahead of the fixed portion of the prompt or switching the effort level mid-session. Models should be compared on cost per completed task, not price per token. OpenAI charges 0.1 times the input price for cache hits on GPT-5.6 and newer, and 0.05 times on the GPT-6.1 models. Google passes the savings on automatically with Gemini but doesn't list a rate on its documentation page.

The new Opus 5.5 guide, reported on by The Decoder, does away with old habits: drop "Think hard" from your prompts and start with the medium effort level. Effort is the setting for how much the model thinks and checks its own work. According to Anthropic, Opus 5.5 on "Medium" matches Opus 5 on "High." Those are vendor claims. The task description matters too: a preprint by Jakub Smékal analyzed 2,700 runs with Kimi K3. When a full specification was cut down to a terse user story, usage rose by 29.7 percent, and by 13 to 115 percent depending on the task.

How companies measured, and what's left of it

The Amazon memo contains a line that fits the study well:

"If your work doesn't benefit from current AI tools, that's a finding, not a failure."

What the authors recommend instead

The recommendations are concrete. Token usage should be used for cost accounting at most. Don't measure performance by it, and don't expect usage targets to meaningfully boost output. "Tokens per commit" doesn't work as an efficiency metric either, because the ratio automatically falls the more someone commits. To track AI adoption, the authors recommend metrics that don't scale with context size, such as days of use. Budgets should be summed over several weeks, since five-week totals rank people fairly reliably (reliability 0.87). Budgets need adjusting after new model releases. Before introducing caps, manage context, cache recurring context and route routine work to cheaper models. Track billed costs rather than tokens. And at the gateway, record which task each session served.

The survey shows why commits alone aren't enough. In 89 percent of responses from technical staff, they also used the agents for work beyond coding, such as data analysis (52 percent), brainstorming (49) and research (44). The median self-reported time saving was 10 hours per week. That comes with costs: almost half of the reported time losses involved wrong or poor results. And 55 percent of respondents had to rework AI-generated material from colleagues, 1.6 hours per week on average.

The bigger picture

The study is the best evidence yet that token counts mainly measure usage intensity, and that it's very unevenly distributed. They don't measure the value of the work. The authors name the limitations themselves: all results are correlations, not causes, since people chose their own usage. Commits are a crude measure, and pull request data was missing. The data covers only five weeks. JetBrains builds AI tools itself, so some of the usage may be testing. And the survey figures are self-reported. Whether the findings carry over to companies with usage targets is an open question. There, extra usage is likely to deliver even less, because idle work is deliberately generated. That's a hypothesis the study doesn't test.

Our take: if you're planning AI budgets for 2027 right now, take three things from the paper. First, look at billed costs per completed task instead of token leaderboards. Second, check your cache hit rate before introducing caps. Third, reassess budgets after every major model switch. Cost-cutting tricks on agent responses may be fun, but the bill is driven by context.