Earlier this year, tech companies actively encouraged employees to maximize their use of artificial intelligence tools, a practice informally dubbed “tokenmaxxing.” The term “token” refers to units roughly equivalent to word fragments in A.I. processing. Employees at firms such as Meta and Amazon engaged in competitions, using leaderboards to track token consumption, signaling a push to integrate A.I. broadly into daily workflows.
However, as the year progressed, the surge in A.I. usage led to escalating costs. Providers like Anthropic and OpenAI charge companies based not only on subscription fees but also by the volume of tokens consumed, with prices rising steeply for more advanced models. Tasks ranging from summarizing meeting transcripts, which require a few hundred tokens, to developing software code that can consume tens of thousands, have added financial pressure. Anthropic’s latest model, Fable, for instance, costs twice as much as its predecessor, Opus, contributing to growing expenses.
In response, companies have begun to scale back unrestricted A.I. usage. Meta recently informed employees that it would impose limits on A.I. tools after identifying an “exponential increase” in related costs. Uber reported exceeding its annual A.I. budget within the first four months of 2024 and has subsequently implemented monthly caps on coding tool usage. Walmart followed suit, setting limits on several A.I. applications. Additionally, Amazon and Meta have removed the tokenmaxxing leaderboards that incentivized high token consumption.
This shift has catalyzed what some call “tokenminning,” or the strategic reduction of token use. Experts note the challenges companies face in navigating the rapid evolution of A.I. integration, with early metrics focusing on token volume often promoting quantity over efficiency. “C.E.O.s who did not know how to measure the A.I. savviness of their employees thought, ‘Well, who’s using the most tokens?’” explained Rob May, chief executive of A.I. consultancy Neurometric and author of “The Tokenminning Manifesto.”
Companies are now emphasizing more strategic A.I. deployment. Uber’s COO, Andrew Macdonald, highlighted the difficulty in linking A.I. usage to tangible business outcomes, underscoring the need for clearer returns on investment. Meanwhile, Meta stated it remains committed to spending billions on A.I in 2024 but aims to reduce costs without compromising results. Salesforce CEO Marc Benioff noted his company is shifting from tracking token consumption to monitoring “agentic work units,” a metric designed to assess output rather than raw usage.
The move away from unrestricted A.I. use reflects broader shifts in how organizations manage these tools. Engineers increasingly deploy “A.I. agents” capable of working autonomously for extended periods, resulting in substantial token consumption and costs. Industry voices advocate reserving advanced A.I. models for complex tasks, while utilizing more cost-effective alternatives for routine functions. AT&T’s chief A.I. officer, Andy Markus, pointed out that deploying less powerful models for the majority of use cases can generate savings of up to 90 percent, noting “the latest greatest frontier model isn’t needed” in most scenarios.
The impact of this trend on A.I. providers remains uncertain. While Anthropic and OpenAI experienced record revenues during the height of tokenmaxxing, the recent shift toward cost containment may alter future revenue models. Meta, for example, has encouraged employees to use its internal coding assistant, MetaCode, instead of third-party options where possible.
Meta and Walmart’s new A.I. limitations have been reported previously, but both Meta and Anthropic declined to comment on the recent changes. OpenAI did not respond to inquiries. The ongoing adjustments underscore the dynamic and still-evolving relationship between corporations and artificial intelligence technology.
