Hello and welcome to Eye on AI. This is Jeremy, covering for Sharon who is away on vacation. In this episode...CNN files a lawsuit against Perplexity...IBMand RedHat initiates a $5 billion bug fixing project…Snowflake enters into a $6 billion agreement with AWS…and the White House allocates $9 billion to U.S. intelligence agencies for developing their own AI chip cluster.
Only a few weeks back, 'tokenmaxxing' was the trend in numerous companies. The concept was: to identify which employees were most innovative in using AI agents, you should monitor their token consumption. (Tokens are the data units that AI models handle; one token is roughly equivalent to a word-and-a-half of English text.) The higher the number of tokens used, the more productive the employee's AI agents were, or at least, the more they were attempting to be AI-focused and creative. That was the intention, at least.Meta, Amazon, OpenAI, and numerous other companies have set up formal or informalleaderboardsRegarding token consumption and motivating engineers and developers to challenge each other in terms of who can utilize the highest number of tokens within a specific timeframe.
Of course, Goodhart’s Law remains valid (it states that when a measure becomes a target, it stops being a good measure) and tokenmaxxing led to some expected negative outcomes. At Amazon, the Financial Timesreported, some employees created AI agents to perform completely pointless or unneeded tasks simply to maintain their token usage metrics, which were now being utilized by managers to evaluate employee performance.
Additionally, none of these tokens are truly free, and some companies have experienced surprise at their expenses from Anthropic and OpenAI. As a result, many businesses are now scaling back their focus on maximizing token usage and are even restricting which employees can access third-party AI agents, particularly those that rely on the most sophisticated AI models as the core of the agent's functionality. Meta removed the unofficial token-maximizing leaderboard that its employees had developed.Microsofthas terminated Claude Code subscriptions for staff in multiple important product departments, according toreportingfrom The Verge. Uber stated that it hadburned throughits full 2026 "token budget" within the first four months of the year, partly because of heavy use of Claude Code. Meanwhile,SalesforceCEO Marc Benioff has stated that his company's Anthropic bill will beabout $300 millionthis year, and he hoped there was a "smart router" that could identify which queries truly needed the most advanced and costly models, and which could be managed by smaller, less powerful but sufficient, more affordable options.
Several executives are also stating that spending on tokens is not resulting in a return on investment across the company. Uber Chief Operating Officer Andrew Macdonaldtold a podcastLast week, it was noted that the ride-hailing company has been having difficulty linking the productivity gains of certain employees to any broader effect on the organization. "If you're not actually able to establish a clear connection between how much valuable features and functions you're delivering to your users," he stated. "[The token costs are] harder to defend." The overall outcome is that the era of tokenmaxxing has come to an end.
Why is AI spending still not generating a return on investment?
But that still raises the larger question of why there is this gap between AI spending and return on investment? Clearly, explicitly rewarding tokenmaxing doesn't help, as it fails to align employee incentives with company objectives (see that Amazon example). Azeem Azhar, the author of theExponential Viewnewsletter, which is as insightful about the economic and business effects of AI as any other source,arguesthat the present AI productivity puzzle might merely be the anticipated "productivity J-curve" that typically occurs with any new, broad-use technology.
In contrast to technology created to enhance a specific process, which typically leads to immediate improvements in productivity, it often requires significant time for individuals to determine the most effective way to apply a general-purpose technology. During this "learning" phase, productivity may actually decline instead of rising. This occurs because businesses must invest time and resources into testing how to utilize the new technology, frequently without observing any positive financial results. Only after people discover the best methods to restructure business processes around the new technology does productivity see a rapid increase.
A well-known example that Azhar discusses in detail is the development of electricity and its influence on manufacturing. The initial step factories took with electricity was to substitute gas lighting with electric lighting. This resulted in some cost reductions, but it didn't significantly affect the overall output of the company. (Additionally, there were installation costs involved in setting up the lights and wiring the factory, which somewhat reduced those savings.) The nature of steam power meant that pre-electric factories were designed around a central engine that drove multiple, or even all, of the factory's machinery through a single drive shaft. Therefore, the second action factories took was to replace the large central steam engine with big electric motors, which they still used to power groups of machines via central drive shafts. This approach was more economical than completely redesigning the factory. However, it proved to be inefficient and not very operationally cost-effective. Improvements in productivity in one area of the production floor often led to bottlenecks in other areas of the assembly line, resulting in minimal overall gains for the factory. It wasn't until companies started powering individual machines with electricity and reorganizing the entire factory layout that significant productivity increases were observed.
Only a small number of companies are reaching Stage 3
Azhar believes that the same pattern will occur with AI, but most companies are currently stuck in stage one or stage two of this development. I think he's likely correct. Tokenmaxxing is simple. Redesigning processes is challenging. Even more difficult—and something Azhar doesn't address—is reevaluating entire business areas, meaning what products or services the company offers, and even its overall business model. This touches on the core purpose of the company. This is where the significant value from AI lies. It's about transformation, not just modification. However, most companies are still not thinking on a large enough scale.
Since many current businesses are not thinking big enough about their use of AI, AI-focused companies have a major opportunity at this time. They can operate more swiftly and capture substantial market share from established companies before these older firms can properly react. It's far simpler to create a new business from scratch than to attempt to completely revamp an existing one. (This also explains why it might be harder than some private equity firms expect to just introduce a bit of AI into their investment portfolio and aim to sell the businesses for higher prices.)
Okay, with that, here's additional AI news.
Jeremy Kahn
jeremy.kahn@@jeremyakahn
No comments:
Post a Comment