Will falling cost per token increase aggregate accelerator spending or reduce it?

Thesis

Falling cost per token is more likely to support higher aggregate accelerator spending than reduce it over the near and medium term—but only if the resulting workload growth outruns efficiency gains and available capacity. That is the base case suggested by the supplied evidence as of September 10, 2026, not an established industry-wide causal relationship.

The mechanism is the Jevons effect: efficiency lowers unit costs, which can stimulate enough additional consumption to increase total resource use. A partial rebound merely offsets some efficiency savings; spending growth requires more than that. [1] Crucially, token spending, compute consumption and accelerator purchases are three different quantities. Lower token prices can expand the first two without immediately increasing the third.

The supplied research reports a 40% decline in usage-weighted token prices from end-June alongside an approximately 50% month-over-month increase in July AI spend per employee among the top 1% of enterprises. [2] This supports the possibility of strong demand response, but the sample is concentrated among leading adopters. The supplied excerpt does not identify the year, the endpoint of the token-price comparison, or precisely how the top 1% is defined. The two observations therefore should not be combined into an implied token-volume growth rate or an industry demand-elasticity estimate.

Why cheaper tokens can raise hardware demand

Lower inference costs expand the set of economically viable applications. Longer generations and more token-intensive workflows become affordable; potential examples include repeated agent loops, richer context and more frequent model calls. [3] The spending chain is conditional:

1. Model and infrastructure efficiency reduces the cost of useful AI work, and providers pass some of that saving to customers.

2. Developers and enterprises increase task volume, context length or task frequency.

3. Aggregate compute consumption grows if that workload expansion exceeds the reduction in compute needed per task.

4. Providers buy additional accelerator capacity if existing capacity and utilization gains cannot absorb the increase.

5. Dollar spending rises if additional purchases outweigh changes in hardware prices, performance and product mix.

The right framework is not token price multiplied again by model size and context length. Customer inference spending is approximately price per token multiplied by token volume, with appropriate adjustments for different token types and services. Compute consumption is workload volume multiplied by compute required per workload. Model size, context length and architecture affect that compute requirement; they are not independent, universally linear multipliers.

For a like-for-like workload, the exact compute-demand condition is:

New workload volume / old workload volume × new compute per workload / old compute per workload > 1.

For example, a hypothetical 40% reduction in compute required per task needs more than a 66.7% increase in task volume for total compute consumption to rise. That example is not an inference from the observed 40% token-price decline: selling prices can fall because of competition, subsidies, model mix or lower costs, and need not track physical compute efficiency.

Even rising compute consumption is not sufficient to establish rising accelerator spending. Better hardware throughput, higher utilization and spare capacity can accommodate more work with fewer incremental purchases. Procurement and replacement cycles also separate current usage from current capital spending. Conversely, training and other workloads can sustain aggregate accelerator purchases independently of inference token economics.

Tokens themselves are not homogeneous. The economically meaningful price is the quality-adjusted cost of completing useful work, not simply the price of one token. A lower-cost model that needs more attempts or produces less useful output may not deliver the apparent saving. [2]

What the supplied evidence says

The strongest direct upstream evidence in the supplied material is NVIDIA’s reported performance. On August 26, 2026, the company reported fiscal Q2 2027 revenue of US$96.2 billion for the quarter ended July 26, 2026, including US$89.0 billion of Data Center revenue. Total revenue increased 106% year over year and Data Center revenue increased 117%; GAAP and non-GAAP gross margins were both 75.0%. [4]

Its fiscal Q3 2027 outlook called for US$108.0 billion of revenue, plus or minus 2%, and GAAP and non-GAAP gross margins of 74.0%, plus or minus 50 basis points. The revenue midpoint is approximately 12.3% above Q2 revenue. The outlook assumes no Data Center compute revenue from China. These are management expectations, not realized results or an independent forecast. [4]

These figures are consistent with strong spending on NVIDIA’s products and an expected further increase. They do not isolate inference accelerator demand, measure spending across all suppliers, or demonstrate that lower token prices caused the growth. Data Center revenue is not a clean measure of inference-only accelerator purchases, and hardware orders can reflect earlier investment decisions rather than contemporaneous usage.

The rental-market signal is directionally supportive: GPU rental prices reportedly rebounded while token prices continued to decline. That divergence suggests that cheaper AI services can coexist with sustained demand for underlying compute. It is not conclusive evidence of rebound, because rental pricing also reflects supply and product mix. The source itself cautions that these markets are young, fragmented and opaque. [2]

The token-price, useful-work, rental-price and overbuilding observations come from the same research article, despite having separate citation IDs. They should be treated as related observations, not independent confirmations.

Revenue, margins and cash flow: who benefits and who bears the risk

The most direct potential beneficiary is an accelerator and systems supplier whose capacity remains scarce—not every company associated with AI. NVIDIA, listed on Nasdaq as NVDA, is the clearest named exposure in the supplied material. [5] Its Data Center business accounted for approximately 92.5% of fiscal Q2 2027 revenue, calculated as US$89.0 billion divided by US$96.2 billion. [4] That makes it highly exposed to infrastructure demand, but does not make all its revenue a pure inference exposure.

If demand remains ahead of supply, accelerator suppliers may retain pricing power and translate workload growth into revenue and cash generation. The supplied figures establish revenue and gross margins, however—not operating cash-flow conversion or the attractiveness of the stock at its market valuation. Rising industry spending alone is not a sufficient investment case.

Cloud providers and AI application companies face a more mixed outcome. Lower underlying inference costs can improve margins when selling prices hold, or stimulate volume when savings are passed through. Margin pressure arises when selling prices decline faster than costs, or when utilization and returns on newly installed capacity disappoint. A lower selling price per token does not by itself imply a lower gross-margin percentage.

Businesses with differentiated products, durable software advantages or scarce capacity are better positioned to retain part of the productivity gain. Commodity inference providers and undifferentiated applications are more vulnerable to price competition. Even substantial usage growth may fail to produce attractive returns if it requires excessive capital spending or does not generate adequate revenue per task.

For accelerator vendors, the main risks are that efficiency advances faster than workloads expand, or that customers shift toward custom silicon and alternative architectures. A shift to custom accelerators can reduce NVIDIA’s share without reducing aggregate accelerator spending; better price-performance can also reduce total dollars required for a given workload. These are distinct risks.

NVIDIA’s Q3 gross-margin guidance midpoint is one percentage point below its Q2 result. That supports caution about assuming unlimited margin expansion, but does not establish that token-price declines caused the moderation. [4]

Investment conclusion and tests of the thesis

My base case remains higher aggregate accelerator spending, with inference and agentic workflows offering a plausible channel through which cheaper useful work unlocks more demand. Confidence is stronger in the existence of that mechanism than in its eventual magnitude. The supplied evidence does not provide an industry-wide spending forecast or a measured elasticity sufficient to settle the question.

The most useful tests are:

• Workload growth versus efficiency: Does quality-adjusted task volume grow fast enough to raise aggregate compute consumption after software and hardware improvements?

• Capacity absorption: Are utilization and rental prices resilient as new capacity arrives, or can existing infrastructure absorb demand without substantial new purchases?

• Monetization and capital returns: Are cloud AI revenue and customer spending keeping pace with infrastructure commitments, rather than usage growing primarily through price cuts or subsidies?

• Supplier breadth and guidance: Do orders, revenue and forward guidance remain strong across the accelerator market? NVIDIA weakness alone could reflect a share shift rather than an aggregate downturn.

A sustained combination of weaker enterprise spending, falling utilization and rental prices, and deteriorating supplier demand would undermine the rebound case. No single indicator proves or disproves it. Overbuilding remains a genuine risk: strong utilization today says relatively little about the supply-demand balance several years forward. [2]

For investors, the distinction is therefore between direct supplier exposure such as NVIDIA, cloud and infrastructure businesses that can earn adequate returns on incremental capacity, and undifferentiated inference or application providers exposed to price competition.

Cheaper tokens are a potential demand catalyst, not an automatic hardware-spending multiplier. The supplied evidence favors the rebound case, but aggregate accelerator spending ultimately depends on workload growth, compute intensity, utilization, hardware price-performance and procurement—not token prices alone.