Inference Is Becoming a Commodity

The strongest case for inference becoming a commodity begins with a particular workload, not a claim that every model has become interchangeable. If several suppliers can deliver the required output at an acceptable level of quality, speed and reliability, a buyer can increasingly choose between them as providers of capacity. The identity of the supplier matters less because the service can be obtained elsewhere on sufficiently comparable terms.

Falling prices make that development commercially significant, but do not establish it by themselves. Stanford's 2025 AI Index reported a reduction of more than 280-fold between November 2022 and October 2024 in the inference price associated with approximately GPT-3.5-level performance on MMLU. That benchmark-specific comparison demonstrates a substantial price-performance improvement, not a universal measure of intelligence or a finding about provider profitability.^[1]

The more direct evidence of substitutability lies in how requests can be allocated. OpenRouter's provider-routing documentation allows selection by price, throughput and latency, with additional controls for parameter support, data policies and other requirements. These features make it possible to redirect suitable requests while also revealing why an API connection alone does not establish that every provider is an adequate substitute.^[2]

A company buying inference is not simply buying a quantity of tokens. It needs a result that works within an application. The relevant requirements may include dependable tool calls, appropriate treatment of data, sufficient context and performance during busy periods. A cheaper endpoint that does not meet those requirements belongs outside the comparison, however attractive its advertised rate appears.

Even endpoints serving the same named model can behave differently. Artificial Analysis evaluates endpoint accuracy across tool use, reasoning and long-context tasks, identifying serving choices such as quantisation and context or output limits as potential reasons for differences. Its methodology supports testing the delivered service rather than assuming equivalence from the model label. It does not establish the outcome for every customer's workload.^[3]

Once a buyer has established several acceptable alternatives, however, routing can change the commercial relationship. A hypothetical application able to move routine requests between qualified providers can respond to a price or performance change without replacing the whole product. That does not eliminate evaluation, monitoring or contractual work, but it can make the threat of switching more credible than a migration requiring substantial redesign.

This is the relevant commodity-like development: a service becomes more contestable because another supplier can satisfy the same requirement. It need not be identical in every respect, and it need not be cheap in an absolute sense. What matters is the range of alternatives a buyer can actually use and the cost of moving between them.

Measuring a difference does not make it easy to reproduce. A provider's low latency, dependable capacity or unusually efficient implementation can remain valuable precisely because competitors cannot match it at the same cost. Customers may be able to compare those advantages perfectly and still have good reasons to remain. Transparency strengthens competition where alternatives exist; it does not create equivalent alternatives by itself.

Nor does technical sophistication cease to support commercial advantage because it is invisible behind an API. Engineering can produce a durable cost advantage or a level of service others struggle to supply. The correct question is whether that advantage persists and matters to customers, not whether they see the machinery that produces it. A standardised interface and a differentiated operation can coexist.

The price and cost curves must also be considered separately. Suppose, illustratively, a request sells for ten units and costs eight to deliver. If its selling price later falls to eight while its delivery cost falls to five, the contribution per request rises from two to three. Price competition has occurred, yet the operator's economics have improved. Falling prices alone cannot establish shrinking margins.

Higher volumes create another distinction. An operator might earn less on each request but more in total, or increase revenue while earning an inadequate return on the capital required. Utilisation can help spread fixed costs, but pushing infrastructure harder can also affect the service customers receive. There is no unlimited scale advantage independent of workload mix, capacity constraints and performance obligations.

Competition can consequently reward both large efficient operators and specialists. A provider might serve a narrow class of workloads particularly well, offer deployment arrangements others cannot match or maintain spare capacity for customers who value certainty. Whether those differences justify higher prices is a purchasing question. They should not be dismissed merely because they can be described in measurable terms.

Model ownership adds a separate layer. A customer requiring a particular proprietary model cannot necessarily purchase its inference from any operator it chooses. Executable open weights can permit more suppliers to host the same underlying model, subject to licence and implementation constraints. Substitution between hosts and substitution between models are distinct developments, and either can progress without the other becoming universal.

The proprietary model may nevertheless face competition from a different model that meets the customer's requirements. That competition is workload-specific. A system essential to one application may provide unnecessary capability in another. The existence of a technological frontier therefore does not mean every request must be bought at frontier terms, just as inexpensive routine inference does not establish that the frontier has lost its value.

Market structure can counteract substitutability too. If buyers face scarce capacity, concentrated supply or restrictive contracts, the ability to compare services may not translate into strong price pressure. An intermediary that makes switching easier could itself become a point of dependence. These are reasons to examine the organisation of the market rather than infer its eventual margins from an API standard.

For independent providers, the strategic consequence is not inevitable commercial decline. It is that access to a widely available model may be insufficient differentiation on its own. Returns will depend on the cost of delivering a qualified service, the value of capabilities competitors cannot readily match and the conditions under which customers can move their demand.

For buyers, the corresponding opportunity is to separate genuine dependency from habit. Some workloads require distinctive models or arrangements; others may support several validated suppliers. Treating the latter as contestable capacity can improve purchasing power without pretending the former have become interchangeable.

Inference is becoming commodity-like where those practical conditions hold. That could support a large and technically demanding infrastructure market rather than a small or unprofitable one. The important change is that a provider must increasingly justify its position through the service and economics it delivers, not through the assumption that access to machine intelligence is scarce everywhere.