On-device model inference in 2026: the platform is ready, the use is not there

Three technologies make in-browser computation possible, and their support differs:

caniuse data as of 24 August 2026. WebAssembly is available practically everywhere and has been for years; WebGPU in five browsers of six, but still a draft.

On WebNN separately, because this is where mistakes are easy. We looked for it both in the support reference and in Chrome’s list of usage counters. There is no entry in the reference. There is no mention in Chrome’s feature enum either. Which means WebNN prevalence cannot be measured today by any instrument, and any figure quoted for it did not come from observation.

Chrome publishes the share of page loads on which a given capability is invoked. As of 25 August 2026, work is submitted to a graphics processor on 0.003967% of loads — roughly four in a hundred thousand.

Side by side:

A gap of roughly twenty-one thousand times. The two figures have different denominators, so read it as an order of magnitude — but the order of magnitude is the point.

The growth is real: 0.000003% in August 2023, 0.000550% in 2025, 0.003967% now — roughly sevenfold in a year. The base is simply near zero, and WebGPU has been in Chrome since May 2023. Three years and four months later, fewer than four loads in a hundred thousand do GPU work.

More on how that gap is structured in our separate WebGPU breakdown.

The reasons are visible in the figures themselves, and they are not about fashion.

The model has to be delivered. Any model capable of something useful weighs tens or hundreds of megabytes. That is comparable to a whole application and orders of magnitude more than the median page, which in our Russian measurement weighs 2.33 MB. Downloading and caching once is possible; but the first visit becomes unacceptably expensive.

The hardware is not guaranteed. Support "by browser version" and the presence of a usable adapter are different things. Chrome on Linux counts as supporting, while a given machine may have no adapter. So a fallback path is always needed — and once it is needed, staying on it is often simpler.

The specification is still a draft. WebGPU holds working-draft status; WebNN is not tracked even by reference works. Few teams commit a product to something that may change.

And above all: calling a server works. In our own operation the median model response is 3.8 seconds. That is slow by web request standards but acceptable for an action a user deliberately triggers. On-device computation has to deliver something a server cannot — privacy or offline operation — or there is no gain.

From the above follows a narrow but real list of cases.

1. The data must not leave the device. This is the one argument that server speed does not override. Processing a document, a photograph or a voice recording without sending it anywhere is a task where local computation wins by definition.
2. Offline operation is mandatory. An application that must keep working on a plane or in a basement cannot rely on a server.
3. The task is small and constantly repeated. Gesture recognition, input suggestion, classification into a few categories — here the model weighs megabytes rather than hundreds, and a server call on every keystroke is impossible.

In every other case calling a server is simpler, more predictable and works for everyone. The figures above are not a verdict on the technology but a description of what the market is choosing so far.

To check what your page serves and what it weighs, use the speed test.