MolmoWeb 8B Beats GPT-4o on Web Navigation — and Scales to 95%

An 8-billion parameter open-weight model just outscored GPT-4o on a live web navigation benchmark. Then test-time scaling pushed it to 95%. The browser agent frontier is no longer proprietary territory.

The WebVoyager Result

MolmoWeb 8B, developed by Allen AI, hits 78.2% on [WebVoyager](https://benchmark.space/benchmark/webvoyager). GPT-4o sits at 65% on the same test. That gap — 13 percentage points — is not a rounding error. It is a smaller, fully open model beating a proprietary API on a task that requires understanding web pages, clicking the right elements, and completing multi-step navigation goals.

"MolmoWeb 8B sets a new record because it achieves a 78% success rate on the WebVoyager task. Surprisingly, this outperforms larger proprietary systems like GPT-4o, which only reaches 65% on the same test." — [Alex, 3:06](https://www.youtube.com/watch?v=diXpkVgJjac&t=186)

The 4B variant is close behind at 75.2%, suggesting the architecture scales efficiently downward.

Test-Time Scaling Changes the Math

Single-pass accuracy tells part of the story. The other part is what happens when you let the model try more than once.

"By running four parallel attempts and picking the best outcome, the 8 billion parameter model reaches nearly 95% accuracy on the WebVoyager benchmark." — [Alex, 3:06](https://www.youtube.com/watch?v=diXpkVgJjac&t=186)

Four attempts. Best-of-four selection. 78% becomes 95%. That is a 17-point jump from inference-time compute alone, no retraining, no architectural changes. For context, the current [WebVoyager](https://benchmark.space/benchmark/webvoyager) leaderboard leader (Alumnium MCP + Claude Code) sits at 98.5%. MolmoWeb 8B with test-time scaling is within striking distance — at a fraction of the cost per query.

This mirrors what the reasoning model literature has shown for math and coding benchmarks: small models with more inference compute can close the gap with larger models. The difference is that web navigation is a multi-step agent task where errors compound. The fact that best-of-four works this well suggests that MolmoWeb's failure modes are not systematic — they are stochastic, and retrying fixes them.

Vision-Only, and Why That Matters

MolmoWeb takes screenshots and grounds its actions on pixels. No HTML parsing. No accessibility tree. No DOM access. This is a deliberate design choice with two consequences.

First, it generalizes. A model that reads pixels can navigate any visual interface — a web browser, a desktop app, a mobile screen. HTML-bound agents break the moment the page structure changes or the site uses heavy client-side rendering. Vision does not care about your DOM.

Second, it is simpler to deploy. No integration with browser automation APIs. No access tree extraction pipeline. Screenshot in, click coordinates out. The inference path is short and deterministic.

The tradeoff is precision. HTML-grounded agents can click exact elements by ID. Vision-grounded agents sometimes click the wrong button when two are visually similar. But the WebVoyager numbers suggest this tradeoff is manageable, especially with test-time scaling to catch occasional misfires.

Where It Stands on the Leaderboard

MolmoWeb 8B ties GUI-Owl-1.5-8B-Thinking from Alibaba, another open 8B model, and beats GPT-5.4 by over 7 points. The proprietary frontier — Surfer 2, Alumnium — is still ahead. But the gap is closing fast, and those systems are expensive closed APIs. MolmoWeb is weights-available.

The Open Agent Trajectory

The browser agent space has moved quickly. A year ago, the idea that an open 8B model could compete with GPT-4o on live web tasks would have seemed unlikely. Today it is a data point. The trajectory matters more than the snapshot.

Two forces are converging. High-quality web agent training data is becoming available — MolmoWeb's gains come partly from better data curation, not just more parameters. And test-time scaling is proving effective for agent tasks, not just reasoning benchmarks. Put them together and the economics shift. Running four passes of an 8B model is cheaper than one pass of a frontier API, and it gets better results.

The remaining gap to systems like Surfer 2 and Alumnium is real. Those systems use proprietary base models and custom harnesses. But the delta — 20 points from MolmoWeb 8B single-pass to the top — is exactly the kind of gap that test-time scaling, better data, and slightly larger checkpoints tend to close. Open browser agents are not at parity yet. They are on pace.

Model	WebVoyager	Params	Open?
Alumnium MCP + Claude Code	98.5%	—	No
Surfer 2	97.1%	—	No
Gemini 3.1 Pro	88.6%	—	No
GLM-5V-Turbo	88.5%	—	No
GUI-Owl-1.5-32B-Thinking	82.1%	32B	Yes
GUI-Owl-1.5-8B-Thinking	78.1%	8B	Yes
**MolmoWeb 8B**	**78.2%**	**8B**	**Yes**
MolmoWeb 4B	75.2%	4B	Yes
GPT-5.4	70.9%	—	No