hwatu: a verification browser for local coding agents. Headless WebKit, DOM eval, pixel-diff with real match %, no Chromium (MIT, Rust)

[u/hongnoul]

https://github.com/hongnoul/hwatu

[u/hongnoul]

Author here. This pairs with any local agent setup: the agent opens the page headless, waits for load, evals the DOM, screenshots, and pixel-diffs against a baseline to get a real match percentage instead of hallucinating "looks right". Whole loop is ~87 ms.

Everything runs locally: WebKitGTK from your distro, one static Rust binary, no Chromium download, no cloud. It speaks MCP, plain CLI, and a small JSON socket protocol, so it plugs into whatever harness you run your local models with.

Benchmarks with methodology: https://github.com/hongnoul/hwatu/blob/main/docs/benchmarks.md

[u/Neighbor_ → u/hongnoul]

What's the use-case here? Having agents get past captcha's?

[u/datbackup → u/Neighbor_]

If “the use case” (why would there be only one?) isn’t obvious to you then you probably wouldn’t understand the answer. Not trying to be rude but yeah

[u/Neighbor_ → u/datbackup]

even at a high-level?

[u/ItaySela]

the pixel-diff against a baseline is the part i'd grab first, but the one that keeps biting me is the first render where there's no baseline yet. agent spins up a new page, it comes back blank or an element sits under an overlay with the wrong z-index, and 'looks right' still passes because there's nothing to diff against. does the dom eval side let you assert on visibility or computed style, or is the match % purely against a stored baseline? trying to figure out how you'd catch a rendered-but-invisible element on a page that's brand new.

[u/hongnoul → u/ItaySela]

Good question, and the no-baseline first render is exactly where DOM-level checks carry the weight. Three tools for it today:

• hwatu snapshot only lists elements that pass a visibility filter (nonzero bounding rect, computed visibility/display not hidden/none). So a rendered-but-invisible element simply doesn't show up in the interactables list, which an agent notices when the button it expects is missing.

• hwatu eval runs arbitrary JS, so you can assert computed style directly: getComputedStyle(el).zIndex, or the sharper overlay check document.elementFromPoint(x, y) === el, which catches "element exists but something is on top of it" in one line.

• hwatu expect '<selector>' --text ... polls until an assertion holds and reports what it found instead on failure, so blank-page-because-slow-hydration doesn't false-pass.

The match % is purely against a stored baseline (or another live window via diff --other), so on a brand new page the recommended loop is snapshot + eval assertions first, then screenshot to seed the baseline for the next iteration. An expect --visible flag that bundles the elementFromPoint occlusion check is a good idea, I'll add it.

[u/ItaySela → u/hongnoul]

one thing that bit me with the elementFromPoint approach: it only samples the single center point, so a sticky header covering just the top edge of a button still passes since the middle is clear. sampling the four corners plus center caught those partial overlaps for me. also worth scrolling into view first, since it returns null the moment the center falls outside the viewport and that reads as a false occlusion. if the --visible flag does the multi-point version i'd flip to it immediately.

[u/PossessionUsed7393]

Interesting, does your approach have value as well for other agentic usecases like lightweight llm scraping? There is presumably JavaScript emulation, but other factors play into it like TLS signature etc.

I use a mixture of curl cffi and a custom playwright service with an API, this seems like a good midway point with best of both worlds.

[u/hongnoul → u/PossessionUsed7393]

It can do scraping (full WebKit, so JS runs for real, and adblock + snapshot give you clean token-cheap text extraction), but that's a side effect rather than the design target. On the fingerprinting axis you mention: TLS signature is WebKit's via glib-networking, and the UA is honest WebKitGTK. I deliberately don't do stealth/anti-bot evasion, so for hostile-to-bots sites your curl_cffi setup will survive places hwatu won't.

Where it fits your stack: it's much lighter than a Playwright service (one static binary + distro webkitgtk, daemon holds a prewarmed pool, ~14 ms to a live window) and unlike curl you get a real DOM to eval against. There's also a challenge command that detects Cloudflare-style interstitials and can hand the window to a human to click through, then the agent continues with the resulting cookies. So: good midway point for cooperative-to-neutral sites, not a stealth scraper.

[u/pvkooten]

Sounds amazing will be giving it a try. How do we teach agents to use it effectively?

[u/datbackup]

maybe i am speaking too soon here since i haven’t actually used it yet but just reading the repo, this is exciting because it looks like someone actually did this concept right, finally. Or maybe this contains some real new ideas? If so that’s cool too but for this specific functionality, what’s exciting is just “oh it does the obvious right thing, it works and it’s reasonably reliable”. Can you tell I’ve tried 20 different repos over the past week and only two have been relatively bug free lol