A reproducible benchmark that puts numbers on browser-automation stealth
Scores browser-automation stealth setups (configs) against open, self-hosted bot detectors on your own localhost, then commits a versioned results file and a regenerable report. It measures detectability only: no crawler, no login, no target site, and every detector runs offline.
Update 3 of the stealthbench build log. stealthbench scores browser-automation stealth setups against self-hosted bot detectors on your own localhost and commits every result. This update widens the matrix: two more detectors (a Sannysoft-style panel and rebrowser-bot-detector) and a fifth config, nodriver. nodriver is async and the bench's BrowserHandle contract is sync, so the new handle drives each call to completion on a private event loop. Wiring it in surfaced a real bug: nodriver's tab.evaluate(return_by_value=True) drops falsy results (its unwrap tests `if remote_object.value:`, so 0/false/'' vanish) and leaves plain objects wrapped in a RemoteObject, so detector signals came back as None or garbage. The fix is to stop fighting its unwrap: wrap the script in JSON.stringify and json.loads the returned string, which round-trips every value faithfully. With that fixed, the two new panels mostly corroborate the tells ranking (undetected-chromedriver and nodriver ace both, vanilla and selenium-stealth get dinged), and camoufox runs only 9 of rebrowser's 10 tests because one is Chrome-specific.
Update 2 of the stealthbench build log. stealthbench scores browser-automation stealth setups against self-hosted bot detectors on your own localhost and commits every result. Update 1 reported one number per config; this update ran each config ten times and drew the spread on the chart. Two of the four scores don't repeat: stock Selenium and undetected-chromedriver wobble run to run (undetected-chromedriver even hits a clean 100% on some trials), while selenium-stealth and camoufox land on the same number every time. The bars are descriptive spread (min/max/mean/population stdev), deliberately not a confidence interval. Also in this update: the results schema bumped to v2 while still reading every v1 snapshot, a history layer aggregates committed snapshots into a trend, and run metadata records browser and driver versions from the environment (numbers only, no IP or fingerprint).
Update 1 of the stealthbench build log. stealthbench is a reproducible benchmark that puts numbers on browser-automation stealth: it scores stealth configs (how you set up an automated browser) against self-hosted detectors (how a site decides you're a bot), on your own localhost, and commits every result. This update adds the fleet's first non-Chrome arm, a stealth Firefox (Camoufox driven through Playwright), and does it the way the architecture was built to allow: a new Config plus a new BrowserHandle adapter, with zero changes under detectors/. The BrowserHandle Protocol is the Ports and Adapters seam; a test (test_seam.py) fails the build if a browser SDK is imported outside configs/, and the sprint's proof is an empty `git status src/stealthbench/detectors/`. The one real roadblock was Firefox's Xray wrapper hiding the detector pages' window globals, fixed by reaching through window.wrappedJSObject in the evaluate translation. Camoufox lands at 82% on the tells panel and passes BotD and CreepJS; the three tells it 'fails' are Chrome-specific, so a genuine Firefox doesn't match them.