What gives a bot away: notes from testing a bot detector
Research note ·
WattSaid publishes numbers that people and AI assistants both read, so we wanted to know what a website can tell about who is reading it. A detector built by limingzi scores each visit from 0, which looks human, to 100; we ran automated clients against it on a test site. These are the results and what we took from them. Nothing here comes from visitors to starfindai.com.
Network checks catch the clients that don't try
| Client | Score |
|---|---|
| Python requests, default settings | 80.1 |
| Python requests posing as Chrome | 95.5 |
| Python urllib, no cookie store | 92.2 |
| curl_cffi impersonating Chrome | 70.1 |
The first client says it is a Python library and asks for pages in order. The second claims to be Chrome, but its TLS handshake, the first message it sends, is the one Python's OpenSSL sends; it also lacks the headers a browser sends, runs no JavaScript and spaces its requests evenly. The third shows the same mismatch and never keeps a cookie.
The fourth is the lesson. Tools such as curl_cffi copy Chrome's handshake exactly, so a TLS fingerprint only catches clients that did not bother to copy one. That client was caught only because it runs no JavaScript and asks for pages in order at even intervals. What is left to check is what such a client does not do: run the page's scripts and keep its cookies.
Hiding the automation flag is itself a trace
| Client | Score |
|---|---|
| Playwright, headless | 99.9 |
| Playwright, headless, flag hidden | 99.9 |
| Playwright, headless, acting like an AI agent | 100 |
| Playwright, visible window, flag hidden, acting like an AI agent | 93.9 |
Headless Chrome names itself in its user agent and sets an automation flag, and its clicks arrive with no pointer travel. Removing the flag leaves signs that it was removed. The third run looked at the page, waited, then acted, over and over, and was read as an AI agent. The fourth is the interesting one: a normal visible browser that passed every check of the browser itself still scored 93.9, on how it moved and clicked alone.
AI agents don't move like people
An agent that works from screenshots follows a loop: take a picture of the page, think for a few seconds, then act. On the page that shows up as long still pauses, then a click that lands at once in the middle of a button, with no pointer travel and a press that lasts no time at all. Text arrives in a field all at once, with no key presses. People wander, overshoot and type one key at a time.
Screenshots can leave a mark too. When Playwright takes a screenshot it hides the blinking text cursor by default, and a page can notice that change. Telling Playwright to leave the cursor alone, or capturing the screen another way, avoids it. With that signal added, the visible-window agent above went from 93.9 to 99.5.
A careful bot passed a single visit
| Client | Score |
|---|---|
| Playwright, visible window, human-like input | 0 |
| The same bot, six runs with fresh cookies | 0, 0, 0, 0, 35, 37.5 |
This bot moved the pointer along curved paths with small jitter, held each press for a moment and typed with a rhythm, and it was rated human. Run six times with fresh cookies, it scored zero four times; from the fifth run the same device kept starting new sessions, which raised it to medium risk.
That is the limit. A bot that drives a real browser and imitates human input passed a single visit, and repetition only got it to medium. The thresholds in these tests came from reasoning and test runs, not from labelled real traffic, so the scores show which kinds of evidence work, not how a detector would do on a real site.
What we do on our own site
We want AI assistants to read our numbers: starfindai.com/llms.txt and the live numbers page exist for them. So we do not block AI readers on starfindai.com. For us, telling bots from people is about knowing who is reading, not shutting the door.
Detector and browser tests by limingzi. Scores are the detector's 0 to 100 risk estimate on a test site running on one machine, not traffic on starfindai.com. The four HTTP clients were run again on 3 November 2026 with the same results. The browser runs were made in late October 2026 with Chromium 148 and have not been repeated. The detector's thresholds have not been checked against real visitors.