Website Blocks Outdated Browsers to Combat LLM Training Crawlers
By
birdculture
Baker's choice. Dense with flavour, light on filler.
Summary
A website owner explains that visitors are seeing an error message because their browsers are being blocked by anti-crawler measures. The site is implementing these blocks to combat a surge of high-volume crawlers in early 2025, many of which use old browser user agents (particularly Chrome versions) to scrape content, likely for LLM training data. The message is directed at legitimate users who encounter the block due to using outdated browsers that match the crawler patterns.
Key quotes
· 3 pulledYou're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions of Chrome)
As of early 2025 there's a plague of high volume crawlers (apparently in part to gather data for LLM training) that use a variety of old browser user agents, especially Chrome user agents
To reduce the load on Wandering Thoughts I'm experimenting with (attempting to) block all of them, and you've run into this
You might also wanna read
Site Blocks Outdated Browsers to Combat LLM Training Crawlers
The article explains that users are seeing an error message because they're using outdated browsers that the site owner's anti-crawler syste
Website Implements Anubis Proof-of-Work System to Protect Against AI Scraping
The article explains that the website uses Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressively
Website Implements Anubis Proof-of-Work System to Protect Against AI Scraping
The article explains that the website is using Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressiv
Website Implements Anubis Proof-of-Work System to Protect Against AI Scraping
The article explains that the website is using Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressiv
Site Blocks Outdated Browsers to Combat LLM Training Crawlers
The article explains that users are seeing an error message because they're using outdated browsers that the site owner's anti-crawler syste
Browser Blocking Notice: Anti-Crawler Measures for Old Browser Versions
This article explains that users are seeing an error message because their browser is being blocked by anti-crawler measures. The author is