Website Blocks Outdated Browsers to Combat LLM Training Crawlers
By
zdw
Crisped on the outside, thoughtful enough on the inside.
Summary
This article explains that users are seeing an error message because they're using outdated browsers that the website's anti-crawler system flags as suspicious. The site owner is blocking old browser versions (particularly Chrome) to combat high-volume crawlers that are gathering data for LLM training, which has become a significant problem in early 2025.
Key quotes
· 4 pulledYou're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old
As of early 2025 there's a plague of high volume crawlers (apparently in part to gather data for LLM training)
They use a variety of old browser user agents, especially Chrome user agents
To reduce the load on Wandering Thoughts I'm experimenting with (attempting to) block all of them
You might also wanna read
Website Blocks Outdated Browsers to Combat LLM Training Crawlers in 2025
A website owner explains they are blocking access to users with outdated browsers as an anti-crawler measure. The site is experiencing a pla
Understanding WebAuthn credential protection policy and discoverable credentials
This article explains the WebAuthn credential protection policy, specifically how developers can use the `residentKey` option to control whe
Let's Encrypt's Challenge: Creating Intentionally Broken Certificates for Testing
Let's Encrypt, as a Certificate Authority, faces unique challenges in testing certificate validation systems. While most tools focus on main
Website Uses Anubis Proof-of-Work System to Protect Against AI Scraping
The article explains that the website uses Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressively
Firefox 148 Introduces Standardized Sanitizer API for Enhanced XSS Protection
Firefox 148 introduces the standardized Sanitizer API as a security enhancement to protect against cross-site scripting (XSS) attacks. The n
Website Blocks Old Browsers to Combat LLM Training Crawlers
A website owner explains that visitors are seeing an error message because their browsers are being blocked by anti-crawler measures. The si