Website Blocks Outdated Browsers to Combat LLM Training Crawlers in 2025
By
zdw
9mo ago· 2 min readenNews
75/100
Toasty
Bagelometer↗
Properly proved. Has structure, has flavour, has a point.
Score75TypenewsSentimentneutral
Summary
A website owner explains they are blocking access to users with outdated browsers as an anti-crawler measure. The site is experiencing a plague of high-volume crawlers in early 2025 that use old browser user agents (particularly Chrome versions) to scrape content, apparently for LLM training data collection. This blocking is an experimental measure to reduce server load on the Wandering Thoughts blog and CSpace wiki.
Key quotes
· 4 pulledYou're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old
As of early 2025 there's a plague of high volume crawlers (apparently in part to gather data for LLM training)
They use a variety of old browser user agents, especially Chrome user agents
To reduce the load on Wandering Thoughts I'm experimenting with (attempting to) block all of them
You're probably reading this page because you've attempted to access
some part of my blog (Wandering Thoughts) or
CSpace, the wiki thing it's part of. Unfortunately
you're using a browser version that my anti-crawler precautions consider
suspicious, most
