How a Nonsense Maze Trap Revealed LLM Training Scrapers Consuming 99.9% of Server Traffic
By
chmaynard
Front-window bakery material. Catches the eye, delivers the goods.
Summary
A programmer describes setting up a dynamically generated nonsense maze to detect LLM training scrapers. The trap didn't increase overall server traffic but made bot requests easily identifiable, revealing that 99.9% of traffic comes from these scrapers. The author analyzes bot behavior and concludes that feeding them garbage content is the most cost-effective approach to waste their resources and protect legitimate content.
Key quotes
· 4 pulledNow, requests to it make up 99.9% of my server's traffic, but my total request volume did not increase.
The bots didn't come to my site because of the trap. The trap made them easier to spot.
These aren't indexing bots, they are scrapers collecting data to train LLMs.
This gave me a lot of information on the bots, and it turns out that sending them garbage is the cheapest and easiest thing I could have done.
You might also wanna read
Website Implements Anubis Proof-of-Work System to Block AI Scraping
The article explains that the website is using Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressiv
Website Implements Anubis Proof-of-Work System to Block AI Scraping
This article explains that the website is using Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressi
Website Implements Anubis Proof-of-Work System to Block AI Scraping
The article explains that the website is using Anubis, a Proof-of-Work system similar to Hashcash, to protect against AI companies aggressiv
Why Anubis Bot Protection Is Ineffective Against LLM Scrapers
The article argues against using Anubis, a proof-of-work based bot protection system, specifically for blocking LLM scrapers. It explains th
fxgn.dev·7mo ago
jqwik maintainer embeds protestware targeting AI coding agents in open-source library
The article reports on a controversial incident in the open-source software world where the maintainer of jqwik (a Java property-based testi
AI Tools Don't Make Engineers Lazy — They Risk Replacing Judgment Through Abdication
The article argues that the real concern about AI in engineering isn't dependency or laziness, but abdication of judgment. It distinguishes