Countermeasures Against Web Scrapers and Bots: Fighting Back with Creative Techniques
By
HermanMartinus
Hand-rolled, kettle-boiled, baked to perfection. Worth every minute at the bakery.
Summary
The article discusses techniques for fighting back against web scrapers and bots that inadvertently DDoS websites. The author describes various methods to waste scrapers' time and resources, including setting up Markov chain babblers to generate endless streams of nonsense data, creating bogus PHP files that appear legitimate but serve no real content, and implementing other creative countermeasures. The piece focuses on practical approaches for website owners to protect themselves from malicious or poorly-behaved bots by making scraping attempts costly and unproductive.
Key quotes
· 5 pulledscrapers are, inadvertently, DDoSing public websites
This post is about fighting back
set up a Markov chain babbler to feed the scrapers endless streams of generated data
bogus php files
making scraping attempts costly and unproductive
You might also wanna read
Anubis: A Proof-of-Work System to Protect Websites from AI Scraping Bots
This article describes a website protection system called Anubis that uses a Proof-of-Work scheme (similar to Hashcash) to defend against ag
Website Uses Anubis Proof-of-Work System to Block AI Scrapers
This article describes a website protection system called Anubis that uses a Proof-of-Work scheme (similar to Hashcash) to defend against ag
Website uses Anubis Proof-of-Work system to block AI scraping bots
This article describes a website protection system called Anubis that uses a Proof-of-Work scheme (similar to Hashcash) to defend against ag
Website uses Anubis Proof-of-Work system to block AI scraping bots
This article describes a website protection system called Anubis, which uses a Proof-of-Work scheme (similar to Hashcash) to defend against
Website uses Anubis Proof-of-Work system to deter AI scraping bots
This article describes a website protection system called Anubis, which uses a Proof-of-Work scheme (similar to Hashcash) to defend against
Anubis: A Proof-of-Work System to Protect Websites from AI Scraping
This article describes a website protection system called Anubis, which uses a Proof-of-Work scheme (similar to Hashcash) to defend against