Developer Shuts Down Self-Hosted Git Server After AI Scrapers Overwhelm It
By
dzulp0d
Reliable enough to start your morning with. Toast it again tomorrow.
Summary
A developer announces the end of their self-hosted public git server that had been running since 2011, citing AI scrapers as the cause of its demise. The scrapers flooded the cgit frontend with excessive requests, overwhelming the server. After months of issues, the author decided not to rebuild and instead migrated repositories to GitLab and GitHub. They also mention maintaining only one self-hosted service (a static blog) and fixing configuration issues caused by AI scrapers filling up disk space with 404 logs.
Key quotes
· 4 pulledAI scrapers have hammered the poor, little server to death by flooding the cgit frontend with tons of pointless requests.
I don't feel like taking up the fight with the scrapers in my spare time, I leave that to people who are in a better position to do so.
Now I'm down to one self-hosted service, which is the webserver hosting mainly this blog and a few more little things.
Apparently millions of 404 answers where not enough to convince the bots that there is no cgit service (any more).
You might also wanna read
Developer Abandons OpenJDK Patch Contributions After Frustrating Oracle CLA Process
A developer shares their frustrating experience trying to contribute patches to the OpenJDK codebase, detailing the bureaucratic hurdles wit
Netflix engineer's open-source tool cuts AI token usage by up to 90%
Netflix senior engineer Tejas Chopra created software called "Project Headroom" that prunes redundant tokens from AI agent instructions befo
theregister.com·4h ago
Copyparty: A Lightweight File Server That Runs as a Single Python Script
Copyparty is a lightweight, full-featured file server that runs as a single Python script, making it extremely easy to set up without needin
hackaday.com·6h ago
Researcher's "ADHD" tool for Claude Code claims 2x improvement; experts call for more evidence
Solo researcher Udit Akhouri released a third-party Agent SDK tool called "ADHD" for Claude Code on Reddit, claiming it makes the coding age
bit.ly·1d ago
ReactOS open-source Windows NT clone reaches ARM64 boot milestone on Raspberry Pi 5
ReactOS, the open-source project aiming to recreate Windows NT, has achieved a new milestone by booting on ARM64 architecture. The experimen
theregister.com·1d ago
Zig Devlog: Build System Rework Separates Maker and Configurer Processes
This devlog entry from the Zig programming language project announces a major rework of the build system, separating the maker process from
ziglang.org·1d ago