EPendragon

Revisiting the Impact of Robots.txt on My Blog: Lessons Learned

The author reflects on their decision to ban crawlers from their website using robots.txt, leading to unintended consequences such as broken LinkedIn posts and decreased visibility of their content.

evgeniipendragon.com10mo ago