MSN bot was smashing Trac server, here is how I blocked it
Just wanted to put a quick post out there. I host about 50 trac environments on an EC2 server, several rather large ones of which are public and pretty high traffic. Yesterday the trac server started having issues, and most of the logged errors had a seemingly random type of python trace data, followed by:
IOError: failed to write data
I dug around online for a bit but did not see much in the way of help for this issue, then I noticed that nearly all the errors were showing from a client that started with 65.55.X.X. Flipping through other logs, it appeared to be an MSN bot agent hitting Trac pages, and in some cases even hitting an infinite recursion (somehow linking to /report2/report2/report2/report2/……). The quick and dirty fix to solve this issue was to block the entire subnet from accessing my vhosts, which was done with this little gem inside of each vhost <directory> tag in Apache:
Order Allow,Deny
Allow from all
Deny from 65.55.0.0/16
Granted this is not a perfect solution, but keeping the site up for real users was more important at the moment than allowing a rampant recursive spider to keep slamming the site as soon as I brought it back up (for nearly 2 days straight now).
Happy Friday!
Source:MSN bot was smashing Trac server, here is how I blocked it