MSN Search bot

Fri Feb 4 11:56:15 2005 EST (-0500 GMT)

My web site used to get along pretty well with Google. They searched my web site a lot and I gaind some pretty good rankings. I couls just mention some one I know in my blog…. say…. Chris Court A.K.A. Monkey, and pretty soon I was the number one Google result for that person.

Well now that MSN has revamped their search engine it’s been getting a little greedy. As per this thread over on , the bot which Microsoft uses to index the world’s web pages appears to needlesly index all kinds of content.

Here’s what it’s been doing to this site in the last four days, versus all the other search engine bots:

Robots/Spiders visitors (Top 25)  
9 different robots* Hits Bandwidth Last visit
MSNBot 305+44 3.11 MB 04 Feb 2005 – 09:49
Googlebot 138+14 1.96 MB 04 Feb 2005 – 09:31
AskJeeves 35+7 214.61 KB 04 Feb 2005 – 00:41
Inktomi Slurp 19+22 117.74 KB 04 Feb 2005 – 03:41
Unknown robot (identified by ‘crawl’) 19+9 140.79 KB 04 Feb 2005 – 03:30
Unknown robot (identified by ‘spider’) 6+1 76.38 KB 04 Feb 2005 – 08:01
Unknown robot (identified by hit on ‘robots.txt’) 0+7 610 Bytes 03 Feb 2005 – 06:42
Netcraft 4 0 03 Feb 2005 – 16:36
Alexa (IA Archiver) 1+2 5.98 KB 04 Feb 2005 – 01:48

* Robots shown here gave hits or traffic “not viewed” by visitors, so they are not included in other charts. Numbers after + are successful hits on “robots.txt” files

Comments are closed.