{"id":106,"date":"2005-02-04T11:56:15","date_gmt":"2005-02-04T16:56:15","guid":{"rendered":"http:\/\/mattclare.ca\/wordpress\/?p=106"},"modified":"2005-02-08T09:17:00","modified_gmt":"2005-02-08T14:17:00","slug":"msn-search-bot","status":"publish","type":"post","link":"https:\/\/mattclare.ca\/blog\/2005\/02\/04\/msn-search-bot\/","title":{"rendered":"MSN Search bot"},"content":{"rendered":"

My web site used to get along pretty well with Google. They searched my web site a lot and I gaind some pretty good rankings. I couls just mention some one I know in my blog…. say…. Chris Court A.K.A. Monkey, and pretty soon I was the number one Google result for that person.<\/p>\n

Well now that MSN has revamped their search engine it’s been getting a little greedy. As per this thread over on , the bot which Microsoft uses to index the world’s web pages appears to needlesly index all kinds of content.<\/p>\n

Here’s what it’s been doing to this site in the last four days, versus all the other search engine bots:<\/p>\n\n\n\n
Robots\/Spiders visitors (Top 25) <\/td>\n <\/td>\n<\/tr>\n
\n\n\n\n\n\n\n\n\n\n\n\n
9 different robots*<\/th>\nHits<\/th>\nBandwidth<\/th>\nLast visit<\/th>\n<\/tr>\n
MSNBot<\/td>\n305+44<\/td>\n3.11 MB<\/td>\n04 Feb 2005 – 09:49<\/td>\n<\/tr>\n
Googlebot<\/td>\n138+14<\/td>\n1.96 MB<\/td>\n04 Feb 2005 – 09:31<\/td>\n<\/tr>\n
AskJeeves<\/td>\n35+7<\/td>\n214.61 KB<\/td>\n04 Feb 2005 – 00:41<\/td>\n<\/tr>\n
Inktomi Slurp<\/td>\n19+22<\/td>\n117.74 KB<\/td>\n04 Feb 2005 – 03:41<\/td>\n<\/tr>\n
Unknown robot (identified by ‘crawl’)<\/td>\n19+9<\/td>\n140.79 KB<\/td>\n04 Feb 2005 – 03:30<\/td>\n<\/tr>\n
Unknown robot (identified by ‘spider’)<\/td>\n6+1<\/td>\n76.38 KB<\/td>\n04 Feb 2005 – 08:01<\/td>\n<\/tr>\n
Unknown robot (identified by hit on ‘robots.txt’)<\/td>\n0+7<\/td>\n610 Bytes<\/td>\n03 Feb 2005 – 06:42<\/td>\n<\/tr>\n
Netcraft<\/td>\n4<\/td>\n0<\/td>\n03 Feb 2005 – 16:36<\/td>\n<\/tr>\n
Alexa (IA Archiver)<\/td>\n1+2<\/td>\n5.98 KB<\/td>\n04 Feb 2005 – 01:48<\/td>\n<\/tr>\n<\/table>\n<\/td>\n<\/tr>\n<\/table>\n

* Robots shown here gave hits or traffic “not viewed” by visitors, so they are not included in other charts. Numbers after + are successful hits on “robots.txt” files<\/span><\/p>\n

<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

My web site used to get along pretty well with Google. They searched my web site a lot and I gaind some pretty good rankings. I couls just mention some one I know in my blog…. say…. Chris Court A.K.A. Monkey, and pretty soon I was the number one Google result for that person. Well… Continue Reading MSN Search bot<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1,3],"tags":[],"_links":{"self":[{"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/posts\/106"}],"collection":[{"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/comments?post=106"}],"version-history":[{"count":0,"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/posts\/106\/revisions"}],"wp:attachment":[{"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/media?parent=106"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/categories?post=106"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mattclare.ca\/blog\/wp-json\/wp\/v2\/tags?post=106"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}