Does Googlebot Hump Your Websites Too?
Filed Under web marketing · Tagged:
Yea, that’s right. I just noticed Googlebot is humping another one of my sites. The real traffic has accounted for just over 200 MB of data transfer this month so far. Googlebot, on the other hand, accounts for over 550 MB. I really don’t understand why, the site is www.ncexchange.org and I haven’t even added a new article to it in ages. Got to say, though, I’d rather have the Googlebot humping my sites than ignoring them.
Peter Davis is a web developer, investor, author, entrepreneur, and most importantly a father. 
Heh, well put Peter. It is kind of sad that we all feel that way about good old Google Bot.
Not necessarily Google but I see the same thing with AskJeeves. They are crawling sites like crazy wasting tons of bandwidth. And the bad thing - they don’t deliver visitors in return.
Peter, the website seems to be unreachable.
When Google bot hits, it hits. I’d prefer them doing a little bit all the time.
Yes, google bot and so many other have been crazy lately!
[...] Matt Cutts posted this on his blog, which is pretty much exactly what I remember him explaining in a session at Pubcon. It does explain why I’m seeing less of the Googlebot at my sites recently, and I’m very happy to have it explained. The short version (see his site for the long version) is that instead of Googlebot, Newsbot, and Mediabot (Adsense) all doing a crawl of your websites, Google will now have something called a “Crawl Caching Proxy” which will allow all of the bots to share the same information, thus reducing the number of times that Google needs to visit your site. Thus, if Mediabot just crawled, Googlebot wouldn’t need to do it again right then. Benefits? The reduction in the number of times your site is crawled should save you and Google on bandwidth, and probably Google on processing power too. I have already noticed the difference, and I used to notice the Googlehump a lot on my sites. Have a look at this site if you want to see the Slurphump. I have it set up so it will identify each of the major bots, as well as registered members. Over the past few months I’ve been seeing Slurp at a ratio of ten to one over registered users. I hope Yahoo! is working on a more efficient bot. 5:34 pm | [...]