You might want to block this fucker

dr rockerdr rocker Regular
edited September 2011 in Help and Suggestions
As there were 209 users online, I thought I would have a look at 'users online'. Baidu Spider (a chinese indexing bot) appeared numerouse times. Now, bots indexing things is no bad things, however, being that:

1. Totse is probably banned in China
2. Its probably ripping images and content that will bring in low to 0 traffic - being as totse is probably banned in PRC, why let it index shit, let alone content rob (which it is known to do)?
3. Its probably responsible for eating bandwidth - a shit load of bandwidth. Some web site operators have said in some months is has eaten all BW allowence very quickly

Now, that is three probably's, but I am probably correct.

On top of that, letting it do such shit is helping China to dominate and so potential subvert the internets.

KILL IT WITH FIRE.

Or modify robots.txt or block its IP ranges.

Comments

  • edited September 2011
    That's funny, I was just looking at our online users as well and I noticed that Baidu Spider crawling around. Maybe we should look into restricting it or something - have you got any links to other places which are commenting on this particular spider?
  • dr rockerdr rocker Regular
    edited September 2011
    Just google Baidu Spider - lots of shit on how to block it comes up and descriptions of what it does.
  • edited September 2011
    I think we should add something into robots.txt - it's a good idea. Having a look through Google results is proving to be very interesting and it's obvious that Baidu is a problem to a lot of website administrators.
Sign In or Register to comment.