Topic: Controlling bots and other crawlers - excess hits to my site (Page 1 of 1) Pages that link to <a href="https://ozoneasylum.com/backlink?for=30944" title="Pages that link to Topic: Controlling bots and other crawlers - excess hits to my site (Page 1 of 1)" rel="nofollow" >Topic: Controlling bots and other crawlers - excess hits to my site <span class="small">(Page 1 of 1)</span>\

 
Karl
Bipolar (III) Inmate

From: Phoenix
Insane since: Jul 2001

posted posted 04-05-2009 03:09

In reference to: http://www.meetscoresonline.com (primarily the page /results.aspx)

I am tracking hits to my site. I'm concerned because some of the hits are excessive. The side effect is of course my bandwidth costs are THROUGH THE ROOF!

Interestingly, the 2nd largest count in the list (still 5 times less the number 1 offender) is Google.
Guy's what should I do? I don't want to block a site that is crawling my pages for good reasons, but then again, I can't afford the excessive bandwidth charges!

Here is the hits by IP (top 10 offenders):

Page UserIP ipcount
/results.aspx 220.181.61.216 40969
/results.aspx 66.249.67.55 7154
/results.aspx 72.30.142.114 5032
/results.aspx 208.36.144.6 3489
/results.aspx 38.99.44.105 2657
/results.aspx 72.30.81.181 1968
/results.aspx 72.30.142.105 1106
/results.aspx 208.111.154.15 998
/results.aspx 208.111.154.16 804
/results.aspx 72.30.142.233 731

Here is the location for each from IP2Location.com.

IP Address Country (Short) Country (Full) Flag Region City ISP Map
220.181.61.216 CN CHINA BEIJING BEIJING CHINANET BEIJING PROVINCE NETWORK
66.249.67.55 US UNITED STATES CALIFORNIA MOUNTAIN VIEW GOOGLE INC
72.30.142.114 US UNITED STATES CALIFORNIA SUNNYVALE INKTOMI CORPORATION
208.36.144.6 US UNITED STATES NEW YORK NEW YORK XO COMMUNICATIONS
38.99.44.105 US UNITED STATES CALIFORNIA SAN FRANCISCO PSINET INC
72.30.81.181 US UNITED STATES CALIFORNIA SUNNYVALE INKTOMI CORPORATION
72.30.142.105 US UNITED STATES CALIFORNIA SUNNYVALE INKTOMI CORPORATION

Adding hits by date. I've only been tracking since about 3/28.

HitDate UserIP ipcount
20090404 208.36.144.6 863
20090403 208.36.144.6 893
20090402 208.36.144.6 893
20090401 208.36.144.6 908
20090404 220.181.61.216 4908
20090403 220.181.61.216 4865
20090402 220.181.61.216 2838
20090401 220.181.61.216 5570
20090331 220.181.61.216 7193
20090330 220.181.61.216 6883
20090329 220.181.61.216 6246
20090328 220.181.61.216 2724
20090404 38.99.44.105 541
20090402 38.99.44.105 544
20090401 38.99.44.105 622
20090404 66.249.67.55 523
20090402 66.249.67.55 845
20090401 66.249.67.55 1453
20090331 66.249.67.55 1869
20090329 66.249.67.55 971
20090328 66.249.67.55 1119
20090404 72.30.142.105 983
20090403 72.30.142.114 785
20090402 72.30.142.114 889
20090401 72.30.142.114 588
20090331 72.30.142.114 668
20090330 72.30.142.114 743
20090329 72.30.142.114 956
20090402 72.30.81.181 522


(Edited by Karl on 04-05-2009 03:12)

(Edited by Karl on 04-05-2009 03:25)

Karl
Bipolar (III) Inmate

From: Phoenix
Insane since: Jul 2001

posted posted 04-05-2009 03:30

Well, pursuing the topic myself a bit further, I find this discussion:
http://www.webmaster-talk.com/the-other-search-engines/53089-what-spider-is-this.html

Very interesting. I'm now considering a ban on China domain IP addresses.

Karl..



Post Reply
 
Your User Name:
Your Password:
Login Options:
 
Your Text:
Loading...
Options:


« BackwardsOnwards »

Show Forum Drop Down Menu