i know its not executed but if he includes an AV+malware he could detect which website has sh.. inside and sell such to AV companies ;)
Printable View
Congrats on 2 Billion OldChap!! :clap::up:
Cheers, It did seem that it would never happen during December-January but I got there in the end even though I now have to stop during the day on my main "unlimited" line. apparently unlimited to them means if you dl more than 8Gigs a day between 9am and 9pm they will terminate the connection.
Big number, I'll try and follow your grand example, and reach 2 billion.......in Turtle tempo : ) But when I'm there you're long gone, so I'll shout it out loud with the wind, so you can hear me up there in front..... : )
Congrats !
Gratz all! Missed a few milestones it seems.
Shame I still cant get my weird error issues fixed, boo :( Crunching WCG 24/7 at the mo on 6 cores though. Gotta love AMD for hex!
Will grab this build over the next week to see if the errors have disappeared, unlikely though. Bar that, must be my ISP blocking traffic or something :(
Hi Mekoa :) It's been really quiet around here, welcome back :up:
When did you start having those error issues? Was it after the big beta node update around December?
Hey Deadly :D
Its the below thread, mostly started after I upgraded to Windows 7. Have no issues with internet using anything else, just MJ12 (same ISP as before also).
http://www.xtremesystems.org/forums/...d.php?t=261795
I've just signed up :D
And now have a question. I'm on a 4096/512 line and after setting M12 to use 100% of the line I have done 21,500 after 3h15m which seems low. I've yet to see the download rate pass around 650 and it usually hovers around 200 to 300, often dipping down to under 50 (last edit, I've just seen download as 9, 0, 83, 14). What's up with that?
EDIT: In those three and a bot hours I've used 250MB down/15MB up. I KNOW that I can pull around 1.4GB/hour over HTTP/FTP and upload around 150MB/hour. Nothing else is using the network other than a bit of browsing (no flash/very few images).
Hey oj, tell us what you chose for crawler settings.
I put my laptop to work on a 8meg/0.7meg line with 100 workers and 35 buckets a few days ago and it seems to run ok but took 6 days for 9mill
On my 50.1.75 line I rarely see more than 25kbits sec this one seems to run at maybe 2
After the next time the NIC goes down, go into control panel and open the 'network and sharing' button. Then on the left side 'change adapter settings' and then find your NIC in the next window. Right-click on it and click 'status' and then the 'details' button. Take note of the IPv4 address before and after the crash and the lease obtain/expire time. It might show no change at all but it's worth a try to see what happens.
What OC said ^^ What operating system are you running? For a 4mbit line I'd suggested 100 max # workers and 20-50 max open buckets(start low and keep raising until you're satisfied with the speed). The general idea is keep a low amount of buckets open but a higher number of workers. Also, under the 'More crawler' tab within options, I would set maximum priority buckets to something low like 5 or 10, if you don't have a beastly router those kinds of buckets can cause the router to hang. Be sure to check if your ISP allows unlimited or large amounts of downloading!!
Happy crawling and thanks for joining! :up::welcome:
One HUGE milestone for one of our smaller yet more determined crawlers :up: 100mil may not seem that big for some of us but for someone who chugs along at 100k urls/day it's a helluva milestone
http://i.imgur.com/u7Bby.png
100,028,744
Congrats eme! :clap::clap:
:up: congratulations :up:
DeadlyFire : you are very right. Congratz to eme ! Very well done man!
I followed the Majestic12 guide (the sticky) to the T, as while all options are explained a lot are still over my head :D
I'm running a Billion BiPAC 7300GA which can handle several tens of thousands of seeds are leeches connected in uTorrent so these packets shouldn't be a concern. Current router uptime is several days so it's not causing issues as far as I can tell :)
A good day for us yesterday, 40mil/over 1TB crawled and second place :clap: :
http://i.imgur.com/nokDc.png
Good work guys :up:
I'll be glad when I can look at my output and it stays constant again though....Just now one day can be 50% more than another just now
I had a weekend of discovery.....
My original setup of running 2 nodes at 350/100 to max out my connection has still been going at it quite well but with the current mix of buckets I find my connection (at least the upload which limits me) to be only partially used and the download was sometimes high sometimes low. As an interim measure I had increased buckets to 125 for slightly better performance.
I have now added a third node at 300/100 and the upload is now balls to the wall whilst the download is staying pretty constant. First full day yesterday and much better numbers.
I will report back here if my percentages suffer but overnight at least they are still in the 80's
Congrats elvis on hitting the Billionaires club!! :clap::clap::clap:
http://i.imgur.com/61sR9.png
1,000,733,396 to be precise :up:
:woot: http://www.dogproductshop.co.uk/smil.../party0019.gif http://www.dogproductshop.co.uk/smil.../party0018.gif A bit of a push going on lately....Congrats.
WHAT THE F:::::.
http://i.imgur.com/t2UwO.png
is it just me ??
Just today i run out of ritalin so I might be just me ( not joking)