Looking for " A Few Good Men"

[[XC] moddolicous]

i got a 10mbit connection over here, but after the tweaks, the Current value is hovering around 1000 kbit/s ... whats going on

Current: 500-1400 kbit/s
network: 900-2400 kbit/s
overall: 200-500 kbit/s
limit: 10000 (10 mbits x 1000kbits/mbit = 10000 kbits)


some problems here: setting the number of async workers and buckets. i set 100:30, and it said "starvation". so i set it to 60:20, still starvation, 300:200, still starvataion, "WE APPEAR TO NEED TO GET MORE URLS!" cpu usage is hovering around 0 for MJ12node.exe .....

ok, some screenshots.




edit:damnit forgot to close the back window and the log.

Here is your problem: Your disk space has NO ROOM. Look at the low disk error. You could try getting rid of some unnecessary stuff, but unless it is 2gig +, it might still come back. When it says that your disk has low space, that means that it tries to upload any buckets that are left.

[mcflurry4321]

Reporting for duty!

12/1 line.

EDIT: My download bandwidth is like 500-2000. I have it set to use 100% of my 12mbit down speed so why would it be downloading so slowly?

[Okda]

what are ur settings for max bucket and open url ?

[mcflurry4321]

what are ur settings for max bucket and open url ?

30 workers with 15 open url buckets.

[Okda]

try increasing it to 30/20 for example and keep increasing while keeping ur error rate to normal

also maybe u will need two machines to fill all of ur bandwidth

[Movieman]

set to max..300/100 and let it rip!
You have 12/1, I have 15/2 and I need 3 machines set at max to grab all my line..Try it and if it's too much, then cut back a bit at a time.

[mcflurry4321]

set to max..300/100 and let it rip!
You have 12/1, I have 15/2 and I need 3 machines set at max to grab all my line..Try it and if it's too much, then cut back a bit at a time.

Is it normal for it to jump down to like 500 every few seconds?

[Movieman]

Is it normal for it to jump down to like 500 every few seconds?

yup..a network, pulses..normal..open task manager and on network even when sending any file it jumps up and down.
Back later..

[mcflurry4321]

yup..a network, pulses..normal..open task manager and on network even when sending any file it jumps up and down.
Back later..

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

[Martijn]

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

It depends. Do not worry too much about it, I have it every now and then as well.

[Movieman]

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

If your at close to 3 mil a day on that line your pretty close to correct settings for one machine..2 machines will put you over 5 mil..3 will pretty much max the line out at maybe 6-7 million..
Joe is better at this than I am. He and frisch understand this much better.
I just set it at max and let it fly..Brute force approach!

[mcflurry4321]

My node seems to keep restarting or something. Has that ever happened to any of you?

[Paladin]

I've noticed lately that the exact same settings as I used a few months ago are sluggish now. My first guess was that SBC Yahoo is doing some kind of dynamic choke. Reading the MJ12 forums points to maybe just the type of URL buckets being issued now (retries of past failed URLs) or DNS troubles with the current client/node version. Adding the OpenDNS IPs to my DNS pathing seemed to help:

208.67.222.222
208.67.220.220

LAN Prop->TCP/IP->Properties->Advanced->DNS->Add

Haven't tried setting the router to use the alternate DNS yet.

(IP phone still prohibits me from seeing how much my 6000/600 can do before that C&D order from the ISP comes.)

[Movieman]

My node seems to keep restarting or something. Has that ever happened to any of you?

yup, not abnormal..It will do an auto reset if it hangs too long on a url..

I've noticed lately that the exact same settings as I used a few months ago are sluggish now. My first guess was that SBC Yahoo is doing some kind of dynamic choke. Reading the MJ12 forums points to maybe just the type of URL buckets being issued now (retries of past failed URLs) or DNS troubles with the current client/node version. Adding the OpenDNS IPs to my DNS pathing seemed to help:

208.67.222.222
208.67.220.220

LAN Prop->TCP/IP->Properties->Advanced->DNS->Add

Haven't tried setting the router to use the alternate DNS yet
.

(IP phone still prohibits me from seeing how much my 6000/600 can do before that C&D order from the ISP comes.)

Good to see your ugly puss here!
There's ways around them. Turn the app on one day..then shut down the next time your at the PC maybe 6-8 hours later..wait a day or so, then do again....Keeps you from looking like one of those despised "Bandwidth Hogs" and torturer of children that the ISP's would make everyone think we are.
My answer to them is yea, and look at the growth of your companies..
30-60% growth in profits per year in a dead economy..and they say they aren't making money..They can kiss my backside they aren't..
Greediest group I've ever seen!

Oh Gentlemen:
Something you might want to see and we haven't seen in a while:
Top teams (Today)

# Name.............. URLsdone Data(MB)
1 XtremeSystems 24,159,290 332,156
2 Team Norway 18,684,847 285,937


Nice work guys!

[[XC] riptide]

When i started MJ12 a while back on a good 2meg down line it was solid 100% usage anytime i had it on. I think My ISP crucifies me when I have it on more than 20 mins ie slows it down, but I can still let it cruise all the same

[[XC] moddolicous]

If your at close to 3 mil a day on that line your pretty close to correct settings for one machine..2 machines will put you over 5 mil..3 will pretty much max the line out at maybe 6-7 million..
Joe is better at this than I am. He and frisch understand this much better.
I just set it at max and let it fly..Brute force approach!

I do what I can Dave.

[Movieman]

I do what I can Dave.

and I appreciate what you do..
I don't know all the inns and outs like you do so i let it fly and hope for the best..

[mcflurry4321]

Help me out then mod

12 mbit down/1 mbit up

What are my idea settings?

[Okda]

we are waiting for his guide to optimize our clients

[Okda]

well i can admit that this thread was/is/will very helpful, cause of the new added members ( like me ) our daily output is putting us on the top of the list ad we can even increase it by a large margin if we can invite more users

[Magnj]

My 3 mil will be offline for a few days.

[Okda]

hoe it will be back soon

[mcflurry4321]

What do these log messages mean?

[7/4/2007 10:04:51 AM] SaveWorkerData can't operate on oItem that is null!
[7/4/2007 10:04:51 AM] Worker #12 failed to save its data. Original BID=155 Original URL=http://www.innermost.co.kr/bbs/zboard.php?id=visual&page=1&sn1=&divpage=1&sn=off& ss=on&sc=on&select_arrange=reg_date&desc=desc&no=2 54&PHPSESSID=


I have like 750 of them in my log over 7 hours.

[[XC] moddolicous]

Help me out then mod

12 mbit down/1 mbit up

What are my idea settings?

Well,for crawlers, I would definitely go with something like 200+, maybe even 300 like Dave said. Just hope that your router can handle it. 100 open URL buckets. Raise the threshold to 10 and check off all the hard error options. Check off no delays between upload. Under misc., make everything normal except upload, which should be set to high. Under archiving, check off LZMA and set word size to 128 and dictionary size to 16mb. Those settings should definitely raise your output.

[Paladin]

Under archiving, check off LZMA and set word size to 128 and dictionary size to 16mb.

Is this a good idea? The tool tips say not to set higher than 64/8. I'm wondering what kind of load it would create on alex's results parsing server if we ignored the requested limit.