Looking for " A Few Good Men"

[Movieman]

or women..:D
We need a few people I think to get back on track and chase down those nice folks from Norway..
If you got any friends with a decent line give them a yell..
Even a 3mbit/256kb cable line can do a million in a day and we don't necessarily want to alienate anyone that can't do 24/7..
I'll take anything someone can offer..10-20K a day, it all adds up.
Let's go get those herr...nice folks from Norway!:D

[RAW-Raptor22]

Noob alert: What do you need us to do?

[[XC] riptide]

^^^^^^^ We need a sticky in here./ Theres one in the Dimes section... I think?


EDIT;

Noob - http://www.xtremesystems.org/forums/showthread.php?t=99061:)

Moderator, move the sticky?

[Frisch]

"Good Men"

I'll have to disconnect.

[[XC] DragonOrta]

I guess that I could give it try again. Hopefully Comcast won't say anything this time...

:p:

[glennpat]

I cranked mine up some more and letting it run a couple of hours longer during the night. Looks like it may have doubled my small piece of the action.

[Movieman]

I cranked mine up some more and letting it run a couple of hours longer during the night. Looks like it may have doubled my small piece of the action.

I think the key for guys with comcast is to run maybe at night when sleeping or to turn it down so that it does maybe 200-300K a day and no more..
That should keep them off of your backside..

Sticky moved..Short version is to have net 2.0 installed and then go here and download MJ12 and then join the team:
http://www.majestic12.co.uk/projects/dsearch/download.php

[Frisch]

let's make a pot, and buy comcast. ;)

BTW
The correct Title of this thread is as follows....

Looking for "A few good men, superior Turtles, the head of comcast, and a contract man for a mission to norway"

[Movieman]

let's make a pot, and buy comcast. ;)

BTW
The correct Title of this thread is as follows....

Looking for "A few good men, superior Turtles, the head of comcast, and a contract man for a mission to norway"

I'll go with most of that but we already have the best turtle available and as to carl in Norway, he seems a good guy..
No contract man, just someone with cable cutters to visit his house!:rofl:

[Frisch]

I'll go with most of that but we already have the best turtle available and as to carl in Norway, he seems a good guy..
No contract man, just someone with cable cutters to visit his house!:rofl:

You're right, i went a little overboard, we would hurt Alex's feelings too, hmmmmm..........

I got It, we buy team norway, now we're buying comcast.
I'm sure we could pay them with some of your fishies.

[Movieman]

You're right, i went a little overboard, we would hurt Alex's feelings too, hmmmmm..........

I got It, we buy team norway, now we're buying comcast.
I'm sure we could pay them with some of your fishies.

Wouldn't work..Commiecast is too greedy.
What we need is to find people that can do a couple hundred K a day and a bunch of them...spread the load..
30-50 guys each doing 100-300K a day would do it easily..
50 times 200K is 10 million..add with what we can do and we could overtake them!:D

[njkid32]

I dont have much of a line but I guess I could help out:D Damn I miss my 12mb line:mad:

[[XC] DragonOrta]

A contract man, eh?

I killed a rabbit with a BB gun tonight...




:p:

[Frisch]

DragonOrta;2275334']A contract man, eh?

I killed a rabbit with a BB gun tonight...

:p:

You're hired.......

First thing to do....take the rabbit, write norway on its chest, write a letter which says "JOIN OUR TEAM, OR A NORWEGIAN RABBIT WILL DIE" and send it to them, that'll give them something to think about.......:yepp: :D

[hixie]

Movieman, we could buy comcast, all we have to do is sell Frisch to a circus, a walking, talking, overclocking turtle is sure to make some money :p:

[Movieman]

Movieman, we could buy comcast, all we have to do is sell Frisch to a circus, a walking, talking, overclocking turtle is sure to make some money :p:

I tried...They read his posts and only offered $5.00..
Then they saw the hat and reduced the offer to $2.00....:rofl:

[Okda]

will my 2mbit/512kbit be any good ??

also will it affect my online gaming performance or do i have to close it while playing ? what about normal usage ?

[Frisch]

will my 2mbit/512kbit be any good ??

also will it affect my online gaming performance or do i have to close it while playing ? what about normal usage ?

Will my 2mbit/512kbit be any good ?? Yes

also will it affect my online gaming performance ? Yes

do i have to close it while playing ? Yes

what about normal usage ? No problem

[Okda]

thanks :)

installing now

[Movieman]

will my 2mbit/512kbit be any good ??

also will it affect my online gaming performance or do i have to close it while playing ? what about normal usage ?

It will use as much as you possibly have if you let it.
Whe I let MJ12 run free I have a hard time opening a web page on a 15mbit/2mbit line!:D
You can limit how much bandwidth it can "have" to work with and when gaming, all you have to do is right click the tray icon and "snooze" and there's presets from 5 mins to a day.

Your line will definately help.
What I would do is set it to use no more than 70% of your up and down and snooze it when gaming.

[Frisch]

Movieman, we could buy comcast, all we have to do is sell Frisch to a circus, a walking, talking, overclocking turtle is sure to make some money :p:

Please ignore, and continue your daily business. The Royal Turtle forces are on their way to eliminate this man.


I tried...They read his posts and only offered $5.00..
Then they saw the hat and reduced the offer to $2.00....:rofl:

Please ignore, and continue your daily business. The Royal Turtle forces are on their way to eliminate this man.

[Movieman]

Please ignore, and continue your daily business. The Royal Turtle forces are on their way to eliminate this man.



Please ignore, and continue your daily business. The Royal Turtle forces are on their way to eliminate this man.

:sofa:
FYI: Got one machine with MJ12 down today..Just the DX3600 running so that is why output is off..Just too hot here, that other machine is in a room off of this one and I shut it down and shut the door to let the AC work better..

[Frisch]

I dont have much of a line but I guess I could help out:D Damn I miss my 12mb line:mad:
Where did he go, arrest this man, and force his line on to the battle field. ;) :D


:sofa:
FYI: Got one machine with MJ12 down today..Just the DX3600 running so that is why output is off..Just too hot here, that other machine is in a room off of this one and I shut it down and shut the door to let the AC work better..

BTW, we're close to a celebrating of the first man to the 1,000,000,000 mark

A milestone made by a little Irish fisherman, so we all have a chance here :D

[njkid32]

:wasntme:

[Movieman]

Where did he go, arrest this man, and force his line on to the battle field. ;) :D



BTW, we're close to a celebrating of the first man to the 1,000,000,000 mark

A milestone made by a little Irish fisherman, so we all have a chance here :D

yea, I'll get there, and soon!:D
As to NJKID, I've tried, he says yes but nothing happens..
You PM him and yell him as an admin he needs to set an example..:up:

(and tell him I'll turn on my FTP for him AFTER he starts running MJ12!
That will kill him!):ROTF:


ahh,well,gee, How you doing Eric? We were just speaking about you.
What a wonderfull guy you are and a great admin, salt of the earth,etc....:sofa:

[Okda]

from the stats i noitced that there are only 7 active users today

is that normal ? we sure do need more members

[Movieman]

from the stats i noitced that there are only 7 active users today

is that normal ? we sure do need more members

Unfortunately, it is normal..and better than normal..
Usually 3-4 guys.
That's why I made this thread.
We need maybe 20-30 guys that can do a little every day.
Even adding a little like 100,000 a day times 30 guys is 3 million a day and that gets us close to the output of Norway.
If we can get lucky and get 2-3 guys out of that 20-30 with decent lines that can add 1-2 mil a day each we can catch them easily.

[Okda]

i can't really understand how to find the best settings for my system

I am running a 2mbit/512kbit on an opteron 165 @ 2.4Ghz

my settings are showed in the screenshots below, anything i can change to increase the output ? at night i give it 100% of the bandiwdth and by day only 70% & 50%

[Martijn]

Installing right now. I've got a spare system I can let this run on. We're uncapped, with a fair use policy at 400GB/month. Shouldn't be too difficult to reach, but I think watching it a little will be enough.

Good ol' days :)

EDIT: I got a 4/1 line.

[[XC]melymel]

I'll try and throw a few hundred K in again, forgot about mj12 and now I have to re-find my optimum settings... bah...

Safrout I have the same down speed as you do but only 256 up. Therefore try the following 2 tweaks:

Set down and upstream to 44,000
Max workers to 40
Also on the page where you enter your user and pass tick the box that says "prefer all TLD's apart fomr .com/.org/.net"


These will use your line to the fullest. Basically you want to keep the success rate at 70&#37; so if after 20 minutes you are >75% then increase your workers by 5. If you are <65% then decrease your workers by 5. This is just a rough guide but any further tweaks are just for squeazing the last bits out of the line.


Great work to all those that kept going while many of us stopped and I look forward to the congratulatory 1,000,000,000 point thread for the old guy :up:

[Okda]

i set my down stream to 2048kbit and upstream to 512kbit, why should i set it to 44,000?


what about Max open url buckets ? should it also be 40 ?

and where to check my success rate ?

EDIT:

what is the best settings for both

Max number of async workers
&
Max open url buckets

and anything else i need to change in my screenshots above ?

[[XC]melymel]

i set my down stream to 2048kbit and upstream to 512kbit, why should i set it to 44,000?


what about Max open url buckets ? should it also be 40 ?

and where to check my success rate ?

I find/found that setting my speeds to the corretc ones meant the client used a bit less than max even with the sliders on 100%, where as at 44,000 it uses all I have.

10 buckets is plenty for the amount of workers you will be using, 6 is the most I've had open at 1 time and that's with all workers working.

The success rate is on the main page (status tab) of the client, 2nd one down in the list entitled "Crawled".

[Movieman]

melymel;2279177']I'll try and throw a few hundred K in again, forgot about mj12 and now I have to re-find my optimum settings... bah...

Safrout I have the same down speed as you do but only 256 up. Therefore try the following 2 tweaks:

Set down and upstream to 44,000
Max workers to 40
Also on the page where you enter your user and pass tick the box that says "prefer all TLD's apart fomr .com/.org/.net"


These will use your line to the fullest. Basically you want to keep the success rate at 70% so if after 20 minutes you are >75% then increase your workers by 5. If you are <65% then decrease your workers by 5. This is just a rough guide but any further tweaks are just for squeazing the last bits out of the line.


Great work to all those that kept going while many of us stopped and I look forward to the congratulatory 1,000,000,000 point thread for the old guy :up:

We don't have any "old" guys here..All young at heart!:up:

[Martijn]

We don't have any "old" guys here..All young at heart!:up:

Are you sure? You really seem to be loosing it now, Dave... :rofl:

[Movieman]

Are you sure? You really seem to be loosing it now, Dave... :rofl:

I lost it a long time ago when I read some of the things that go on here ar XS.
Guy's with P4D's that keep losing boards with bad PSU's when I've never had an issue with a PS in 10 years..
Guys losing $400.00 sets of ram from trying to run 2.4v thru what is rated for 2.1v max..
More MB's lost in the average month than I've owned total..
Yea, I lost it but the question is why?
I think it's from trying to find some logic in this insanity!:rofl:

[Magnj]

I have FiOs as you can see in my sig....is faster better?

[Movieman]

I have FiOs as you can see in my sig....is faster better?

You got PM:D

[[XC]melymel]

I have FiOs as you can see in my sig....is faster better?

Yes 20mb.....:slobber:

[Movieman]

melymel;2279450']Yes 20mb.....:slobber:

Just got off the phone with him..:hrhr:
And you guy think this old fart is slow..:ROTF:

[Magnj]

:)
Working on install now, Have to let the laptop live by the router since wireless will butcher the connection.

[Movieman]

:)
Working on install now, Have to let the laptop live by the router since wireless will butcher the connection.

Sounds good and thanks for helping..
Any questions, yell..
Now don't blame me if it becomes addictive!:D

[[XC]melymel]

Just got off the phone with him..:hrhr:
And you guy think this old fart is slow..:ROTF:

Quick work on getting on the phone :up: however a 28 minute phone call still very much falls into the slow category, getting better though! :lol:

Magnj: Thanks very much for helping :).

[Movieman]

melymel;2279507']Quick work on getting on the phone :up: however a 28 minute phone call still very much falls into the slow category, getting better though! :lol:

Magnj: Thanks very much for helping :).

Nah, maybe a 10-15 min call..I was easy on him..
You know, first time always go gentle...:rofl:

[Magnj]

Edited out of guilt by a MOD..:wasntme:

Had to put in a second router...it's getting crowded in this house lol

[Martijn]

21 min 25 seconds to be exact :p

Had to put in a second router...it's getting crowded in this house lol

2x 24port 3Com Superstack 610 here. It can be worse :p:

[Magnj]

ha, its all good.

Ok it seems to be up and running and I joined XS through the MJ12 site.

Does someone want to point me to a settings help thread or guide...there's a lot of buttons here lol

[Movieman]

ha, its all good.

Ok it seems to be up and running and I joined XS through the MJ12 site.

Does someone want to point me to a settings help thread or guide...there's a lot of buttons here lol

http://www.majestic12.co.uk/projects/dsearch/mj12node/manual/

[Martijn]

ha, its all good.

Ok it seems to be up and running and I joined XS through the MJ12 site.

Does someone want to point me to a settings help thread or guide...there's a lot of buttons here lol

Trial and error is best here I suppose. Every connection needs to be tweaked to it's own capabilities. We can tell you what everything does, but not how to set it right. :)

[Magnj]

ok I set max workers to 100 and max buckets to 40 for now. I have it set as 20,000 and 5,000 for my bandwidth at 100&#37;

[[XC]melymel]

Where's moddo lately? :shrug: He used to be good at tweaking the client I also seem to remember him saying he'd get a guide together a "while" ago :p:.

[Okda]

ya i can't figure out the best settings or even what needs to be tweaked

[Magnj]

Even with 100 max workers and 40 buckets im only working on 5 buckets with 30 workers.

[Magnj]

Okie now I'm cookin, 11th place for the day and I've only been running for about 4 hours

[Movieman]

ok I set max workers to 100 and max buckets to 40 for now. I have it set as 20,000 and 5,000 for my bandwidth at 100&#37;
If your at 20,000 and 5000 and that slide bar to the right is at 100% your going to use that whole line and can set the workers to 300 and the buckets to 100 BUT that will slow your line I beleive.
Try that when no one else in the house is on a PC and see what happens.
I would set those numbers in BUT move the slide bars to 70% max to keep enough banndwidth for the rest of the machinery and maybe slide them over to 100% when sleeping.

melymel;2279579']Where's moddo lately? :shrug: He used to be good at tweaking the client I also seem to remember him saying he'd get a guide together a "while" ago :p:.

Moddo is tied up with looking at colleges I think..

[Magnj]

Ok I'll try that out Movieman.
Do my numbers look about right so far?

[Okda]

we sure need more members

this project is very good and doesn't consume power like crunching or anything else so ur elec bill won't suffer, it just need an uncapped internet connection which i am sure 90&#37; of the members @ XS do have

[[XC] moddolicous]

Moddo is tied up with looking at colleges I think..
I was, but I'm back now.


melymel;2279579']Where's moddo lately? :shrug: He used to be good at tweaking the client I also seem to remember him saying he'd get a guide together a "while" ago :p:.
I'll try to make it as soon as I can.

[Okda]

WOW today we are kicking some butts :D

we just need more few members and no one then will be able to stop us

http://www.majestic12.co.uk/projects/dsearch/stats.php

[[XC] moddolicous]

Kicking butt would put us at number 1 producing team today safrout. We are just having an OK day;) . Hopefully I'll be able to tweak a few things and really get rocking. Keep my spot warm Hixie.

[Okda]

ya u r right :S

will be waiting for ur guide to how to optimize the settings :)

[Magnj]

Yea your guide should help me out a lot. I think I have the fastest line here but without someones help its not really working that way.

[Martijn]

Wow!! This could be my second day of pumping 1 million URLs :D

[Okda]

/me runs searching for a place to hide with his 2mbit/512kbit connection

[Martijn]

Wow!! This could be my second day of pumping 1 million URLs :D

So I did it :D

4/1... 1m/day? Still can't believe it. Last time I did 1m I saved up for 2 days? Has the client improved that much? :shrug:

[[XC] moddolicous]

Well, the only parts I have done right now are the personal and connection tabs. I can finish the crawler, profile and Misc sections tommorrow and sunday I'll finish the archiving sections.

[Okda]

NICE
waiting for sunday :)

[carlfrs]

Good to see XS back in business. Team Norway has ruled MJ-12 the last six months! I have some good news for you..., I will need to reduce my crawling in a few weeks time because of change of work. Then you will have the chance to overtake us, maybe this year.
Good luck!:D

[hixie]

moddolicous;2280709']Kicking butt would put us at number 1 producing team today safrout. We are just having an OK day;) . Hopefully I'll be able to tweak a few things and really get rocking. Keep my spot warm Hixie.

What spot? Oh! you mean the one i'm standing in right now? It's only for the mean time, i'll return it to you monday, when i turn my BT off ;)

[Magnj]

3rd :( I want first!
It seems like my one computer can't saturate my line....do I have to keep adding nodes to completely saturate the line?

[Frisch]

3rd :( I want first!
It seems like my one computer can't saturate my line....do I have to keep adding nodes to completely saturate the line?

Yep, one won't swallow that line.

[Magnj]

crap. I can run my desktop for some of the time but this thing is like a damn space heater and I don't have central air.

[[XC] moddolicous]

3rd :( I want first!
It seems like my one computer can't saturate my line....do I have to keep adding nodes to completely saturate the line?
In order to completely saturate a 10mb line, you would need 2 conroe rigs (possibly dedicated). It is tough to completely saturate a line. Of course there are exceptions (movieman's clover could saturate an 8mb line, maybe 10mb).

[Movieman]

Ok I'll try that out Movieman.
Do my numbers look about right so far?
If your set at 50% your doing fine..Those numbers make sense for a laptop on your line.

moddolicous;2280046']I was, but I'm back now.


I'll try to make it as soon as I can.
Glad to see your ugly puss!:clap:

Wow!! This could be my second day of pumping 1 million URLs :D
go Martijn, GO!:cheer:

/me runs searching for a place to hide with his 2mbit/512kbit connection
Never think that way. It's the combined group that makes it work.
Imagine someone coming in with 1000 PC's that could only do 200K a day each?
I think I'd need to go and buy some Depends!:rofl:

Good to see XS back in business. Team Norway has ruled MJ-12 the last six months! I have some good news for you..., I will need to reduce my crawling in a few weeks time because of change of work. Then you will have the chance to overtake us, maybe this year.
Good luck!:D
Hi Carl! Good to see you.
Don't tell me this. We want to catch you guys at your prime but I hope the new job works out well for you.:up:

3rd :( I want first!
It seems like my one computer can't saturate my line....do I have to keep adding nodes to completely saturate the line?
One laptop won't use all of that line.
I had to add another machine to use all of mine..Just a little machine.:wasntme:

moddolicous;2282268']In order to completely saturate a 10mb line, you would need 2 conroe rigs (possibly dedicated). It is tough to completely saturate a line. Of course there are exceptions (movieman's clover could saturate an 8mb line, maybe 10mb).
:hrhr:..Yup!

Thank God for multi-quote!:up:

[Magnj]

I have 2 on 100&#37; right now...they seems to be pretty slow...but maybe I don't have enough horsepower

[Movieman]

I have 2 on 100% right now...they seems to be pretty slow...but maybe I don't have enough horsepower

2 machines set to 100% on your line should do at least 5-8 mill a day..
If you want I'll walk you through the settings..

[Magnj]

K thx.

I'll wait for the guide to come up this weekend and see if that will help.

[Movieman]

yup:D

[SkylinGTR26]

yeah i joined the team.
Ill check this out for a while.
Help you guys out... :D

[Martijn]

yeah i joined the team.
Ill check this out for a while.
Help you guys out... :D

Thanks for joining!! What speed is your line?

[Movieman]

yeah i joined the team.
Ill check this out for a while.
Help you guys out... :D

Thanks! We can use all the help we can get!:clap:
yell if questions..Don't be shy.;)

[Movieman]

Yea, a double post..Just to grab some attention!:D
was looking at the XS page over at MJ12 and had an idea:idea:
We've had at one time or another over 60 people from XS run MJ12..
What if we tried to do a "Family Reunion" for one day?
Get everyone( or most, a couple no longer do DC and one has passed on) and do MJ12 flat out, balls to the wall for one 24 hour day and knock their socks off?
I think we could easily top 50 million and if everyone got into it flat out for the day we might top 100 million.:shocked:
I'm finding out that if I let this thing run flat out here I can crank over 10 mil in a day myself so I think the 100 mil is doable.
Comments? Suggestions? Timeframe?
Talk to me and tell me if you like this or if you think I'm pissing in the wind.:D

[Okda]

ya a super day will be nice

but we should find a way to get about 20~30 more users to run MJ12 for 24/7 with internet speeds around 2mbit at least

what about a global PM like what was done for WCG ?

[[XC]melymel]

We could try it but I think a lot that ran it had to stop due to comcast crapping themselves :p:, e.g. DAK and his T3.

[[XC] moddolicous]

Make sure that Alex says his servers can handle it. I remember we were going to do this before, but Alex said his servers would freak if we did it. BTW, as far as the guide goes, it might take me longer. I can put up a rough one tomorrow and fine tune it later if that is OK with you guys.

[Movieman]

moddolicous;2283030']Make sure that Alex says his servers can handle it. I remember we were going to do this before, but Alex said his servers would freak if we did it. BTW, as far as the guide goes, it might take me longer. I can put up a rough one tomorrow and fine tune it later if that is OK with you guys.

Yup, ahead of you there..or great minds think alike.
emailed Alex this afternoon and now is a bad time, maybe by september but I'll talk with him in advance on it..

[Okda]

so what is the maximum limit that the servers can take ?

[Oc-Ghost]

Helping out with my 512/512 :cool:

[Okda]

thanks for joining :)

[Okda]

we are the first today :)
http://www.majestic12.co.uk/projects/dsearch/stats.php

[Frisch]

Helping out with my 512/512 :cool:
Thanks :up:


we are the first today :)
http://www.majestic12.co.uk/projects/dsearch/stats.php

Brilliant !! Excellent !! Magnificent !! marvelous !! superb !! jolly good flawless a$$ kicking

http://img48.imageshack.us/img48/8083/smimc1.gif

[Martijn]

Yes, yes, that was my 500k of today that did it!!!!! :p: J/K

Onward and upward!! :toast:

[Frisch]

Yes, yes, that was my 500k of today that did it!!!!! :p: J/K



It's all your fault Matijn, where's the last 200 K :down: :shakes:

http://img518.imageshack.us/img518/5654/jjjvy3.gif

;) :D

[[XC]melymel]

It's all your fault Matijn, where's the last 200 K :down: :shakes:

http://img518.imageshack.us/img518/5654/jjjvy3.gif

;) :D

Sorry that's me, forgot to restart the client and lost about 250K's worth :p:

[Frisch]

melymel;2283890']Sorry that's me, forgot to restart the client and lost about 250K's worth :p:

You'll report at the principals office, for immidiately spanking.

Thanks for your honesty, it's always best to come forward....... I think...... anyway it keeps our principal in form.....btw .....who is he :confused:

[Okda]

---^
Lol

[Movieman]

There's a bunch of sick puppies in this group!:rofl:
Hey, we're growing. In a week we've jumped close to 10 mil a day.
I'm pissed though. Tried for 10 mil yesterday and failed.
got to a little over 9.5 mil but couldn't make it..Maybe today.

As to the max for Alex's servers he didn't tell me but I got the impression that he had them working on another part of the project that would bring in some cash to help him pay for this huge connectivity fee he must have each month.
He's a great guy who talks straight and those are the kind of people I enjoy helping.

[Okda]

today is not a good day :(

[[XC] moddolicous]

today is not a good day :(
Yea, my main computer restarted and I don't have MJ12 set to auto-start. My bad.

[justin_c]

i got a 10mbit connection over here, but after the tweaks, the Current value is hovering around 1000 kbit/s ... whats going on

Current: 500-1400 kbit/s
network: 900-2400 kbit/s
overall: 200-500 kbit/s
limit: 10000 (10 mbits x 1000kbits/mbit = 10000 kbits)


some problems here: setting the number of async workers and buckets. i set 100:30, and it said "starvation". so i set it to 60:20, still starvation, 300:200, still starvataion, "WE APPEAR TO NEED TO GET MORE URLS!" cpu usage is hovering around 0 for MJ12node.exe .....

ok, some screenshots.

http://img181.imageshack.us/img181/7397/mj122qm1.th.jpg (http://img181.imageshack.us/my.php?image=mj122qm1.jpg)
http://img224.imageshack.us/img224/3846/mj121bn6.th.jpg (http://img224.imageshack.us/my.php?image=mj121bn6.jpg)

edit:damnit forgot to close the back window and the log.

[Okda]

i think u will need more than a single machine to be able to use 100&#37; of ur connection


moddolicous;2286668']Yea, my main computer restarted and I don't have MJ12 set to auto-start. My bad.

also i am afraid that our hero ( a.k.a Dave ) might be facing problems cause his output is less than his normal by atleast 75%

[[XC] moddolicous]

i got a 10mbit connection over here, but after the tweaks, the Current value is hovering around 1000 kbit/s ... whats going on

Current: 500-1400 kbit/s
network: 900-2400 kbit/s
overall: 200-500 kbit/s
limit: 10000 (10 mbits x 1000kbits/mbit = 10000 kbits)


some problems here: setting the number of async workers and buckets. i set 100:30, and it said "starvation". so i set it to 60:20, still starvation, 300:200, still starvataion, "WE APPEAR TO NEED TO GET MORE URLS!" cpu usage is hovering around 0 for MJ12node.exe .....

ok, some screenshots.

http://img181.imageshack.us/img181/7397/mj122qm1.th.jpg (http://img181.imageshack.us/my.php?image=mj122qm1.jpg)
http://img224.imageshack.us/img224/3846/mj121bn6.th.jpg (http://img224.imageshack.us/my.php?image=mj121bn6.jpg)

edit:damnit forgot to close the back window and the log.
Here is your problem: Your disk space has NO ROOM. Look at the low disk error. You could try getting rid of some unnecessary stuff, but unless it is 2gig +, it might still come back. When it says that your disk has low space, that means that it tries to upload any buckets that are left.

[mcflurry4321]

Reporting for duty!

12/1 line.

EDIT: My download bandwidth is like 500-2000. I have it set to use 100&#37; of my 12mbit down speed so why would it be downloading so slowly?

[Okda]

what are ur settings for max bucket and open url ?

[mcflurry4321]

what are ur settings for max bucket and open url ?

30 workers with 15 open url buckets.

[Okda]

try increasing it to 30/20 for example and keep increasing while keeping ur error rate to normal

also maybe u will need two machines to fill all of ur bandwidth

[Movieman]

set to max..300/100 and let it rip!:D
You have 12/1, I have 15/2 and I need 3 machines set at max to grab all my line..Try it and if it's too much, then cut back a bit at a time.

[mcflurry4321]

set to max..300/100 and let it rip!:D
You have 12/1, I have 15/2 and I need 3 machines set at max to grab all my line..Try it and if it's too much, then cut back a bit at a time.

Is it normal for it to jump down to like 500 every few seconds?

[Movieman]

Is it normal for it to jump down to like 500 every few seconds?

yup..a network, pulses..normal..open task manager and on network even when sending any file it jumps up and down.
Back later..

[mcflurry4321]

yup..a network, pulses..normal..open task manager and on network even when sending any file it jumps up and down.
Back later..

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

[Martijn]

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

It depends. Do not worry too much about it, I have it every now and then as well.

[Movieman]

I only ask this because I never seem to remember MJ12 jumping around so much the last time I ran it for a few days. Not sure though.

If your at close to 3 mil a day on that line your pretty close to correct settings for one machine..2 machines will put you over 5 mil..3 will pretty much max the line out at maybe 6-7 million..
Joe is better at this than I am. He and frisch understand this much better.
I just set it at max and let it fly..Brute force approach!:rofl:

[mcflurry4321]

My node seems to keep restarting or something. Has that ever happened to any of you?

[Paladin]

I've noticed lately that the exact same settings as I used a few months ago are sluggish now. My first guess was that SBC Yahoo is doing some kind of dynamic choke. Reading the MJ12 forums points to maybe just the type of URL buckets being issued now (retries of past failed URLs) or DNS troubles with the current client/node version. Adding the OpenDNS IPs to my DNS pathing seemed to help:

208.67.222.222
208.67.220.220

LAN Prop->TCP/IP->Properties->Advanced->DNS->Add

Haven't tried setting the router to use the alternate DNS yet.

(IP phone still prohibits me from seeing how much my 6000/600 can do before that C&D order from the ISP comes.)

[Movieman]

My node seems to keep restarting or something. Has that ever happened to any of you?
yup, not abnormal..It will do an auto reset if it hangs too long on a url..

I've noticed lately that the exact same settings as I used a few months ago are sluggish now. My first guess was that SBC Yahoo is doing some kind of dynamic choke. Reading the MJ12 forums points to maybe just the type of URL buckets being issued now (retries of past failed URLs) or DNS troubles with the current client/node version. Adding the OpenDNS IPs to my DNS pathing seemed to help:

208.67.222.222
208.67.220.220

LAN Prop->TCP/IP->Properties->Advanced->DNS->Add

Haven't tried setting the router to use the alternate DNS yet
.

(IP phone still prohibits me from seeing how much my 6000/600 can do before that C&D order from the ISP comes.)

Good to see your ugly puss here!;)
There's ways around them. Turn the app on one day..then shut down the next time your at the PC maybe 6-8 hours later..wait a day or so, then do again....Keeps you from looking like one of those despised "Bandwidth Hogs" and torturer of children that the ISP's would make everyone think we are.
My answer to them is yea, and look at the growth of your companies..
30-60% growth in profits per year in a dead economy..and they say they aren't making money..They can kiss my backside they aren't..
Greediest group I've ever seen!:soap:

Oh Gentlemen:
Something you might want to see and we haven't seen in a while:
Top teams (Today)

# Name.............. URLsdone Data(MB)
1 XtremeSystems 24,159,290 332,156
2 Team Norway 18,684,847 285,937


Nice work guys!:up: :clap:

[[XC] riptide]

When i started MJ12 a while back on a good 2meg down line it was solid 100&#37; usage anytime i had it on. I think My ISP crucifies me when I have it on more than 20 mins ie slows it down, but I can still let it cruise all the same

[[XC] moddolicous]

If your at close to 3 mil a day on that line your pretty close to correct settings for one machine..2 machines will put you over 5 mil..3 will pretty much max the line out at maybe 6-7 million..
Joe is better at this than I am. He and frisch understand this much better.
I just set it at max and let it fly..Brute force approach!:rofl:
I do what I can Dave.

[Movieman]

moddolicous;2288649']I do what I can Dave.

and I appreciate what you do..:clap:
I don't know all the inns and outs like you do so i let it fly and hope for the best..

[mcflurry4321]

Help me out then mod :)

12 mbit down/1 mbit up

What are my idea settings?

[Okda]

we are waiting for his guide to optimize our clients :)

[Okda]

well i can admit that this thread was/is/will very helpful, cause of the new added members ( like me :) ) our daily output is putting us on the top of the list ad we can even increase it by a large margin if we can invite more users

[Magnj]

My 3 mil will be offline for a few days.

[Okda]

hoe it will be back soon :)

[mcflurry4321]

What do these log messages mean?

[7/4/2007 10:04:51 AM] SaveWorkerData can't operate on oItem that is null!
[7/4/2007 10:04:51 AM] Worker #12 failed to save its data. Original BID=155 Original URL=http://www.innermost.co.kr/bbs/zboard.php?id=visual&page=1&sn1=&divpage=1&sn=off&ss=on&sc=on&select_arrange=reg_date&desc=desc&no=254&PHPSESSID=


I have like 750 of them in my log over 7 hours.

[[XC] moddolicous]

Help me out then mod :)

12 mbit down/1 mbit up

What are my idea settings?

Well,for crawlers, I would definitely go with something like 200+, maybe even 300 like Dave said. Just hope that your router can handle it. 100 open URL buckets. Raise the threshold to 10 and check off all the hard error options. Check off no delays between upload. Under misc., make everything normal except upload, which should be set to high. Under archiving, check off LZMA and set word size to 128 and dictionary size to 16mb. Those settings should definitely raise your output.

[Paladin]

moddolicous;2289560']Under archiving, check off LZMA and set word size to 128 and dictionary size to 16mb.

Is this a good idea? The tool tips say not to set higher than 64/8. I'm wondering what kind of load it would create on alex's results parsing server if we ignored the requested limit.

[Frisch]

Homer simpson here to clear a few things up.

There's so many of you today, please don't eat my pie.....ahhhhhhhhhh....pie........arggggghhhhh

I got 300 workers to feed, (they'll believe me those suckers), and it's very important that they get their pie.....ahhhhhh...pie....argggghhhhh



First day for a long time, that one actually had to scroll to see who was on......:D


Regards
The scrolling Turtle.

[Movieman]

Homer simpson here to clear a few things up.

There's so many of you today, please don't eat my pie.....ahhhhhhhhhh....pie........arggggghhhhh

I got 300 workers to feed, (they'll believe me those suckers), and it's very important that they get their pie.....ahhhhhh...pie....argggghhhhh



First day for a long time, that one actually had to scroll to see who was on......:D


Regards
The scrolling Turtle.

Teams growing my friend and that's nice to see.
Must be a very bad batch of urls today.
I left the machines flat out and didn't go on the web for the last 12 hours so no bandwidth taken from MJ12 at all and only 3 mil added in 12 hours..:(
Good news is that we're ahead of Norway today!:up: :clap:

[Frisch]

Teams growing my friend and that's nice to see.
Must be a very bad batch of urls today.
I left the machines flat out and didn't go on the web for the last 12 hours so no bandwidth taken from MJ12 at all and only 3 mil added in 12 hours..:(
Good news is that we're ahead of Norway today!:up: :clap:

Nice isn't strong enough, it's ...............WEEEEEEEEEEEEEEEEEE :D

You want me to pity you, you the greediest, you the incarnation of greed....MARGE HE'S EATING OUR PIE....

Norway is going down...DOWn

[Movieman]

Nice isn't strong enough, it's ...............WEEEEEEEEEEEEEEEEEE :D

You want me to pity you, you the greediest, you the encarnation of greed....MARGE HE'S EATING OUR PIE....

Norway is going down...DOWn

D O W N is right! It will take a while but it will happen!
I think I will try to get the whole forum on this for a week come september.
Hope Alex isn't reading this!:ROTF:
I can see him wondering what is happenning when he gets 500 mil in a day just from XS!:shakes:

[mcflurry4321]

In 5 hours of uptime I've gotten 560 of this pair of log messages:

[7/5/2007 8:21:31 AM] SaveWorkerData can't operate on oItem that is null!
[7/5/2007 8:21:31 AM] Worker #17 failed to save its data. Original BID=313 Original URL=http://zonewm.biz/forum/editpost.php?s=&action=editpost&postid=33368


How do I get rid of them?

[Martijn]

I had a lanparty yesterday, I should be going full whack again tomorrow :)