MMM
Results 1 to 13 of 13

Thread: Just joined up!

  1. #1
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174

    Just joined up!

    Movieman urged me to join up with the addition of me new internet connection.


    http://www.speedtest.net/result/1043692486.png

    http://www.speedtest.net/result/1059591355.png

    I have been reading through the posts and went through the tweaks. Is this beneficial to run this on multiple workstations?



  2. #2
    Xtreme Enthusiast
    Join Date
    May 2007
    Posts
    648
    Welcome Jerry!! Yes running many nodes helps to saturate your line and crawl more URLs. Is that 100mbit line uncapped? *starts calculating days til I'm steamrolled

  3. #3
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174
    Quote Originally Posted by DeadlyFire View Post
    Welcome Jerry!! Yes running many nodes helps to saturate your line and crawl more URLs. Is that 100mbit line uncapped? *starts calculating days til I'm steamrolled
    Yes I was told it was uncapped. No one in the area is currently on the same segment and was told it was not "regulated" yet. Just hope all of my activity doesn't arouse suspicion. :-)



  4. #4
    Xtreme Enthusiast
    Join Date
    May 2007
    Posts
    648
    Quote Originally Posted by CrazyJerry View Post
    Yes I was told it was uncapped. No one in the area is currently on the same segment and was told it was not "regulated" yet. Just hope all of my activity doesn't arouse suspicion. :-)
    Very cool If you need help or suggestions on the client settings let us know!

  5. #5
    Xtreme Legend
    Join Date
    Mar 2008
    Location
    Plymouth (UK)
    Posts
    5,281
    I reckon you can do some damage with that connection even if you just run on it overnight I'm thinking you could overtake me in weeks few.

    Glad to have you on board


    My Biggest Fear Is When I die, My Wife Sells All My Stuff For What I Told Her I Paid For It.
    79 SB threads and 32 IB Threads across 4 rigs 111 threads Crunching!!

  6. #6
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174
    Quote Originally Posted by DeadlyFire View Post
    Very cool If you need help or suggestions on the client settings let us know!
    I see some of your downstreams are 12,812kbs. How can I achieve speeds like this? Are you only crawling specific domains?

    I used the basic sticky guide i saw on the main page. Could you recommend any settings for my connection type?



  7. #7
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174
    Quote Originally Posted by OldChap View Post
    I reckon you can do some damage with that connection even if you just run on it overnight I'm thinking you could overtake me in weeks few.

    Glad to have you on board
    I have one goal to start off. This is to run over Movieman

    I will be going live with this connection after the first of the year for all of my users. We are currently utilizing 2 t'1s for internet bandwidth. The users will be happy with 20mb and till think it is blazing fast. Just need to setup some scheduling with my offsite backups and i'll be good to utilize this connection to its full potential. Work does come before play...



  8. #8
    Xtreme Enthusiast
    Join Date
    May 2007
    Posts
    648
    Quote Originally Posted by CrazyJerry View Post
    I see some of your downstreams are 12,812kbs. How can I achieve speeds like this? Are you only crawling specific domains?

    I used the basic sticky guide i saw on the main page. Could you recommend any settings for my connection type?
    Are you going by the stats page on the Majestic 12 site? Those speed readings aren't entirely accurate because they take a 'snapshot' of the speed at a point in time and speeds within the node bounce all over the place(generally not steady). Though the bottom speed shows a rough approximate speed of the user's connection(my stat page shows 43mb currently, I have a 50mb line). I have .ca .org .us and .com currently set as my domains. Post a screenshot of the 'crawler' tab under Tools>Options in the client and we can check your settings

    Quote Originally Posted by CrazyJerry View Post
    I have one goal to start off. This is to run over Movieman
    Dave might regret recruiting you after hearing that He's already got me hounding after him, let alone 2 of us

  9. #9
    Xtreme Cruncher
    Join Date
    Dec 2008
    Location
    Los Angeles/Hong Kong
    Posts
    3,054
    Since you have a fast connection, you can set the async worker tab to "999" and the amount of open buckets top "333." Those values are all under the crawler tab.
    Team XS: xs4s.org



  10. #10
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174
    Quote Originally Posted by DeadlyFire View Post
    Are you going by the stats page on the Majestic 12 site? Those speed readings aren't entirely accurate because they take a 'snapshot' of the speed at a point in time and speeds within the node bounce all over the place(generally not steady). Though the bottom speed shows a rough approximate speed of the user's connection(my stat page shows 43mb currently, I have a 50mb line). I have .ca .org .us and .com currently set as my domains. Post a screenshot of the 'crawler' tab under Tools>Options in the client and we can check your settings
    Screeny Attached bellow!

    Quote Originally Posted by DeadlyFire View Post
    Dave might regret recruiting you after hearing that He's already got me hounding after him, let alone 2 of us
    I sent Dave a PM of my speed tests he told me about MJ12 I signed up. But it's time for some friendly competition. I am setting up a few rigs at other sites to run at night. They all have a minimum of 30mb down. I may call and see if we can get in on better promotional deals. Just saying if I get a good deal whats more bandwidth?
    Attached Images Attached Images



  11. #11
    Xtreme Cruncher
    Join Date
    Dec 2008
    Location
    Los Angeles/Hong Kong
    Posts
    3,054
    You definitely will need more async workers and buckets.

    Check my post above, you would need to set 999 async workers and 333 open buckets. You might need to install MJ12 to more computers since 999 is the max you can go, that means each rig will crawl at about 12mbit/s
    Team XS: xs4s.org



  12. #12
    Xtreme Enthusiast
    Join Date
    May 2007
    Posts
    648
    lkiller is right. Try putting your number of workers to greater than 350 and number of open URL buckets to higher than 300. You can set both even higher but I tested running only 1 node on my connection and after 300-350 or so workers the gains are very minimal if any. What kind of router do you have? A connection that big will swamp the NAT table on most consumer routers and force you to reset them every couple hours. A linux firewall/routing solution is probably the most powerful thing you can use next to a multi-thousand $ Cisco router.

  13. #13
    Xtreme Cruncher
    Join Date
    Jun 2008
    Location
    Crown Point, IN
    Posts
    174
    Quote Originally Posted by DeadlyFire View Post
    A connection that big will swamp the NAT table on most consumer routers and force you to reset them every couple hours. A linux firewall/routing solution is probably the most powerful thing you can use next to a multi-thousand $ Cisco router.
    For the time being I am using a Cisco 871 and am looking to demo a Sonicwall NSA.



Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •