MJ12 Claimed another internet victim

[[XC]thewildblue]

Got home from work last night after a terrible day to find that I had no internet. Tried my different routers none of them would work. So I pop out and get a new router. When I get in I check for answer phone messages and find that I have a message from griffin my ISP saying that I have been cut off for excessive useage. Now I find this out of order apparently Im doing 90gb a week, but Ive been paying for an uncapped connection. Well apparently under the terms of the fair usage Ive been overdoing it. So I called them up they refuse to turn me back on last night, I have 2 domains on this connection with mail servers etc. I explain that I know what the bandwidth was and its turned off and wont be on again. No Ive been given my port code. I find this unreal, no warning no letter just they felt I was using too much. Apparently last year at this time users doing 1TB a month were cut off, now its 240gb or more, except it doesnt state this anywhere and doesnt give you access to see what you and put through your pipe.

I am going to be phoning up this morning to speak to a customer service rep but I dont hold any hope of getting back.

Anyone know of any uncapped ISPs in the UK that offer static IPs.

[DAK1640]

Sorry to hear this TWB...I know the BS ur going thru...Hope it all gets sorted out.

[Movieman]

Sorry to hear it James.:(
I think the answer to things like this is setting up to do 6-8 hours a day in the off peak usage time and maintain a low profile.
So far no issues with Verizon here.
Crossing my fingers that they don't complain.

[[XC] moddolicous]

Yea, sorry to hear that TWB. I have no problems with Optimum (1TB used for all of november so far).

[[XC]thewildblue]

I had already set it to only run between midnight and 7am. The ISP advertises as an uncapped connection, yet in the small print has a 50gb cap a month. So I have called the ASA over here in the UK.

[jimwah]

..under the terms of the fair usage...
Anyone know of any uncapped ISPs in the UK that offer static IPs.

Fair usage :upset: ISP's seem to love this, suitably vague, term. I have a static 1mb home line that's uncapped (although I haven't spanked it to find out truly what the deal is) from Zen. They seem helpful & the guy I spoke to was happy to give me a breakdown of my GB's/month when I rang them recently.

Good luck getting it sorted man :up: Hopefully the ASA will spank those useless griffin boys :slapass:

[[XC]thewildblue]

I have got it turned back on now. But I have now migrated to eclipse internet. So I at least have a connection. They apologised etc. No more mj12 for me.

[njkid32]

I wondered about this.. I am using much more bandwidth than I should be.. It seems durring the day I get huge amounts anywhere from 14-19mb... It varies though. But at night sometimes its only at like 2mb just for a bit though. I am scared that Cox will see this consumption and turn me off... I guess only time will tell:)

[[XC]melymel]

It seems a bit unfair for them to have just cut you off like that with no prior warning :(. Did they mind it when you went full steam on DIMES?

@njkid; Try running MJ-12 and see if they notice ;) :p:

:toast:

[njkid32]

***cough***cough***Borg #3***cough***

[[XC]thewildblue]

Well from what I can make out Dimes doesnt take that much bandwidth its just makes the line slightly laggy, where as mj12 proper sucks the bandwidth, thats what made me pop up on the radar.

I will wait till I move to eclipse before doing Dimes in a big way I think.

[Bloody_Sorcerer]

dimes bandwidth isn't even close to on the same scale as MJ12. one dimes client pulls on average about 1 KB/s. MJ-12, on the other hand... well, we tend to measure it in mb/s :)

[jspace]

I have worried about this since March when I got DSL. Ever since then I've tried to keep the connection utilized in some way. MJ12, Bittorrent, etc, one of those is always running.

Session: 21,016
Today: 15,540
Month: 215,234

215234/1024=~210GB
+ web browsing
+ FileFront
+ torrents
+ everything else
= lotsa bandwith?

I'm not going to continue 'cuz it will bring me to what I pulled back in the day on dialup:D

[[XC] gomeler]

I had to back down on my output also as I believe I was cresting 300 gigs in November and I hear comcast is very picky about these issues. I miss my college connection where I could routinely go through a terabyte or more a month and not worry about the connection being frozen. I hate how ISPs have the ability to deem what is a fair amount of bandwidth to use but then again the same thing happens in webserving in regards to processor usage.. I just hate the internet backbone in general.

[Majestic-12]

Hi thewildblue and all,

I am sorry to hear about your ISP - it is very unfortunate that some ISPs, especially in the UK, claim to offer unlimited broadband but in reality they cap it at some ridiculous figure - reporting them to ASA is a good move, they should not be able to get away with such misleading advertising that gives them unfair competitive advantage!

Good unlimited ISPs in the UK are Telewest (cable), BeThere.co.uk (ADSL 2+) and (yeah I know it sounds aweful) AOL... there ought to be some more.

However the main point is that while running MJ12node is perfectly legitimate, it is best to keep under the radar - we have a lot of people crawling, and while I appreciate the desire to crawl as much as possible and max out things, it might be better to crawl less just to be sure: better crawl at 1 Mbits whole year, than crawl at 5 Mbits for a month: as we get more poeple joining it will allow individuals to crawl less URLs thus being under the radar of some of those ISPs that are not particularly honest about what they sell.

Traffic counter shown in node should be reasonably accurate, so don't push it hard - unless you are sure you are on Unlimited ISP of course.

Sadly there is no way around making MJ12node use less bandwidth - we can't crawl page without having it transferred and sadly vast majority of servers do not even support gzipping content :(

[Movieman]

Hi thewildblue and all,

I am sorry to hear about your ISP - it is very unfortunate that some ISPs, especially in the UK, claim to offer unlimited broadband but in reality they cap it at some ridiculous figure - reporting them to ASA is a good move, they should not be able to get away with such misleading advertising that gives them unfair competitive advantage!

Good unlimited ISPs in the UK are Telewest (cable), BeThere.co.uk (ADSL 2+) and (yeah I know it sounds aweful) AOL... there ought to be some more.

However the main point is that while running MJ12node is perfectly legitimate, it is best to keep under the radar - we have a lot of people crawling, and while I appreciate the desire to crawl as much as possible and max out things, it might be better to crawl less just to be sure: better crawl at 1 Mbits whole year, than crawl at 5 Mbits for a month: as we get more poeple joining it will allow individuals to crawl less URLs thus being under the radar of some of those ISPs that are not particularly honest about what they sell.

Traffic counter shown in node should be reasonably accurate, so don't push it hard - unless you are sure you are on Unlimited ISP of course.

Sadly there is no way around making MJ12node use less bandwidth - we can't crawl page without having it transferred and sadly vast majority of servers do not even support gzipping content :(
Glad to see you here.
Yes, it's a problem here in the US also.
I'm just lucky so far that Verizon is so hot to cut into the cable market that they don't care how I abuse their line!:D

[Majestic-12]

The best strategy is stay below the radar - I understand that the temptation to push more urls is hard to resist but no need to start from maxing out one's line, plus soon more people will join the project and it will allow individually lower crawling...good theory which kind of discounts desire to push things to the max. :cool:

It really annoys me that like 1/3rd of all Internet traffic is bittorrent, but some ISPs would pick on 100% legit web crawling activity, instead of wasting money on big marketing and CEOs, they should just add more capacity to the system!!! :(

Comcast - it is best to keep it low with this ISP as there was one disconnection early this year, however they did warn in advance and by all means never ignore warnings that ISP may give - scale down crawling to zero, I can't expect anyone to sacrifice their connection for the project, this is certainly not intention!

[vaio]

Freeola may be worth a try.
Mine is capped but they do offer unlimited too....and static IP.

Oh, and the initial contract is only 3 months so if it doesn't suit.....

[Movieman]

The best strategy is stay below the radar - I understand that the temptation to push more urls is hard to resist but no need to start from maxing out one's line, plus soon more people will join the project and it will allow individually lower crawling...good theory which kind of discounts desire to push things to the max. :cool:

It really annoys me that like 1/3rd of all Internet traffic is bittorrent, but some ISPs would pick on 100% legit web crawling activity, instead of wasting money on big marketing and CEOs, they should just add more capacity to the system!!! :(

Comcast - it is best to keep it low with this ISP as there was one disconnection early this year, however they did warn in advance and by all means never ignore warnings that ISP may give - scale down crawling to zero, I can't expect anyone to sacrifice their connection for the project, this is certainly not intention!
HMM..Ahh, now that your a registerd XS member, we have this WCG team that always has room for another member.
Yes, I am brutal!:ROTF:
And it plays well with MJ12!;)

[Majestic-12]

My brain can barely handle this project, if anything extra is added it might explode :)

[mad mikee]

Like Me & AAdjuster :D

(/me goes off to write idiot notes to get alex on XSTM :D )


HMM..Ahh, now that your a registerd XS member, we have this WCG team that always has room for another member.
Yes, I am brutal!:ROTF:
And it plays well with MJ12!;)

[vaio]

They want your rigs not your brains :D

[Movieman]

They want your rigs not your brains :D
What he said! :rofl:

and MikeE: You stay away from him or it's :slapass: time!:D

[mad mikee]

And can't you tell when I'm yanking your chain yet (maybe :wasntme:)

Alex - YHPM

You just go play w/ the Scot ....

Speaking of which - Where is Mc Intosh :shrug:, another fun person :hehe:

And to vaiodeux: We want him for his rigs AND his brain (wit), just like you contribute here :D


What he said! :rofl:

and MikeE: You stay away from him or it's :slapass: time!:D

[Movieman]

And can't you tell when I'm yanking your chain yet (maybe :wasntme:)

Alex - YHPM

You just go play w/ the Scot ....

Speaking of which - Where is Mc Intosh :shrug:, another fun person :hehe:

And to vaiodeux: We want him for his rigs AND his brain (wit), just like you contribute here :D
Me->:cord:<--Mike the Mad One!:rofl:

[Majestic-12]

They want your rigs not your brains :D

Ah... I don't have many rigs yet, and they are either shutdown at the moment or crunch data (processing new urls) - shortly they will be indexing 24/7 and then merging data into searchable index, once done they will be used for searching - that's heavy stuff guys, I am aiming to reach 10 bln index and it must search as fast as Google, or at least very close to it - don't like crap search engines and can't expect others to!

So can't just now put rigs to do other crunching 24/7 as crawling for url is just part of the problem, unfortunately :(

[vaio]

Best stress test them with folding in the meantime :D
MJ12 and folding play nice ;)

[Majestic-12]

It is tough call - I will need to be sure that search servers respond very fast to search queries, a lot of work in this direction will be done start pretty much in a week time, I will see what can be done in Q1 2007 - I should get more boxes by then, and once I know search patterns I might be able to put high cpu usage clients on a few of them. Sorry if this sounds disappointing and rather indecisive :(

[mad mikee]

Never forget we can always use witty statements around here :D
(Not to mention good insights :fact: )
& the stuff I was talking about was mostly from a 'marketing' standpoint, so didn't know if it would be worth it anyway.

Keep popping in, always nice to have visitors :hehe: :fact:


Ah... I don't have many rigs yet, and they are either shutdown at the moment or crunch data (processing new urls) - shortly they will be indexing 24/7 and then merging data into searchable index, once done they will be used for searching - that's heavy stuff guys, I am aiming to reach 10 bln index and it must search as fast as Google, or at least very close to it - don't like crap search engines and can't expect others to!

So can't just now put rigs to do other crunching 24/7 as crawling for url is just part of the problem, unfortunately :(

[vaio]

It is tough call - I will need to be sure that search servers respond very fast to search queries, a lot of work in this direction will be done start pretty much in a week time, I will see what can be done in Q1 2007 - I should get more boxes by then, and once I know search patterns I might be able to put high cpu usage clients on a few of them. Sorry if this sounds disappointing and rather indecisive :(

I don't mind........not in their team anyway :D
Just passing thru ;)

[Majestic-12]

It is a pleasure to drop by :)

I am sorry I could not firmly promise I will take part in other projects because very heavy processing will start here very shortly, in a matter of weeks and what's worse Google spoiled people (and this includes me) to offer very fast searches, personally I have very low tolerance for slow search and currently MJ12 search is slow - this must be fixed and I have good plan for that, but having processes running at 100% CPU (even at idle priority) may or may not seriously increase time required to do search, that's why I can't commit myself before I do more work and see what the score is.

I think however it will be possible to dedicate couple of boxes to other distributed projects, I used to take part in distributed.net RC64 challenge :)

[mad mikee]

I've seen one of google's 'satellite' sights :slobber:
30,000+ servers in racks as far as the eye could see (& that is just ONE site)

That makes it a very tough act to follow....


It is a pleasure to drop by :)

I am sorry I could not firmly promise I will take part in other projects because very heavy processing will start here very shortly, in a matter of weeks and what's worse Google spoiled people (and this includes me) to offer very fast searches, personally I have very low tolerance for slow search and currently MJ12 search is slow - this must be fixed and I have good plan for that, but having processes running at 100% CPU (even at idle priority) may or may not seriously increase time required to do search, that's why I can't commit myself before I do more work and see what the score is.

I think however it will be possible to dedicate couple of boxes to other distributed projects, I used to take part in distributed.net RC64 challenge :)

[Majestic-12]

That makes it a very tough act to follow....

Well worth trying to though :)

And if it happens that CPU cycles could be spared then I would certainly put some other stuff to run :)

[[XC] riptide]

Well worth trying to though :)

And if it happens that CPU cycles could be spared then I would certainly put some other stuff to run :)
Why not try your spare CPU cycles on Seventeenorbust, or 'SOB' as it is called. http://www.seventeenorbust.com/. We have our own sub section here.... http://www.xtremesystems.org/forums/forumdisplay.php?f=210
What type of machines do you/will you have at your diposal?

That project is a setup and forget project, perfect for a busy man...

Its all about finding the last of the Sierpinski Primes. 9 discovered, 8 to go...

:D

EDIT: There's loads of people viewing the MJ-12 section tonight :)

[Martijn]

Why not try your spare CPU cycles on Seventeenorbust, or 'SOB' as it is called. http://www.seventeenorbust.com/. We have our own sub section here.... http://www.xtremesystems.org/forums/forumdisplay.php?f=210
What type of machines do you/will you have at your diposal?

That project is a setup and forget project, perfect for a busy man...

Its all about finding the last of the Sierpinski Primes. 9 discovered, 8 to go...

:D
SHUT UP!!! :D :D :D :D :D


I don't care what he does, as long as he joins XS :toast:

[vaio]

SHUT UP!!! :D :D :D :D :D


I don't care what he does, as long as he joins Crunchers Inc :toast:

:D :D :D

[Martijn]

:D :D :D
StFU, n00b :stick:



:rofl:

[vaio]

noob? :eek:

Huuuuuuuuuuuurumph :p:

[Majestic-12]

The main constraint for me is to be sure that search speed is not affected, which might not be possible under condition when CPUs are 100% used even if they are on idle priority - problem is with low latency required by searchers, ie fast response from server. Sorry again for not giving straight committment - I don't like promising things that may not be done, but I can promise that I will look into it in Q1 2007: I expect to get 4 more dual core servers, and think a couple of them might be loaded up with some CPU crunching.

Going out to pub for a few beers, cheers all! :toast:

[vaio]

I find MJ12 slightly slows other projects....not the other way around.

[Martijn]

The main constraint for me is to be sure that search speed is not affected, which might not be possible under condition when CPUs are 100% used even if they are on idle priority - problem is with low latency required by searchers, ie fast response from server. Sorry again for not giving straight committment - I don't like promising things that may not be done, but I can promise that I will look into it in Q1 2007: I expect to get 4 more dual core servers, and think a couple of them might be loaded up with some CPU crunching.

Going out to pub for a few beers, cheers all! :toast:
Of course we understand. Cheers on the beer :D :D
:toast:

[[XC] riptide]

The main constraint for me is to be sure that search speed is not affected, which might not be possible under condition when CPUs are 100% used even if they are on idle priority - problem is with low latency required by searchers, ie fast response from server. Sorry again for not giving straight committment - I don't like promising things that may not be done, but I can promise that I will look into it in Q1 2007: I expect to get 4 more dual core servers, and think a couple of them might be loaded up with some CPU crunching.

Going out to pub for a few beers, cheers all! :toast:
Enjoy the beers. I'll be putting a note in my diary for the 1st of Jan :D HAHA! 4 Dual cores. Niiiiice.

Oh Martijn... your not going for beers! Get inside and do your homework :slap: Little brat!

[Martijn]

Oh Martijn... your not going for beers! Get inside and do your homework :slap: Little brat!
I'm 16, you bastard :D


Oh, I forgot drinking is over 18 in Ireland :rofl:

Have a :toast:

[Majestic-12]

I find MJ12 slightly slows other projects....not the other way around.

Yes, that's because you run MJ12node - this is just part of the problem: when data is crawled it needs to be indexed, merged and then searched, plus I need to regularly (about once a month) deal with new urls to create new work units, last time it took whole week to do it (deduplicating 50 bln urls is not fast) :(

So, basically what I am saying is that I have to run very heavy CPU tasks here, and worse once search is up on many boxes I will need to be sure latency is not affected, I think this is doable with new index format I will work on this month.

:toast:

[[XC] riptide]

I'm 16, you bastard :D


Oh, I forgot drinking is over 18 in Ireland :rofl:

Have a :toast:
:rolleyes: Most people start here at 13-15 .. anyways

so .... the argument stands...GET BACK IN AND DO YOU HOMEWORK!!!! :slapass: :slap: HA HA`!

[Martijn]

so .... the argument stands...GET BACK IN AND DO YOU HOMEWORK!!!! :slapass: :slap: HA HA`!
Hmmm.... Homework?




:idea:

Can you eat it?

[Mc Intosh]

You just go play w/ the Scot ....

Speaking of which - Where is Mc Intosh :shrug:, another fun person :hehe:



Reporting :banana:


This Scot is still going full speed, without complaints from ISP.. I don't even want to know what i downed last 30 days... not counting numerous DVD's from usenet...

Very sorry to hear some people are cut off, i'd go :banana: 's

Mean while M&M are :slapass: everyone besides that PY222... can't seem to get him.... time that more members join the M&M manglers :stick:

[Durzel]

Can someone explain to me how you can pull 90GB a month with legit web traffic?

If you're just leeching warez off newsgroups 24/7 then you deserve to be cut off.

[[XC] moddolicous]

Can someone explain to me how you can pull 90GB a month with legit web traffic?

If you're just leeching warez off newsgroups 24/7 then you deserve to be cut off.
Try running majestic and you'll be able to answer that questios. On a good day, 90gig will fall in 2 days, maybe less. I have used 1tb for the month of November. Time for a defrag.

[Vapor]

Heh, I was doing 90GB every 10 hours :p:

[[XC]thewildblue]

Well Im back on now after a week of downtime. Running mj12 on another connection but very limited now.

I went with a provider called eclipse in the end, but it seems that was a mistake , I paid for the service thats uncapped off peak, with peak being 6pm to 12pm, but its dog slow at the moment and I cant get full speed on my usenet access. Im going to give them a week then I will go elsewhere if it doesnt improve. The problem is Im out in the sticks so no cable or adsl2. I may take a look @ Freeola as that might be what I want.

[Movieman]

Well Im back on now after a week of downtime. Running mj12 on another connection but very limited now.

I went with a provider called eclipse in the end, but it seems that was a mistake , I paid for the service thats uncapped off peak, with peak being 6pm to 12pm, but its dog slow at the moment and I cant get full speed on my usenet access. Im going to give them a week then I will go elsewhere if it doesnt improve. The problem is Im out in the sticks so no cable or adsl2. I may take a look @ Freeola as that might be what I want.
We'll take all you have to give pal..10,000 a day or 10 mil..all adds up!
Good to see you back!:toast:

[[XC] riptide]

So..... I wonder how Majestic-12 is getting on getting in new servers. Although this post is pretty early, I'm doing this to get that 'Fart' thread title off the main page. Sorry Frisch. :p:

[Frisch]

^^It's ok riptide, it's not proper behavior for a man of my age, to let a thread title like that, shine all over XS. ;) :D

[[XC] riptide]

Well Frisch its ok.... I am actually curious about the servers seeing as there MAY be CPU power available from Maj-12.

[perry_78]

(not part of these teams, but nevertheless an interesting read :p:)

My ISP generally doesn't mind, but I seldom go through more than 5gb a day.. My hosting however, despite being "unlimited", is a little :banana::banana::banana::banana::banana:y about such matters..