View Full Version : Linux Guru's want to help me? (WCG Cluster)
[XC] mysticmerlin
09-11-2006, 06:49 PM
I have like 10 or so 1.5Mhz and less (down to 233mhz P2) linux capable boxes. I was wanting to do a cluster to run WCG on. I do not have the funds to get a 10 port KVM but I do have a 12 port switch and an extra monitor.
So is it possable to put all 10 to one screen and have a Boinc/WcG crunch farm with it? I am not new to linux but this is a new idea to me.
What do you think? Am I out of my mind?:slap:
If possable it may help other's who are having to change the crunch3r (if that ain't :bs:) Boinc files and only have to do it once and still have all the power running.
KaptainBlaZzed
09-11-2006, 07:56 PM
just load windows on them all and set up romote desktop so you can access them all from one computer.
that is how i run my farm.
plus the cluster will cost you in production.
[XC] mysticmerlin
09-11-2006, 08:11 PM
I only have 2 legit copys of Windowz
Linux = Free
rob725
09-11-2006, 08:23 PM
Does Linux not have something akin to Remote Desktop?
[XC] mysticmerlin
09-11-2006, 08:28 PM
yes but cluster makes them all like one super comp. If they all do one at a time they are a waist of electric on the grid. Make a cluster and they are one 5+Mhz box. So I understand it. Might be wrong.
Maybe I should just flebay them all and take the $50 bucks I make and get a Dell :( :rofl:
[XC] hipno650
09-11-2006, 08:36 PM
what are the specs on them? i use my kvm for my comps and let them run individual wu's. and it only takes like 48h for a p266 to run one wu:D
[XC] mysticmerlin
09-11-2006, 08:39 PM
Wow 2 days per WU? :rofl: I am thinking that a Baywolf cluster @ 5Mhz will be better off. If not then I will save my pennies and get a new puter and try and keep up with you all.
[XC] hipno650
09-11-2006, 08:42 PM
ya mabye it's not worth the week or more for it to get a wu done:D
[XC] mysticmerlin
09-11-2006, 08:49 PM
Nice Oc's in our sig BTW there hipno650. A 650 @4.1 is not a bad deal at all on air. Thought of going water on it?
Ok :)
As far as i understood you want your bunch of pcs to act as one "supercomputer". This is possible, and it isnt even that hard.
http://clusterknoppix.sw.be/ should do the trick.
What this does is: It says it has a computer with X cpus, and it is trying to divide the work equally.
You dont even need harddisks for the computers as if you start one of them as the "master" the others can boot over the net. Very cool software if you ask me. If you are intrested to try it, feel free to ask.
-
And there is also the more normal solution: install linux on every device and manage it from any pc you want. As in linux everythin can be done over the net, you can even just open the boinc manager app on the machine you want to control, but let the window display on the pc you are sitting.
[XC] mysticmerlin
09-12-2006, 06:15 AM
I will download the cd when I get my DSL this weekend ::woot: Direcway sucks!
KaptainBlaZzed
09-12-2006, 06:24 AM
Does Linux not have something akin to Remote Desktop?
Yes it does, it is called VNC
I use a program called ultra VNC viewer.
Yes it does, it is called VNC
I use a program called ultra VNC viewer.
VNC isnt really what you should use if you want performance...
NomachinesNX or FreeNX is really really great. It allows smooth control of your desktop, even if you only have a 56k dialup. Very cool s/w indeed...
Firechicken
09-12-2006, 08:40 AM
Is there any non-Java based viewers around that anyone has used and liked?
[XC] hipno650
09-12-2006, 03:11 PM
Nice Oc's in our sig BTW there hipno650. A 650 @4.1 is not a bad deal at all on air. Thought of going water on it?
the best part is that it's on a nforce 4 chip board!!!!:rocker: :rocker: :rocker: :rocker: :rocker: :rocker: :rocker: :rocker: thanks man:D no my temps are ok like 50c load and i don't have the money to go water:( i am saving my money for a conroe (part time) in which case i could have a p4 2.4 (northwood) on 24/7 and mabye my p4 650 24/7 as well but my ma and pa might have somthing to say about that:( :D
Bloody_Sorcerer
09-12-2006, 03:15 PM
SSH anyone? :fact:
rcofell
09-12-2006, 03:44 PM
SSH anyone? :fact:
I was waiting for that to show up, about the best aspect of using a *nix variant, especially for crunching where everything can be done from command line.
Haven't made any shell scripts before, but I'm sure setting one up would be extremely helpful for installing/configuring boinc on a large amount of machines(use a scp client to transfer the script file over ssh and then run it in a ssh terminal).
Is there any non-Java based viewers around that anyone has used and liked?
tightvnc is doing a really good job here.
ssh is remote console, not remote desktop ;) (ok ok, X11 forwarding.. but thats not really a desktop ;) )
Would be intrested in what you are doing now merlin. Going for a Cluster or setting up a Farm...
[XC] riptide
09-13-2006, 02:14 PM
I will download the cd when I get my DSL this weekend ::woot: Direcway sucks!
Mystic. Be sure to inform the thread here how this goes. I'm very intersted myself as i have a few wrecks of things around here. The idea of no-HD makes it all the more appealing! ;)
[XC] mysticmerlin
09-13-2006, 05:45 PM
Mystic. Be sure to inform the thread here how this goes. I'm very intersted myself as i have a few wrecks of things around here. The idea of no-HD makes it all the more appealing! ;)
Will do, If I can figure it ou :slap:
[XC] mysticmerlin
09-18-2006, 12:00 PM
Ok, here is what I am going to try just incase any one wants to try it also.
Just might work. I will let you all know.
http://www-128.ibm.com/developerworks/linux/library/l-clustknop.html
:) I am very anxious to hear if everything worked as expected,.. a real cluster would be soo cool. Think about it. You have like one 20 core machine, and not 20 one core machines. Looks like less work to me ;)
[XC]melymel
09-18-2006, 01:41 PM
I really want to kow how you get on mysticm, I can't afford anymore hdd's so a cluster would be cool :cool: :toast:
[XC] riptide
09-18-2006, 04:06 PM
Tell us! Very interested.
rob725
09-18-2006, 04:36 PM
Extreemly nubish linux question:
After reading the clustknop link I wondered why you have to run the script on each of the drones. Can't you just run several instances on the master and let it automatically balance to the rest of the cluster?
[XC] mysticmerlin
09-18-2006, 04:48 PM
Extreemly nubish linux question:
After reading the clustknop link I wondered why you have to run the script on each of the drones. Can't you just run several instances on the master and let it automatically balance to the rest of the cluster?
This is my first forea into cluster. I will let you know if I figure it out. :toast:
gearhead364
09-18-2006, 06:48 PM
I wish I could run this on my school's beowulf cluster, like 52 cpu's running simultaneously. Anyway, I have always been interested running Linux, does have any suggestions on a version to try. Also is it easy to setup my PC so that I can choose to boot from two different OS.
[XC]thewildblue
09-18-2006, 10:52 PM
Hmm this is a great idea, I might try and have a play with VMware and simulate it. Love the idea of a cluster.
rob725
09-18-2006, 11:31 PM
Do you mean tricking one machine to think its many and clustering the many back to one?
Fr3ak
09-18-2006, 11:48 PM
I was so close in building a 16 nodes Sempron cluster a while ago, but after a lot of reading, setting it up is troublesome and the worst thing: Its slower than running all PCs with their own OS.
Sure its great you dont need HDDs, but you would only need a 3gb hdd and they are very cheap on ebay.
[XC] riptide
09-19-2006, 01:27 AM
Well... before I even consider looking into one has to bear in mind that network speed may be a limiting factor. If the interconnects are not fast enough.... it basically starves. Depending one what speed you can get the trafic up to and what class of machines they are. But I'm gonna have a read into it now.
[XC] riptide
09-19-2006, 02:13 AM
Well having read the description it now makes a little more sense. In terms of me doing DC projects. Since all my machines would be always at 100%, there is no need for a load balancing cluster really. I mean, why share out processes to a node that are already at 100% ;) However its advantage is that the very small drone program has a very small footprint and maybe in this instance are resources saved overall. But other than that....
Well, as far as i understand it is so, that you just install boinc on the master, and then boinc thinks it runs on a lets say 100 cpu pc. it opens up 100 processes of the working threads on the machine and those get migrated to the other nodes.
Since all my machines would be always at 100%, there is no need for a load balancing cluster really. I mean, why share out processes to a node that are already at 100% ;) However its advantage is that the very small drone program has a very small footprint and maybe in this instance are resources saved overall. But other than that....
Jup. Once after the thread got moved it should run as fast as it would be with a normal linux installation.
[XC] mysticmerlin
09-19-2006, 08:04 AM
that is how I am getting it brot. My only thing is if my lan cards will do pxd or what ever it is called. Worth a shot.
Better ideas for a one headed monster?
Any one got a say 10 spot KVM they want to get rid of?
[XC] Malkec
09-19-2006, 01:51 PM
Why don't you use screen command on your linux? You dont't need desktop to run boinc client. All you need is ssh (console) access.
I run it on my linux server with command "screen ./boinc_client" and it works great.
[XC] mysticmerlin
09-19-2006, 04:14 PM
can you elaberate?
Also is this better than clustering?
this isnt really what you want mysticmerlin ;) - screen is just a shell multiplexer, that means that it makes the use of multiple shells on just one terminal. Imagine like multiple desktops but only without graphics ;)
I have tried clusterknoppix on some virtual machines.. i havent installed boinc on them yet, but clustering works. Really a great livecd.
[XC] mysticmerlin
09-21-2006, 12:04 PM
I downloaded it and have them burnt. I am going to put them on 2 test rigs and see what I can get with it.
[XC] mysticmerlin
09-22-2006, 07:42 PM
How would I go about making linux and boinc come on when I hit the power button and keep it headless. I only have one CRT and 4 rigs to set up. Like a Knoppix that starts boinc in my name?
The cluster will not work with my NIC cards :(
You only need to start boinc on the master machine, the others will get added to the cluster and the threads running on the master (with the crt on) will move his threads to the nodes.
download chaos here: http://midnightcode.org/projects/chaos/
and run it on the three pcs without a monitor. They should get added to the cluster (visible in the master when opening the white symbol in the kicker) and start working for you.
[XC] mysticmerlin
09-23-2006, 01:48 AM
But do they still need PX? cappable cards? If so mine don't support it and I can not afford at this point to get 4 or 5 of them. :(
no, booting from cdrom is enough :)
[XC] mysticmerlin
09-23-2006, 02:03 AM
but they have to talk right? so the NIC's have to be capable.
yes, but not pxe capable. pxe is only used to boot from network.
[XC] mysticmerlin
09-23-2006, 03:03 AM
got yha! I will give it a go then on the 2 and see what I get. Will it work on a 802.11b lappy also? And one more, Is there a Mac version of something like this that you know of to put them in the node also?
uhm. Mac version isnt possible because it has to be the same cpu arch as on your master, as it will put its threads on the other pcs, and an x86 prog wont run on ppc mac.
It will work on your wireless laptop too, but you need to get your wlan running in linux, which may be not so easy..
[XC] mysticmerlin
09-23-2006, 03:38 AM
Ok, skip both, I want to make this work some time this year. LoL Thanks for all you help brot. Hope to get it running soon.