D2OL and linux guide!

alright so i recently switched a couple rigs over to linux for various reasons and in the process have come up with some nice info about running D2OL in linux that i thought i would share. i know someone was asking about this awhile back, anyway maybe its will even convince some of u to convert lol. anywho here goes...

1. loading the GUI version on a linux system (pretty basic)
go here: http://www.d2ol.com/dOL/download_instructions.html
download the GUI installer with Java
*in this example the installer was downloaded to /usr/src/
once downloaded just enter this into terminal window
cd /usr/src (wherever u downloaded the file to)
chmod 755 installDDOL.bin
./installDDOL.bin
the setup proggy will start and just follow the instructions to install

2.load the non-GUI version
download the non-GUI version of the client with java from the link above
its a tar.gz file, when i downloaded it, i just opened it directly from the internet with a filemanager
and un-tared it to /usr/src but you can also save the tar.gz file to ur hard drive and un-tar from the
terminal with this comand:
tar zxvf /usr/src/guilessInstallDDOL.tar.gz
once un-tared u have a folder called SengentD2OL (in my case /usr/src/SengentD2OL)
now make a directory called /D2OL and copy the contents of SengentD2OL into the new directory
your should have two folders in your /D2OL folder /D2ol/D2OL and /D2OL/jre
now go into D2OL/res and open the general.prp file chage the field "LaunchMain" to false
at this point if you are reinstalling d2ol copy your node.prp file into /D2OL/D2OL/res folder
now for the moment of truth!!
pop open a terminal window and type the following:
cd /D2OL/D2OL
./D2OL controller:cli
(u must type the above exactly to run in text mode)
if all is well it should start up no problem. this should run fine on almost any major distro, but if u get error post here and ill try to help.
running in text mode has many advantages besides taking up less resources, you can also run it on a system that doesnt have a Desktop Manager.



much more to come! running as a service more complicated stuff i just wanted to get this up for now. will add more soon!

:toast:

chipmonk010 is there any gain in runing Linux over MS on D2OL?

to tell you the truth i didnt notice much difference, but with linux your can easily set up a terminal server and boot nodes diskless. this was the main motivation to migrating some machines to linux. you can also get away with less ram since the linux footprint is so small, and it may speed things up slightly since linux is much less bloated and takes up less resources. so between running in windows and running in linux text mode i would think there would be an increase in output but i have not tested this so im not sure...

Let us know if this helps!

Keep the pointers coming... If it looks good I'll switch all of my dedicated crunchers over to Linux...

Did I follow that guide, you can run d2ol diskless ????
If so , I wanna know how to do that. Would everything then be stored in physical memory? Could you setup a vitual ramdrive with enough physical memory and run without a drive of any sort?

[edit]
Guess I did read it wrong, too much soma.
Anyway, would be cool to boot off of network card and then do all the batch/setup diskless and use a virtual ramdrive to store all the stuff!! :slobber:
[/edit]

Quote:

---

Originally Posted by lv_dicedealer

Did I follow that guide, you can run d2ol diskless ????
If so , I wanna know how to do that. Would everything then be stored in physical memory? Could you setup a vitual ramdrive with enough physical memory and run without a drive of any sort?

[edit]
Guess I did read it wrong, too much soma.
Anyway, would be cool to boot off of network card and then do all the batch/setup diskless and use a virtual ramdrive to store all the stuff!! :slobber:
[/edit]

---

by following what i wrote up there u wont be able to boot disklessly that is much more complex, but the reason i migrated to linux was to be able to boot disklessly. in order to do this u need a linux server, and you dont even need a ram drive, you can mount a folder remotely on another comp as your hard drive so if theres a power outage u dont loose any cands :D i just recently set this up n its
pretty cool for more info check out the my thread at free-dc.org, PCZ over there was nice enough to walk me through setting it up. http://www.free-dc.org/forum/forumdisplay.php?forumid=6
when its fully setup, add new nodes is quite easy and your only need a mb with built in NIC, cpu and ram. so its cheap cheap cheap! :toast:

How do you boot into Linux on the slave nodes? From a CD?

You should use PXE to boot over the network, then NTP to mount your / to a ramdrive. After that, it's all gravy.

Quote:

---

Originally Posted by smokey

You should use PXE to boot over the network, then NTP to mount your / to a ramdrive. After that, it's all gravy.

---

Is it all automated... or do you have to go through a bunch of BS every time you need to reboot or power down?

Define 'a bunch of bs'.

ok.. I turn on my XP rig... it boots... goes into windows... and D2OL starts...

All I have to do is press the power button...

Diskless Linux setup??

you dont need a cd, i do it through pxe check the link i posted for more info ^ all your need is a mb with nic, cpu and ram, nothing else not even video once its setup. just hit the power button and it boots up n starts d2ol :D that being said it took me awhile to get setup right. i have it setup so u can telnet each client (log on to the client over the network) and check to make sure everythings working. very cool stuff :cool:

telnet :eek: ! Even if it is just a small LAN, ssh, ssh, ssh. It's faster, smarter and stronger, so to speak.

@Rodzilla: There really isn't much to it. You can achieve the same effect through Windows, though the optimizations for a 'dumb' client are not there... PXE booting is simple and there seem to be at least two of us around that would be willing to help you (and anyone else along).

would be nice to be able to boot off a cd or something, saves on hard drives :D

Why do you need a hard drive? See: http://www.kegel.com/linux/pxe.html

Nice work!! Time for me to actually get it working on linux. Wait... does this apply to all distros??

smokey - thats what I was saying.. it would be nice because then we wouldnt need HDD's.

Quote:

---

Originally Posted by shadowing

Nice work!! Time for me to actually get it working on linux. Wait... does this apply to all distros??

---

i pretty sure this applies to all linux distros, BSD might be different but i havent tried bsd so i dono, what distro do u have?
ive loaded d2ol on fedora and slackware with no problems using the steps above

PXE is distro-agnostic. In fact, it is even OS-agnostic, to a certain extent. Remote booting can be configured on Windows/UNIX/Mac/Linux machines. All of these operating systems support thin clients, and can be configured as such. The great advantage of using BSD/Linux for this is that the GUI and all of the unnecessary bloat is optional. Really, the only thing that needs to be running is a network interface, a terminal daemon (ssh or telnet) and the kernel/system loggers. Pretty much everything else commonly found on a system is just gravy in this situation.

i think shadowing was asking about the guide to loading d2ol, but i agree 100% ^ :D i use ltsp 3.0 and clients boot in text mode and automatically start d2ol as a service and using nohup i apend the progress output to a text file :cool: it works quite nicely, i set it up with the help of PCX over at free-dc (i already posted the link) its very minimalist which i like, im also looking into openMosix which is a clustering software, which if i can set it up basically makes a lan into one big supercomputer so you could have 11 nodes and one server u start 12 instances of d2ol on the server and it automatically spreads the load over the networkl so each comp does one instance, this can be done with a bootable cd or pxe boot, it looks pretty cool. smokey u ever try this? know anything about it?

LOL. That's my job. I'm a sysadmin for a small compute cluster. The cluster is running Solaris, since it has the best performance on a small cluster environment. Other Unixes and Linux are fine for it too, but they don't begin to pull ahead on the performance until the amount of cpus goes over 1000, or so.

[EDIT] Just some linkage for you: MPI. [/EDIT]

I'm using Knoppix. I can't find any other distro's I like.

1000 cpus :slobber: thanks for the linkage ill check it out.

shadowing if ur looking for a good beginner distro, try ubuntu its very newbie friendly but also stable and robust, only thing i dont like about it is it come with gnome and no support for kde. i use slackware myself, i just finished a couple days of distro testing and im reloading slackware 10.0 as we speak great stable distro, lil tough for beginners tho. xandros is supposed to also be a good newbie distro, i dono much about it tho and have never tried it....

:toast:

Quote:

---

Originally Posted by chipmonk010

1000 cpus :slobber: thanks for the linkage ill check it out.

shadowing if ur looking for a good beginner distro, try ubuntu its very newbie friendly but also stable and robust, only thing i dont like about it is it come with gnome and no support for kde. i use slackware myself, i just finished a couple days of distro testing and im reloading slackware 10.0 as we speak great stable distro, lil tough for beginners tho. xandros is supposed to also be a good newbie distro, i dono much about it tho and have never tried it....

:toast:

---

Thanks for the noob info! :up: :D now maybe I can give linux a go!

If I were you Rodzilla, I would just 'suck-it-up' and install Slack. We can help you solve any problems you have along the way. I feel like it's best for you, in the long run, to just sit down and start learning Linux. Stop searching for an 'easy' way out. Remember, we're here to hold your hand, if you need it, too. ;)

Is there a Distro that was the clear advantage with D2OL? Something that will make it fly?

I would think It'd be a distro that is simple not complicated. I'm trying out Slackware and Knoppix to see which one would be better.

There really aren't any noticable performance differences between Debian, Slackware and Gentoo. I would suggest using the Con Kolivas kernel patchset, if you'd like to eke some extra performance out of a _server_.

im not sure about d2ol but overall the speed difference between slackware and say fedora 3 is very noticable. its not really hard to install, im sure u could do it Rodzilla, it just becomes more complicated to customize settings since theres no gui tools for such things, personally thats the way i like it so it doesnt bother me. i say give it a shot like smokey said we can help ya out if u have trouble.

smokey what distro do u use? slack or somethin else?

I am currently running Slackware 9.1 on three rigs at home, and one NetBSD 2.0 server. I haven't felt the need to upgrade the Slacken Boxen to 10.x, and my laptop is the only one running 2.6.x, but if you're running a server full-time, I would _definately_ go with the 2.6.x-ck patchset.

Re the clustering: before you get too far into clustering, you should know that it is very expensive in both time and money. Clustering tends to bring out the worst in hardware and software and exposes _any_ manufacturing flaws that may be present in either. Hence, it is highly advisable to keep spare hardware around and have the source available for whatever software is running for debugging purposes (that's why most clusters don't run commercial OSes ;)). Also, if you're planning on needing X number of clients, you should increase your cluster size by 10-15%. This will account for all the downtime for hardware swapping that will occur, and it _will_ occur. A cluster is very fruitful and a helluva a lot of fun, but it takes a bit of work to build and maintain.

Quote:

---

Originally Posted by smokey

I am currently running Slackware 9.1 on three rigs at home, and one NetBSD 2.0 server. I haven't felt the need to upgrade the Slacken Boxen to 10.x, and my laptop is the only one running 2.6.x, but if you're running a server full-time, I would _definately_ go with the 2.6.x-ck patchset.

Re the clustering: before you get too far into clustering, you should know that it is very expensive in both time and money. Clustering tends to bring out the worst in hardware and software and exposes _any_ manufacturing flaws that may be present in either. Hence, it is highly advisable to keep spare hardware around and have the source available for whatever software is running for debugging purposes (that's why most clusters don't run commercial OSes ;)). Also, if you're planning on needing X number of clients, you should increase your cluster size by 10-15%. This will account for all the downtime for hardware swapping that will occur, and it _will_ occur. A cluster is very fruitful and a helluva a lot of fun, but it takes a bit of work to build and maintain.

---

Why would it cost so much to do?

The cost is entirely in the hardware. As a node dies, you'll need to replace the hardware with new materials. You can expect that in a cluster of 10 nodes, one node will always be down for hardware failure. This is not an unreasonable or uncommon figure.

But I have 10+ PCs running and my hardware never dies... why would running them as a cluster kill the hardware :confused:

Perhaps you're just lucky. ;)

Quote:

---

Originally Posted by smokey

I am currently running Slackware 9.1 on three rigs at home, and one NetBSD 2.0 server. I haven't felt the need to upgrade the Slacken Boxen to 10.x, and my laptop is the only one running 2.6.x, but if you're running a server full-time, I would _definately_ go with the 2.6.x-ck patchset.

Re the clustering: before you get too far into clustering, you should know that it is very expensive in both time and money. Clustering tends to bring out the worst in hardware and software and exposes _any_ manufacturing flaws that may be present in either. Hence, it is highly advisable to keep spare hardware around and have the source available for whatever software is running for debugging purposes (that's why most clusters don't run commercial OSes ;)). Also, if you're planning on needing X number of clients, you should increase your cluster size by 10-15%. This will account for all the downtime for hardware swapping that will occur, and it _will_ occur. A cluster is very fruitful and a helluva a lot of fun, but it takes a bit of work to build and maintain.

---

so clustering stresses components, much like folding does basically, im not surprized, to tell u the truth i havent made much progress with it yet im still trying different distros and screwing up linux installs...lol just loaded freebsd on a box, gonna mess with that a little maybe make it the server for the cluster. we shall see.
i was gonna try n use open mosix for clustering anything else out there thats better? (and free) since this is just for fun i dont have any actual need for it lol.

I decided to try out three distro's in the end. Slackware, Gentoo, and Knoppix. I'll see which one is the fastest.

'fastest' meaning... what? And are _you_ doing the tuning? Not to imply that you are not capable of optimizing any particular distribution, but comparing one distro to the next (especially when custom kernels ala Knoppix) are involved, and extrapilating useful, applicable results is, at the least, near impossible. :rolleyes:

Just for the 'heads up' Knoppix is going to be slow-as-heck compared to _any_ hard disk-based distro. Even when you do a HDD install, it's still got all of the ramdrive and rare hardware tweaks that aren't necessary and slow the ship down.

Of course, they are all hard drive installations. I just realized I can't compare speed. Heck, i don't think there will be much of a difference.

If there is any noticable output improvement from the linux/cluster platform, since I am basically a m$ slave, could someone makeup an ISO for me or any other team member that is in the same boat (the m$ one with no drain plug) to use to install the server and the nodes?

Which distro? There would probably be no difference.

NOTE: I have the Slackware 10.1 distro ISO, Gentoo EXPERIMENTAL AMD64 DVD iso, Gentoo STABLE AMD64 CD Iso.

Quote:

---

Originally Posted by shadowing

Which distro? There would probably be no difference.

NOTE: I have the Slackware 10.1 distro ISO, Gentoo EXPERIMENTAL AMD64 DVD iso, Gentoo STABLE AMD64 CD Iso.

---

I was reffering to a difference in output going from m$ to linux. And is there a special distro for Xeons? (See, I am the linux n00b)

What I really wanted to know was if someone could make an ISO that would be an unattended install that also would install D2OL (plug 'n play linux, if you will)?

I dunno if anyone can do that. We'd probably have to ask one of the distro's developers to help us in that.

By specific distro for Xeons, it would only mean the itanium's and 64 bit ones.

Umm... don't confuse the question, shadowing. There is no specific distro for any x86 hardware. They all run the same kernel and utils, so it's really just a matter of tuning the kernel _for_ Xeons, once it's installed. The 'Big Distro Choice' is more one of personal preferance than performance issues.

On the building an ISO issue, it's possible, but not really feasable to do so. I would suggest that you just download one of the major distros default ISO, and we will hold your hand through the installation and setup.

Quote:

---

Originally Posted by smokey

Umm... don't confuse the question, shadowing. There is no specific distro for any x86 hardware. They all run the same kernel and utils, so it's really just a matter of tuning the kernel _for_ Xeons, once it's installed. The 'Big Distro Choice' is more one of personal preferance than performance issues.

On the building an ISO issue, it's possible, but not really feasable to do so. I would suggest that you just download one of the major distros default ISO, and we will hold your hand through the installation and setup.

---

I think a Distro ISO that we could distribute to the team would me major! Can someone make this happen? :D Might be something in it for you ;)

I think the only place you may notice a difference is with a 64bit cpu and a 64bit OS. That might help in an advantage, but I dont own a 64bit CPu yet! I think this is worth looking into.

@Rodzilla: Yeah, it would be so major that it would require it's own production cycle and development team. Creating a distrobution isn't something one guy can just do in a weekend. Hence the reason that Slackware-like situations aren't common by the furthest stretch of the imagination.

@mod: 64bit computing does provide higher numerical accuracy, and may allow for a better crunching drone, I agree. Somebody with Linux experience and a 64bit CPU should give us some numbers. I would, but the most recent 64bit CPU I played with was an Alpha. :P

I didnt know they still make alpha's. I looked into a 64 bit system, and mite be able to get one together by June something (birthday's in may) so I'll try that out then. What 64 bit OSes are out there. I know Windows Xp 64 edition, and one of the suse linux versions. What else is these? Also, since D2OL is a 32 bit app, would it maybe not be able to take advantage of it?

Umm... HP bought DEC and killed the Alpha. I was referring to the amount of time that has passed since I last played the 64bit game. Also, I don't personally run D2OL, so I'm not sure about whether it can take advantage of the numerical advantages of 64bit computing. Once again, this is something for someone with an available CPU to play with. Anything, in theory, can be crosscompiled into a 64bit ELF on a 32bit machine, thus any distro could be ported to 64bit. So far, I'm aware of 64bit versions of SuSE, LFS (obviously) and Gentoo. Slack64 is a work in progress, but has reached the point of being beta tested through ##slackware frequents on IRC. Since Pat has been under the weather, development toward that sort of thing has taken a backseat to security and stability updates to the 10.x series.

And then, there's also Fedora Core that supports 64. Mind you, for Gentoo, it is very complicated and takes a long time to set up 32 bit apps, or installation in the 64 bit environment. :(

Quote:

---

Originally Posted by smokey

@Rodzilla: Yeah, it would be so major that it would require it's own production cycle and development team. Creating a distrobution isn't something one guy can just do in a weekend. Hence the reason that Slackware-like situations aren't common by the furthest stretch of the imagination.

@mod: 64bit computing does provide higher numerical accuracy, and may allow for a better crunching drone, I agree. Somebody with Linux experience and a 64bit CPU should give us some numbers. I would, but the most recent 64bit CPU I played with was an Alpha. :P

---

You misunderstand... I mean use an existing distro that is on a CD with the D2OL Agents all set to run (Maybe minus the node.prp) so you just pop the CD in or copy it to a HD...

Quote:

---

Originally Posted by Rodzilla

You misunderstand... I mean use an existing distro that is on a CD with the D2OL Agents all set to run (Maybe minus the node.prp) so you just pop the CD in or copy it to a HD...

---

just download the gui-less version with JVM un-tar it and its ready to go just copy n paste, no install required and it can run with or without the gui. the folder could easily be burned to a cd. im on my way out the door but later ill see if i can put an iso together or something :D

make sure to check if crunching under a 64bit OS on a 64 bit CPU increases cands a day.

Quote:

---

Originally Posted by moddolicous

make sure to check if crunching under a 64bit OS on a 64 bit CPU increases cands a day.

---

The app isn't 64 bit... so it won't even run will it? If it does... I'm sure it won't help.

.:EDIT:. and by app I mean D2OL...

Ah... the disadvantages to a closed-source system...

Thread bump here, but, I got D2OL working on Linux at last, after trying to work out why the installer's inbuilt JVM wouldn't work. Running it on my P4EE, running Gentoo 2005.0, with ~x86 testing packages, gcc 3.4.4, kernel 2.6.12-nitro5 (Nitro-sources patchset, basicly, Con Kolivas plus lots of extras including some patches from the Andrew Morton patchset, extra patches donated by Con, Genetic libraries, Genetic-Anticipatory IO scheduler, etc etc). I'm not overclocked TOO much, atm I am running at 4.4GHz. It's kinda terrible by my standards, but ah what the heck, it's 24/7 stable :) If I went any higher I would have to reboot every day or two to stop it from crashing. I am not at home now, so I can't check on it. I hope it's still up, but I haven't produced any candidates today, so I am kinda worried.

how is all those patches going to hwlp you on a closed source jre and a closed source d2ol client, sounds like a case of ricer syndrom ;) theyre going to do so little its not worth the instability imo

Just cause the client is closed source, it doesn't mean that I can't gain some extra candidates from a faster kernel. The patches have been proven stable. The CK patchset is safe to be used in a production enviroment, and the genetic routines are also stable, as are the Andrew Morton patches. It does also contain some experimental stuff such as Software Suspend 2, and Ricer4 (eww), but I don't use that ;) ext3 + writeback + b-*trees all the way! I've never had any instability with this patchset.

All right, I'm making my brothers old comp into a 24/7 cruncher with gentoo as the operating system. Only a couple of probs:
1: Will Linux run with ONLY 128mb ram, or do I have to buy a 256 stick?
2: Will linux install on a 2gig hard drive, or will I have to buy a new one as well?
No, WinXP is not a choice because if I always use it I'll never want to use linux. This is a perfect opportunity.

Gentoo will run on a system with 64MB of RAM, but if you wish to have it crunching, 256MB would be recommended. As for HDD space, that won't be enough if you want to crunch on it. Maybe 4GB.

alright, so I need to get a cheapo 256ram and a 10gb hard drive. Will that work?

Probably yes.

I hate to resurect an old thread, but if anybody has problems running D2OL in linux, it is probably due to a missing library, available at http://home.nyc.rr.com/computertaiju...6.2-2.so.3.bz2. It is bziped. In my case, D2OL would complete one candidate, then java would slowly take over 100% of the cpu leaving DockWin none. This fixed that problem. I would suggest all you linux users check cpu usage via top for like 2-3 days after you initially start running D2OL to make sure 99% of the cpu is still going to DockWin and not java. Also, if you are running with controller:cli, there should be NO java processes running at all. I use D2OL in ubuntu and arch linux, both needed the library listed above to work correctly. Arch is very nice because it is so small, boots fast (if you have to shutdown a lot) and is pretty stable. No major issues. Ubuntu is a little larger, but it runs on my desktop, which I know can handle it.

Considering the fact that D2OL is multithreaded, I think a cluster would be pretty cool. As soon as I can afford it (like a couple of weeks) I am going to start setting up a debian cluster with openmosix, and FAI, so I can just plug in a new node and everything will automatically install. When I start D2OL, all the processes should migrate. I haven't done a whole lot of cluster stuff, like none, but I think I should be able to figure it out. If I end up getting it done I will post my progress here.

Has anyone tried to run D2OL on a cluster yet?
If yes, how did it run?