FAH Setups

Quick-find
Post #1 --- CPU Client Console guide - jimwah
Post #2 --- CPU Client Service guide - SparkyJJO
Post #17 -- GPU Client install guide - SparkyJJO
Post #22 -- VMWare 64-bit Linux SMP guide - jimwah
Post #23 -- Managing Folding Clients across a Cluster - angra
Post #38 -- Simple, Non-service, Linux Client guide - jimwah
note - Authors will not take responsibility for any hardware/software failures that may occur following these guides, install & configure at your own risk - etc etc

So I'm thinking we should post up how we set up our folding rigs, since we get asked quite a bit, and I'm curious to see how others do it. This is not the only way to set up folding@home, this is simply how I tend to do it.

I run the majority of my rigs on Windows2000 [I only hang a CD/DVDrom temporarily to install OS] running VNCserver, this way I can boot them with a monitor & keyboard/mouse, then unplug the lot [apart from network of course] & log in from my main machine on the network [100mbit switch hanging from a 4-port ADSL router & DHCP enabled] from then on.

folding@home client: the jimwah <non-service> console method
[typically used to set up the standard client on windows machines, to start & stop folding manually]

- Make folder C:\fah\one\
- Download Folding client to this folder (from http://folding.stanford.edu/download.html)
- Create shortcut to the FAH_Console.exe on my desktop, rename this shortcut Core1, just for clarity
- **Optional** Specify any special flags in this shortcut (see Console Client Configuration Options here> http://folding.stanford.edu/console-userguide.html) as shown below:

http://i3.photobucket.com/albums/y68/jandz/shortcut.jpg

- Double-click shortcut & run through initial configuration as below; crucial parts are the username of your choice, and the team number 36362!

http://i3.photobucket.com/albums/y68.../FAH_setup.jpg

Note - I only tend to enable the enable Large WU's over 5mb option on machines with at least 256mb per core - again, it's just the way I do it. I also don't do a service install, I like to see the client chugging away :)

..And that's it, you have a core folding for team XS. Simply click the X top-right of the title bar, to close the client, and double-click your shortcut to restart the client where it left off.

..time for a celebratory drink :toast:

Multiple Core Systems
- Repeat from beginning for each core, next time creating C:\fah\two\ folder, and naming the desktop shortcut Core2, and finally altering the Machine ID option to 2 in the configuration shown above. Then C:\fah\three\ folder, shortcut Core 3, Machine ID 3, etc.. for further cores.

Remote Monitoring
- I also share the C:\fah\ folder from each machine, and ensure all machines are on the same workgroup, this way I can point FAHmon at the shared folder of each machine [eg \\tomcat\fah\] & monitor the WU's progress. This is something I've let get a bit untidy recently & need to reconfigure :slapass:

If you've got a way that makes sense to you, or how you set up your GPU client, or even how you got the SMP client running in MacOSX, post it up & let everyone have a look. Considering the recent spate of new folding clients we have added into the mix, folding can look like a complicated business from the outside - lets make it a bit simpler if possible :up:

1. Download the Windows NT/2000/XP "No nonsense" text-only console (I'm assuming you are running windows).
I only use the console version, the GUI takes extra system resources IMO.

2. Make two folders, one for each client and put one FAH504-console.exe file in each. Let's assume you have C:\FAH1 and C:\FAH2 for example for the two folders. If you have only a single core CPU, just make one folder.

3. Start menu > Run > Type in "C:\FAH1\FAH504-console.exe" -configonly
include the quotes!

Here are the settings I use:
> User name: your name
> Team number: 36362
> Launch as a service: yes
> Ask before fetching/sending work: no
> Use IE settings: no
> Use proxy: no
> Allow 5MB or larger WU's: yes
> Change advanced options: yes
> Core priority: idle
> CPU usage requested: 100 (you can set this a little lower if the system sems to slow down much, but I've never noticed a slowdown)
> Disable highly optimized code: no
> Pause if on battery power: no (I don't run it on any laptops so this doesn't matter much)
> interval between checkpoints: 5 (default is 15, use whatever you prefer)
> Memory to indicate: full amount
> request WU's without deadlines: no
> set -advmethods always: yes
> ignore deadline info: no
> Machine ID: 1

4. If you have a dual-core CPU, repeat for the second client in C:\FAH2 and use the same settings except have Machine ID set to 2.

5. After that, go to Start menu > run > regedit. Do a search for -svcstart, this will bring up the first client service command line. Edit that line, and after the -svcstart I add -local -advmethods -forceasm
the -local is very important for running multiple clients on one computer, it makes sure they don't get confused at all. the -advmethods just makes sure that it is turned on, -forceasm is good in case you have a system crash (like a power outage) it won't allow the client to disable SSE optimizations when restarting from the improper shutdown.

6. For a dual-core CPU, continue the search for the -svcstart and on the second client command line in the registry add the same stuff.

7. Exit the registry editor.

8. Reboot and you should have the client(s) running as a background service.

For monitoring the clients I use FahMon


Some extra stuff:
All my systems run WinXP, and I have remote desktop enabled so I can access them that way if I need to. Also I have shared each of the FAH folders so I can monitor them with FahMon.

On my main system I have priority set to idle as you can see above, but on my dedicated folders I've changed that to low. Also, under My computer > Properties > Advanced tab > Settings under Performance > Advanced > Processor scheduling I have it set to optimize for the best performance of background services. I'm not sure if that has done much of anything for the folders but it can't hurt. Just DO NOT set that on your main rig, it can slow stuff down bigtime!
http://i58.photobucket.com/albums/g2...rfsettings.jpg
http://i58.photobucket.com/albums/g2...scheduling.jpg

Great stuff SparkyJJO, some good tips in there :woot:

Thanks!

And I've been using Hamachi to create virtual-LAN and to monitor remote computers... just share that fah-folder...
...naturally with "goodish" firewall config.
BTW: using always console as a service...

jimwah & SparkyJJO;

Great work...:toast: :toast:

What's the significance of these 2 settings?

request WU's without deadlines: no
> set -advmethods always: yes
Thx

The dealines setting is less relevant at the moment, since I don't think they have any WU without deadlines - but these were handy for very old machines, or machines that would only fold for 30mins a day or something; so they had a chance of getting work done, where with a normal WU they'd run over the expiry date.

Setting -advmethods means you're eligable for new/experimental work units, sometimes these include bonus points over standard WU; but again, this seems less relevant at the moment as everyone is bombarded with the same units regardless of settings :shrug:

:D

From my experimentation recently with disabling -advmethods I found it made things worse. I started getting amber cores. Enabling it again got me some tasty 1495, 1166, and 1167 WU's. Seems that the 2124/2125/2126 WU reign is coming to an end for a little while at least :D

I have a couple questions. I am new to F@H and have joined your team. When a person uses a dual core system, are they doing one WU per core or is it 2 instances of F@H using the same WU?

And also when a person shares the folder on their network so that they can get more CPU power do they too have separate WU's or are they sharing the same folder and helping complete the first WU form the original computer?

Also I have the F@H_core program running as a service in the background taking up idle CPU time, which I don’t mind at all, just curious I don’t need to open the actual program too, do I? Or is it just doing its thing as a service now and that’s good?

Also if I know I am going to step away from my computer for 2+ hours and I set the priority of the service in Task Manager to High from Low, will that make a difference?

1. You have 2 instances of F@H, and they each work on their own WU's. Sometimes the 2 cores will be each working on their own copy of the same WU, but they are still going to be doing 2 separate WUs.

2. When you run F@H on another computer it is another WU. Sharing the folder just allows a monitoring program on one computer to monitor the other as well. For example I have 8 computers running F@H right now, and I have 9 separate WU's being crunched (my main computer is a dual core).

3. I am assuming you are using the text-only console version of the F@H client. When you run that as a service you don't need to run anything because it automatically starts when Windows starts.

4. I don't recommend messing with the priority in task manager, because setting it from low to high can freeze up the computer. I messed with it once and it wasn't pretty :p: It won't make much of a difference anyway because when you aren't using the computer it is using 100% CPU, so whether it is set to low or high won't affect that much when the system isn't being used.

Hope that answers your questions :toast:

When I run the program you use FahMon, it says that currently the folding as you call it is stopped or red. Then I open the client (console version only) and it shows Orange, but if i close the console its back to red (stopped) even though the service is running in the task manager.


Also im working on project 2124 ><;;

Right click on My Computer and hit Manage. Then go to Services and Applications > Services, and see if in the list there is something that starts with FAH@.... If there is then try rebooting and see if FahMon shows it as being active again. If it isn't in there, run the console file with the -configonly flag, and tell it to install as a service. Then follow steps 4-6 in my guide above.

When it is running in task manager there should be the console file and a core file.

My Install was bugged when I switched directories, I saw it when I looked at the service. I ran configonly again and now im restarting to see if it fixed it.

I fixed it, but in FahMon it no longer says the Core Name that is used, although all other info is correct and the light is orange, so im assuming its working correctly.

Nevermind, I had to reset my preferences again and set it to update the project databases in FahMon. Everything is workign great.

Anywho this is only a test cause after x-mas i will have a very (hopefully) high overclocked C2D E6400.

glad to know it is all worked out SeeKnaY :) That new CPU will be of help too :D

Can someone write out a guide for the GPU install, because I have a feeling thatin the future we will get many more GPU's than CPU's (IMO).l

I could probably put up a basic one in a few days time when my replacement PSU comes through - my GPU WU has probably expired by now anyways so I wont potentially be losing any folding time by setting up a new client & screenshotting the process etc. If anyone wants to step in, in the meantime, feel free. I myself could do with a clear guide on the overclocking aspect of GPU folding.

I got it jimwah :)

Known working drivers are the Catalyst 6.5, 6.10, 6.11, 7.2, 7.3, 7.4, and 7.5. If one version doesn't seem to work for you, try another one of these. Don't bother with any others because they won't work at all, or they won't work as well.

1. Download the GPU client console version from Stanford's site.

2. Put the file where you want to run it, i.e. C:\FAHGPU

3. Go to run and punch in "C:\FAHGPU\FAH5.91beta2-console.exe" -configonly (don't forget the quotes, and the filename will change with new versions)

4. Here are the settings I used that seem to work well:
> User name: your name
> Team number: 36362
> Launch as a service: no (this client does not support service mode)
> Ask before getting work: no
> Use IE settings: no
> Use proxy: no
> Allow 5MB or larger WU: yes (not sure if this affects anything)
> Change advanced options: yes
> Core priority: idle
> CPU usage requested: 100
> Disable optimized code: no
> Pause if battery used: no (shouldn't matter, I don't know of any laptops with X1900's)
> Interval between checkpoints: default
> Memory to indicate: full amount
> request WU's without deadlines: no
> set -advmethods: no (again, not sure if that affects anything)
> ignore deadline info: no
> machine ID: normally 1, but anything different than any other clients

5. Create a shortcut someplace - like on the desktop so you don't forget to start it ;) - and put in "C:\FAHGPU\FAH5.91beta2-console.exe" -local as the command line.

6. Run it!

Notes:
1. You MUST have a whole CPU core dedicated to the GPU. This is just a limitation at this point and there isn't much we can do about it. It is possible this may be reduced in the future but I'm not sure.
2. If you start up a game it'll automatically kill the client, so you'll have to start it back up afterwards. I've noticed that sometimes it won't kill the client, if it doesn't you'll be able to tell because the game will run like crap ;)
3. My X1900XT with no tweaks would do a WU in about 14 hours, or about 565 PPD. The X1900GT will be slower because it has 12 fewer pixel shaders which is what the client uses to fold.

The latest version of ATItool forces the card into 3D mode, which can help you complete WU's much faster. Using it got me down to about 12 hours for a WU, but the downside is the fan is running faster (more noise) and the card gets MUCH hotter. I used the fan control in ATItool to set my fan at 55% which was noisier but kept the card in the 70's, only a few degrees warmer than before. Of course you can use ATItool to overclock but MAKE CERTAIN that it is FULLY STABLE otherwise you'll end up with system crashes, EUE's, or both.

Quote:

---

Originally Posted by SparkyJJO

I got it jimwah :)

---

:up: Ask & thou shalt receive, cheers Sparky. Added a kind of contents to the original post to point to guides.

bump :thumbsup:

Thanks for the bumpage sc00p, I've done a few succesful units in VMWare over the last few days, & written a guide, which I'll post up when I get back from work. Hopefully get a few members on these 1700 point units :D

Anytime m8 :)

But p2604&-05... whooah :slobber: Just noticed you mentioned 1760/wu :party3: :hump: :shakes: :hitself:

This is a guide to setting up VMWare server to run a 64-bit linux OS, from within Windows; allowing us folders to run the 64-bit Linux SMP (ie, multi-core/multi-CPU) client without the dangers/hassle of a full linux install or dual boot.

Why?
Because the SMP folding client can provide greater points yeild for contributors like you and me, & also helps Stanford run different projects at a greater speed than the standard windows client alone.

You will need...

• A dual core, 64-bit enabled CPU - with VT enabled on Intels (eg C2D e6x00, AMD x2, Pentium D 9x0) Check compatibilty using Tools provided at VMWare if unsure; I personally have only used C2D's & have only read about the usability of other CPU's so far. Recommend 1Gb RAM imo.
• Windows installed.
• VMWare Server installed http://www.vmware.com/products/server/ (register to receive serial codes for this product, which is free & unrestricted) :)
• A copy of a 64-bit Linux OS, of your choice, on bootable media (eg CD, or even as a raw ISO).
• Some time, a couple of hours should cover it, depending on your familiarity with VMware, Linux, and the folding client.
• I would suggest a reasonable DSL-like internet connection, as these are large units & the upload can take some time on slower connections.

VMWare Setup
So you've got the serials & installed VMWare, now we need to configure the Virtual PC that our linux distribution will run in. VMWare Server is generally very straightforward to use, create a Virtual Machine via the wizard, hit play to turn it on, and install an OS as if your were sat in front of any normal PC :) Create a new Virtual Machine and follow the steps to suit your needs. I used the following options:

• Created a hard disk of 10Gb, allocated space now (this took a few minutes).
• 512Mb RAM Available to the Virtual Machine.
• Bridged Networking - enables the Virtual machine to gain a DHCP IP like any real machine on my network. ** NAT is recommended if you have issues with Bridged Networking, or if your PC is connected direcetly to the internet. **
• CDrom enabled, no floppy or sound enabled. You can also mount an ISO image here instead of your PC's optical drive.
• 2 CPU's available to the Virtual Machine, we certainly need this option selected.
• Selected a suitable OS, to match the linux OS that is being installed.

Here's some pics of the interface, for reference: [OS Selection Options][Networking Options][System Summary]

Fire it up
So the Virtual Machine's configured, time to fire it up & get the linux install going. Insert the linux OS disk (or mount the ISO image as a CDRom, in the VM's setup) and press the Play button on the VMWare interface to power on; click within the window to get your mouse/keyboard 'inside' the Virtual Machine (press Ctrl-Alt to release them back to Windows at any time). Hit F2 as the VMWare POST screen flashes up & check the BIOS boot order is correct, picture here [VMWare BIOS] Save & Exit the BIOS.

OS Install
After the restart I'm greeted with the Redhat FC4 installation screen [Linux Install from CD] and can begin this install just as I would on a real PC. Linux installers these days are largely self-explanatory, and well documented, so I won't bore you with too many details here; and of course the steps will vary depending on what 64-bit distribution of Linux you opted for.
I accept the defaults for almost everything during the install, except that I set my hostname manually to anything apart from the default, and ensure that I have the 'Development Packages' 'Compatibility Support' 'Admin Tools' package groups installed in Fedora Core. If you see an option for 'Compatibilty' or '32-bit Libraries' I advise you install them, based on this thread from FAH Forum.

Time to Login
The install completed, the machine booted & is now waiting for a login [Login to Linux], so login as root or the non-root user you created as part of the setup and do some exploring in your new OS install :) At this point I also spend some time stopping services such as sendmail, ssh, cups for printing, etc (I just type ntsysv in a terminal, or find the relevant GUI in Fedora)

You may need to install the VMware tools, Fedora appears to include drivers for running inside VMware, but other distributions may not. There's an option in the VMWare Tools drop down iirc, that will effectively mount a driver disk as a CD in your linux OS - from which you can install the tools/drivers (needed to get high-res display and more responsive mouse etc).

Find and open a Terminal, and if you couldn't login as root, type su and enter your root password [Terminal Open].Now you can configure the SMP-Client from Stanford &, with a bit of luck, start folding some monster units :D

SMP Download & Configuration
Some installation details can be found at the bottom of Stanford's page, the SMP FAQ. I simply run the following commands in a terminal:

mkdir folding/ make directory called folding
cd folding change directory
wget http://folding.stanford.edu/release/FAH_SMP_Linux.tgz download SMP Client
tar -xzvf FAH_SMP_Linux.tgz unzip SMP Client
./fah5 -local -configonly run client configuration *Enable large units over 5Mb*
echo "./fah5 -local -forceasm -verbosity 9" > FAHSMP make a script to run the client with desired flags, -advmethods can cause problems apparently, but has worked fine for me
chmod +x FAHSMP make that script executable
./FAHSMP run the script & get folding!

To shutdown the folding client, highlight the terminal window & hit Ctrl-C (this is also good practice on windows boxes I understand, for a clean shutdown).

Notes/Problems:
SMP - Check Stanford's Linux SMP forum for help with the client.
Linux - General Linux problems are best Google'd for, and there's a few Linux experts around here too somewhere I'm sure.
Distributions - Custom made distributions are available for 64-bit folding purposes apparently, and possibly even ready-made VMWare image files - I might look into over the coming weeks, a pre-made XS 64-bit linux VMWare file <100mb would be very useful tool imo.
Performance - Running Linux inside VMware is rumoured to cost between 10%-20% in terms of performance hit, compared to running it natively; not too bad in my opinion.
Disaster? - Remember that anything that goes wrong within the Virtual Machine is no big disaster - at worst you can delete the VM & start again.
VT - Disabling VT (Vanderpool?/Virtualization Technology) on my E6400 seemingly ruled out any chance of running a 64-bit OS within a 32-bit Windows environment - so do not disable it! Once off, it may need a hard reset to re-enable, and simply locked out the option on one of my boards completely.
Networking - VMWare adds some interfaces to your list of network connections, like so [VMWare interfaces] which it uses to get a connection for the VM's. The SMP client works on big units, which can be slow to upload for me (1mb down, 256k up) usually taking about 5 minutes & lagging internet usage quite severely. Depending on the number of machines folding SMP & their speed; and on your internet usage & connection, this may or may not be an issue for you.
CPU Utilisation I'm noticing CPU usage of around 90-92% in windows currently, I believe the usage is higher currently on the larger units (1760pts) and falls around the 80% mark for the smaller units (587pts).
Time Sync I've noticed that an OS running inside VMWare can sometimes get it's clock out of sync, for example, mine was sometimes 30mins ahead after running 24hrs roughly. I changed the clock properties to sync with an NTP Time server, which seems to have solved the issue so far.

Conclusion
I'm only just starting out on the VMWare method myself, and have yet to encounter any real problems. Is it worth it? Well its hardly apples Vs apples, but here is the situation on my e6400's at the moment, by my rough estimations...

• 17mins/Frame - Project 2604 in VMware (e6400@2.8), over 1K ppd (machine is down a few hours each day)
• 12mins/Frame - Project 2605, native Linux 64 (e6400@3.2), approaching 2K ppd
• And my work machine (e6400@stock) crunches normal FAH windows units, yesterday 2x3038's for a total of 300-400ppd.

Hope I didn't bore you to death with all that, good luck if you choose to go this route, and as always, fold on :up:

edit - altered known compatible CPU's, Intel 9x0 should be ok, 9x5 exclude VT feature. Appended NAT Networking for VM setup.

PLACEHOLDER

by making this post, I am committing myself to explaining how my setups are managed for a distributed memory/Beowulf-style cluster configuration. This will probably be of interest to exactly zero other XS users, but maybe it will have some useful info. I will probably need some advice and/or prodding to make this into a useful post, so please ask questions and make suggestions. I'll try my best to get this into a visually appealing format.

For those who do not know, I have access to a Beowulf cluster at work, and when we aren't fully utilizing it for its primary purpose (high performance computing software methodology research), I have it fold. My system consits of a pool of compute nodes that are on a private network, and one "front" node thatis on the private network and also exposed to the internet. The front node exposes several NFS file systems, including the user home directories. The compute node disks and setups are treated as disposable, so no state information is ever kept directly on the compute nodes. My configuration has each machine operating in a separate, globally visible directory, and each client running under "screen" on the host node. This allows me to keep tabs on everything from the main login node, and makes it easy to reattach a terminal to any one of the clients without too much hassle. I also run "ganglia" so that I can get a quick look at the loading on all of the nodes.
I have a directory "FAH" in my home directory. Within that directory there is are 2 directories for each of the two-cpu nodes, and 4 for the quad cpu nodes. They are named $HOSTNAME-a, $HOSTNAME-b, etc., where HOSTNAME is the hostname of each machine.

I then ran setup from within one of the -a directories, and copied the config file to every -a directory. I modified the machine ID option, and copied that to all of the -b directories.

I created a script "fold-here" that has:

Code:

---

cd ~/FAH/$HOSTNAME-b
screen -dms B ./FAH502-Linux.exe
cd ~/FAH/$HOSTNAME-a
screen -dms A ./FAH502-Linux.exe

---

I then have a script that one of my GRA hacked up for me ("spawn"):

Code:

---

#!/bin/bash

# Script for starting jobs on multiple machines
# input arguments are command(s) ($1) and machines ($2)
# machines argument can be:
#  1) all  -- processes the command(s) on all the nodes in /etc/hosts
#  2) c1-1:c1-31  -- range.  will only work with the same family of
#      nodes.  ie: c1's, c2's, c0's....
#  3) c1-1 c1-2 c1-3  -- space deliminated list

COMMAND="$1"
MACHINES="$2"

if [[ "$MACHINES" == "all" ]]; then
  for NODE in $( cat /etc/hosts | grep 255.255 | grep -v manager | grep -v network | cut -d" " -f3); do
      echo "processing" $COMMAND on $NODE
      rsh $NODE $COMMAND
  done
elif [[ $MACHINES == *:* ]]; then
  STARTNODE="`echo $MACHINES | cut -d: -f 1`"
  ENDNODE="`echo $MACHINES | cut -d: -f 2`"
  i="`echo $STARTNODE | cut -d- -f 2`"
  j="`echo $ENDNODE | cut -d- -f 2`"
  while [ $i -le $j ]; do
      NODE="`echo $STARTNODE | cut -d- -f 1`-$i"
      echo "processing" $COMMAND on $NODE
      rsh $NODE $COMMAND
      i=$(($i+1))
  done
else
  for NODE in $MACHINES; do
      echo "processing" $COMMAND on $NODE
      rsh $NODE $COMMAND
  done
fi

exit 0

---

if you copy this directly, you'll have to tweak it a bit as it is to some degree explicitly hard coded for our internal network configuration.

I then run execute:

Code:

---

spawn "fold-here" c0-0:c0-15

---

or whatever other machine range I want to use.

There are variants for the 64 bit machines, and the SMP client, but this is the more complicated scenario. It should be fairly obvious how to simplify this for the singe-client-per-node case.

hi jimwah

nice guide!!

i have a small problem though.. i keep getting this errors and cant continue further.. it seems that i dont have an internet connection inside vmware..

http://i75.photobucket.com/albums/i2...ma/kubuntu.jpg

tnx

Hey ferds, glad you've found the guide useful so far :up: check out your network interfaces in the Ubuntu GUI & see if there's an eth0 or eth1 interface, and that it's configured properly for your network (eg DHCP enabled, device starts at boot time). Network cards can also be managed using the command ifconfig in a console.

VMWare uses a very common 'virtual' network card iirc, so ubuntu shouldn't have any issue detecting it & loading a suitable driver :shrug: I know a few people tend to use NAT networking for their VM's, it might be worth trying that also & see how the network configuration changes. I don't claim to be a linux guru by any means - but I hope we can muddle through & sort it out :)