Crunching tips

[sierra_bound]

Points - This always confuses new members. WCG's point system is different from BOINC's. When the WCG project first started, it did not use BOINC. It was only later that BOINC was added as a client. But since BOINC uses a different point system, WCG had to come up with a way to convert BOINC points into WCG points. The current formula is one BOINC point equals approximately seven WCG points.

BOINC uses a validation process. The same work unit must be processed error-free by at least two different computers (hosts) before credit is awarded. The validation process can take anywhere from a day to a week. So when you install BOINC you will probably notice that you weren't awarded many points the first couple of days. This is normal.

One other important thing about points:

From the WCG website:

Will I lose points if a work unit finishes slower than usual?

No. Points are not tied to number of work units, but amount of computational effort exerted. Therefore, if your computer worked for three days on one work unit, or in those same three days completed five work units, you would get the same amount of points if the machine worked at the same level of effort.

[sierra_bound]

Caching work units - In the Device Profiles section of your My Grid page, there is a setting that allows you to increase (or decrease) the number of work units that you have cached. Some people like to have at least a 2-3 day supply of WU's. This is protection in the event the WCG servers crash for an extended period of time. About a year ago, the servers were down for three straight days!

[twilyth]

Yes, it is.


I've had RAM fail in two different crunchers recently and I suspect a problem in another one. Will take a look at that machine later today. Anyone have any recommendations for cruncher RAM?

I've used Patriot in 2 builds - including one quad. The last 2 builds are using Wintec (with heat spreaders) but the final one is still under construction. The other is also a Q6600. Both quads are less than 6 months old though. I don't remember the fsb settings but the quad with patriots is at 3.6ghz. The other quad I can't get past 3.2ghz but according to memtest, the memory isn't the problem. I suspect the mb - it's a cheaper P35 than the patriot build.

[G.Foyle]

I have used Patriot LLK's for several months in my cruncher without a problem, but their overclockability was disappointig at least; they were advertised to run @ 1000 5-5-5-15 @ 2.1V, but they barely made 950MHz CL5 at that voltage Older LLK sticks use Elpida chips, but if you are lucky you might get newer ones with Powerchip, they are very good for their price.

[sierra_bound]

Windows vs. Linux

Many people have asked whether it's worthwhile to run Linux on WCG. Many BOINC users already do. As I mentioned earlier, Windows will usually generate more points per day. But the 64-bit versions of Linux do fairly well.

Advantages of Linux:
- Free (many versions)
- Good OS for older systems because hardware requirements are less demanding than Windows

Even if you're not familiar with Linux, current distros like Ubuntu make it fairly easy because BOINC is included in the software package. There are usually just one or two steps to install it. Don't even need to use command line. Instructions for installing BOINC on Ubuntu can be found here.

With 64-bit Linux operating systems, depending on the type of processor you're using, you may need to install the 32-bit libraries. Even though you can crunch with 64-bit clients and operating systems, the WCG project applications themselves are 32-bit.

If you are using Ubuntu x64, open up the Terminal and type in this command, then hit enter:

sudo apt-get install ia32-libs

If you have an AMD 64 or Xeon processor, you probably don't need to do this. And no need to install libraries if you are using Windows x64.

[David_L6]

Caching work units - In the Device Profiles section of your My Grid page, there is a setting that allows you to increase (or decrease) the number of work units that you have cached. Some people like to have at least a 2-3 day supply of WU's. This is protection in the event the WCG servers crash for an extended period of time. About a year ago, the servers were down for three straight days!

Excellent advice SB. I remember the server crash in December '06. I ran out of work. I now have my crunchers set to connect every .5 days and cache 2.0 extra days of work.

[sierra_bound]

Yes, I ran out of WU's during that outage too. The servers are located in Colorado which just had a blizzard. When the servers went down no one could check it out right away because the roads were snowbound. Now, I always keep a 2-day supply of WU's, just in case.

[D_A]

Windows vs. Linux

...

If you are using Ubuntu x64, open up the Terminal and type in this command, then hit enter:

sudo apt-get install ia32-libs

If you have an AMD 64 or Xeon processor, you probably don't need to do this. And no need to install libraries if you are using Windows x64.

I'm running Ubuntu 7.10 AMD64 and while the client and manager from the repositories will "run" as is, they will trash every work unit in under 5 seconds flat if I don't have the ia32-libs installed. With them installed the machine is as stable as a rock. hth

[sierra_bound]

I think it depends on the set-up. Another member here is running Kubuntu x64 with an AMD processor and I believe he did not have to install the 32-bit libraries. But if you run into a problem, installing the libraries is the likely solution.

[123bob]

"Use no more than: XXX% of processor time" Parameter This setting is found as a parameter in your WCG profiles. A common starting question we get is "why isn't the CPU showing 100% usage in the taskmgr?" Set this to 100% if you want to max out your CPU. Another use for this parameter is as a way to control CPU heat in laptop crunchers. I'm running a work lappy 24/7 at 60%. Since you can't control the heat as well in a laptop, adjust this parameter to prevent burning the thing up.

To the question of max point production, I'm getting extremely good results running HCC on 64 bit vista with 64 bit BOINC version 5.10.28. It generates relatively low amounts of page fault counts (thousands) and efficient (read low) run times for HCC WUs. The WUs score around 70 BOINC points each, but I produce a lot of them since they only take about 2.7 CPU hours to run. I've been very consistent and stable with this combination. I currently run 10 quads this way.

To the issue of D9 Micron, I've burned a few sticks of D9 out too. I've had very good luck with just promo stuff, I don't OC it, or put more than 2.1VDC to it ever. Crunching is hard on RAM....

Excellent thread Sierra. Thank You!!

Regards,
Bob

[brot]

That's interesting . Where do you select this? When installing the OS?

Its basically done in 2 entrys, once in the /etc/fstab and once in the kernel bootoptions, which are in the /boot/grub/menu.lst for me.

/dev/md1 /boot ext3 noatime,data=journal 1 2
/dev/md2 / ext3 noatime,data=journal 0 1

title=gentoo - 2.6.24-rc8
root (hd0,0)
kernel /kernel-2.6.24-rc8 root=/dev/md2 rootflags=data=journal vga=0x31B video=vesafb:mtrr3,ywrap

edit: that does only work for ext3 filesystems, others will ignore that flag. You can look up if your fs uses data=journal when looking at dmesg, if one line says something like "EXT3-fs: mounted filesystem with journal data mode." it works.

[WrigleyVillain]

Wow thanks for this...

xs wcg ftw

[deepsilver]

Nice sticky.

[sierra_bound]

Patience & Perseverance - When you first start crunching, it will seem like you're not making much progress. But 10,000 points/day, for example, becomes 300,000 after a month, and 3,600,000 after a year. Also think about all those work units you are submitting to researchers. Even if you have just one machine crunching, you may end up processing the WU that leads to a scientific breakthrough. So don't become discouraged. We are all here to support each other.

[naokaji]

how to avoid getting addicted, if you got this far its too late

sorry.. coudnt resist

[[XC] Hicks121]

Thanks SB & Bob & whom ever else contributed to this thread!! This is top notch.

[jspace]

Installing the ia32-libs is probably a good idea. I couldn't get the client to even run without them on Debian 4.0r1 Etch x64.

(BTW: I got Beryl running on that system. I'm proud of myself)

[TheGoat Eater]

I just got done installing windows server 2008 64 bit RC1 to try a 64 bit client on it (Athlon 64 3500+ winnie)

[sierra_bound]

CPU Benchmark

I'm sure most of you have seen the CPU Benchmark that's included in the BOINC Manager. The benchmark is run automatically after you install BOINC or when you change the client version. It will then automatically run every five days on Core 0 (first core) only. So having a multi-core processor will not give you higher benchmark scores.

So what do the results mean?

The benchmark has two components:

- Whetstone is the floating point calculation. Each work unit delivered to your machine includes an estimated number of floating point operations (FLOPS). BOINC divides this by the Whetstone benchmark number to estimate completion time. So the higher the Whetstone number, the lower the estimated time to complete the work unit.

- Dhrystone is the integer calculation. The term Dhrystone is actually a play on words. Since it has no floating point component like Whetstone, it's considered "dhry".

So do higher benchmarks mean more points? That's often the case. Though some operating systems, like Linux 64-bit, seem to give inflated Dhrystone results. Also, a ten percent increase in your benchmark score does not necessarily mean your daily point output will also increase by ten percent.

People like to try different clients because some give higher benchmark scores than others. I have always felt you should pay more attention to the daily output of your machine. That has always been the best benchmark.

[sierra_bound]

Uploading work units

This issue has come up a number of times. WCG uses a scheduling server to determine when to take in your completed work units and when to send out new ones to your computer. If you feel the WCG servers are not taking in your completed WU's fast enough, there are several things you can do.

1) In the Device Profiles section of your My Grid page, there is a setting called "Connect to the network about every _____" (see pic below). By lowering the setting, the servers should take in the WU's at shorter intervals. But keep in mind that even if you set it to 0.1, that's still one-tenth of a day or 2.4 hours.

2) On the BOINC Manager, go to the Projects tab and hit Update. This will force the servers to take in the completed WU's.

3) Create a batch file. One of our members, meshmesh, has a how-to guide. You can find it here.

Sometimes, your finished WU's are not being uploaded because the WCG servers are down. You can verify this by checking the Messages tab on the BOINC Manager.

[sierra_bound]

Virtual memory usage

As many of you have noticed, during crunching the paging/swap file is constantly being used. Virtual memory usage is even higher with some operating systems like XP x64. Always make sure you have enough virtual memory allocated. If you're using XP, right click on My Computer ---> Properties ---> Advanced ---> Performance Options ---> Advanced.

WCG recommends that the maximum paging file size should be at least 2 or 3 times your real memory size, and should preferably be at least 512MB, whichever is greater.

[sierra_bound]

What does preempted/waiting mean?

Occasionally you will have a work unit that's been paused. Depending on a client version, it will say "preempted" or "waiting" in the BOINC Manager. There are a couple of reason why this may happen. Sometimes you will be sent work units with earlier deadlines than the others in your queue. Those always get crunched first. Also, if you are running more than one project, you will notice that BOINC switches its crunching between projects. It is done this way to give equal CPU time to all projects, set in accordance with your preferences and with the amount of time until your tasks reach their deadlines.

Computer is overcommited

This message will occur if BOINC finds that the total time required to finish all work units in your queue exceeds one or more deadlines. For example, you might have ten work units downloaded and each will take about 24 hours to complete. But all deadlines are within 4 days. Unless you have a quad-core, there's no way you'd be able to meet all the deadlines. BOINC starts with the earliest deadline, attempting to complete as much work as possible. You sometimes see this problem with slower machines. The solution is usually to lower the setting "Connect to the network about every ____" (see above post).

[don_xvi]

Windows vs. Linux
With 64-bit Linux operating systems, depending on the type of processor you're using, you may need to install the 32-bit libraries. Even though you can crunch with 64-bit clients and operating systems, the WCG project applications themselves are 32-bit.

If you are using Ubuntu x64, open up the Terminal and type in this command, then hit enter:

sudo apt-get install ia32-libs

Interestingly enough, having just forgotten to do this on a new Ubuntu x64 install, HPF seems to work without installing the 32 bit libraries.

[jcool]

What values can you recommend for the "write to HDD every XX at most"?
I don't think it's good to set that too high, because if the drives are spinning up and down every few minutes I doubt they'll last in 24h operation. Someone once mentioned crunching onto a usb stick, I found that to be quite a cool idea, because you can let your HDDs power down for long periods of time.

About the D9, I have built lots of rigs with different brands and using 4gb myself for 24/7 crunching - just don't give them over 2,1V and nothing will happen - they still do 1100 CL5 or so just don't be too Mhz greedy with a cruncher

[sierra_bound]

What values can you recommend for the "write to HDD every XX at most"?
I don't think it's good to set that too high, because if the drives are spinning up and down every few minutes I doubt they'll last in 24h operation. Someone once mentioned crunching onto a usb stick, I found that to be quite a cool idea, because you can let your HDDs power down for long periods of time.

I have it set to 60 seconds. Have not had any problems with my HDD's.