Want to run 100% CEP2 WUs? Here's how...

[josh1980]

So you want to run the Clean Energy Project Phase 2 (aka CEP2) but find your disk thrashing too much? Here's how you can get around it. As shown here the Clean Energy Project needs our help. This is an idea I came up with on my own to try to help others that believe in the CEP2 project to donate more of their resources. With a little luck someone else can post a detailed screenshot guide or PDF of instructions on how to do this setup.

This guide assumes that you are fairly knowledgeable with BOINC and WCG. You also want to maximize your CEP2 WUs for your machine.
The key is using block level caching with an extremely long time delay for writing data to the drive. The intent is that the WUs will thrash the heck out of the block level cache and thereby minimize the hard drive thrashing. Of course, data will still get written to the drive, but hopefully the writes will be more sequential.
System specs for what I used: Dual CPU, Quad Core(8 thread) with 16GB of RAM and 160GB hard drive. I used Windows 7 x64 Home Premium, the latest version of BOINC(6.12.34 x64 version) along with Supercache 5.0.524.0. Supercache can be purchased for $79.95 at www.superspeed.com. A fully functional 14 day trial version is also available at their website.

1. First, you want to set up world community grid to give you CEP2 WUs. I recommend you create a device profile. For me, I created the new "school profile". Set up a custom profile and uncheck all projects except CEP2. Set up the other characteristics of the profile as you see fit. Make sure that for project specifics you choose the number of work units for CEP2. I chose unlimited since I want 100% crunching for CEP2. Don't forget to change your device to use the new profile you created. I also set the profile to cache only 0.25 days of work because I want to cache the projects in use without caching files that are not.

Note: The machine I tested this on has 8 cores(2 CPUs) and I found that I could achieve 100% utilization using just 7 of the 8 cores. The machine also handles other low CPU usage programs for me, so I can only expect that the caching program I used utilized the 8th core. Feel free to experiment and figure out how many cores work best for you.

2. Now that WCG is setup to give you all the CEP2 WUs you want it's time to setup your computer to receive these units. Install Supercache 5. After rebooting if necessary it is time to setup a partition for your BOINC data. I resized my C partition to make about 40GB of space available for BOINC. The partition for BOINC needs to be dedicated to BOINC data(you don't want other data being cached) and it needs to be big enough to not end up filling up. CEP2 uses alot more drive space than other projects. My opinion based on using a RAM drive previously is to create a partition between 2 and 4 times the number of threads you will create in GB. If you intend to run 16 threads simultaneously, I would recommend between 32 and 64GB for the BOINC partition. I'd recommend 32GB because you don't want to partition too big for the cache. Format the partition to 4k sector size NTFS.

3. Next I installed BOINC, pointing the data directory to the E drive. I do not know how you can change the data directory if BOINC is already installed. If someone posts this information I will update this post. After BOINC is installed reboot if necessary.

4. Next it is time to setup the cache. Install supercache and on installation when it asks you to setup the cache for the windows partition uncheck the box.
I determined that I wanted 80% of my RAM utilized. I've seen CEP2 WUs use as much as 300MB per thread. Since I was expecting 8 threads I figured 3GB would be utilized at the most for CEP2. The WUs use more RAM as the WUs nears completion, typically starting at 60-80MB.

So I setup a cache of 5GB to see how it would perform. The size of the cache can play a part in how well this performs. I setup the granularity as 4kb and set the total allocation of 5GB. I also enabled the deferred-Write Mode and set the latency to infinite. This means that the cache shouldn't try to write data to the drive until the cache is full, a manual flush command is initiated, or something else triggers it by supercache?
For a 5GB cache I found that 2GB of data was written about every 10 minutes. The hard drive would sit idle for about 10 minutes, then for about 30 seconds it would write data at 20-70MB/sec(total was typically 1-2GB). Then go idle again for approximately 10 minutes. For a 10GB cache I got about 20 minutes of idle time.

5. Now that the cache is setup, it is time to download those WUs. Using BOINC, I added the WCG project and began downloading WUs. I ended up with all CEP2 WUs. The computer then ramped up to 100% CPU utilization as the WUs were downloaded. Watching the temperatures, CPU usage, and cache I was able to determine that the cache that I setup worked very well for this project. Even during the times of hard drive activity the CPU stayed at 100% utilization and the CPU temps showed that the CPU was still being stressed by the work.


If all went well, you should see similar results to what I saw. CPU will sit at 100% utilization for about 10 minutes+, then the hard drive will write a bunch of data, then idle again.

If you use a SSD, you MAY find that the total amount of data written to the SSD will be lower using the supercache.

So why does all this stuff work? I believe that the CEP2 project continually concatenates the files on the hard drive with new data as the CEP2 WUs are completed. Because we have a large block cache we are writing all this data that is continually updated to the cache instead of thrashing the hard drive. The built-in Windows cache has a finite time before data is written to the hard drive. Being able to set up a cache that theoretically, might not write to the hard drive for a very very long time is a big bonus for us.

Edit: If I were to put together a guide, what is the recommended format? .PDF? .DOC? I could easily make both of them..

[OldChap]

I don't think it really matters which format the guide were to be written in, I guess those wanting this have the ability to read both.

Nice work by the way.

[OldChap]

Sorry DP.

You should post over at WCG when done too (or even as is)

[Plan.B]

I just got a q6600 rig off hardforum. I'm going to try this.

[haschioz]

thank you very much josh1980! awesome guide. i will try this once i get a new rig.

[Otis11]

Nice write-up! Thanks for putting in the time to do this!

A couple questions though:
1. You said this is utilizing the the 8th core? So that's a 12.5% overhead - how much does the disk thrashing impact it?
2. Does this improve performance over just normal usage on a HDD after the overhead? What about for a SSD?
3. How many WUs can a SSD handle under normal conditions? How many with this tweak? (I realize this is a solution intended for HDDs, just asking)
4. If you're spending $80 on this program, what advantage does that have over a cheap 32GB SSD for say $40?

(Great idea, not trying to detract, these are just the thoughts that came to my head, and of course I don't expect you to figure this all out, just thought someone might know some and we want to get the most performance possible)

[josh1980]

Nice write-up! Thanks for putting in the time to do this!

A couple questions though:
1. You said this is utilizing the the 8th core? So that's a 12.5% overhead - how much does the disk thrashing impact it?
2. Does this improve performance over just normal usage on a HDD after the overhead? What about for a SSD?
3. How many WUs can a SSD handle under normal conditions? How many with this tweak? (I realize this is a solution intended for HDDs, just asking)
4. If you're spending $80 on this program, what advantage does that have over a cheap 32GB SSD for say $40?

(Great idea, not trying to detract, these are just the thoughts that came to my head, and of course I don't expect you to figure this all out, just thought someone might know some and we want to get the most performance possible)

1. I was having a problem with my antivirus.. it was causing explorer.exe to max out one core.(this issue was completely unrelated to my testing), but the other cores were all maxed out. The cache program is only single threaded(I believe so anyway), so the "worst case" scenario would be that the cache would max out one core and the WCG threads would be limited to the amount of work the could perform. I retested this today, and with 7 cores dedicated to CEP2 I was getting 94% CPU utilization. The "System" process was using about 15% of one core. I did not see any thread specifically for the cache, so I can only assume that it is running under the System process. I had a few other programs running in the background that are not CPU heavy, (FTP server service, etc), so a few % might be for those.
2. This improves the usage of the CEP2 threads when crunching many CEP2 WUs simultaneously on a HDD. Last year I had some problems with a computer I had setup that was dedicated to CEP2. I couldn't understand why I could not get 100% CPU usage. Later I figured out it was the 8 threads of excessive writes. So many writes were being executed by the 8 cores that the crunching was being hindered. The intent of this guide was to prevent CEP2 dedicated crunchers from being limited to hard drive performance.(I just can't bring myself to recommend people buy an SSD just so they can thrash it with writes for CEP2.)
3. I do not know exact numbers for SSD, but my ANS-9010(Ram-based drive) can do 16 threads of CEP2 without any slowdown(SR-2). Some SSDs can out-perform the ANS-9010. Based on my old Core i7-920 numbers, I was typically getting only 40-60% CPU time based on an average of 14 days of crunching with 8 threads of CEP2 running 24x7 on a 1.5TB HDD. It took me over a week to figure out what the problem was since I didn't have a hard drive LED attached to the motherboard. Believe me, I was pulling my hair out and since I was using Linux and I'm a noob at linux I was even more lost at the cause.
4. The only thing I can think of over the SSD is that the cache will not wear out.

When the hard drive started writing, it wrote large quantities of data quickly(20-80MB/sec for about 20-30 seconds), giving me the impression that the writes were being written sequentially instead of random(which is the problem with CEP2). The CPU usage stayed at 100% while the data was being written to the hard drive. Since the hard drive I used physically was also my boot drive, using the system during the hard drive writing process was very slow. If you wanted to use the system as a workstation I would recommend a separate drive for crunching. One of the WCG threads sometimes goes to 0% CPU usage during the 20 seconds or so when the HDD is being written to. It's never more than one thread that is temporarily stopped and the CPU usage drops accordingly(in my case, about 12%).

If you were really dedicated you could make a 32GB cache and using a file share for the boinc data directory do a boatload of CEP2 crunching over your entire LAN.

[Otis11]

Dude... you are on fire!

Also, if anyone was interested in doing a RAMDISK - the free version allows up to 4GB and the paid version can scale higher ($15) I think I might check it out...

Also going to look into this one when I have time... (unfortunately the only computer I can currently mess with doesn't have enough RAM to do this...)

Edit: also this one... seems simple enough, might try it now... looks like my BOINC data directory only needs 1.5 GB...?

[josh1980]

Remember, you need 2GB free space or the CEP2 WUs will stop downloading. It's some kind of hard coded requirement for the WUs to download. I learned this the hard way with Boinc last year.