The CDT and copywaza lab

[KTE]

OK, yes I have the results saved.

I tested exactly as called by others following the testing methods given by BenchZowner/Hipro5 and Elmor/OPB.

*All standard Windows tweak done on all runs (2 or 3 services running max, 13 processes idle, 118MB) but for the first completely untweaked normal Windows run.
*When I tested 512MB-512MB pagefile vs. 384MB-384MB there was absolutely NO difference in 8M-4M-1M runs so I left it to 512-512MB for all tests but first two.

OPB/Elmor's CDT method: (1)
Hipro5's CopyWazza method: (1) (2)

First thing I'd like to mention is that I could not repeat and get 500MB/500MB for memory/cache out of 600MB installed! The maximum mine went up to after CDT was 365MB/363MB. I "think" this depends purely on your hard drive cache (mine was 8MB/2MB).

For CopyWazza, same method; it did NOT raise my system cache/available memory by much at all. This again I think depends on your hard drive cache size.

Test Setup
P4 Celeron 2800MHz D0 @ 3360MHz (stock volts 1.312V)
512MB x 2 (dual channel) OCZ DDR400 2-3-3-6 1T (4:5)
Hitachi Desktar ATA133 7200RPM 80GB 8MB cache HDD
Maxtor ATA133 7200RPM 80GB 2MB cache HDD

Format
Tweaks
=====
Task Manager Available Memory (SPi Available Memory)
Task Manager System Cache
Best 8M Time

Windows bootup Idle memory/cache/processes


SuperPi Mod 1.5 / XS 8M

Test 1:
Pagefile 250-1000MB
Not real-time
====
469.7MB (439.23MB)
409.67MB
10m 48.212s


Test 2:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 250MB-1000MB
Not real-time
====
469.23MB (472.83MB)
69.99MB
10m 25.469s (sorry missed the screenshot )

Test 3:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 512MB-512MB
Real-time
=====
472.86MB (481.07MB)
92.90MB
10m 22.955s


Test 4:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 512MB-512MB
Real-time
CopyWaza (C/D, 1st partitions, different drives)
====
366MB (377.04MB)
363MB
10m 20.752s


Test 5:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 512MB-512MB
Real-time
FreeRAM XP Pro
====
513MB (528.96MB)
58.5MB
10m 19.881s


Test 6:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 512MB-512MB
Real-time
CDT (D/E, 1st/2nd partitions, same drive)
====
490.60MB (504.48MB)
104.75MB
10m 19.861s


Test 7:
Tweaked Windows
Maxmem=600 (Boot.ini)
Pagefile 512MB-512MB
Real-time
CDT (C/D, 1st partitions, different drives)
====
365.1MB (375.41MB)
360.1MB
10m 15.696s


Lets just say I spent a whole week (before today) trying to beat the CopyWaza time with everything else and it was impossible... until I tried the last technique.

Hope it helps.

[massman]

Great work !!

I'll read it through when I get back home

[saaya]

hey massman... thats weird though, why does it need that much space to calculate pi? it ultimately makes it a hard drive benchmark considering this bench is so old... back then the cache was so slow that the hdd must have played a big role in getting good results. now which uge caches and mem the problem isnt that much the hdd i guess.

again, weird why it needs so much space...
and 632mb... well we are talking about systems with 2gb of mem so why would pi write something to the hdd and not the mem, the mem should have plenty of space for any temp file pile that bench comes up with...

copying what from d to c makes the pi run slower?

[saaya]

KTE, what last technique?

Your running this on a celeron with little cache, which makes your results not directly comparable but interesting! and your not running 32m so maybe theres a tweak or setting that only works with 32m, but i find that unlikely, if there is no gain in 8m its unlikely that theres a gain in 32m i think.

so you went from 10:48 untweaked to 10:15? thats a nice boost...
Are those individual runs? did you make each run only once?
Is this the average or the best results with each technique?

thanks for all the time you spent on this and thanks for sharing

[mrlobber]

KTE, top shelf testing stuff

Too bad you are using a Prescott Celeron, not a C2D... CDT seems to have different impact on various families of processors, just like BeardyMan showed with his A64.

Interesting thing about the hard drive cache impact... I did all my comparisons on a single 20Gb IDE 2Mb drive, time to retest everything with larger cache drives and with 2 different drives That 4second gain on your 2nd CDT run (between 2 drives, not 2 partitions of the same drive) is especially noteworthy.

I think it has a lot more to do with one partitioned hdd or two seperate ones, position of the pagefile and things like that.

Heh, KTE just showed the opposite... although OPB of course knows what he is saying, which makes things even more interesting

I will let you have all the answer, buddy, YGPM,
looks like someone start picking agin

[KTE]

hey massman... thats weird though, why does it need that much space to calculate pi? it ultimately makes it a hard drive benchmark considering this bench is so old... back then the cache was so slow that the hdd must have played a big role in getting good results. now which uge caches and mem the problem isnt that much the hdd i guess.

Yep that was exactly my theoretical thought before running. I thought it 'aint logical for CW/CDT to effect 1M or 32M since SPi is small enough to fit inside most nowadays CPU L2 cache and if not then RAM. What does HDD have to do with it? I can't yet explain my practical results. They are showing HDD cache/buffers and OS/CPU prefetch is affecting SPi result.

KTE, what last technique?

Last one was CDT using two different drives rather than two different partitions on one drive. The latter didn't work much at all.

Your running this on a celeron with little cache, which makes your results not directly comparable but interesting!

True. I only tested to get a sense of "across the board" effect.

and your not running 32m so maybe theres a tweak or setting that only works with 32m, but i find that unlikely, if there is no gain in 8m its unlikely that theres a gain in 32m i think.

Honestly, I've only ran 32MB once on a P4 3GHz Northwood in the last year and it took wayyy too painfully long to want to run again 21-30 times on a Celeron to test. I could run it for further investigation if anyone wants, but these are just some preliminary results. Still at work here.

so you went from 10:48 untweaked to 10:15? thats a nice boost...
Are those individual runs?

Yup. Best result in 3 runs for each technique was given above.

thanks for all the time you spent on this and thanks for sharing

You're welcome mate.

[v|aDy]

Without CDT 14.15.125


With CDT: 14.04.063


With CopyWaza by Hipro5 method: 14.05.563

[saaya]

KTE, was there a lot of fluctuation between the 3 runs you did for each technique? was this a fresh windows install? with the latest service packs?

several people seemed to have a lot of problems to reproduce their results.
talking about this, has anybody heard back from elmor yet?

[Zeus]

hey massman... thats weird though, why does it need that much space to calculate pi? it ultimately makes it a hard drive benchmark considering this bench is so old... back then the cache was so slow that the hdd must have played a big role in getting good results. now which uge caches and mem the problem isnt that much the hdd i guess.

again, weird why it needs so much space...
and 632mb... well we are talking about systems with 2gb of mem so why would pi write something to the hdd and not the mem, the mem should have plenty of space for any temp file pile that bench comes up with...

copying what from d to c makes the pi run slower?

It does not need much harddisk space and only uses very little memory in fact.

Here's a quote from the SuperPi help folder:

Memory requirement:
For the maximum calculation speed, 8 MB/1 million decimal digit is favorable, but 2 MB/1 million decimal digit is completely acceptable. Working memory is automatically adjusted by the software. Anyway, available main memory size is crucial for the processing speed!

Disk storage:
As for the working disk storage, 10.5 MB per 1 million decimal digit is needed. Working disk storage is automaticall freed. As for the permanent data storage, 1 MB per 1 million decimal digit is needed. Elapsed time is very keen to the disk access time. In order to short the elapsed time, you are better to equip high speed hard disk drive!

Having much RAM doesn't have an advantage here so it seems, how interesting is the sentence about the harddisk on the other hand?

[zbogorgon]

IMHO, Spi 32M is much more tweakable than 1M

EDIT: comment removed, please read the first post
-Saaya

[KTE]

Two things:

Does SPi write to the HDD real-time, in the last iteration or after finishing?
Do CW/CDT help with the prefetch buffers in both the CPU and HDD?

KTE, was there a lot of fluctuation between the 3 runs you did for each technique?

Differences between the 3 runs were in milliseconds not seconds. Each of those times were very difficult to get at those clocks/timings with that setup. A typical fully tweaked run will get you 10m 22.9s without those extra methods, repetitively, no matter how hard you bang your head.

The first run I did, something was wrong with Maxmem flag not working in boot.ini (that would give a CopyWaza 8M time of 10m 25s) so I reinstalled XP Pro SP2 32-bit + all updates. Then just ported over many of the files I needed on that install. I went into Start > Run> msconfig and chose to load the "minimum services/drivers/processes" option (which gives you problems later when restarting them, beware).

several people seemed to have a lot of problems to reproduce their results.

I'll do some re-tests a little later just to clarify if those results are repeatable and to what extent on that setup. Not sure how long that PC will be there before being disassembled. 5 of each this next time around. I'll test that later on with 32M too (bloody fire haah, 50 minutes x30 of waiting )

talking about this, has anybody heard back from elmor yet?

Nope. But I would've done the same right thing in his position.

Elmor, (I forgot your name, Robert?) please give us more information here for everyones benefit. And on behalf of myself and others, thank you for defending what you knew and experienced. I don't like weasels or those not man enough to admit what they factually experienced.

Admin has laid down the rule of no ad hominems and kiddy crap, so I believe he will stick to admonishing that in this thread (unlike what happened in the previous).

[BeardyMan]

A64 X2 at 3.4ghz boxed cooled 376mhz ddr2 5-5-5-18
and 5gb maxtor instead of the 160gb with 16mb buffer.
gain = 0.3secs instead of the 0.6secs average
from 25.500 to

[massman]

hey massman... thats weird though, why does it need that much space to calculate pi? it ultimately makes it a hard drive benchmark considering this bench is so old... back then the cache was so slow that the hdd must have played a big role in getting good results. now which uge caches and mem the problem isnt that much the hdd i guess.

again, weird why it needs so much space...
and 632mb... well we are talking about systems with 2gb of mem so why would pi write something to the hdd and not the mem, the mem should have plenty of space for any temp file pile that bench comes up with...

copying what from d to c makes the pi run slower?

- No, the 632MB is used to clean up the memory's matrix from all the other data. It's not about cleaning the HDD, I think. The HDD transfer is just a way to clean the memory.

- Maybe, by limiting the ammount of memory, the data will be less spread over the memory's matrix and thus faster accesible? I believe you can use maxmen=600, 700, 500 as well, you have to alter the 3 files however to 732 and 532. Right, Kevin??

- Not SuperPi runs slower, the copying itself is slower.

[Onepagebook]

Yes Peter, well see someone start yelling and still be allow here? what do we say?

[saaya]

- No, the 632MB is used to clean up the memory's matrix from all the other data. It's not about cleaning the HDD, I think. The HDD transfer is just a way to clean the memory.

how does it clean the memory?
If you overwrite 11101010100101 with 0011101010101001 how does that "clean" the memory?

[KTE]

how does it clean the memory?
If you overwrite 11101010100101 with 0011101010101001 how does that "clean" the memory?

I'm not sure how it cleans memory, but here is one possibility:

Here we have loss of time with these methods. The main purpose of a cache in any device is to decrease access time as well. The larger the cache hit ratio, the better the prefetch, the faster the access time, the quicker the application/instruction time. This may hold us some clues.

You have some RAM address ranges reserved for hardware I/O mapping in Windows by default and others reserved deemed necessary for kernel level code. On top of this you'll have extra taken up by basic system processes and they tend to accrue and not release from memory even if the address range is no longer needed. Thus you'll have a section of memory unaddressable for no reason.

When you force the "cleansing" by something which will either a) defrag the RAM b) force it empty, such as a bigger process requiring the full RAM, naturally the Windows cache and memory will empty all unneeded extra address ranges which were reserved by other applications (prioritize) and start filling up if you've given Windows priority to real-time Programs. FWIW the HDD pagefile is also a cache managed by Windows kernel level. Then rather than the RAR files still retaining in memory as many applications will, after the copying finishes, they are released immediately and that part of memory becomes freely available to everything subsequently. I "suspect" the type of files matter (RAR files) but I'll try it more thoroughly soon.

That's as far as I can see if memory increase does play a role and how. For me, this has all to do with the various caches and prefetch algorithms as this is their well known function - to increase speed and decrease latency.

[Onepagebook]

I'm not sure how it cleans memory, but here is one possibility:

Here we have loss of time with these methods. The main purpose of a cache in any device is to decrease access time as well. The larger the cache hit ratio, the better the prefetch, the faster the access time, the quicker the application/instruction time. This may hold us some clues.

You have some RAM address ranges reserved for hardware I/O mapping in Windows by default and others reserved deemed necessary for kernel level code. On top of this you'll have extra taken up by basic system processes and they tend to accrue and not release from memory even if the address range is no longer needed. Thus you'll have a section of memory unaddressable for no reason.

When you force the "cleansing" by something which will either a) defrag the RAM b) force it empty, such as a bigger process requiring the full RAM, naturally the Windows cache and memory will empty all unneeded extra address ranges which were reserved by other applications (prioritize) and start filling up if you've given Windows priority to real-time Programs. FWIW the HDD pagefile is also a cache managed by Windows kernel level. Then rather than the RAR files still retaining in memory as many applications will, after the copying finishes, they are released immediately and that part of memory becomes freely available to everything subsequently. I "suspect" the type of files matter (RAR files) but I'll try it more thoroughly soon.

That's as far as I can see if memory increase does play a role and how. For me, this has all to do with the various caches and prefetch algorithms as this is their well known function - to increase speed and decrease latency.

Based on what your statement here, what I would like to add is
THe type of RAR file doesn't matter but its size.

[BeardyMan]

Size is what i'm going to test now

around 2Gb vs 632Mb

[hipro5]

Lets just say I spent a whole week (before today) trying to beat the CopyWaza time with everything else and it was impossible... until I tried the last technique.

Hope it helps.

http://www.xtremesystems.org/forums/...&postcount=327


You're "steeling" my techniques....Shame on you....
Now TRY the SAME with a BIG Copy Waza....

[BeardyMan]

using a big file seems to be uselles for me loss instead of gaining...

[hipro5]

using a big file seems to be uselles for me loss instead of gaining...

With TWO SEPARATE HDs.....One with OS and everything in it and one ONLY with the PAGEFILE (clean) and at the BIGINING of the HD.....
Though BEST is if someone could use a Gigabyte iRAM as SECOND HD with the page file IN IT...

[elmor]

Nice to see this back on track again, great work saaya.

Elmor, (I forgot your name, Robert?) please give us more information here for everyones benefit. And on behalf of myself and others, thank you for defending what you knew and experienced. I don't like weasels or those not man enough to admit what they factually experienced.

Nowhere near Robert, that's crotale I do not have so much more info to add but I can sum up what I've experienced so far.

several people seemed to have a lot of problems to reproduce their results.
talking about this, has anybody heard back from elmor yet?

First time I tried the CDT Tweak I had those crazy results that I posted in the previous thread. However after reformat I was never able to get System cache about the same as Avaliable memory again and hence no boost (that's what I believe gives the boost) except a few seconds which could probably be aquired with hipros method aswell. I started using maxmem = 600 and I actually got worse times with that than with 620 so one could do some experimenting there.

Currently I'm playing with P5E3 WS Pro as I got kinda frustrated over not getting this to work again, but I'll get into it once again when I have some time to spare.

//Jon

[massman]

Hmm, I think I'm reaching the point where CDT gives a small boost, though, before I try to explain and rewrite OPB's guide, I need to ask a few questions, make a few statements. I hope that Kevin finds the time to respond and point out the wrong ones .

- CDT basicly is an improved Copy-Waza. It's build on the same fundamentals, but is more elaborate. The average gain depends on what frequency you're running, but at 3.6 - 4.0G, the gain should be +/- 6 to 10 seconds (if copy-waza and cdt are done correctly).

- The superpi run can be split up in several segments, two of them are very recognisable (got some graphs, tomorrow). These two segments seem highly dependant of the harddisk and pagefile. OPB: does increasing the pci freq help? The lower the harddisk's latency, the better, right?

- Everyone who's wondering what file he should use:
1 file which is exactly 3x632MB big. The proces of how to create the files has already been told in the second post. Tomorrow there will be a graph.

- OPB: your theory is based on Server03, right? Just because of the LargeSystemCache tweak, which is standard enabled on 03, or is there another reason?

ELMOR: have you enabled the Largesystemcache reg key? Without that, CDT and Copy-waza are useless. (Beardy: that's the setting we spoke about on MSN. Of course it's obvious :p)

[Onepagebook]

Pete, I will directly quote the whole thing I re-post and integrated CDT-V for you , that 's the prepare job before you doing anything further, so check your Pm 5 ~10 minutes later

[KTE]

http://www.xtremesystems.org/forums/...&postcount=327
You're "steeling" my techniques....Shame on you....
Now TRY the SAME with a BIG Copy Waza....

I didn't know about that BUT it's good that works for both of us.

I'll re-run if you want doing copywaza again.

But how big a CW file?
I tried 4GB already, two different drives, 1st partitions.