The CDT and copywaza lab

[mrlobber]

Sick post KTE Can't wait the time I'll be able to run over everything you wrote and repeat it. I already noticed a couple of very slight differences with what I was doing but to see their possible impact, testing, testing, and again testing is needed

May I ask another question (seems to be trivial but maybe it isn't): which is your system partition? C:\ as usual (which would mean you have Superpi folder on your system partition), or maybe F:\, or even another one?

From what I've experimented with over 4-5 methods of improving memory/cache and getting it to from 580-490/620-500, improving memory/system cache is not all that is happening here with CDT. That's what CW does best.
CDT is improving the memory/cache like CW but also there is something do with the method => something happens to your hardware/OS prefetch/cache by running this method which you do not get by running CW or anything other that I've tried and it is this that improves the time over a CW.

From a pure theoretically speculative viewpoint , with every day it seems more to me that CDT itself is not about the memory/system cache balance at all, it is about those improvements you're talking about and the ultimate goal should be to find what kind of improvements these are and which are the actions creating them. The memory/system cache balance might be just an added bonus when doing together CDT + copywaza = (we know as) "CDT IV", that is, all the copying between drives / partitions is needed mostly for this reason (integration of copywaza into CDT).

[KTE]

C: is also my system partition and where SPi is.

Thanks.

[funkflix]

Thanks for the really detailed post KTE!

Tried it today on my AM2 system.

Without CDT:


With CDT:

[KTE]

Very nice funkflix, no problem. Can you try a copwaza test keeping everything else the same, following this method please: http://www.xtremesystems.org/forums/...&postcount=315

On a side topic; not that I care, but if you had any way of preventing cheating in Super Pi then this name calling would never happen and you'd know who cheated for their time and who not without guessing. It struck me only last night, which shows you how little I think about Super Pi but right now you have no way of verifying the accuracy of someones scores.

That's like Asafa Powell running 9.74s in 100m at the Olympics infront of all and the verification body allowing people to run in their back fields with a monitoring device they install across a measured 100m track distance. The monitoring device only captures when someone crosses the start line, how long they are ontrack and when they cross the finish line and works out the time taken independently. It records the time and uploads it automatically to the governing body database.

There's 3 billion or so people who would've ran 100m at some time or another -> people compete in it. When they compete they are willing to do anything to get a better time and beat his record, especially when it's not standardized.

So bit by bit people start getting 10s, 9.8s, and suddenly a 9.5s creeps in. Not long from then you have 9s, 8s and even lower. Soon you see faster times than ANYONE like Asafa can get near, so he retires a loser at 1000th place.

Why? because the verification and authentication contained massive loopholes.

So the people used a mophead to start and finish the race in their back yards and those times were uploaded perfectly as the device can't tell what exactly crossed the lines. Get me? Now how bad is that.

We need some automated non-user changeable verification of what speed/settings the SPi run is started at and what it finishes at to verify the authentication of every time submitted.

The sad case of the world is, unless you take measures to curb it, many people will try and cheat to a victory in anything, doesn't have to be someone you dislike or suspect.

Otherwise even I could've said 11000MB/s with DDR1600 RAM !!

(make note: some oc bug)

[funkflix]

Very nice funkflix, no problem. Can you try a copwaza test keeping everything else the same, following this method please: http://www.xtremesystems.org/forums/...&postcount=315

Yep, will try it. Result should be in here within the next hour.

[funkflix]

Now a run with the hipro5 copywaza. But i didn't used the registry-tweaks, bcs. of the better comparability.



For me it seems that both tweaks do about the same.

[dinos22]

yeap that's what the gains were that i was getting with CW on AMD
maybe even a touch faster

[KTE]

I can recheck your gains with AMD builds funkflix, I'll do that when I can. I have two one new and one slightly older (5000+ BE) AMD builds lying here. I'm not sure if the BE will get to 3500 stable like yours though. Just no RAM yet on any of them and the second CPU needs a newer board. Thanks for the tests.

[mrlobber]

Couldn't resist and took my home laptop with me to my work for 2 days (after all, I could leave it running Superpi while I was working, right? )

The results... well... again they speak for themselves

Platform:

• Barebone laptop (manufactured my MTC)
• Core Duo T2600 (2.16 Ghz), actually an ES cpu left from the Aopen 975X days
• 2x1 Gb DDR2-667 Mhz ram (5-5-5-15 stock timings)
• Samsung 5400rpm 120Gb HDD
• ATI Mobility X1600
• WinXP SP2

Stock 32M score (24/7 OS, absolutely no tweaks):
27m16.625s (you can compare on hwbot, a couple of similar 27 minute scores there as well - done on laptops obviously)

Tweaked Win score (maxmem=600, pagefile 512Mb, registry tweaks, luna silver theme, many services disabled):
26m27.953s

Copywaza run #1 (did it my usual style, copy of "cdtfile" 3x times in Superpi folder):
26m05.219s

the best CDT I managed:
25m25.250s
KTE, you were clear enough in your post how you did the CDT, and this time I followed it letter by letter . The only difference was that I was on a single HDD, not 2 identical as you were.
(this was also the moment I thought I've finally managed to get that CDT secret working, a 40sec improvement over copywaza )

...until...

Today, I did several copywaza runs again (first, hipros style, which turned out to be a bad one, then I rebooted and tried my usual stuff a couple of times).
And guess what, this time I did:
25m25.390s

Several conclusions from it all:

• again, basically no difference for me between my CW and CDT
• the patterns observed in available memory and system cache were the same as in KTE's runs, that is, CW could get them better balanced in numbers (on that last run: 442/445 Mb) while CDT not so good (~430/441 Mb)
• if I start 32M, then close it on any loop (3rd, 4th or 6th)and do either CDT or copywaza, my next full 32M run always turns out to be very bad.
• I had a Superpi folder on both my HDD partitions, and one time while testing CDT accidentally run Superpi not from C:\, but from D:\. And it didn't make any notable difference just around 0.1 seconds.
• CDT usually makes the first 2 or 3 loops faster than copywaza but later copywaza catches it up, in my case at least (I observed this on my C2D 3600Mhz system as well).

[KTE]

Well done mate.

[*] CDT usually makes the first 2 or 3 loops faster than copywaza but later copywaza catches it up, in my case at least (I observed this on my C2D 3600Mhz system as well).[/list]

Ah, this is something I observe too! But only on the runs where CW gain=CDT gain. In those cases, the first 3-4 loops are quick but after that it slows down to CW time. I can't comment "affirmatively" because I haven't tested other clocks. I might be able to get over and pick RAM up tonight after work to test more, but usually those runs are a bit slower than CDT @ 100% of the technique I follow.

[mrlobber]

Ok, I made a quick Excel comparison of 4 different runs as well... somehow forgot to save that good CW screeny?



Notice that dead precision between the two CDT runs (except for the loops 18 & 19, where a sudden drop as well as an "equalling" jump happens ) Also, indeed, as massman showed in his screenies at the beginning of this thread, the pattern differs if Pi was run from different locations even though the final results might be almost similar...

[KTE]

I get very reproducible results with both CDT and CW, although I honestly never have the time to run it-> it just takes too long.

Also mrlobber, have you never seen a 4s-5s difference in the loop times in a 32M run w/CDT or CW?

I can get a 29s loop, a 30s, a 31s, a 32s, a 33s, many 34s and a few 35s loops in the same 32M run, repeatable. It is usually when CDT is working only <a percentage> of its maximum capability for a reason I haven't found out yet, so some are quick (where its affecting) and others are standard (where its not making a difference). When it works fully, every loop is very consistent and quicker than a CW loop which always (with me) gives you more than 2-3s drop in total time over the 24 iterations. I guess I need to spend hours on it to figure it out but I still think there's more performance gain left to uncover, around the - 8-10s mark over best CW. I just don't understand the missing "cues" yet to know how to go about troubleshooting it.

[massman]

Ok, I made a quick Excel comparison of 4 different runs as well... somehow forgot to save that good CW screeny?



Notice that dead precision between the two CDT runs (except for the loops 18 & 19, where a sudden drop as well as an "equalling" jump happens ) Also, indeed, as massman showed in his screenies at the beginning of this thread, the pattern differs if Pi was run from different locations even though the final results might be almost similar...

I might find some time this weekend to try some things out. Got a 3DMark thread which I have to feed as well .

By the way, interesting graph, it seems that lowering the variability of the time to do one loop is the key to a fast 32M run. I've attached a graph of some other 32M runs I had in my excel sheet.

[KTE]

I had time so I ran some tests.

How much is the hard drive involved with a Super Pi calculation?

I had nothing else running and so measured every query/read/write the application and OS made towards the hard drive.

Well, whadya know.

• Super Pi 1M took 52 seconds (P4C).

• In those 52 seconds Super Pi made 11,075 disk queries/read/writes.

• Of them there were only 19 queries.

• The first Super Pi 1M action was to query and open the file Msimtf.dll.

• Super Pi made 10-25 read/writes per millisecond during the 1M run.

So, anyone using a RAMdisk or something with less latency than a typical HDD, theoretically, will have faster times considering they know how to.

Attached is the starting output of Super Pi.

[kiwi]

OK, now it looks like science

[dinos22]

I had time so I ran some tests.

How much is the hard drive involved with a Super Pi calculation?

I had nothing else running and so measured every query/read/write the application and OS made towards the hard drive.

Well, whadya know.

• Super Pi 1M took 52 seconds (P4C).

• In those 52 seconds Super Pi made 11,075 disk queries/read/writes.

• Of them there were only 19 queries.

• The first Super Pi 1M action was to query and open the file Msimtf.dll.

• Super Pi made 10-25 read/writes per millisecond during the 1M run.

So, anyone using a RAMdisk or something with less latency than a typical HDD, theoretically, will have faster times considering they know how to.

Attached is the starting output of Super Pi.

theory and practice are very different

i can run ERAM and have zero HDD use

time don't improve
in fact they are worse in 1M

[STEvil]

Had a thought, how might HDD cache size affect these tweaks?

Also will different HDD controllers affect performance?

edit - and what about block write size/etc?

[massman]

theory and practice are very different

i can run ERAM and have zero HDD use

time don't improve
in fact they are worse in 1M

That raises another question: How come that the theory is wrong?

Very interesting question, in fact.

[KTE]

True. dinos22 when you say you run ERAM, do you install the OS on it too?

Don't worry, I'll test the "possible" impact of RAMdisks in my build. I have two sets of DDR2 RAM now. It may not make a difference where you already have fast, low latency drive access but it may do if you don't.

BTW that mapping is from the calculation start of 1M not from the opening of SPi.

And another thing for guys benching: I'm not sure why you disable a core and deem it fast (unless you experienced it?) but I get slower times by running on only one core, repetitively. My Spi is by default running 90% on Core0 and 10% on Core1. In 1M, its 0.165-0.200s slower for me no matter which single core I use, even if I just set the affinity to one.

[mrlobber]

True. dinos22 when you say you run ERAM, do you install the OS on it too?

He can't, ERAM is a software RAMdisk

[KTE]

Then it's not the same thing as what I mentioned can possibly see a little gain. OS dll files have to be on the "faster space".

[massman]

iRam with OS installed on should be faster than a regular harddisk, no?

[mrlobber]

iRam with OS installed on should be faster than a regular harddisk, no?

Who knows... I have i-RAM myself, but no DDR modules to put in it at the moment, otherwise this would have been the first thing I'd have tested

However, I'm strongly suspicious that even an ultra fast/zero access time HDD for OS/pagefile/Pi together itself is not the key to 32M power - the "system balance" Kevin is talking about probably is involving much more than just this.

[KTE]

Well, right now I'm fine with just discussing the tweak, its workings and its benefits. Humans are what they are today by discussing among one another and sharing or you'd all still be in caves. But ultimately this is the Xtreme Pi section, meaning people here being extreme addicts of Pi will want to compare absolute times especially those competing day and night.

Again, I made it clear: if you want to "catch" and "stop" cheating by everyone and anyone, develop something which detects the SPi start and stop hardware settings. There's no other way, even video editing movies is easy.

Looking at Kevs time compared to my C2D Pi testing at 3600 and those of others competing, his time is very very very ... very ... very fast. The only way I know I can get that time following what we know is if I "bump" up the clocks at the start and then drop them back again for my ss. Which we know the right name for.

It's like Schumacher being lapped a second behind MacRae in F1.

Because Super Pi mainly depends most on CPU MHz. 1MHz gives you enough benefit.

514x7 3597 1233 5-5-5-4-10 = 13.468s

To those waiting for some sort of magic; I can say right now that I don't think with whichever hardware/software at 3600 I can even get close to 13.8s at those settings. I haven't tried, but I've played about enough to have a good approximation of my own build. But I can most likely show that CDT works for 32M at any settings.

[massman]

Thanks for the really detailed post KTE!

Tried it today on my AM2 system.

Without CDT:


With CDT:

For CTD and A64, try MAXMEM=104 and three files of 136MB