Does super pi 1.4 mod really faster than super pi 1.5 mod?

[guess2098]

After i heard many of my friends said

"xxoooxoxoxoxo 1.5mod is slower than 1.4mod"
"???blahanablah 1.4mod is faster than 1.5mod"
"banana and banana 1.4mod and 1.5mod is the same"

i gave a try today

FX60 ES handpick
G.Skill 2GBHZ handpick
DFi SLi-DR cap mod handpick bios 510-1
EVGA 7800GTX KO SLi stock
PC P&C 1KW
Tower 120
windows xp SP1

both super pi ran at realtime

1st test
super pi 1.5mod with core 0 started 1st
super pi 1.4mod with core 1 started 2nd





2nd test
super pi 1.5mod with core 1 started 2nd
super pi 1.4mod with core 0 started 1st





sry the 2nd A64 i forgot to press the refresh


well.... from this... i think maybe... maybe the 1.5mod is a little bit slower than 1.4mod about 2~3sec.
i am not sure why it made difference.does the code is a bit slower. maybe the time refresh is faster than 1.4mod. as we can see, the repeating decimal is not appear too frequenly. that is why i think 1.5mod is slower than 1.4mod

[dinos22]

that's what me and eva2000 are seeing........2-3 seconds on 32M runs

i wonder if japanese letters/numbers refresh quicker

[guess2098]

that's what me and eva2000 are seeing........2-3 seconds on 32M runs

i wonder if japanese letters/numbers refresh quicker

lol i love ur sig~

jhehehe

oh no no more sig video lol

[Eldonko]

That is strange, I better stick to 1.4

[eva2000]

yeah for me most of time 32M, v1.4 is faster than v1.5

Going after my 32M personal best time which was done with 2x 512MB XMS3500C2v1.1 BH-5 but this time with 2x 512MB OCZ PC3200 Gold UTT BH-5

--------------------------------------------------------------
:: 2x 512MB Corsair XMS3500C2v1.1 BH-5 ::
10x315HTT = 3150Mhz
166 divider
262mhz 2-2-2-5 7-14 2223 at 3.48v vdimm in bios
:: Super Pi v1.4 mod
32M = 22m 56.875s old/current personal best
32M = 22m 57.859s

vs


--------------------------------------------------------------
:: 2x 512MB OCZ PC3200 Gold UTT BH-5 ::
10x315HTT = 3150Mhz
166 divider
262mhz 2-2-2-5 7-14/16 2223 at 3.54v vdimm in bios
:: Super Pi v1.4 mod
8M = 4m 48.765s
32M = 22m 57.984s

:: Super Pi v1.5 XS
262mhz 2-2-2-5 7-16 2223 at 3.54v vdimm in bios
8M = 4m 48.516s new personal best 8M time
32M = 22m 56.890s 0.015s behind my old 32M PB heh






--------------------------------------------------------------

[Freebo]

mmm... 1.5 added bout 6(ish) seconds on to my time

[stummerwinter]

I took a closer look to 1M and made a statistical analysis...

System:
CPU: Opteron 144 @ 333 x 9
MoBo: DFI NF4 Ultra-D with 702-1
RAM: OCZ VX @ 250 MHz @ 1.5-2-2-4
OS: WXP SP2

Test:
- start System
- start 1M and run it once
- run 1M 10 times in a row an take results
- analyse data

Software:
- Minitab 14

First you have to perform a normality test to see, if your data-set is normally distributed. This is important to choose the right method for the comparison.

1.4 Mod:


P-value > 0,05 => normal distribution
Mean: 27,602 s
StDev: 0,026

1.5 Mod:

P-value > 0,05 => normal distribution
Mean: 27,605 s
StDev: 0,018

That means, we can use a so called 2-sample-t-test to compare both data sets, confidence interval set to 99 %:

First a individual plot for the data, second a box-plot:



And here in printed version, the test was set to not equal, which is our hypothesis:



Conclusion

With this high p-value in this case (0,783 - needs to be smaller then 0,05 to keep the hypothesis) we can reject the hypothesis from a statistical standpoint with a savety of 99 %, NO difference between the 1.4 and the 1.5 Version for 1M.

1.4 is sometimes faster, but with a bigger variance compared to the 1.5-Version.

I will repeat this with 25 datapoints each and also for 32M...

[dinos22]

can you tell us more about the timings you used.....whether you used auto or set values

[nachthymnen]

Same OS and same settings(240x10 2/2/2/0)

1.4 version=34.409
1.5 version=34.469

[stummerwinter]

Who? Me?

Set most at manuel, but didn´t change...

1T-1.5-2-2-4-12-16-3-2-1-3-... and the rest I forgot, but can post this evening...

I just saw, that I have a big spread in the results, even after a reboot.

I think it make sence to use more datapoints at 1M, 25 should be enough...

[LenniZ]

This version is slower at my pc also.. fugger must have bananed something when he changed the program :--)

[TaPaKaH]

For my FX-57 SuperPi 1.1M is faster than 1.4 & 1.5.
1M (SuperPi 1.1 , FX57 @ 3.7GHz) - 24s (sometimes 25s)
1M (SuperPi 1.4 & 1.5 , FX57 @ 3.7GHz) - 25.xxx (always)

[tarrcm]

Good work guys! Very interesting, I think I'll try some comparisons as well!

[afireinside]

Denny did you run them seperatly? If you run both at the same time the time varies a bit

[tarrcm]

Thought I'd give it a go as well but for the sake of time I only did 1M's.
SPi 1.1 only showed whole numbers, all the same.
SPi 1.4 gave 26.422s (average of 5 runs)
SPi 1.5 gave 26.431s (ave. 5 runs)
After deleting dat files I reran:
SPi 1.4 gave 26.406s (ave. of 5 runs)
SPi 1.5 gave 26.431s (ave. of 5 runs)

So, I guess I know what I'll be using for 1M. Should be similar for 32M?

[stummerwinter]

Ok, some more comparisons...

@HWL a discusson came up, if it´s accurate enough to run them afterwards or only after a restart.

So i compared this for 1.4 Mod...

First test run was done like this:
- start computer
- run 2 x 16k (was a recommadation from somebody from HWL to give more accurate results)
- run 1M and pic result
- and restart

mean: 27,652 s

standard deviation: 0,069 s





Then I tried i 10 times in a row, means:
- start computer
- run once 1M
- then 10 times in row an pic result

mean: 27,586 s

standard deviation: 0,020 s





Here the results compared in box-plots:



That means, the variation for method one ist bigger, even with a higher mean...

[tarrcm]

Holy cow stummerwinter! Impressive results you have there...I wish I could understand it all as easily as you do, back to school for me?
Could you give more explanation with your conclusions please?

[dinos22]

i really think you need to use results with at least 8M runs....rather than 1M.....too much variation there....

[tarrcm]

I was curious how 1M would turn out between the different versions as I am about 32M. Repeatability is another point of interest and desireable I would think.
There must be so many variables with this, perhaps some of which are particular to an individuals system but I wonder if most peoples results would be similar in terms of which version of SPi produces faster times for any given test?

[stummerwinter]

@tarrcm: Ok, first some background informations...

I joined a 6 Sigma-training at my job last year, so no way to learn that in school...

6 Sigma as a methody to analyze in a strucered why a process (whatever you like) and improve it to a level of maximun 3,4 defects in a 1.0000.000 pieces (whatever).

With help of statistical tools (I used here to analyze Minitab 14) I took a closer look here to the variances of SuperPi.

Coming to the point, every process (here running SuperPi) has a variation. And I tried to figure out, how big this variation is.

For example take the 10 times 1.5 MOD XS run:

I run it 10 times, gave me 10 results.

The average (mean) was 27,605 s (sum up all results and dived it by 10 ). 1 standard deviation (equals to 1 Sigma) was 0,018 s

That means, from statistcal standpoint based on these ten values:

- with a probability of 95 % (1 Sigma) you will get 27,605 s ± 0,018 s at further measurements

- with a probability of 99,9996 % (6 Sigma) you will get 27,605 s ± 0,648 s at further measurements


The original Ide was to have a closer look to the topic, which Pi-Version is faster (first results) and if there ist a difference running Pi in a row (without restarts) or restart it for every run.


Conclusion:

to 1) no difference between 1.4 and 1.5 at 1M

to 2) the variation is smaler when you run 1M in a row instead of restarting the system (in this case, running first 2 x 16k)


@dinos: The variation I measured was very small...

But you are right, would be interesting doing it for 8M or 32M...

I´ll start with 8M...

[tarrcm]

Thanks stummerwinter! I think I'll let you do the analysis & I'll buy the beer

[stummerwinter]

8M is allready in progress...

Beer from America?

Only this one: RAMSTEIN wheet beer

Befor the germans here kick my ass i need to say, that this guy learnd his business in Germany at Erdinger. He made his brewmaster there, really a very good beer!

I had the pleasure during my time in Wayne / NJ in 2003 tasting it.

[high5]

did this a while ago on Sempron @2.5GHz:

Attachment 45728

so no issues here.

[tarrcm]

8M is allready in progress...

Beer from America?

Only this one: RAMSTEIN wheet beer

Befor the germans here kick my ass i need to say, that this guy learnd his business in Germany at Erdinger. He made his brewmaster there, really a very good beer!

I had the pleasure during my time in Wayne / NJ in 2003 tasting it.

Not exactly...I ment Germany, I'll go there
I'll keep my eyes open for that! The best I've had to date was Kulmbacher.

[stummerwinter]

So, took a while (was bussy with my DICE-test), but have now 8M-results ready...

Same system, like posted before...

Test performing: 1 x 1M, then 10 x 8M in a rowe and take results

First, normality tests again:

8M 1.4 Mod



Normaly distributed!

8M 1.5 Mod



Normaly distributed!

Comparison:





Result:

This time, 1.4 Mod is slightly faster then the 1.5 Mod version!

Two-sample T for 8M 14 Mod vs 8M 15 Mod

N Mean StDev SE Mean
8M 14 Mod 10 304,899 0,134 0,042
8M 15 Mod 10 305,010 0,152 0,048


Difference = mu (8M 14 Mod) - mu (8M 15 Mod)
Estimate for difference: -0,111000
99% CI for difference: (-0,296757; 0,074757)
T-Test of difference = 0 (vs not =): T-Value = -1,73 P-Value = 0,101 DF = 17

At leat, it´s just 0,1 s at 8M...

32M is in preparation...