New Multi-Threaded Pi Program - Faster than SuperPi and PiFast

[poke349]

The guy in Japan? There can be only one.**

Shigeru Kondo aka 先生 Pi / 戦艦 Pi

**Ok, we all know there's another..

Daisuke Takahashi aka 天皇 Pi

XmX

Yes it's Shigeru Kondo.

I have no idea where he gets his machines... but whatever the case, they're always top of the line...

You'd think that AT LEAST SOMEBODY on XtremeSystems would be able to beat him...
Though even if there were... I doubt he/she would've seen this program or this thread...

[Movieman]

Yes it's Shigeru Kondo.

I have no idea where he gets his machines... but whatever the case, they're always top of the line...

You'd think that AT LEAST SOMEBODY on XtremeSystems would be able to beat him...
Though even if there were... I doubt he/she would've seen this program or this thread...

[poke349]

uh......

Updated the list with your results

6 new world records

[alicemb]

hi,

The first efficient and scalable Multi-threaded Pi Benchmark:

60x faster than SuperPi on Core i7.
5x faster than PiFast 4.3 on Core i7.
32M Pi in 9.982 seconds on 4.2 GHz Core i7.
10m Pi in 2.996 seconds on 4.2 GHz Core i7.
1 billion digits in 8 minutes on 4.2 GHz Core i7.

hmm...



here a little sample of pi x64 sse3 chudnovsky, but i dont push/optimize it more (means: will not use it in MaxxPI² / www.maxxpi.net),
because it will only perform on intels that much and this is a *nogo* for comparative Benchmarks, but for getting WR it's ok.

nice work, Alex!

cu

[Movieman]

uh......

Updated the list with your results

6 new world records

Hey, someone had to step up for the honor of XS and I'll be damned if I know anyone else on this forum that has W5590's to match that guys machine in Japan!

[poke349]

hi,



hmm...

here a little sample of pi x64 sse3 chudnovsky, but i dont push/optimize it more (means: will not use it in MaxxPI² / www.maxxpi.net),
because it will only perform on intels that much and this is a *nogo* for comparative Benchmarks, but for getting WR it's ok.

nice work, Alex!

cu

Wow. Good work! That beats QuickPi 4.5 by more than 2x.

You might wanna see how it compares with Parallel GMP-Chudnovsky. (if you're willing to compare across platforms)
http://www.amdzone.com/phpbb3/viewto...?f=23&p=168248


As far as hardware comparability goes, I'm not that interested in comparing between different vendors. But the x64 SSE3 binary is still there for anyone who wants to do it anyway.

v0.4.4 is probably gonna have a specially tuned x64 SSE3 binary for Phenom. (whenever my roommate decides to build it...)

Hey, someone had to step up for the honor of XS and I'll be damned if I know anyone else on this forum that has W5590's to match that guys machine in Japan!

Off Topic:
Dave... Seriously... what timezone are you in? Why are you still up?

[Movieman]

Off Topic:
Dave... Seriously... what timezone are you in? Why are you still up?

east coast..5:52am here right now..

[XmikeX]

Is it wise to disturb Japan on this issue?

Remember what happened the last time...



SARABAA CHIKYUU YOOOO....

XmX

[ensu3]

Ran on my 24/7 overclock

[poke349]

Is it wise to disturb Japan on this issue?

Remember what happened the last time...



SARABAA CHIKYUU YOOOO....

XmX

I guess I'm not the only person who watches too much anime...

Both my roommate and I... neither of us speak Japanese.

But we're almost completely immersed in it while we're in our room...
posters... music... desktop themes... etc. All Japanese Anime...

Ran on my 24/7 overclock

Nice...

Maybe if I had better cooling I could try 4 GHz 24/7...

[Movieman]

Is it wise to disturb Japan on this issue?

Remember what happened the last time...



SARABAA CHIKYUU YOOOO....

XmX

Interesting but we're talking brute power here not pretty pics.
First you need a very stable machine, then reduce the overhead so there's no crap running, then you need whatever speed you can get and remember, these gainestown machines aren't overclockable, at least not yet so the mhz is even..
That's what we do in WCG every day.
The machines run at 100% load for months on end without a boot.

[XmikeX]

Interesting but we're talking brute power here not pretty pics.

At this point, I should mention that the only point to that pic was to get a laugh out of poke349.

""MISSION ACCOMPLISHED""

XmX

[poke349]

At this point, I should mention that the only point to that pic was to get a laugh out of poke349.

""MISSION ACCOMPLISHED""

XmX

I think you confused Dave more than you confused me...

[poke349]

So I've been noticing that some people are getting a bit confused over the inconsistencies caused by running a 32-bit OS on an x64 capable processor.

Like someone was confused at why a pair of similarly clocked (and identical) Core 2 Quads would have vastly different times.
And why a Core 2 Quad could be beaten by a slightly higher clocked Core 2 Duo...
(All due to a 32-bit OS running on a 64-bit capable processor.)

So it seems like the launcher binary does too well a job of hiding the fact that there's multiple binaries (and multiple code paths) - to the point that some users are completely oblivious to the fact that their 64-bit processor is being crippled by a 32-bit OS...

So here's what I've added. It'll be in the next release.

[poke349]

Version 0.4.4 is out!

• New binary tuned for AMD K10. (0.5 - 2% faster than x64 SSE3. Hardly worth the effort, but it's the best I can do. It's here to be fair to AMD.)
• Added start and end times.
• Added CPU utilization and multi-core efficiency statistics.
• Ability to disable hexadecimal digit output in the Custom Compute option.
• Slightly faster Log(2), Log(10), and Euler's Constant.

Otherwise, this version is speed-consistent with v0.4.3 for all Pi computations.

Like v0.4.3, this version has not been fully tested before release. (I've hardly tested it at all.) So be sure to let me know of any bugs you find.


Other things:

The CPU utilization and multi-core efficiency %'s include the period where the digits are written to the disk. (which has low CPU usage)
So the %'s will be slightly deflated. This was done due to practical reasons with the way the code works. (And the fact that the computation isn't really "done", until all the digits have been formatted and saved.)
Writing to a ram disk will help out.

I've noticed that it's ridiculously hard to get over 90% efficiency with 8 logical cores.
I haven't been able to get a Pi computation to hit 90% on my i7 rig with the ram that I have. (I haven't tried with a ram drive yet.)
Because of internet problems back at home, I don't have access to my 64GB workstation to see if 10 billion digits is enough for 8 cores to hit 90%. (not gonna get fixed until winter break )






And here's a screenie of the program's error-correction in action.
This time it's for real. (Not staged like the other screenie I posted.) It didn't quite have enough vcore to be stable.



*I've fixed the typo in the error message.

[Serotoninn]

ooo this new version is very nice, over 90% efficiency too
though why did you remove the misc. stuff at the end such as the processor info?
nvm, I'm an idiot... only benchmark mode has the validation

[Damian^]

First run


Second run


Third run

[Damian^]

a fourth run is always higher than the third score. Time to get clocking

[poke349]

It looks like it's got something to do with your background programs.
Notice how your CPU utilization/multi-core efficiency goes up between the runs.
Anything in the background that uses even the slightest amount of CPU will drop your multi-core efficiency and hurt your score.

Running with real-time priority really seems to help.

[ATiFreak]

Q6600 @ 3.3 25m 0.4.4.7760 fix1

[poke349]

Here's the first quad-socket result!

Credits to: WaiKin Wong + Rickie Chang

My friend and I did a run on our computer. An HP DL 785, 4 x Quad core AMD Opteron 8356 (2.3 GHz) with 128 GB of memory. The backing store is a FusionIO 600 GB disk. OS is Windows Server 2008 R2 Datacenter.

Current Settings: (select option # to change setting)

  1     Constant:    Pi
  2     Algorithm:   Chudnovsky Formula

  3     Decimal Digits:        50,000,000,000
  4     Hexadecimal Digits:    41,524,101,187

  5     Threading Mode:    16 threads
  6     Write Digits To:   F:/temp
  7     Compress Output:   Yes - Compress digits and split them into multiple
                           files with  1,000,000,000  digits per file.

  8     Computation Mode:    Basic Swap
  9     Swap Disks:          1

        Memory Needed:        100 GB
        Disk Needed:          118 GB

  0     Start Computation!


option: 0



Constant :  Pi
Algorithm:  Chudnovsky Formula

Decimal Digits    :   50,000,000,000
Hexadecimal Digits:   41,524,101,187

Threads:    16
Mode   :    Basic Swap  ( Disks = 1 )

Start Time: Fri Dec 18 02:30:41 2009


Allocating and Reserving Memory...       100 GB
Constructing FFT lookup tables...


Begin Computation:

Summing Series:  3,525,683,470 terms
Time:    61998.655 seconds  ( 17.222 hours )
InvSqrt...
Time:    2290.942 seconds  ( 0.636 hours )
Final Multiply...
Time:    3421.265 seconds  ( 0.950 hours )

Pi:  67710.880 seconds  ( 18.809 hours )

Writing Hexadecimal Digits:   41,524,101,187  digits written

Constructing Base Conversion Table:
Time:    2944.222 seconds  ( 0.818 hours )
Base Converting (Primary Cutting Parameters):
Time:    14787.232 seconds  ( 4.108 hours )

Writing Decimal Digits:   50,000,000,000  digits written


End Time:   Sat Dec 19 03:01:53 2009

Total Computation Time:                 85443.384 seconds  ( 23.734 hours )
Total Time (including writing digits):  88269.942 seconds  ( 24.519 hours )

CPU Utilization:        1194.89 %
Multi-core Efficiency:  74.6809 %

Last Digits:
3967497298 9229992389 7944203755 6895008004 9953150775  :  49,999,999,950
6599559400 3447556105 3766739199 8513398712 7510930042  :  50,000,000,000


Press any key to continue . . .

Digits appear correct. They match the ones on Shigeru's Kondo's website.

[zalbard]

Since HyperPi fails to run for me, I've been looking for a replacement memory / mem. controller stress testing software.
I am not so much into numerical benchmarks, but I gotta say this one has a great potential as a stress testing tool.
LinX doesn't seem to work so well with HT due to Intel's libraries, while y-cruncher seems to be perfect. In fact, it managed to crash my LinX stable rig! O.o
So huge props to you for developing it!
Just hope you'll continue optimising the program (higher efficiency usually means higher stress levels) and get the stress testing mode out of beta soon! Thanks again!

[Talonman]

Installed... Love it.

Such a joy to see my system memory using over 6BG when I select opt#6.

Fine job!

[poke349]

Since HyperPi fails to run for me, I've been looking for a replacement memory / mem. controller stress testing software.
I am not so much into numerical benchmarks, but I gotta say this one has a great potential as a stress testing tool.
LinX doesn't seem to work so well with HT due to Intel's libraries, while y-cruncher seems to be perfect. In fact, it managed to crash my LinX stable rig! O.o
So huge props to you for developing it!
Just hope you'll continue optimising the program (higher efficiency usually means higher stress levels) and get the stress testing mode out of beta soon! Thanks again!

Glad you like it!

You bet I'll continue to optimize it. I already have a couple of new optimizations that are almost done. But I'm holding back on them to give the current version a little more "life".

There's a number of forums running y-cruncher competitions using the current release, so to avoid wrecking havoc on them (like I did with this thread), I'm probably going to branch the program into "newest" and "benchmark" versions.

That'll let me continue with optimizations while maintaining the speed consistency needed for competitions.

And maybe once in a while (say when AVX comes out), I'll unload all optimizations at once into the "benchmark" version and push for everyone to switch to it.


EDIT: The stress-test option is pretty much stable. The only reason why it's still in beta is because the 2 threads write independently to the console - which isn't "entirely" thread safe.
(It won't crash because of it, but you could get a situation where the output from the two threads get interleaved with each other.)

Anyhow, I've intentionally offset the two threads by 1 second to make it very rare for them to print at the same time.

Installed... Love it.

Such a joy to see my system memory using over 6BG when I select opt#6.

Fine job!

You can probably figure out what kind of computer I used to write and test this program.

btw, there's a slight chance that I might try to port it to fermi in the future... no promises though...

[Kurumi]

Ran the batch benches with the new 0.4.4. Here are my results:

25000000 8.074 7573850b0bc23f8e1bee78aaa4354166
50000000 17.922 864abf38181e874d028bf4272c443e26
100000000 37.940 48a784fc35471ec408603601ec00df76
250000000 107.450 647c8e7c845bbb4c1b732a2d71c57923
500000000 234.795 1174e0882e3932251d65f55e6d8c44dc
1000000000 519.458 677f930fa3dcd53514e3e5c8dee7ae59
2500000000 2433.400 8f48e6e52fdfca981d82a0005220b7b7


Guess there's no page-trashing this time.