New Multi-Threaded Pi Program - Faster than SuperPi and PiFast

[poke349]

if the code performs different operations for different CPU ,
you dont have a true benchmark, you have a simple test.

PI use floating point unit , Prime use integer unit,

I do not want put this into question,
do not speed race,
however, are convinced that a benchmark should run with the same (identical) code on many CPUs to be true benchmark.

Everyone has different views on what a benchmark should be.
Some believe that you should run identical code on all processors.
That's what SuperPi is for.

On the other hand:
If code A runs faster on processor A, and code B runs faster on processor B.
Then I have no problem with benchmarking A on A and B on B if they do the same task.
This is what y-cruncher is. It is meant to be a different kind of benchmark.

Take your pick.


I'm not a fan of crippling processors by not using their features - since that hides their true potential.
It's just like having a street race: If one car has nitrous, why should you ban the driver from using it? It hides its true potential.


EDIT:
Pi doesn't just use floating-point. y-cruncher uses both, floating-point and integer.

Though the ratios of how much of each will vary depending on what system you run it on.
On Intel processors, it uses more floating-point.
On AMD processors, it uses more integer. This is because AMD's have a stronger integer unit than Intel's. (in the context of this program)

[bonis62]

Everyone has different views on what a benchmark should be.
Some believe that you should run identical code on all processors.
That's what SuperPi is for.

On the other hand:
If code A runs faster on processor A, and code B runs faster on processor B.
Then I have no problem with benchmarking A on A and B on B if they do the same task.
This is what y-cruncher is. It is meant to be a different kind of benchmark.

Take your pick.


I'm not a fan of crippling processors by not using their features - since that hides their true potential.
It's just like having a street race: If one car has nitrous, why should you ban the driver from using it? It hides its true potential.


EDIT:
Pi doesn't just use floating-point. y-cruncher uses both, floating-point and integer.

Though the ratios of how much of each will vary depending on what system you run it on.
On Intel processors, it uses more floating-point.
On AMD processors, it uses more integer. This is because AMD's have a stronger integer unit than Intel's. (in the context of this program)

I'm not a fan of crippling processors by not using their features - since that hides their true potential.
It's just like having a street race: If one car has nitrous, why should you ban the driver from using it? It hides its true potential.


in this case i am in full agreement with you.

[Alpha]

1896.62 - v0.4.4 x64 SSE3 ~ Alpha - 2 x Intel Xeon E5320 @ 2.33 GHz

If is managed to screw the formatting up (i am sure i did), just flog me to death.

[poke349]

I'm just wondering...

Would anyone be interested in a Pi-based hard drive benchmark?

It'll be a more of an all-around benchmark that will utilize CPU, memory, and disk.
The bottleneck will most certainly be the disk... unless... well... if you have like 8 of them running in parallel...


p.s. you might be able to guess what I'm getting at...

[bonis62]

I'm just wondering...

Would anyone be interested in a Pi-based hard drive benchmark?

It'll be a more of an all-around benchmark that will utilize CPU, memory, and disk.
The bottleneck will most certainly be the disk... unless... well... if you have like 8 of them running in parallel...


p.s. you might be able to guess what I'm getting at...

road is arduous,
the results are never stable on disc,
the results of tests vary , also cleaning the cache,
i thing the only way to make stable benchmarks is to
performs many bench and then averaging the final result.

This is just my personal reflection

[poke349]

road is arduous,
the results are never stable on disc,
the results of tests vary , also cleaning the cache,
i thing the only way to make stable benchmarks is to
performs many bench and then averaging the final result.

This is just my personal reflection

Indeed results aren't stable on disk. But the "benchmark" will be extremely long running. Hours to days depending on the size. Any sort of temporary uncertainty would be definitely averaged out quite well by then.

The only problem with the concept of a Pi-based HD benchmark is that the winner will pretty much be whoever has the most ram and the most HDs running in parallel.
So there's almost no point in overclocking unless you have enough disk bandwidth to become CPU-limited.

[bonis62]

Indeed results aren't stable on disk. But the "benchmark" will be extremely long running. Hours to days depending on the size. Any sort of temporary uncertainty would be definitely averaged out quite well by then.

The only problem with the concept of a Pi-based HD benchmark is that the winner will pretty much be whoever has the most ram and the most HDs running in parallel.
So there's almost no point in overclocking unless you have enough disk bandwidth to become CPU-limited.

a small contribute :

#define BLOCKSIZE 512
#define TIMEOUT 30

[poke349]

a small contribute :

#define BLOCKSIZE 512
#define TIMEOUT 30

What does that do?

[bonis62]

What does that do?

make a buffer of 512 byte

writing many times until you go to timeout,

so you have the time,

takes this time to make an estimate
without writing to disk many hours.

-------------

I am interested in your hdu application,
I already tried, but i've failed,
Having spent so much time i can give only these tips,
do not know if they can help you, i hope

[poke349]

make a buffer of 512 byte

writing many times until you go to timeout,

so you have the time,

takes this time to make an estimate
without writing to disk many hours.

-------------

I am interested in your hdu application,
I already tried, but i've failed,
Having spent so much time i can give only these tips,
do not know if they can help you, i hope

Oh, thx. Though that wasn't what I was thinking about.
It'll be on the lines of:

How long will it take to compute 10+ billion digits of pi? (or some other constant)

Provided that you don't have the ram for it, it will rely entirely on disk speed and the size of your working memory. Hence why it will be biased towards machines with a lot of memory.

Obviously, a computation of such a size will take hours when done on disk (heck, it takes an hour and 40 minutes on my workstation - all in ram). But then again, there aren't any benchmarks that last more than a few minutes.
So it'll be interesting to present a longer-running benchmark... more like an endurance benchmark that'll stress CPU, memory, and disk.

So it definitely won't be for the impatient...

Making it suitable for competitive benchmarking will be tricky.
Since the very nature of this benchmark will heavily favor the multi-socket servers that have ton of memory and ton of expansion slots to fit in a ton of hard drives.

So a slower workstation that has enough memory to "ram-only compute" the entire computation or ram-drive all the disk accesses will beat an overclocked desktop anyday...
(well, it's not like multi-sockets haven't already been dominating this benchmark... )

So the most important factors for getting a good time with this Pi-based HD benchmark will be (in descending order of importance):

1. If you have enough memory to not need the disk, or to ram-drive all the IOs.
2. The # of hard drives you have (and they won't need to be in raid). Sustained total disk bandwidth under various different sequential and and non-sequential access patterns will be what matters (though it'll be bottlenecked by the slowest HD).
3. The amount of memory you have. The more memory you have, the less the program will need to access the disk.
4. Finally, clock-speed and memory bandwidth...

[bonis62]

Oh, thx. Though that wasn't what I was thinking about.
It'll be on the lines of:

How long will it take to compute 10+ billion digits of pi? (or some other constant)

Provided that you don't have the ram for it, it will rely entirely on disk speed and the size of your working memory. Hence why it will be biased towards machines with a lot of memory.

Obviously, a computation of such a size will take hours when done on disk (heck, it takes an hour and 40 minutes on my workstation - all in ram). But then again, there aren't any benchmarks that last more than a few minutes.
So it'll be interesting to present a longer-running benchmark... more like an endurance benchmark that'll stress CPU, memory, and disk.

So it definitely won't be for the impatient...

Making it suitable for competitive benchmarking will be tricky.
Since the very nature of this benchmark will heavily favor the multi-socket servers that have ton of memory and ton of expansion slots to fit in a ton of hard drives.

So a slower workstation that has enough memory to "ram-only compute" the entire computation or ram-drive all the disk accesses will beat an overclocked desktop anyday...
(well, it's not like multi-sockets haven't already been dominating this benchmark... )

So the most important factors for getting a good time with this Pi-based HD benchmark will be (in descending order of importance):

1. If you have enough memory to not need the disk, or to ram-drive all the IOs.
2. The # of hard drives you have (and they won't need to be in raid). Sustained total disk bandwidth under various different sequential and and non-sequential access patterns will be what matters (though it'll be bottlenecked by the slowest HD).
3. The amount of memory you have. The more memory you have, the less the program will need to access the disk.
4. Finally, clock-speed and memory bandwidth...

i write this small stability app based on the sum of FP (ArcTan) and Primes,


i've removed the disk IO function because final results too busted in a small times

[poke349]

i write this small stability app based on the sum of FP (ArcTan) and Primes,


i've removed the disk IO function because final results too busted in a small times

Just curious, what algorithm are you using?
I'm currently doing something that requires primes so I also have a tiny program that'll find them. (and their reciprocals and logarithms)

So I knocked together a quick and dirty program do that. It isn't threaded, yet here's what I get on my laptop. (Core i7 @ 1.6 GHz)



It's off by one from your result because it doesn't search even numbers so it missed 2 as a prime.

[Alpha]

what algorithm are you using? i can find all the primes up to 2 billion in about 2 minutes with the euclidean sieve. 15 seconds is a 10 fold improvement!

[poke349]

what algorithm are you using? i can find all the primes up to 2 billion in about 2 minutes with the euclidean sieve. 15 seconds is a 10 fold improvement!

I'm using a very straightforward implementation of this one:
http://en.wikipedia.org/wiki/Sieve_of_Eratosthenes

The only optimization I did was to completely remove even numbers.
No vectorization, no multi-threading.

I'm implementing it as a bitstream that represents every odd number from 1 to n. So the memory requirement is n / 16 bytes.

The straightforward approach to the algorithm isn't parallelizable and is horrific when it comes to memory access.
So I came up with a way to modify it so that will parallel and ensure near 100% cache hit.
But seeing as how it's fast enough for my purposes so far, I'm probably gonna leave it as it is for now.

So a fully optimized version will probably go under 4 seconds on my laptop. (Core i7 720QM @ stock)
And even faster on my workstation or my i7 rig.

[Alpha]

well, that's that i use too.

because you are skipping odd numbers, try this:


(BTW, this is in VB)
data is an array of booleans.
everything else is an integer.

x = 3
While x * x <= endingPoint
    If data(x) = True Then
        a = x
        b = 2 * a
        a = a + b
        While a <= endingPoint
            data(a) = False
            a = a + b
        End While
    End If
    x = x + 2
End While

[poke349]

well, that's that i use too.

because you are skipping odd numbers, try this:


(BTW, this is in VB)
data is an array of booleans.
everything else is an integer.

x = 3
While x * x <= endingPoint
    If data(x) = True Then
        a = x
        b = 2 * a
        a = a + b
        While a <= endingPoint
            data(a) = False
            a = a + b
        End While
    End If
    x = x + 2
End While

The first issue with using array of booleans is that VB doesn't implement it as a bitstream. I don't know a thing about VB, but I'm reading that it does it using an array of 16-bit integers?

If that's the case, then it'll use 16x more memory than it should?

[mattkosem]

Here's what I get at my 24/7 clock.

--Matt

[poke349]

Here's what I get at my 24/7 clock.

--Matt

No HT?

That's very impressive for a 24/7 OC.

[mattkosem]

No HT?

That's very impressive for a 24/7 OC.

Nah, I ditched it a couple weeks ago. Aside from the cpu tests in 3dmark vantage, everything I run is faster at 4.5Ghz wo/HT than it is at 4.2Ghz with it. Thanks. It's 20 pass linpak stable and runs through 400% without errors with 4 memtest instances. It's not as strong of a chip as some of the D0s out there, but does pretty well.

--Matt

[poke349]

Nah, I ditched it a couple weeks ago. Aside from the cpu tests in 3dmark vantage, everything I run is faster at 4.5Ghz wo/HT than it is at 4.2Ghz with it. Thanks. It's 20 pass linpak stable and runs through 400% without errors with 4 memtest instances. It's not as strong of a chip as some of the D0s out there, but does pretty well.

--Matt

Ah. I think y-cruncher only gets about 8 - 9% improvement from HT at 50m.



Anyways:
Here's a little update on v0.5.x:

Of the optimizations that I've done so far... here are the results:

Core i7: 2 - 5% faster than v0.4.4.
Core 2: 2 - 6% faster than v0.4.4.
Phenom II: 5 - 10% faster than v0.4.4.

AMD wins here.

I think AMD K10 is gonna be able to pull ahead of Core 2 clock-for-clock.




More to come. You may have noticed that I've added a few new categories to the rankings on the first post of this thread.

[bonis62]

Just curious, what algorithm are you using?
I'm currently doing something that requires primes so I also have a tiny program that'll find them. (and their reciprocals and logarithms)

So I knocked together a quick and dirty program do that. It isn't threaded, yet here's what I get on my laptop. (Core i7 @ 1.6 GHz)



It's off by one from your result because it doesn't search even numbers so it missed 2 as a prime.

try this on your laptop and post the screen please

(this is for the prime only..and 32 bit)

http://www.xstreme.it/oprimes.zip

My algo is custom, derivate from "even numbers" and the "square root" (true / false)

[zalbard]

Here's a little update on v0.5.x

Very nice!
Any idea about the approximate release date?

[poke349]

try this on your laptop and post the screen please

(this is for the prime only..and 32 bit)

http://www.xstreme.it/oprimes.zip

My algo is custom, derivate from "even numbers" and the "square root" (true / false)

Here ya go:

Very nice!
Any idea about the approximate release date?

I'm hoping by the end of February. But it's not a promise since there's one VERY LARGE feature that still needs to be tuned and tested.
And with my workstation down until at least spring break, things are gonna be extra slow...

For that reason, I've halted all optimizations that affect ram-only computations.

There's one major algorithmic improvement that I'm working on. (It requires prime numbers - hence this whole discussion about them...)
But I'm gonna push that back to a later release. There's enough to test already in v0.5.2.
I might try it for v0.5.3, but I'm expecting a lot of issues to show up in v0.5.2 once it gets into the hands of someone who can let it rip through some very high-end workstations.

So the next few v0.5.x might be solely bug-fixes.

[bonis62]

difference of six seconds , not bad, my 32-bit code is good, but
your code is truly remarkable.........

[poke349]

difference of six seconds , not bad, my 32-bit code is good, but
your code is truly remarkable.........

Well... I compiled it for x64...