hwbot multipi - multithreaded superpi (open beta)

[Gautam]

Frederik and Kunaak-

This isn't an issue about people being upset over extra stuff to install. It's a question of which language will work best to create a taxing benchmark. Java is inherently too inefficient to be optimal for the task at hand. This current version fails to even stress the CPU 100% at realtime. (Check task manager) Java is bloated and a little clunky with memory allocation. It has a lot of nice uses, but IMHO, it is less than optimal to be used in such an application. I too feel that C/C++ would be better choices, with seperate versions compiled for Windows, Linux and MacOS. The loss of portability is unfortunate, but I highly doubt that even a Java version would result in identical performance across all operating systems given the same hardware.

[dig412]

The 100% cpu thing is odd, but it uses 98% on my (single core) pc. The problem with having the c versions for every platform is maintaining it. RichBa5tard has already said its too much hassle, and if it works well enough in java, which i think it does, then its no problem.

[dr_sharp]

2:32.938 - AXP-M @ 2420MHz

These times are still too quick IMO. If this program is to test the next generation of cpus... its going to need to hold conroes to 2min+

[Gautam]

I think the duration is all right...takes me around 40 secs. A bit slower would be nice though to take into account quad scores. Here's my score for what its worth.

[RichBa5tard]

Frederik and Kunaak-

This isn't an issue about people being upset over extra stuff to install. It's a question of which language will work best to create a taxing benchmark. Java is inherently too inefficient to be optimal for the task at hand. This current version fails to even stress the CPU 100% at realtime. (Check task manager) Java is bloated and a little clunky with memory allocation. It has a lot of nice uses, but IMHO, it is less than optimal to be used in such an application. I too feel that C/C++ would be better choices, with seperate versions compiled for Windows, Linux and MacOS. The loss of portability is unfortunate, but I highly doubt that even a Java version would result in identical performance across all operating systems given the same hardware.

Gautam,

The goal of superpi/multipi is not to be the most efficient Pi calculation app. The algoritm is filled with checks for bad memory / calculation errors to dedect unstable overclocks, that's why it's so 'slow'.

As for the not stressing the processors 100%, that's intentional too. The first version (0.1) stressed all the cores at 100%, but that gave such a huge performance gain on a quad core, it just wouldn't be fun. A mediocre 2ghz A64 needed 4 minutes, a quad core 30 seconds... those differences are just way too high to make it fun for everyone. v0.2 taxes the memory a lot more, which causes a 100% cpuload with one core, and ~80% with 2 cores. I think this is much more fun as now your memory speed matters too.

I am, in no way, claiming that I've used the most effecient algoritm to calculate pi, nor that I've used the most efficient language to do it.

My goals are:
- fun to benchmark
- advantage with multiple cores, but single core should still be 'fun' and quad core should not obliterate all at stock speed
- cross platform
- easy integration with hwbot.org

My goals are not:
- calculate pi in an efficient way

SuperPi doesn't uses the most efficient algorithm either... i haven't seen anyone complaining about that either.


One last thing Gautam, do you know how much time it takes to develop it in C++ for 3 different platforms properly? I'm with Kunaak on this, i've seen too many great benchmark apps go down the pooper because the community was demanding too much. I can write it in java in my spare time, I can not do it in C++ without taking a break from work.

[hrvojezg00]

54 sec on dual dual 265 opterons

[Gautam]

I didn't realize that that was deliberate...so you ended up "cheating" ultimately. But you know that I'm in support for leveling the playing field, so I retract what I said earlier in that case. Good concept.

I wasn't expecting this to be such a sore subject. You know I appreciate all of your hard work, but I always try to share my thoughts as well. The efficiency isn't a big deal like I said...just the stressing of both the CPU and memory subsystem. If you feel Java is the best path to take then no worries.

[RichBa5tard]

Yeah, cross platform is a sore subject for me. I don't want to install windows just for benching! Give us mac/linux users a break!

[Gautam]

Riddle me this though, do you think that *nix results will be comparable with Windows results?

Anyone with a Core-based Mac run this yet?

[M.Beier]

RichBa5tard, you have PM.

EDIT:
Got a 53 sec time on my 3.6ghz conroe.

[RichBa5tard]

Riddle me this though, do you think that *nix results will be comparable with Windows results?

Anyone with a Core-based Mac run this yet?

I honestly think so, yes.

I've got a core based mac, with dual boot windows. I'll compare it tonight.

[RichBa5tard]

Well, I've tested 0.2 thoroughly and there seems to be a difference between platforms I think it has mainly to do with the amount of open processors primarely caused by the window manager. The heavier/fancier your GUI, the slower multipi gets.

Ranked by speed:
linux terminal > linux KDE > windows XP > MacOSX Tiger

The gain for a linux terminal was too much to call it fair, so I started tweaking the algorithm again.

The new algorithm (0.3, to be released early next week), will tax memory less, but will be comparable between platforms. I haven't had the chance to test it on Mac OSX yet, but results on a linux terminal and windows gui are nearly identical (I did 5 runs on each and fastest run on linux was not more than 200 milliseconds faster than slowest run on windows xp). If I get the same, identical results on macosx i think we have a winner.

It doesn't tax the memory as much anymore though, but I find consistency between platforms much more important than taxing the memory.

[Gautam]

Yeah, I was expecting mostly the same. In case you haven't noticed, all experienced Pi benchers use Windows Server 2003. Despite using the same kernel and otherwise being virtually identical to XP, its much leaner and as a result will nearly almost win, albeit by a tiny amount. Linux would be one step further.

Once again this is quite a feat- a multithreaded benchmark comparable across virtually any platform. Quite unprecedented I believe.

[RichBa5tard]

w00t

I'll see if I can get hold of a Win2003 somewhere.

[Gautam]

180 day eval free from Microsoft.

[RichBa5tard]

Thanks!

[kiwi]

Sorry what do you mean by "taxing" memory?

I can see there is a switch to VM how much ram to use but not sure what is optimal

[dinos22]

Well, I've tested 0.2 thoroughly and there seems to be a difference between platforms I think it has mainly to do with the amount of open processors primarely caused by the window manager. The heavier/fancier your GUI, the slower multipi gets.

Ranked by speed:
linux terminal > linux KDE > windows XP > MacOSX Tiger

The gain for a linux terminal was too much to call it fair, so I started tweaking the algorithm again.

The new algorithm (0.3, to be released early next week), will tax memory less, but will be comparable between platforms. I haven't had the chance to test it on Mac OSX yet, but results on a linux terminal and windows gui are nearly identical (I did 5 runs on each and fastest run on linux was not more than 200 milliseconds faster than slowest run on windows xp). If I get the same, identical results on macosx i think we have a winner.

It doesn't tax the memory as much anymore though, but I find consistency between platforms much more important than taxing the memory.

lol you just gave me a good idea

*searches for my copy of linux that installs in RAM heh

[dig412]

lol you just gave me a good idea

*searches for my copy of linux that installs in RAM heh

What you need is a live cd, they dont use the hard drive at all :P
Try something like SLAX, its pretty easy to use, and it can copy itself entirely to ram.

[AAdjuster]

1:54.254 sec's. On my Newark at 3060 ghz.

[Peen]

I havent posted here in like a year but decided to try this out

P4 Prescott w/HT 3.6ghz - 2m 31s

P4 Prescott w/o HT 3.6ghz - 3m 11s

Nice benchmark!

[CCUABIDExORxDIE]

conroe really doesnt dominate here for the mhz. i wonder what an x2 at like 3.6ghz would do.

[massman]

conroe really doesnt dominate here for the mhz. i wonder what an x2 at like 3.6ghz would do.

certainly not 40s. I guess about 50s

[dsl]

DC Opty 175 @ 2.25GHz - 1m 39.266s
24/7 OS, a lot of apps running in background

[bsodmike]

This doesn't work right in OS X 10.4.7... it's really screwed up. ran as sh file.sh

[01:53:54] kurosaki-ichigo:~/Desktop/multipi michaeldesilva$ sh multipi.sh 
**** MULTIPI 0.2: a multithreaded pi calculator ****

USE: /MultiPi <numberofiterations> <workunitsize>
<numberofiterations>: total number of iterations to find pi - DEFAULT: 100000000
<workunitsize>: divise iterations into blocks of this size to spread work over cores - DEFAULT: 5000000

WARNING: this is a beta version, do not distribute unless you're a betatester. Results may vary in future versions.

Starting Pi calculation
Available cores dedected: 2
Calculation on core 1 started.
Calculation on core 2 started.
30m 08.295s - Loop 1 of 20 finished
30m 16.048s - Loop 3 of 20 finished
^Z
[1]+  Stopped                 sh multipi.sh

it goes towards 60secs, and ends up with a 32min end time..it's messed up