AMD Opteron 48 Core system

[Sheik]

Been having some great fun with this system the last week. Thought I'd post a few scores

System:

Tyan B8812F48W8HR
4 x 12 Core AMD Opteron 6168 processors (1.9 GHZ)
64GB Kingston DDR3 1333MHZ ECC RAM
8 x 600GB Seagate 15.700rpm SAS drives
LSI MegaRAID SAS 9261-8i RAI card
Windows 2008 Server R2 Enterprise Server

Unfortunately my graphics card doesn't even begin to do this system justice so no 3D testing

Node interleaving and NUMA and can have quite a big effect on performance in certain tests.

Some of the better results achieved are as follows:

Cinebench 11.5

Single CPU - 0.58 points
Multi-CPU (48 threads) - 23.94 points
MP Ratio - 41.64x

Best results were achieved with Node interleaving enabled (i.e. non Numa mode)

YCruncher 10 000 000 000 test (10 000 000 000 chart)

2,121.99 seconds

Again the best results were achieved in non Numa mode.

ECC memory mode also made a huge difference on some of the synthetic benchmarks like Sandra.

Sandra Memory Bandwidth

ECC mode - good
Numa disabled

Benchmark Results
Aggregate Memory Performance : 26.21GB/s (25.85GB/s - 26.21GB/s)
Integer B/F iSSE2 Memory Bandwidth : 26.17GB/s (25.85GB/s - 26.17GB/s)
Float B/F iSSE2 Memory Bandwidth : 26.25GB/s (25.86GB/s - 26.25GB/s)
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Memory Performance : 20.12MB/s/MHz
Integer B/F iSSE2 Memory Bandwidth : 20.09MB/s/MHz
Float B/F iSSE2 Memory Bandwidth : 20.15MB/s/MHz
Results Interpretation : Higher scores are better.

ECC mode - Good
Numa enabled

Benchmark Results
Aggregate Memory Performance : 59.63GB/s (47.68GB/s - 59.63GB/s)
Integer B/F iSSE2 Memory Bandwidth : 59.65GB/s (47.6GB/s - 59.65GB/s)
Float B/F iSSE2 Memory Bandwidth : 59.6GB/s (47.75GB/s - 59.6GB/s)
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Memory Performance : 45.77MB/s/MHz
Integer B/F iSSE2 Memory Bandwidth : 45.79MB/s/MHz
Float B/F iSSE2 Memory Bandwidth : 45.76MB/s/MHz
Results Interpretation : Higher scores are better.


ECC mode - Basic
Numa enabled

Benchmark Results
Aggregate Memory Performance : 96.66GB/s (92.75GB/s - 96.66GB/s)
Integer B/F iSSE2 Memory Bandwidth : 96.62GB/s (92.38GB/s - 96.62GB/s)
Float B/F iSSE2 Memory Bandwidth : 96.71GB/s (93.13GB/s - 96.71GB/s)
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Memory Performance : 74.20MB/s/MHz
Integer B/F iSSE2 Memory Bandwidth : 74.16MB/s/MHz
Float B/F iSSE2 Memory Bandwidth : 74.24MB/s/MHz
Results Interpretation : Higher scores are better.

Sandra Cache and Memory

ECC - Good
Numa enabled

Benchmark Results
Cache/Memory Bandwidth : 615.5GB/s (342.32GB/s - 615.5GB/s)
Results Interpretation : Higher scores are better.
Speed Factor : 27.20
Results Interpretation : Lower scores are better.

Cache Information
Integrated Data Cache : 1.69TB/s
L2 Cache : 974GB/s
Results Interpretation : Higher scores are better.

Performance vs. Speed
Cache/Memory Bandwidth : 331.73MB/s/MHz
Results Interpretation : Higher scores are better.

Performance vs. Power
Processor(s)/Chipset(s)/Memory Power : 394.40W
Cache/Memory Bandwidth : 1598.08MB/s/W
Results Interpretation : Higher scores are better.

iSSE2 Cache/Memory Results Breakdown
Data Item Size : 16bytes
Buffering Used : No
Offset Displacement : Yes

Detailed Benchmark Results
1kB Data Set : 1.71TB/s (547.2GB/s - 1.71TB/s)
4kB Data Set : 1.7TB/s (544.17GB/s - 1.7TB/s)
8kB Data Set : 1.7TB/s (804.13GB/s - 1.7TB/s)
16kB Data Set : 1.7TB/s (827.3GB/s - 1.7TB/s)
32kB Data Set : 1.71TB/s (793.31GB/s - 1.71TB/s)
64kB Data Set : 1.7TB/s (737.55GB/s - 1.7TB/s)
128kB Data Set : 1.72TB/s (697.3GB/s - 1.72TB/s)
256kB Data Set : 1.82TB/s (711.51GB/s - 1.82TB/s)
512kB Data Set : 1.6TB/s (716GB/s - 1.6TB/s)
1MB Data Set : 1.46TB/s (822.71GB/s - 1.46TB/s)
4MB Data Set : 1.25TB/s (718.17GB/s - 1.25TB/s)
16MB Data Set : 668GB/s (383.14GB/s - 668GB/s)
64MB Data Set : 252.33GB/s (212GB/s - 252.33GB/s)
256MB Data Set : 90.4GB/s (85.8GB/s - 90.4GB/s)
1GB Data Set : 89.66GB/s (82.39GB/s - 89.66GB/s)
4GB Data Set : 85.29GB/s (76.75GB/s - 85.29GB/s)
16GB Data Set : 68.39GB/s (62.33GB/s - 68.39GB/s)

Sandra Multi-Media

Benchmark Results
Aggregate Multi-Media Performance : 496.17MPix/s
Multi-Media Integer x16 iSSE2 : 392.42MPix/s
Multi-Media Float x8 iSSE2 : 627.34MPix/s
Multi-Media Double x4 iSSE2 : 345.18MPix/s
Results Interpretation : Higher scores are better.

Benchmark Results
Aggregate Multi-Media Performance : 193GOPS
Multi-Media Integer x16 iSSE2 : 152.6GIPS
Multi-Media Float x8 iSSE2 : 244GFLOPS
Multi-Media Double x4 iSSE2 : 134.22GFLOPS
Results Interpretation : Higher scores are better.

Sandra Processor Arithmetic

Benchmark Results
Aggregate Arithmetic Performance : 293.4GOPS
Dhrystone ALU : 333GIPS
Whetstone iSSE3 : 258.48GFLOPS
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Arithmetic Performance : 154.42MOPS/MHz
Dhrystone ALU : 175.28MIPS/MHz
Whetstone iSSE3 : 136.04MFLOPS/MHz
Results Interpretation : Higher scores are better.

Performance vs. Power
Processor(s) Power : 223.83W
Aggregate Arithmetic Performance : 1310.78MOPS/W
Dhrystone ALU : 1487.84MIPS/W
Whetstone iSSE3 : 1154.79MFLOPS/W
Results Interpretation : Higher scores are better.

Sandra Crypto

Benchmark Results
Cryptographic Bandwidth : 4.24GB/s
AES256-ECB ALU Cryptographic Bandwidth : 4.45GB/s
SHA256 ALU Hashing Bandwidth : 4GB/s
Results Interpretation : Higher scores are better.

AIDA64 Extreme Edition (Benchmark only detects 32 cores)

CPU Queen - 65515
CPU PhotoWorxx - 35265
CPU ZLib - 733.5 MB/s
CPU AES - 170024
CPU Hash - 10107 MB/s
FPU VP8 - 3649
FPU Julia - 38706
FPU Mandel - 19693
FPU SinJulia - 9838

Some fun screenshots

[massman]

If I add all your hardware to the hwbot database, will you go for a couple of Wprime runs?

[[XC] Oj101]

Why does Task Manager only see 2GB RAM?

[PatRaceTin]

wow that massive core

[Sheik]

Why does Task Manager only see 2GB RAM?

Haha, knew someone was going to ask that. Took the screenshot with just 2GB while I was waiting for the 64GB to arrive.

[[XC] Oj101]

Come on, give me something to REALLY drool over

[Mafio]

man, that rig is asking for wprime

[blindbox]

Haha, knew someone was going to ask that. Took the screenshot with just 2GB while I was waiting for the 64GB to arrive.

2 GB RAM on a 48-core system. You're cruel!

[hyc]

64GB is still pretty light for that many cores. My 16 core HP DL585 G5 (4 Opteron 8354s) has 128GB of DDR2 ECC and I'm still wishing we had gone for more. Been juggling a bunch of 300GB data files just tonight. Ah well, the money we saved by not buying higher density DRAM can go to a socket G34 box I guess.

[[XC] Synthetickiller]

G34 is a fun socket to play with. I have a single 8 core that I can't max at all.

What are you using this rig for? Just a really expensive toy/benching set up?

[EniGmA1987]

WOW. Thats just insane. I wish I had a system like that to play with

[slaveondope]

Will you be going to 64 cores here shortly

If I add all your hardware to the hwbot database, will you go for a couple of Wprime runs?

Don't forget to do this ^

[Pestilence]

That's Hot

[Sheik]

What are you using this rig for?

Animation creation.

Hopefully 64 cores in the near future

[FlanK3r]

wprime, wprime, wprime please (I thinking, wprime 1.55 can minimal 16 threads, test it please)

[Sheik]

Will run it as soon as I can. Machine on site right now, so no access at the moment.

[verndewd]

throw in a cuda 4 card set up and takeover the world