Supermicro X9DAX-iF..Almost perfection!
This review started out like any other. Build the system, test it, get a "feeling" for the systems pro's and con's , then get some hopefully good quality pictures and write it up but something happened along the way. When I started testing I started to see that the potentials inside this system were much more than I had expected. That with a little work and experimentation on my part I might just see results far in excess of what I had expected and this turned out to be true.
First a couple pics of the case and how it comes packed. It is in a double box and encased in foam insulation:
http://img823.imageshack.us/img823/2557/24617913.jpg
http://img844.imageshack.us/img844/3766/62416360.jpg
Here is the Supermicro X9DAX-iF motherboard. Dual socket 2011, C602 chipset.
http://img196.imageshack.us/img196/9012/35950725.jpg
Diagram of all specifications:
http://img266.imageshack.us/img266/2930/33021815.jpg
Let's take a look at this new board. Take a fast look here at the company's site and then I'll get into the specifications:
http://www.supermicro.com/products/m...0/X9DAX-iF.cfm
This is a dual socket 2011 board using Intel's C602 chipset.
Basically what you get is in their words is a "Enhanced Extended EATX " board that is 13.68x13"or for you metric folks 34.74cmx33cm.
It has everything one would expect on a modern dual socket workstation style board .
Here's a list of the features on the board:
Dual socket R(LGA 2011) supports Intel Xeon E5-2600 series
Intel C602 chipset,QPI up to 8,0GT/s
Up to 512GB DDR3-1600MHz EDD REG DIMM,16 DIMM sockets
2 PCI-E 3.0x16,3 PCI-E 3.0x8,1UIO PCI-E x8( for SMC UIO HBA) or 1 PCI-E 2.0x4( in a x8)
Intel i350 Dual port GbE LAN
8xSATA2 and 2xSATA3 ports
Integrated IPMI 2.0 and KVM with dedicated LAN
Dual port front and rear USB 3.0
Performance boost with Hyper-Speed ( BCLK adjustment)
The board has support for SATA 2.0(3Gbps) and 3.0(6Gbps)RAID 0,1,5 and 10
Onboard video through Matrox G200eW
Onboard audio through Realtek ALC889 7.1
Next I need to show a picture of this board and it is sitting in Supermicro's monster of a case, the CSE-747TG-R1400B-SQ .
I know, that's a mouthful but there is logic behind the name.
No comments about wire management. I make them work. I never claimed to make them pretty!
http://img15.imageshack.us/img15/3007/15958881.jpg
This case has it all. Eight hot swap drive bays, a fan bar down the middle to keep things cool and those fans are good ones. 92x38mm. The system comes with two with two more optional for a total of four available if needed. There is another two 80x38mm rear exhaust fans that are also hot swappable and all fans are 5000rpm units.
I made a couple small changes because I was using memory with tall heat spreaders. I used the "older" SM heat sinks that came out last year for socket 2011 versus the new monsters Supermicro just brought out for use with this board. Since I was going to be pushing for the absolute maximum possible computational power I replaced the factory 92x25mm Nidec fans with a pair of 92x38mm Vantec tornadoes to get that little extra bit of cooling power.
Now if you are going to use shorter memory then the new Supermicro heat sinks are the way to go.
Can I say slick and hidden in the middle of them unseen are 92x25mm Nidec fans that attach to a top plate that removes with two screws:
http://img404.imageshack.us/img404/5072/77609057.jpg
Let's get back to what is important and that is the overclocking potential on this board. Now we all know that Intel has the E5-26xx SB-E's multi and BCLK locked BUT not 100%.
The BCLK can be adjusted upwards and a couple companies have already tried this but not with the success I saw here.
I can run with my E5-2687W 8 core Sandy Bridge Xeons at 34x106.25 BCLK for 3612MHz. To the best of my knowledge no other company has been able to do this and stable.
As of today I have run my DC app for over a month solid at 100% load with HT enabled so 32 threads loaded at this speed.
http://img560.imageshack.us/img560/105/47886381.jpg
Now the interesting part of this is that I can run at those settings with default voltage and the picture above was taken while running WCG( DC app) with 32 threads at 100% load. My cpu temps in a 75F room never exceeded 64C.
Now let's get back to what makes this board special and that is it's BIOS.
For the first time in many years Supermicro has over clocking options and voltage adjustments in their bios. I know, it shocked me too.
Supermicro has always had a companywide image of absolute reliability above all else and to maintain that quality they had tight controls on what one could and couldn't adjust on their boards.
With this series they have stepped into a new direction realizing that there is a market for systems that can be adjusted for more performance than what you get at default speeds.
Here's a few pics of what you have for options in the BIOS and you see that not only can you add vcore to the cpu's but also vdimm to the memory and have control of all the memory sub timings and last but not least you can increase the BCLK from default 100 up to 106.
http://img542.imageshack.us/img542/1471/90183184.jpg
http://img405.imageshack.us/img405/5025/62548614.jpg
http://img12.imageshack.us/img12/5735/19658920.jpg
http://img259.imageshack.us/img259/9171/12972155.jpg
Now we've seen the tools that Supermicro has put into the BIOS to allow overclocking. Time to see what can be done with those tools.
First the build materials so you understand what went into this.
Supermicro CSE-747TG-R1400B-SQ with redundant hot swap 1200W PSU's
Supermicro dual socket 2011 X9DAX-iF motherboard
Asus HD7850 video card
Crucial M4 SSD
Memory: four types tested
Centon DDR3-1333 ECC REG 8x2 gig
Corsair GTX1's. C9-2400's 8x2 gig
Corsair Platinum C10-2666's 8x4 gig
Corsair Platinum C9-2133's 8x4 gig. These IMO are THE sticks to get:
http://www.corsair.com/en/dominator-...m4b2133c9.html
This pic is of the C10-2666's but the C9-2133's look identical:
http://img541.imageshack.us/img541/7475/32231198.jpg
I first started with the Centon's and they were good but just had this gut feeling that there was more to be seen.
Then I went to the Corsair GTX1's and although they performed better I still had that feeling that something was missing.
Then I borrowed the Corsair Platinum C10-2666's and things started looking up. Now the max you can get to for speed on memory is 1983 using the Force 1866 BIOS setting and the added effect of the 106BCLK setting brings you to 1983. Not too shabby for a dual socket machine.
The C10-2666's were excellent but they were designed to work in a range that I couldn't get to and that is where the C9-2133's proved they were the best choice.
I was all of a sudden seeing times on benchmarks I couldn't even get close to before. The Corsair sticks are hand binned Samsung chips and using them you see why Corsair is known for the quality of the products they sell. Now this review isn't about the Corsair memory but that memory played a big part in my ability to get the most from this system. Had I gone with generic DDR3-1600ECC REG as most would do with this system I'd have missed a huge amount of the performance potential.
Let me back that last statement up.
Benchmarks are the best way to show in numbers what your system can do. I decided to run Geekbench to satisfy a friends curiosity and posted the top score ever done by a 16 core machine of 45,043 which is very close to what the 40 core systems are doing and 2000-3000 points above other E5-2687W machines on this link.
http://browser.primatelabs.com/geekbench2/top
http://img543.imageshack.us/img543/6223/81123909e.jpg
Next I went to my favorite Cinebench 11.5 and posted a 27.27 score which is the best ever by a 16 core system and keep in mind I did this before I got the C9-2133's. I think more is still available.
http://img23.imageshack.us/img23/4340/32434651.jpg
Then to another of my favorites wPrime32m:
2.218 seconds.. Next best recorded at HWBot for a machine using these same cpu's is 2.324 seconds
http://img543.imageshack.us/img543/2290/67032980.jpg
I then went on a spree to see just how many records I could take at HWBot with this machine.
Highest validated speed with 2-E52687W's at 3612MHz
http://img266.imageshack.us/img266/9897/41928836.jpg
Multi core Pi record:
http://img838.imageshack.us/img838/3167/47798248.jpg
UC Bench record:
http://img16.imageshack.us/img16/4271/51150009.jpg
and last but not least there is a very smart young guy at the XtremeSystems forum named Alexander Yee who wrote a Pi app called y-cruncher. This is a pi app that can be done from 25 million decimal places up into the trillions BUT for me with 32GB of memory I was limited to 5 billion.
I took every record on his page from 25 million to 5 billion:
http://www.numberworld.org/y-cruncher/#FastestTimes
Now the difference between my times and others grows as the longer the benchmark the greater the difference. The benchmark score for 5 billion was held by a very nice gentleman named Shigeru Kondo from Japan using a Supermicro dual socket 2011 board similar to mine but without the overclocking options in the bios.
He was also using E5-2690's that max out at 3300MHz while my E5-2687W's with the 106 BCLK would run at 3612MHz.
That extra 312MHz while it sounds small gave me a time of 530 seconds vs. his 589 seconds..yes, a full 10+% improvement in time needed on an under 10 minute benchmark:
http://img823.imageshack.us/img823/6226/31243257.jpg
I spent the best part of two months trying every combination I could think of with this system and when all the testing and benching is done it's just the most solid system I've ever worked with and most certainly the fastest dual socket machine in terms of pure power I've ever seen outside of maybe something running unlocked multiplier chips on liquid nitrogen cooling.
This you can run all day at 3612Mhz on all cores with HT enabled and not any issues at all.
it is just that good and it's been a pleasure to work with.
Now if you decide to build one of these I'm going to tell you to seriously consider getting the Corsair Dominator Platinum C9-2133's for the system. They really do make it come alive and worth the money spent.
I want to add this but was trying to decide how to say it as I believe in not posting negatives about competing products but the truth is the truth. There is simply no other dual socket 2011 board that will compete with this for delivering the maximum attainable computational power with a pair of E5-26xx chips. It is just more capable than the competition and there is no other dually 2011 that will run at 105BCLK let alone the 106.25BCLK that I see on this system. I'll be blunt and say that if you buy something else you've made a mistake if your goal is maximum computational output. If you're in the financial field this is the one you want period.
There is one last thing to add and that this system is 'Ivy Bridge ready" with its current BIOS.
That means that when the new 22nm 12 core IB xeons come out this fall you can just pop them in and you're golden.
There are four people I'd like to thank for their help on this review that turned into a incredibly interesting project:
Jake Crimmins and Robert Pearce from Corsair
Doug Herz and Bert Shen from Supermicro
Thanks to all of you for your help, expertise and willingness to share your knowledge.
