jcool
09-25-2009, 11:56 AM
Ok fellas, since I have seen a lot of confusion and misleading info on Dual Nehalems over here these last couple of days, I decided to post a sort of FAQ for the benefit of those looking to go Dual Socket Intel. If you have any additions with what you think might be relevant, post up and I'll add it here.
Q: What kind of CPUs can I use for a DP (Dual processor) LGA1366 setup?
A: Only series 5500 Xeons from Intel will work together on a dual mobo. There are 4 different kinds of these CPUs: L55xx (low voltage), E55xx (Entry), X55xx (Server) and W55xx (High End). Series 3500 Xeons like the w3520 will N O T run together! They are based on the Bloomfield Core and, apart from the ECC support and different binning process, virtually identical to a Core i7 920. They are missing the Dual QPI bus that only the Gainestown and upcomping Gulftown (core i9 Hexacore) CPUs feature.
Q: Really? No way to mod them or something? What about a single w3520 on a DP board?
A: Definitely no way to mod or trick them into working together. Much like Intel's locked multiplier, this limitation cannot be surpassed. As for running a single W35xx on a DP LGA1366 board, it might work - provided the mobo manufacturer implemented the correct microcode. More often than not, your Dual LGA1366 won't start.
Q: So what CPUs would you recommend for crunching/video editing/other heavily threaded work?
A: As always, that depends on your budget. Cheapest pair that makes sense is the Dual E5520. With Turbo and HT enabled, they will Auto-Overclock to 2,4Ghz on 16 threads, their performance equal to a single i7 clocked at 4,8Ghz, which isn't possible with anything but extreme cooling. With 80W TDP, they are also simple to cool and easy on the power bill. The L5520's have a 60W TDP and are therefore ideal if your elec costs are very high, although the substantially higher initial cost usually doesn't justify getting them.
As for the performance alternative, those of you looking to spend a bit more in order to get better performance, I suggest having a look at the X5550's. Their Turbo mode works differently, giving them a +2 boost on all cores even with 16 threads fully loaded, effectively giving you 16x2,93Ghz of crunching power. Note that Turbo mode only works properly in retail D0 chips, it might behave differently or not work at all with early ES samples like the ones I have. At 95W their TDP is still moderate compared to Desktop i7's, series 3500 Xeons, or W55xx model Xeons which are prohibitively expensive anyways.
Still, if only the best is good enough for you, get a pair of W5590's and enjoy 16*3,5Ghz or more, since they are theoretically unlocked.
Q: "Theoretically unlocked"? What the hell does that mean?
A: Means they don't have multipliers locked by Intel, so they can be ran at any multiplier between 12 and 63 IF the board supports that. Which is a big if, as all DP LGA1366 boards known to me cannot fully unlock these chips. Something to do with the motherboards ability to control Turbo mode multipliers above those intended by Intel. Those DP boards weren't designed with overclocking in mind, remember :rolleyes: :shakes:
Q: Since you mentioned it.. How do they OC?
A: Not at all, I'm afraid. Hardly worth it, some might say. As a matter of fact, the maximum attainable BCLK on all currently known (known to me, at least) boards is exactly 138Mhz using SetFSB in Windows. 138,3Mhz inevitably hardlocks the system. And only if you force the QPI mult down to 36x (4,8GT/s), with 40x or 44x it will hit a wall at 136 or 135,7 respectively. This holds true for all Tyan, Supermicro, and Asus series of boards, probably the Intels as well. The reason for this is fairly unknown, Saaya once told me he thought he knew what the problem might be, apparently some register not being pulled high on startup (whatever the hell that means :D ). Bottom line is, it probably can't be fixed by a simple bios update. Hardmods maybe, we would need someone who understands cuircuitery way better than I do.
Q: So.. anything in bios then? BCLK, Voltages? Heck, ramtimings even?
A: Afraid not. Not even ramtimings, even tho some boards offer to force certain dram dividers (it doesn't have any effect on my Tyan tho)
Q: While we're at it - what Ram do I need?
A: That depends on your needs as well as on your choice of CPUs. Generally you have to choose between standard DDR3 and DDR3 ECC (registered or unregistered) DIMMs. E and L series Xeons will only support DDR3-1066 speeds, while X and W series Xeons can handle DDR3-1333. CPUs should be populated with at least 3 sticks of ram each to attain maximum bandwidth, which comes in handy on a DP platform. Bear in mind that the maximum amount of standard DDR3 memory that you can use is 24GB. If you want more you're gonna have to use REG ECC ram. Also, with 6 DIMMs per CPU, the X and W Xeons will usually switch down from 1333Mhz to 1066, unless you use that "Force 1333" option in bios I was talking about earlier.
Here's the catch tho: 1333Mhz only ever works if the rams themselves offer a correct SPD entry for running at DDR3-1333 at 1,5V. That's why for me, that "force" option has no effect, because my rams don't have the proper programming. So if you want or need the full ram speed, you're gonna need X or W series CPUs, a maximum of 12 Rams total that have the correct SPD programming, and a motherboard that can enforce the 1333 option with 2 Rams per memory channel. Piece of cake huh :rofl:
Finally, when you use more than 2 Rams per channel, the motherboard will reduce the Dram speed to DDR3-1066. If you use 4 rams per channel, it goes down to DDR3-800.
Q: What kind of PSU do I need to power a DP i7?
A: Depends mainly on what kind of GPUs you are planning on using. If you'll just use an onboard or other low-powered solution, something in the 400W range will be more than enough to handle it. One of the main advantages of a Dual i7 is its high efficiency, meaning performance per Watt is excellent and usually better than that of SP desktop rigs.
Note that you will need a PSU with 2x 8pin 12V power connectors on virtually all boards! Lower powered PSUs usually don't have those. You can, however, use a 4 to 8 pin adapter - that's what I do anyway, without any trouble. I recommend the Seasonic S12-II 430 with 4 to 8 pin adapter for an efficient, reliable, affordable and quiet choice if you're not into GPU folding of any kind.
If you are, better get something heavy. The Corsair HX850 would be an excellent choice for a Dual CPU, Dual GPU setup. It's the most efficient unit in its class :up:
Q: What motherboards would you recommend?
A: That depends on your needs, budget, and available space.
If you can't fit an eATX board or just want something affordable to crunch, I'd recommend the Tyan S7002 series or the Supermicro X8DTL-i.
If you want onboard SAS, or loads of ram, you'll want one of the bigger boards with 12 or even 18 ramslots. Tyan S7010, Supermicro X8DAi or even the Asus Z8PE-D12X come to mind. You can also have a look at Intel's very own series of S5520 motherboards.
Finally, if you want the ultimate Multi-GPU platform, there's the Tyan S7025 - a Dual Tylersburg chipset motherboard that features four double-spaced PCIe 2,0 x16 slots without the use of any crappy nvidia NF200 chips. Besides running four GFX cards at full blast, you still got 8 PCie lanes left for dedicated sound, raidcon etc.
Bottom line: The differences between the boards and brands are slim but obvious. Just chose whatever fits your case, budget, and needs, since they all don't OC the differences end here. All the same line of chipsets as well.
Q: Which coolers fit?
A: These motherboards all come with pre-installed backplates, so you gotta make sure you get a cooler with screws, not pushpins. You can remove the stock backplate by unscrewing the socket, but I wouldn't recommend doing that, since you will then have no backplate at all (unlike x58 boards which have a small stock backplate in any case). The coolers below may not work on all motherboards, especially the ATX Duallies are often a tight fit.
- Noctua U12P 1366 Tower - probably works on all boards (need different M3 screws to secure the mounting though)
- Sharkoon C120 (http://geizhals.at/deutschland/a475052.html) - probably not available outside of Germany/EU, but still - good tower cooler at a cheap price, and it fits with no mods using the stock mounting and the stock mobo backplate
- TRUE with LGA1366 mounting - works, but might block topmost PCIe Slot of certain mobos. Also heavy as :banana::banana::banana::banana: so maybe not the best choice here
- Cheapest choice, modded Bloomfield Boxed coolers. Not the best choice though, CPUs will run to 80+C in a closed case while crunching in summer. Ok for E series Xeons I guess, borderline ok for X, no go for W Xeons
- And finally, the Intel thermal solutions STS100C - small, compatible, cheap, but also annoyingly loud, crappy cooling performance without the help of 10000RPM rackmoint case fans. I wouldn't recommend those.
Q: So if it's that hard and expensive to properly configure and run a DP i7 that you can't even overclock, why on earth would you want one?
A: Easy - see for yourself ;) (http://database.he-computer.de/Bilder/dualgainsy/X5570_BOINC.jpg)
That's all for now, getting tired.. am sick as a dog as well, damn flu :down:
If you want more info and pics from my setup, go here (http://www.xtremesystems.org/forums/showthread.php?t=227603)
Q: What kind of CPUs can I use for a DP (Dual processor) LGA1366 setup?
A: Only series 5500 Xeons from Intel will work together on a dual mobo. There are 4 different kinds of these CPUs: L55xx (low voltage), E55xx (Entry), X55xx (Server) and W55xx (High End). Series 3500 Xeons like the w3520 will N O T run together! They are based on the Bloomfield Core and, apart from the ECC support and different binning process, virtually identical to a Core i7 920. They are missing the Dual QPI bus that only the Gainestown and upcomping Gulftown (core i9 Hexacore) CPUs feature.
Q: Really? No way to mod them or something? What about a single w3520 on a DP board?
A: Definitely no way to mod or trick them into working together. Much like Intel's locked multiplier, this limitation cannot be surpassed. As for running a single W35xx on a DP LGA1366 board, it might work - provided the mobo manufacturer implemented the correct microcode. More often than not, your Dual LGA1366 won't start.
Q: So what CPUs would you recommend for crunching/video editing/other heavily threaded work?
A: As always, that depends on your budget. Cheapest pair that makes sense is the Dual E5520. With Turbo and HT enabled, they will Auto-Overclock to 2,4Ghz on 16 threads, their performance equal to a single i7 clocked at 4,8Ghz, which isn't possible with anything but extreme cooling. With 80W TDP, they are also simple to cool and easy on the power bill. The L5520's have a 60W TDP and are therefore ideal if your elec costs are very high, although the substantially higher initial cost usually doesn't justify getting them.
As for the performance alternative, those of you looking to spend a bit more in order to get better performance, I suggest having a look at the X5550's. Their Turbo mode works differently, giving them a +2 boost on all cores even with 16 threads fully loaded, effectively giving you 16x2,93Ghz of crunching power. Note that Turbo mode only works properly in retail D0 chips, it might behave differently or not work at all with early ES samples like the ones I have. At 95W their TDP is still moderate compared to Desktop i7's, series 3500 Xeons, or W55xx model Xeons which are prohibitively expensive anyways.
Still, if only the best is good enough for you, get a pair of W5590's and enjoy 16*3,5Ghz or more, since they are theoretically unlocked.
Q: "Theoretically unlocked"? What the hell does that mean?
A: Means they don't have multipliers locked by Intel, so they can be ran at any multiplier between 12 and 63 IF the board supports that. Which is a big if, as all DP LGA1366 boards known to me cannot fully unlock these chips. Something to do with the motherboards ability to control Turbo mode multipliers above those intended by Intel. Those DP boards weren't designed with overclocking in mind, remember :rolleyes: :shakes:
Q: Since you mentioned it.. How do they OC?
A: Not at all, I'm afraid. Hardly worth it, some might say. As a matter of fact, the maximum attainable BCLK on all currently known (known to me, at least) boards is exactly 138Mhz using SetFSB in Windows. 138,3Mhz inevitably hardlocks the system. And only if you force the QPI mult down to 36x (4,8GT/s), with 40x or 44x it will hit a wall at 136 or 135,7 respectively. This holds true for all Tyan, Supermicro, and Asus series of boards, probably the Intels as well. The reason for this is fairly unknown, Saaya once told me he thought he knew what the problem might be, apparently some register not being pulled high on startup (whatever the hell that means :D ). Bottom line is, it probably can't be fixed by a simple bios update. Hardmods maybe, we would need someone who understands cuircuitery way better than I do.
Q: So.. anything in bios then? BCLK, Voltages? Heck, ramtimings even?
A: Afraid not. Not even ramtimings, even tho some boards offer to force certain dram dividers (it doesn't have any effect on my Tyan tho)
Q: While we're at it - what Ram do I need?
A: That depends on your needs as well as on your choice of CPUs. Generally you have to choose between standard DDR3 and DDR3 ECC (registered or unregistered) DIMMs. E and L series Xeons will only support DDR3-1066 speeds, while X and W series Xeons can handle DDR3-1333. CPUs should be populated with at least 3 sticks of ram each to attain maximum bandwidth, which comes in handy on a DP platform. Bear in mind that the maximum amount of standard DDR3 memory that you can use is 24GB. If you want more you're gonna have to use REG ECC ram. Also, with 6 DIMMs per CPU, the X and W Xeons will usually switch down from 1333Mhz to 1066, unless you use that "Force 1333" option in bios I was talking about earlier.
Here's the catch tho: 1333Mhz only ever works if the rams themselves offer a correct SPD entry for running at DDR3-1333 at 1,5V. That's why for me, that "force" option has no effect, because my rams don't have the proper programming. So if you want or need the full ram speed, you're gonna need X or W series CPUs, a maximum of 12 Rams total that have the correct SPD programming, and a motherboard that can enforce the 1333 option with 2 Rams per memory channel. Piece of cake huh :rofl:
Finally, when you use more than 2 Rams per channel, the motherboard will reduce the Dram speed to DDR3-1066. If you use 4 rams per channel, it goes down to DDR3-800.
Q: What kind of PSU do I need to power a DP i7?
A: Depends mainly on what kind of GPUs you are planning on using. If you'll just use an onboard or other low-powered solution, something in the 400W range will be more than enough to handle it. One of the main advantages of a Dual i7 is its high efficiency, meaning performance per Watt is excellent and usually better than that of SP desktop rigs.
Note that you will need a PSU with 2x 8pin 12V power connectors on virtually all boards! Lower powered PSUs usually don't have those. You can, however, use a 4 to 8 pin adapter - that's what I do anyway, without any trouble. I recommend the Seasonic S12-II 430 with 4 to 8 pin adapter for an efficient, reliable, affordable and quiet choice if you're not into GPU folding of any kind.
If you are, better get something heavy. The Corsair HX850 would be an excellent choice for a Dual CPU, Dual GPU setup. It's the most efficient unit in its class :up:
Q: What motherboards would you recommend?
A: That depends on your needs, budget, and available space.
If you can't fit an eATX board or just want something affordable to crunch, I'd recommend the Tyan S7002 series or the Supermicro X8DTL-i.
If you want onboard SAS, or loads of ram, you'll want one of the bigger boards with 12 or even 18 ramslots. Tyan S7010, Supermicro X8DAi or even the Asus Z8PE-D12X come to mind. You can also have a look at Intel's very own series of S5520 motherboards.
Finally, if you want the ultimate Multi-GPU platform, there's the Tyan S7025 - a Dual Tylersburg chipset motherboard that features four double-spaced PCIe 2,0 x16 slots without the use of any crappy nvidia NF200 chips. Besides running four GFX cards at full blast, you still got 8 PCie lanes left for dedicated sound, raidcon etc.
Bottom line: The differences between the boards and brands are slim but obvious. Just chose whatever fits your case, budget, and needs, since they all don't OC the differences end here. All the same line of chipsets as well.
Q: Which coolers fit?
A: These motherboards all come with pre-installed backplates, so you gotta make sure you get a cooler with screws, not pushpins. You can remove the stock backplate by unscrewing the socket, but I wouldn't recommend doing that, since you will then have no backplate at all (unlike x58 boards which have a small stock backplate in any case). The coolers below may not work on all motherboards, especially the ATX Duallies are often a tight fit.
- Noctua U12P 1366 Tower - probably works on all boards (need different M3 screws to secure the mounting though)
- Sharkoon C120 (http://geizhals.at/deutschland/a475052.html) - probably not available outside of Germany/EU, but still - good tower cooler at a cheap price, and it fits with no mods using the stock mounting and the stock mobo backplate
- TRUE with LGA1366 mounting - works, but might block topmost PCIe Slot of certain mobos. Also heavy as :banana::banana::banana::banana: so maybe not the best choice here
- Cheapest choice, modded Bloomfield Boxed coolers. Not the best choice though, CPUs will run to 80+C in a closed case while crunching in summer. Ok for E series Xeons I guess, borderline ok for X, no go for W Xeons
- And finally, the Intel thermal solutions STS100C - small, compatible, cheap, but also annoyingly loud, crappy cooling performance without the help of 10000RPM rackmoint case fans. I wouldn't recommend those.
Q: So if it's that hard and expensive to properly configure and run a DP i7 that you can't even overclock, why on earth would you want one?
A: Easy - see for yourself ;) (http://database.he-computer.de/Bilder/dualgainsy/X5570_BOINC.jpg)
That's all for now, getting tired.. am sick as a dog as well, damn flu :down:
If you want more info and pics from my setup, go here (http://www.xtremesystems.org/forums/showthread.php?t=227603)