Well I tried!

[fallwind]

Unfortunately the top card couldn't breath (even with something wedged in between to create some spacing) and it ran at a constant 90c even with the fans roaring away. So I pulled the middle card and I'll have to find a home for it in the farm. The bottom card pulls in cool air through a hole cut in the bottom of the case so it's OK where it is.

[OldChap]

I am not so sure about the back of the case slots and clearance there but I do wonder if lifting the card a little using an extender would allow air into the card.

[jeanguy2]

Nice rig Fallwind,

With the cold temperatures you guys are getting in the Prairies now, you may just have to put that third card back in to keep the house warm. I have two Gigabyte 7950 WF3 OC video cards set up in crossfire. One of them (top one) runs about 13 deg C hotter than the lower one (59 vice 46 deg C). I thought that the temp difference was due to different BIOS or less than ideal heat paste application during manufacturing. After looking at your pictures I may be running into the spacing problem you experienced. Must investigate my video card temp differential further.

[pfm3136]

If it had been the Rampage IV Extreme, instead of the Formula, there would be a extra slot in the middle, wich is nice for tri-crosffire (or 3 cards using 3 slots each).
Other way would be to get a gpu block and watercool the top card.
I also have a Thermaltake Armor, that was used as a file server (up to 16 disks in it), and i have a EK-CoolStream RAD XTX (480) on top of it.
But points wise, it makes more sense to just get the card in another pc.

@jeanguy2
I once had 3 6950/70 with Artic Accelero Xtreme coolers, and each card had about 10C more than the lower one.

[NKrader]

I have an open pci express slot if you needed help finding one :-)



sick rig! :-P

[fallwind]

If it had been the Rampage IV Extreme, instead of the Formula, there would be a extra slot in the middle, wich is nice for tri-crosffire (or 3 cards using 3 slots each).
Other way would be to get a gpu block and watercool the top card.

Thanks for the tips, I was thinking along the same lines as well. I have a reference 7970 in another machine I could throw a waterblock on, or I could sell the R4F and swap in an R4E. I'll think about it for a couple days and make a decision.

[shoota]

Cases?? What are those?

[StitchExperimen]

Ya got me now worried about placing 3 Gigabyte 7950s together. I just ordered a 3rd one.
Please paste your app_config file for me to look at. Thanks in advance.
My case is a little different I have two vent fans on the top one 120 on the rear and a 140 on the top and my watercooler is father foward. So maybe I'll pass... hope hope... maybe I'll have to cut a hole for a second fan on the side panel.
Bruce

What about changing that 200mm fan to a pull instead of a push config that's what I did on mine. (120mm fan)

[pfm3136]

Cases?? What are those?

That's a Thermaltake Armor Silver (aluminium version), i've got the black (non aluminium) version.
I don't think they make this cases anymore, but others that looks like this.

[jeanguy2]

Ya got me now worried about placing 3 Gigabyte 7950s together. I just ordered a 3rd one.
Please paste your app_config file for me to look at. Thanks in advance.
My case is a little different I have two vent fans on the top one 120 on the rear and a 140 on the top and my watercooler is father foward. So maybe I'll pass... hope hope... maybe I'll have to cut a hole for a second fan on the side panel.
Bruce

What about changing that 200mm fan to a pull instead of a push config that's what I did on mine. (120mm fan)

As mentioned above, I added a 120mm fan above the 2 X 7950 video cards set to pull air from the case through the side access panel. It made for a significant improvement. Full load video cards temps have gone from 59 /46 deg C to 54 / 42 deg C. (fan speed @ 60%) Both video cards now running 5 deg C cooler. I can now try to push their cores to > 1100 MHz.

Fallwind, you may want to try reversing that side 200 mm fan on your case...

[nanoprobe]

Cases?? What are those?

Things lawyers take.

[fallwind]

I don't think I need to change the airflow around because with 2 cards, they run at 60c with the fans nice and quiet at 45%. It's just that with 3 cards, the top one gets its air intakes blocked and it runs really hot. I also don't want to flip the side fan around because it bathes the entire mb in cool air which is important when overclocking, especially for the VRM's around the CPU. I'm leaning towards swapping the mb to the R4E to get better spacing of the cards. I'll research this option over the weekend and go from there.

Cases?? What are those?

Haha yeah, caseless crunchers rule but this is my personal rig and I have little ones with curious little fingers so it's best to have a case!

[BlindFreddie]

Can you fit an extra fan or 2 or 3 into the lid directly over the GPUs, blasting air from outside onto and between them?
With that setup, they might run cooler if you take the shrouds off them and perhaps their OEM fans too. There would be more clearance between them that way.

Unless you're really into form-above-function that is.

[OldChap]

One last try at a solution for running 3 on the current setup: add an additional fan to the rear of the case to pull more air out of the card(s)




The pic is of an accessory that came with the atcs 840

[StitchExperimen]

I installed 3 Gigabyte HD 7950s and I installed horizontally two 120mm 9 blade 1750 rpm fans that are a bit forward directional in air flow over convenitial fans. First I tried to pull and that was worthless. Pushing I was able to get 86-87c in a 80f bedroom after about 1 hour to settly. I have a very small case so I get good air flow. 5 120mm fans (3 are 9 blade). 4 are intake. And a 6th fan is top exhaust that is 140mm. I have external closed loop water cooling by Corsair H100i cooling a Xeon 8 core 150 watt 3.1 GHz.

[shoota]

Haha yeah, caseless crunchers rule but this is my personal rig and I have little ones with curious little fingers so it's best to have a case!

Well little ones do throw a kink into the caseless cruncher equation

[Movieman]

HMM, I have 3 double spaced PCI-eX16 ver3 slots in my dualie 2011 system and those cards would look purrrty in there..
Also my 2-8 core cpu's with HT will be able to dedicate a thread to each of 32 GPU WU..
Now thats a scary thought yes?

[fallwind]

It's running fairly decently now with the help of a Rampage IV Extreme. The slot layout allows 3 cards to be installed with 1 slot of space between each for the fans to draw in cool air. The bottom card sits right on the bottom of my case which is fine since I cut a 2x120mm intake vent there. GPU temps are 72C-64C-59C top to bottom with the fans running at reasonable noise levels. I can see already though that's she's going to get noisy in the warmer months...lol!

Only 2 small issues to report: PPD is not what I hoped. Three 7970's in separate machines would net about 2.55 Million PPD and this looks like it will top out at around 2.2M. I have room to OC the GPU's but I'm thinking the CPU is the bottleneck here and there's not much OC room left without pushing the volts way up.

The other issue is I'm seeing lots of errors saying "No process is on the other end of the pipe. (0xe9) - exit code 233 (0xe9)" Approximately 10-20 per day. I've turned off the Windows Watchdog and I have plenty of RAM and swap space. Others have speculated that it's caused by running too many threads simultaneously. I'm running 24 but I also ran 24 with 2 GPU's and it didn't cause these errors back then. Initially I tried running 36 threads but all the WU's would error out so there's definitely an upper limit to how many threads you can run. I'd like to figure out what the bottleneck is that causes this limit though. I'll have to play around with it for a while I guess.

[Movieman]

It's running fairly decently now with the help of a Rampage IV Extreme. The slot layout allows 3 cards to be installed with 1 slot of space between each for the fans to draw in cool air. The bottom card sits right on the bottom of my case which is fine since I cut a 2x120mm intake vent there. GPU temps are 72C-64C-59C top to bottom with the fans running at reasonable noise levels. I can see already though that's she's going to get noisy in the warmer months...lol!

Only 2 small issues to report: PPD is not what I hoped. Three 7970's in separate machines would net about 2.55 Million PPD and this looks like it will top out at around 2.2M. I have room to OC the GPU's but I'm thinking the CPU is the bottleneck here and there's not much OC room left without pushing the volts way up.

The other issue is I'm seeing lots of errors saying "No process is on the other end of the pipe. (0xe9) - exit code 233 (0xe9)" Approximately 10-20 per day. I've turned off the Windows Watchdog and I have plenty of RAM and swap space. Others have speculated that it's caused by running too many threads simultaneously. I'm running 24 but I also ran 24 with 2 GPU's and it didn't cause these errors back then. Initially I tried running 36 threads but all the WU's would error out so there's definitely an upper limit to how many threads you can run. I'd like to figure out what the bottleneck is that causes this limit though. I'll have to play around with it for a while I guess.

Isn't the rule of thumb one GPU WU per cpu thread available? Correct me if I'm wrong on this..
If your going to run three of those monster 7970 cards maybe I should sell you my dualie..

[fallwind]

You can run more GPU WU's than available CPU threads by changing the <avg_ncpus> parameter. A value of 1 gives you 1 WU per thread. A value of 0.5 with give you 2 WU's per CPU thread, 0.333 gives 3. The CPU portion of the WU is quite short so it makes sense to run multiple WU's per CPU thread to try and keep the CPU loaded. And 7970's are very hungry cards so 8-12 threads is about right for them. Maybe I'll just reduce the number of parallel WU's to 12 (4 per card) and see what happens in terms of PPD but I have a feeling that 4 isn't enough to keep them busy.

[Movieman]

You can run more GPU WU's than available CPU threads by changing the <avg_ncpus> parameter. A value of 1 gives you 1 WU per thread. A value of 0.5 with give you 2 WU's per CPU thread, 0.333 gives 3. The CPU portion of the WU is quite short so it makes sense to run multiple WU's per CPU thread to try and keep the CPU loaded. And 7970's are very hungry cards so 8-12 threads is about right for them. Maybe I'll just reduce the number of parallel WU's to 12 (4 per card) and see what happens in terms of PPD but I have a feeling that 4 isn't enough to keep them busy.

ahh, I learned something today!

[bearcatrp]

ahh, I learned something today!

So you can teach old dogs new tricks!

[Movieman]

^^ woof!