Daisy Chain water cooling. 6 PCs, 1 Pump. Any Advice?

[Fuzzy_3D]

They story goes something like this. I’m looking at adding a few more machines to our render farm (my accomplice and I), and we’re gonna be outfitting them with E6600 Core 2 Duos.

The last nodes we added were D820s. We all know what the heat output on those was like, so we tacked some aftermarket Zalman HSFs on each one of those for about $50 a pop. This let us squeeze quite a fair bit of speed out of them, which my friend was reluctant to do at first, but when it proved to take about 30% off of the stock rendering time, he came around.

So this time, we’re seriously considering the idea of water cooling for the E6600 stack. Price is obviously the issue here, were looking at about 4-8 separate nodes, depending on the final rendering load. So I got the idea of hooking all the blocks into a single loop, with one pump and a few rads. I’m fully aware that we’re burdening about 6 systems onto a single point of failure, but I’m sure we’ll think of something in terms of a failsafe.

I’d really like to get some feedback on what you guys think of this, so I can start bouncing a few ideas around. We figured a relatively cheap water cooling solution would surpass any aftermarket air cooling at the same price.

Besides that, I’d love to get some advice on the kind of components that would fit the bill. Keeping in mind that at a certain price point, we’d prefer to add another node to the system and just use stock cooling, rather than have an expensive/elaborate water cooling setup.

First of all, I’m looking to use a single swiftech MCP655 as the pump. This forum seems very fond of the DDC+Petra top, for good reason I presume, and I’d be glad to spend the extra $25 if you guys reckon it’s a worthy upgrade.

Looking at the cheaper CPU blocks, I figure we’d go with the Apogee over the AX MP1. Simply because the MP1 has a built in accelerator nozzle that would cause a huge pressure drop, considering there’ll be 6 of them routed through a single pump. And besides that, the Apogee looks to have a larger cooling area.

Now for the complicated part. We need to find a “cost effective” rad arrangement.
The only headway I’ve made on this so far is that we’ll probably need a 120mm cooling area for each of the satellites.
Assuming we have 6 slaves, that’s either 6 1x120mm, 3 2x120mm, or 2 3x120mm rads. Obviously having a separate rad for every block would be optimal in regards to cooling efficiency, the main concern here though is raw rendering power/dollar.
We can get 6 Swiftech MCR 120s for about $150 ($25ea). However, 3 HWL BIP2s will only cost us $100 (~$33ea), and furthermore, 2 HWL BIP 3s would be $84 ($42ea).

My partner in crime insists on getting 3 MCR-220s (~$120), and I’m beginning to think this would work pretty well, since the MCRs have (apparently) relatively low restriction, and 3 120x2 rads “probably” have less flow resistance than 6 120mm rads.

Needless to say we’ll be going with a rather large res, and a T-Line is out of the question, unless someone can convince us otherwise.

Pricing it all out, aftermarket HSFs would run us about $300, the water cooling setup
(MCP655-$75, 6xApogee-$276($46ea), 3xMCR-220-$120($40ea), Res-$25) adds up to about $500, minus the shipping, tubing, fans, etc. Lets say $600 to be sure.

If we were to add another MCP 655 and use a single 120mmMCR rad for each node, the price of water cooling the array would jump to $700, and that’s where we would be inclined to just use stock cooling and throw another node onto the stack.

It all comes down to whether the crude method of water cooling is really worth twice as much as cooling with air on aftermarket heat sinks (Zalman CNPS7700-Cu).

Also to note, we’re looking to set this up some time Q2 2007.
My experience with water cooling is rather limited, and I’m feeling a little in over my head here, so I figured I’d ask sooner rather than later, so as to wrap my head around it beforehand.

Anyway, I’ll appreciate any constructive input or advice on this. Nothing is set in stone yet, so feel free to throw anything at me.

[SparkyJJO]

For something like that, I'd go with 2 DDC+ with Petra's top and apogee blocks. Thermochill rads would be better but the 3 MCR 220s wouldn't be bad. THe two pumps are so that you don't loose too much flow and pressure through all that.

Something like this: Res > DDC+ > DDC+ > apogee > apogee > rad > apogee > apogee > rad > apogee > apogee > rad > res.

I think it is a kind of cool idea. Now I am no pro at watercooling so you'll probably want the input of the real pros around here.

[Levish]

6x 50$ (cost of heatsink) = 300$ and more redundancy and considering you probably aren't going for max effort o/c due to need to be 100% stable, you'd be better off sticking to aircoolers.

furthermore to be honest you may be better off going with quad cores and cutting the number of nodes down to 3-4

[Bun-Bun]

Originally Posted by SparkyJJO

For something like that, I'd go with 2 DDC+ with Petra's top and apogee blocks. Thermochill rads would be better but the 3 MCR 220s wouldn't be bad. THe two pumps are so that you don't loose too much flow and pressure through all that.

Something like this: Res > DDC+ > DDC+ > apogee > apogee > rad > apogee > apogee > rad > apogee > apogee > rad > res.

I think it is a kind of cool idea. Now I am no pro at watercooling so you'll probably want the input of the real pros around here.

That is exactly what I was going to say.

I also wish I could be there to help you set that up....

EDIT: MCR220's < PA120.2 and the GTX240 is somewhere in there as well probably in the middle... but those three rads would be all I would recommend. Is noise an issue for your render farm?

or since its going to be a big res anyway, you could get fun and build a bong fish tank type thing.

[Clay]

I'm by no means any kind of an expert either. Are the graphics to be cooled also? Are these seperate computers or are they all inna' rack? If seperate computers then wouldn't that be a pretty long loop? I think I would also find myself greatly concerned with preasure drop. Peeps are saying the apogees preasure drop is about 5% per block. With a bunch of blocks in the chain it could be alot more. (30% at least anyway.Maybe more) Then 3# swiftech 220 rads 17% or so each? (+51%? maybe more) So 1 D5 pump might not be the best idea. Then theres a chance that the processors will run warmer as the water warms down the loop. (Kinda' sure.How much? No clue.LOL) 2#res's and pumps and rads might be better. Maybe sumptin' like...
Res>Pump>Rad>Block>Block>Block>ResPump>Rad>Block>B lock>Block>Res.
But then there is also the length of tubing to consider. (And for every 90 degree bend another kitten implodes.LOL)
By the time you get done you could have probably spent alot more time rendering.
This is all speculation on my part. I know it's been done before but myself I have no experience with that. (Someone here will know though)

Edit: PA's have the lowest preasure drop of all rads.(Sofar) 2#120.3"s 2#D5's Might be good.

[Xeon th MG Pony]

6 systems? you use larg diameter tubing and make a sucking / pressure lines that tap into a 3/8th bleeder lines to the systems, use an iwaki or other such hi powered pump to suply the 6 systems parallel.

[Xeon th MG Pony]

I do the same I cool(ed) (My stuff was ripped off a couple or so years ago and am trying to rebuild any way) Same way you do not do it series too many issues.

You select a high pressur pump that has high flow, and a huge rad system, or cooling tower. You use a 1 inch branch lines, now every system taps into the branch suply lines with 3/8 tubing. Every block in the whole system must be identicle as with the piping lengths inside the system as to maintain system balance. You can use the shut off valves as balancing valves for each parallel branch you add.


As a back up system you can have a parallel pump that kicks in if flow drops by X amount, and then sounds an alarm whilst spinning up the back up pump.

[Bun-Bun]

Originally Posted by Xeon th MG Pony

I do the same I cool(ed) (My stuff was ripped off a couple or so years ago and am trying to rebuild any way) Same way you do not do it series too many issues.

You select a high pressur pump that has high flow, and a huge rad system, or cooling tower. You use a 1 inch branch lines, now every system taps into the branch suply lines with 3/8 tubing. Every block in the whole system must be identicle as with the piping lengths inside the system as to maintain system balance. You can use the shut off valves as balancing valves for each parallel branch you add.


As a back up system you can have a parallel pump that kicks in if flow drops by X amount, and then sounds an alarm whilst spinning up the back up pump.

ooh that makes sense. Then you wouldnt need RAD's inbetween computers... you just have a huge one at the end... wouldnt even have to use a computer purpose built rad just get some huge one with a large fun...

And you wouldnt have to worry about pressure losses as long as your pump has enough flow.

This could be a lot of fun

[Fuzzy_3D]

Regarding Quad Core, we've done our calculations (based on Cinebench), and the numbers show that a E6600 rig will yield 0.963 marks/$, while a QX6700 rig only gets us 0.800 m/$. That's in CND, and taking into account the cost of the whole computer, not just the chip.
If the price/performance ratio of Quad Core comes close enough to that of the dual core at the time we buy the systems, rest assured we'll be opting for the Quad.

The most puzzling part is getting efficient water cooling, and hopefully getting a bit more power out of the nodes while keeping them stable. For the 820 nodes we went 10% under a 74hr stable clock.

As for the loop order, wouldn't something like this be more ideal?

Res > Pump > Rad > Block > Block > Rad > Block > ?Pump? >Block > Rad > Block > Block > Res

The nodes are all using onboard graphics, I know how terrible onboard GPUs are, but the client software is a purely 2D GUI, so we don't see the need for anything fancy.

ThermoChill rads in N.America are Ridiculously over priced. Unless they cool 3 times better than the MCRs, we'll be sticking with Swiftech. Unless someone can suggest something better of course.

Each node will be housed in a seperate m/ATX tower. Hard as we tried, there just wasn't a viably cost effective way to use 4U Racks.
Though sound is not an issue, within reason.

Does anyone have any suggestions for going with Series VS Parallel?

The series loop is outlined above, I'm thinking somehting like this for parallel:

................. /> Rad > Block > Block \
res > Pump1 -> Rad > Block > Block - > Pump2 > res
..................\> Rad > Block > Block /

I'm pretty sure that a parallel setup will absolutely need 2 pumps.

[CedricFP]

Thermochills do not cool 3x better. They do, however, cool better and that's what XS is all about.

I would recommend swiftechs myself. The HWL rads (all of them, BIP/BIX/GTS/GTX are all worse performing (with low CFM fans such as a Yate Loon) than the MCR series.

I would also recommend instead of going for 3xMCR220 (total of 6 120m fans) go for 2xMCR320 (also total of six) and this means slightly less restriction.

As for the pump, I would get 2xDDC-02 w/ Petras top (which should be more than enough).

Your lineup would become:

Res - DDC-02 - MCR320 - Apogee - Apogee - Apogee - MCR320 - Apogee - Apogee - Apogee - Res

That's my 0.2

[NickS]

Apogees, an Iwaki and a few MCR-220's or MCR-320's.

[Aerou]

I agree with NickS, maybe you should put an Iwaki there,
I guess that putting such an amount of watter into movement will need a very powerfull pump = Iwaki.

That is if you REALLY wish to stick with the 6 rig 1 loop idea.

I would definetly prefer seperate a loop for each rig, cuz if something gets bad and your flowrate would collapse, all 6 rigs would go to hell ... Its risky man ...

I guess we should get some input from the gurus here ...

[NickS]

Ultimate reliability = Iwaki too.

[SiGfever]

For that many systems I would go for the Iwaki for power and rock solid reliability.

ThermoChill is the best but the MCR-320QP is a very good rad.

[STEvil]

You might consider using a gravity fed setup.

A res placed at a high place could drain down to each system seperately and collect in a res at the bottom. A pump would then lift from the lower res to the top res. A decent AC sump pump rated for continuous use would probably be ideal for flow/lift height and you could make use of a large copper radiator between the res's.

Alternately you could make the worlds largest PC cooling bong...

[_G_]

I will probably burn for posting this in here but maybe you could build something
like this in an empty 4u case.
http://www.performance-pcs.com/catal...ducts_id=21026

[STEvil]

can build an identical setup for 1/5th the price probably.

[BobMaximus]

Originally Posted by _G_

I will probably burn for posting this in here but maybe you could build something
like this in an empty 4u case.
http://www.performance-pcs.com/catal...ducts_id=21026

Koolance!



Burn complete

[Bun-Bun]

Originally Posted by CedricFP

I would recommend swiftechs myself. The HWL rads (all of them, BIP/BIX/GTS/GTX are all worse performing (with low CFM fans such as a Yate Loon) than the MCR series.

While there is proof that your claim is true for the BIP/BIX/GTS rads, there is no such proof for the GTX that I know of (if there is that you know of please let me know) so please stick to the facts.

Also noise is not a huge issue here so running 70+CFM fans could be a possibility bringing the HWLabs into play. Although the MCR220 would be the better bet as far as flow is concerned.

Personally I like the idea suggested above and going parallel loops with a good Iwaki pump. That is the way to go here. You get performance and reliability at the same time. With series you are adding heat from one CPU to the next in the loop, parallel you are not, and you are effectively only have the equivalent of one WB worth of pressure drop with parallel loops. And then you can have a giant bong or RAD right before it all and then back to the res/pump. If you must use purpose built rads then run 2 x MCR320 and have the rads run in series before you split to the processors and then after the rads split into parallel to each processor and then meet back up in the resevoir.

I think some variation of that would be great.

[Mr. Peanut]

All I can say is make sure none of your water-cooling components are flow-restrictive ones. Here is where you might have to sacrifice a little bit of performance for some feasibility. If it means Apogees instead of Storms, do it. If it means single-pass instead of dual-pass rads, do it. (Not that I'm saying it DOES mean that you have to make these sacrifices. I am only providing examples of what I mean by "sacrificing a little bit of performance for some feasibility.") Also, I would definately go with 7/16" ID tubing over 1/2" ID tubing for this one.

[Bun-Bun]

Originally Posted by Mr. Peanut

All I can say is make sure none of your water-cooling components are flow-restrictive ones. Here is where you might have to sacrifice a little bit of performance for some feasibility. If it means Apogees instead of Storms, do it. If it means single-pass instead of dual-pass rads, do it. (Not that I'm saying it DOES mean that you have to make these sacrifices. I am only providing examples of what I mean by "sacrificing a little bit of performance for some feasibility.") Also, I would definately go with 7/16" ID tubing over 1/2" ID tubing for this one.

You want the least restriction so why 7/16" over 1/2"?

[Fairydust]

When I still had a folding farm, I used a setup with 2 D5s, a Mora2 with 9 fans, 6 custom CPU blocks (pingrid, no jet impingement) and Colder quick disconnects on the radbox and PCs, 1 2.

My main goal was managing the thermal load on the small room, as during summer the heat output from PCs in a 9 m2 room was raising ambient temps past viable levels, even with an airtunnel at the PSU exhausts. The airtunnel was working well though and if you do get problems with raised ambient temps from the farm, it's an easy and cheap way to direct loads of thermal energy away.

None of the D5s ever failed, but the dual setup put me at ease if 1 had. After passing through 6 CPU blocks on Athlon XPs running at 2200 MHz, the difference between input/output temp at the radiator never exceeded 1 K. The heat output of 6 (oced) Core duos will be higher, but your flowrate will most likely be higher compared to my 0.65 gpm with the very restrictive radiator and moderately restrictive blocks.

Using a centralised radbox with all electrical components made installation and maintenance pretty easy. I chose and like the simplicity of a a completely serial setup as I find even adjusting flowrates for individual sections a rather strenuous task. The setup Xeon th MG Pony posted sounds good though, especially for loops with very high pressure drop.

[CedricFP]

Originally Posted by Bun-Bun

While there is proof that your claim is true for the BIP/BIX/GTS rads, there is no such proof for the GTX that I know of (if there is that you know of please let me know) so please stick to the facts.

Also noise is not a huge issue here so running 70+CFM fans could be a possibility bringing the HWLabs into play. Although the MCR220 would be the better bet as far as flow is concerned.

Personally I like the idea suggested above and going parallel loops with a good Iwaki pump. That is the way to go here. You get performance and reliability at the same time. With series you are adding heat from one CPU to the next in the loop, parallel you are not, and you are effectively only have the equivalent of one WB worth of pressure drop with parallel loops. And then you can have a giant bong or RAD right before it all and then back to the res/pump. If you must use purpose built rads then run 2 x MCR320 and have the rads run in series before you split to the processors and then after the rads split into parallel to each processor and then meet back up in the resevoir.

I think some variation of that would be great.

Why would a radiator designed for high CFM fans be better than a radiator designed for low cfm fans when we are talking about low cfm fans??



I was under the impression the OP wanted acceptable noise levels. Forgive me if I'm mistaken.

[Turtle 1]

Iwould look at something like this . Its expensive but so are render farms. Better safe than sorry.

http://www.koolance.com/shop/product...roducts_id=372

[NickS]

That might actually be a good idea there, Turtle. Expensive but clean, easy, etc.