Water cooling business servers?

[Extigy]

Watercooling small servers is probably not worth it - just get some good AC - this would be a precursor to more extreme cooling anyway as even if you are watercooling, that heat is going to be transfered to the air eventually (albiet alot slower) which will raise the ambient temperature which without some sort of AC, will heat the room up gradually!

If you are looking for perfomance and high server density, watercooling is brilliant (see ibm 575) but again, without proper infrastructure and design, you would run into heat problems.

I fail to see how air cooling is more reliable than water cooling tho. Air cooling consists of a lump of metal with no moving parts (heatsink) and fans, which yes may break down but get some good quality ones you reduce this (mtfb's of 30,000+ hours on good brands) - but this problem is apparent in both solutions. With watercooling you have to worry about leaks, cleaning the loops, pumps failing. Reading some people have had very good experiances water cooling, but being reasonable, watercooling is more complicated, introduces more variables and thus, greater margin for error.

[Serpentarius]

could anyone elaborate more on server watercooling?
i always wonder how they installed it? where the radiator is placed? how they perform maintenance? would they change thermal paste periodically? could they perform maintenance with the server running non-stop? when is the usual periodic maintenance? monthly? quarterly? or yearly?

anyone care to elaborate their work experience?

[Xeon th MG Pony]

There's this issue of warranty.
If some hardware fails and you call to get it replaced, they're not going to help you once they see you water cool your server so you should think about that point

One of the hydro dams here has an entire field control cabinet and central gate control cabinet water cooled by design, believe it or not you can buy them prewater cooled for industrial control systems.

[Xeon th MG Pony]

Watercooling small servers is probably not worth it - just get some good AC - this would be a precursor to more extreme cooling anyway as even if you are watercooling, that heat is going to be transfered to the air eventually (albiet alot slower) which will raise the ambient temperature which without some sort of AC, will heat the room up gradually!

If you are looking for perfomance and high server density, watercooling is brilliant (see ibm 575) but again, without proper infrastructure and design, you would run into heat problems.

I fail to see how air cooling is more reliable than water cooling tho. Air cooling consists of a lump of metal with no moving parts (heatsink) and fans, which yes may break down but get some good quality ones you reduce this (mtfb's of 30,000+ hours on good brands) - but this problem is apparent in both solutions. With watercooling you have to worry about leaks, cleaning the loops, pumps failing. Reading some people have had very good experiances water cooling, but being reasonable, watercooling is more complicated, introduces more variables and thus, greater margin for error.

Fans have several failure points:
1 - Berrings - Failure modes: Dust, Heat stress, Lubrication failure which leads to seizing.
2 - Dust - Cuases over heating of the berrings, absorbs the lubricants, throwes out spacing, unbalances the blade assembly
3 - electrical - Failed windings, failed tach, Failed hall effect.

Dust all so radicaly reduces Hs cooling efficincy and such. Now we still have this pit fall in wc but the differance is we have a whole lot more time to switch over the fan and it is easier to spot to boot.

Providing proper design and install and testing leaks are a non issue inside the case.

Pump failure is a non issue with a redundant cold spare with auto fail over.

cleaning again is a non issue with proper coolant, I haven't cleaned mine in over 5 years! of near 24/7 (Only times it was off was when I moved & when we lost power for more then a day)

Water cooling is actualy verry simpple when you look at it in a modulare view.
1 - The server system - 3 blocks (4 if dual proc) tubing leading out the back of the case

2- Interconect system (The headers with a ball valve to ballance flow and or close off for removal or work on the rack)

3 - Cooling system - Radiator & fan assembly & pumps External to the cluster out side the server room and or building.

[Xeon th MG Pony]

could anyone elaborate more on server watercooling?
i always wonder how they installed it? where the radiator is placed? how they perform maintenance? would they change thermal paste periodically? could they perform maintenance with the server running non-stop? when is the usual periodic maintenance? monthly? quarterly? or yearly?

anyone care to elaborate their work experience?

A - you never change the paste! You clamp on the blocks and that's it.

B - it is installed very securely and care fully

C - Maintenance? Ensure the rad is clean along with fans and check coolant levels, check the pumps power draw and ensure it matches the lable specs, and check the guage to ensure head pressure is stable, thats it.

D - Yes you can do maintenance on the PC hard ware while the system is hot if you have to. Thats the wonderus thing about water cooling, you can swap fans and have a good amount of time to do so with the system running not to mention every thing in the server case stayes much cleaner!

As for when it is don depends, cleaning - more often the better, general soft ware maintenance depends on the servers duties and load. Higher the load the more often it needs to be checked up on.

Scheduling is very dependant on many things, budget, load, time tables, availibility and such.

Here are some pics of my home server set up.

First pic is of the fire wall system, only the cpu is cooled rest is passive it ties in preallel to the main loop, pressure diffriential drives the flow (plenty to keep it happy)

Second pick is how it ties into the main loop with 2 Y fittings and some reducers

Third is the main cooling loop comming out of my computer, 4th pic didn't turn out will try again.

[skinnee]

Thats in your house, not a production environment.

[NaeKuh]

last time i talked to dave about watercooling a SMP, he almost chopped my head off.

Me: Dave i wanna watercool sammy
Dave: WHY?!?!?!? ARE YOU NUTS!! 35W!

[Xeon th MG Pony]

No the production ev one is lucious and has sweet manifold with Eq valves that feed each 2U server and such, if I had a rack I'd build the same set up, frankly I was too lazy to make my home core look the same, but I based the flow system off the production run, the diff is zerro.

It works, it is reliable, it is hugely bennificial too you save power with the huge reduction in fans, as a bonuse you get free additional cooling by pre-heating incomming water to the hot water tank (IE Heat reclaiming)

(FYI the systems we've sold have never had a complaint, A sexy 19U ssystem (Switch 3 servers 1Aux space) All water cooled. I've given you as much info as I am allowed to as to how we did it!

[IanY]

I admire your level of passion even though I would debate the merits and flaws either direction.

My company's servers process over two trillion dollars of funds flow daily, and not a single machine is liquid cooled. And its not my company, as in ownership, or I wouldn't be here on XS fooling around lol

[Xeon th MG Pony]

I admire your level of passion even though I would debate the merits and flaws either direction.

My company's servers process over two trillion dollars of funds flow daily, and not a single machine is liquid cooled.

No suprise, look how long it takes industry to upgrade any thing if they can avoid it, all so larg industry is verry hesitant to try new things, thats what our company is trying to do show them that there is a better more reliable way and how that way enables them to re-capture that energy and re-use it!

There is a better way, and that way not only is better but it saves you money, heat reclaim, lower A/C costs, Easier maintanace, so maney great benifits, not to mention better for our environment thanks to the ability to recapture energy other wise wasted!

[IanY]

The way the bozos set up my office machine, or rather their general lack of knowledge of how to set up a simple desktop pc, I think the bank would best stick to heatsinks lol

I have visited two of the secret datacenters. They have overkill a/c and a dust free environment (to the extent possible). Mostly IBM blade servers. Some obnoxiously large, some small.

[skinnee]

Don't get me wrong...would love to WC everything, but its just not viable in the large business environment.

[Xeon th MG Pony]

Don't get me wrong...would love to WC everything, but its just not viable in the large business environment.

It would be if implimented at the start, to retro fit I would agree.

The people I work with we aim for the fledgling buisnesses that are geting their first rack in hopes that when they expand they'll continue to use high efficincy cooling rather then relying on brute force methods IE over kill A/C.

You still need the A/C but when dumping the larger heat load out side you end up needing to consume much less power for the same cooling effect and using heat reclaim on both systems just multiplies the savings.