Bloodrage 4 slot quadfire ain't working - EDIT: it works! :D

[Patch]

Can't get into CCC with 4 X 4890's installed. Tells me ATI drivers are not installed or drivers failed.

GPU-Z sees all 4 cards.

Have uninstalled and reinstalled last 2 CCC updates at least 3 times each, with and without clearing CMOS.

Any Bloodrage BIOS updates that make a difference?
Any driver suggestions?

Currently using G25.

Gonna go take the kids swimming. When I return I guess I'll try a few different BIOSes.

[The-Fox]

Maybe a driver issue.
What driver version are you running ? what OS ?

BTW, are you using any PCI-E riser for the 4th GFX card ?

[Mescalamba]

I'm kinda afraid thats possible, that ATi didn't think about 4x HD 4890 setup.. and if Im correct, its not first time they "forget" to do this..

If mobo works and GPU-Z sees all graphic cards.. its definetly driver problem. Probably wait for next catalyst.. :(

[Patch]

Been trying 9.6 and 9.5 CCC.
Vista 64. XP won't do more than 2 cards.

This board is so tight I'm using 4 risers. Even need a riser on the first card to get the second riser in.

I was having a bear getting decent CPU clocks on the classified with quadfire 4890's (trifire was fine), so I thought I'd try the trusty Bloodrage.

Also tried just doing Tri-fire 4890's on the BR. No luck. Still says CCC failed and won't load it.

Thinking that the 8x slots might be a problem with tri-fire or quadfire, I slapped on my 4870X2 with one 4890 just using the 2 16x slots. It at least recognizes CCC, but won't let me trifire with the 4890.

Edit: I could get all four 4890's going on the Classified. But performance was way worse than Tri-fire, and for some reason it really hindered my CPU clocks.

[Mescalamba]

I guess that X58 chipset wasn't really designed to hold this load.. wait for HD 4890X2 or just use 2x HD 4870X2 :)

That poor performace and low CPU clocks are cause chipset is so much stressed.. (and maybe NF200 chipset isn't happy from that:) )

Sorry to hear that.. maybe ask directly AMD? Or get another mobo.. :( (Im for DFi if it has 4 PCI-E)

[Patch]

Well, a fresh clean install of Vista got 4890 trifire going on the Bloodrage. It wasn't even accepting ONE card, so something clearly wasn't right. Figured a fresh OS might help.

But when I put 4 cards on it won't even let me into Windows without a BSOD. POSTS and boots Windows to just before login - then BSOD everytime.

Tried underclocking the CPU, over and undervolting it.

Tried over and undervolting Chipsets. Same problem. I think you're right Mescalamba, chipset has a hard time handling this load.

I wonder if the power mod to the 24 pin connector Shamino described for quadfire 4890's on the Classified would help with the Bloodrage?

[zanzabar]

ati supports 4x 4890 with 9.5 and i think 9.6, chew has it on his am3 he might be able to help u if its a driver problem.

[saaya]

Been trying 9.6 and 9.5 CCC.
Vista 64. XP won't do more than 2 cards.

This board is so tight I'm using 4 risers. Even need a riser on the first card to get the second riser in.

I was having a bear getting decent CPU clocks on the classified with quadfire 4890's (trifire was fine), so I thought I'd try the trusty Bloodrage.

Also tried just doing Tri-fire 4890's on the BR. No luck. Still says CCC failed and won't load it.

Thinking that the 8x slots might be a problem with tri-fire or quadfire, I slapped on my 4870X2 with one 4890 just using the 2 16x slots. It at least recognizes CCC, but won't let me trifire with the 4890.

Edit: I could get all four 4890's going on the Classified. But performance was way worse than Tri-fire, and for some reason it really hindered my CPU clocks.

hmmmm might be power, if all cards try to suck a lot of power through the pciE slot... wiring from the psu directly to the pciE slot power might help... but i dont think this is a power issue... then it would work but crash or you couldnt even boot with 4 cards installed...

i think it has to do with memory sizing... the way memory is adressed and remapped... so its probably a vga bios/mobo bios compatibility thing... unfortunately i doubt foxconn will test with 4 vgas even if you ask them... but pm carl, lordtrident, he might be able to help :toast:

and yeah. drivers sure arent tested and debugged for this either, so im not surprised its not working well and you need several attempts to get it working...

btw, if you use riser cards that might actually cause the problem... cause the lowest pciE slot has way long traces already... id recommend you to watercool the first and third card, that gives you enough space to fit 4 4890s in there, and the 2nd and 4th dont need to be watercooled. try if that works, also on the classified...

longer pciE traces = worse performance... not always, but it can cause worse perf

also increase x58 ioh 1.5v volts, ie the pciE volts...

and lower pciE clocks on classified with 4 cards?
you mean lower bclocks ie lower qpi clocks right? well thats to be expected if you load the chipset more... :D

kinda like running loads of memory also means lower max memory clocks...

[The-Fox]

I'm with saaya here.
Might be Power issue, what PSU are you trying to feed this monster with ?

Also, Risers can cause problems as saaya said.
Instead of Waercooling, you can use a 3rd party cooler which takes 1 slot on the 3rd card, and then your 4th card will fit without needing a riser, but it is going to be tight.
here is an example from dpa (http://www.xtremesystems.org/forums/showpost.php?p=3851579&postcount=1), an XS user managed to pulled it off:

http://www.diskusjon.no/index.php?act=attach&type=post&id=307477

[Mescalamba]

Seriously brutal config. :D

Hm.. DFi on their AMD based mobos, has another floppy like power plug next to each PCI-E slots.. I guess that now I know why.. Idea about cards eating too much PCI-E power might be correct..

Cause I don't think you get BSOD just cause you have four cards in system.. BSOD's are pretty usually cause by some kind of HW fault..

Btw. this config needs some serious ampers on each rail.. plus I wouldn't try it with less than 5 rail PSU..

[Patch]

Saaya,

Yeah, it was bclocks that were hindered on the Classified big time with 4 cards on.

I don't think it's a driver issue after all. It's power. People have been blowing their 24 pin ATX connectors with quadfired 4890's on Classifieds. Trifire is now working great on the Bloodrage, but still can't get into Windows with 4 cards on. Even with IOH at 1.6. (with and without ICH up high, too).

I've been using dual PSU setup with a Corsair HX1000 and a PCP&C 750W Silencer splitting up the cards.

On the Classified I was using one riser, and just like DPA's setup you posted, the first card intake was tight. The Bloodrage is so tight I need more risers.

[saaya]

mascalamba, i dont think you need a certain amount of rails, its more about 12v wattage... and those extra power connectors on the boards might help...
i once meassured power on the pciE slots with 2 vgas and there was a notable voltage drop from the atx connector to the pciE slots... i think .3v or .4v... with more vgas thats probably even worse...

lol props to dpa! :toast:
great job at cramping those cards together without any riser cards, respect! :up:

patch, risers dont really work that well with pciE 2.0 anymore afaik... at least the flexible ones, and i havent seen a hard pcb one that would make sense to use...
i think you really have a better chance of getting 4 cards to work if you do what dpa did... though the BR has a sligjtly diferent slot layout...
if i were you id try the first card with a riser, second normal, third mod the heatsink, water or any other single slot heatsink, and 4th, normal...

also, you might wanna check each slot individually, cause ive heard from 2 people who had one of the x8 slots or even one x16 anbd one x8 slot not working at all or unstable... seems quality control has a few lazy days and lets boards slip with not fully tested pciE slots...

i have 6 vgas here, but no riser card and no waterblock... and no heatsink like that weird thermalright thing either i could mod somehow...

i was benching with sof at gc last year in germany though and we were running 4 4850s iirc... and that worked alright... but that was on a79as... anyways, no bios probs, no psu probs... and those suckers pulled quite some current, and it was on an amd phenom1 board so yeah, the cpu was pulling LOADS of current too...

im sure 4890s pull more current than that but i cant really imagine thats the problem here... hmmm
so what, before you could get into windows but now you cant even get into windows with 4 cards?
or wehats the status atm?

[Patch]

im sure 4890s pull more current than that but i cant really imagine thats the problem here... hmmm
so what, before you could get into windows but now you cant even get into windows with 4 cards?
or wehats the status atm?

It's this thread that keeps me coming back to power issues:
http://www.xtremesystems.org/forums/showthread.php?t=226942

Yep, benches and overclocks just fine on BR with 3 cards on risers. If I connect the 4th, it posts fine and starts Windows load but always freezes just before login screen. Doesn't matter if I underclock/overclock/default settings the GPU's and CPU. Same thing with 4 cards.

Here's my setup:

http://img140.imageshack.us/img140/1666/brquad1.jpg
http://img197.imageshack.us/img197/3193/brquad3.jpg
http://img205.imageshack.us/img205/8177/brquad4.jpg

I have a few MCW60's and can put 2 cards on water. But since it's now 95 F in my garage right now, I think I'll wait till the heat wave subsides.

Then it's going on DICE/LN2. I'd like to try out the supplied BR LN2/DICE pot using 3 cards before I come back to 4 gpus.

BTW, any suggestions for insulating the NB/SB complex for extreme cooling with the supplied pot?

I haven't actually seen anyone use it yet. I like that that the vregs and SB can also benefit from the cold with the way it's mounted, but it does make for interesting insulation issues.

[Utnorris]

Just use Artisit's eraser around the NB and SB. Easy to put on and get off and just put a paper towel between the eraser and the layer of foam, just like you would do your CPU for extreme cooling. Not sure if you need to put anything in the PCIe slots.

http://www.xtremesystems.org/forums/showthread.php?t=224912

Not the NB, but gives you a good idea.

[Patch]

That is a very nice guide that I haven't seen before. I've used eraser, but never thought of putting it in the microwave first. Great stuff!

To be more specific about my insulation concern:

Typically when freezing a NB you need to cut any heatpipes connecting to vregs and SB because you're replacing the NB heatsink with the pot.
But the Bloodrage heatsink/waterblock/LN2 pot sits on top of a block that is contiguous with the SB heatsink. And you don't need to cut any heatpipes to the vregs when using the LN2 pot. The SB heatsink itself is gonna be a condensation magnet. Seems like the whole SB sink complex needs to be covered over with insulation, but it seems unnatural to cover up an entire heatsink with insulation. Also seems like the heatpipes to the vregs need to be wrapped with insulation too.

I'll certainly find out through trial and error. Just wondering if anyone has used the provided pot before and what their experience was.

Now I'm off topic...........

[saaya]

hey patch, looks nice... those are the same risers i used a while back... was so hard to find a shop that had them at a decent price, only found one in the uk and raja ordered a pair for me back then :D

anyways, so as soon as you use a card in the 4th slot its a problem even with no cards in slots 1-3?
have you tried the 4th card without riser?

about power... well... check out xbitlabs, they did a detailled test and the 4890 pulls 24W from the pciE slot under load...
thats fine and within specs, and the 4870 actually pulls almost double that, but with 4 cards your around 100W and thats with stock speed... when oced your probably looking at 125W or more and since the atx plug only has 2 measily 12v connectors... thats definately a weak link... and if your psu is not using good cabling or a good connector, then that can definately melt the atx connector, no doubt about it!

the weird thing is that if you look at the cards, i cant see where the 12v from the slot actually goes... it seems its somehow bundled with the 12v from the 6 and 8 pin connectors and used together? but thats the weird thing, why would they do this? why mix two diferent 12v rails that are clearly known to have diferent voltage levels and spinkes in them? and then it gets even more weird...

correct me if im wrong but the 6pin is rated for 50W max and the 8pin for 75W max, right?
they fully saturate the 6pin and suck 50W from it, but suck even less than that from the 8pin... 0_o only 45W...

anyhow... it seems they mix the pciE slot power and 6+8 power cause thats the only way they could get around using two 8pin connectors which would reduce system (psu) compatibility big time, especially for xfire...

http://www.xbitlabs.com/misc/picture/?src=/images/video/radeon-hd4890/4890power_full.png&1=1


about the included pot...
yes that insulation works great, if its expensive, go get the office poster gum stuff, its dirt cheap usually and works equally well.
just DONT mix that with vaseline or any other dielectric grease, it gets really messy then and gets softer and sticky like hot chewing gum rotting in the sun on the street :S :lol:

the pot doesnt work well, i tried it myself... it works good enough and i didnt want any further dealys with BR so i said its ok...
basically its too short and ln2 spills over quickly... but theres a way to use the pot and thats by adding an extension to it...
i used a paper cup once and a styro foam cup another time... a pepsi bottle cut up should work too...

paper cup worked best for me though since you can really sqeeze it in there and have it close the gaps.
there will still be some ln2 spilling over the boarders of the pot, but if you add a propper extension thats ok, some ln2 will squirt on the board but even if you dont insulate at all i never had a problem with that... but i do recommend insulating it of course!

actually what worked best for me was cutting up the plastic extension for the blackops 4in1 cooler to use with ln2 and dry ice, and then squeeze that into the BR pot... it was fully watertight after that and tall enough so no ln2 spilled over, but i dont think you have those around... and modding the BO extension took quite some effort lol :D

if you have a thin pipe pot, then id recommend you to use that... the bundled pot is more for people who havent licked blood yet and wanna give it a try... it doesnt work well but it comes in for free and is good enough for some basic sessions :D

[Patch]

anyways, so as soon as you use a card in the 4th slot its a problem even with no cards in slots 1-3?
have you tried the 4th card without riser?



4th slot works fine by itself (with riser) and will do Crossfire and Tri-fire (with risers). It was tough to get it enabled, though. Sometimes it helped to boot into Safe Mode first, then restart into normal. Still can't get into Windows with all 4 cards, though.

Thanks for the NB pot info. It sure does look short.
Think I'll try it. Paper plate makes a good funnel (cut a hole in the middle and roll it up :up:) and can be longer than a paper cup.

Gonna give your power suggestions a try this week. Tantalizing stuff.

[The-Fox]

Saaya,
PCI-E 6 Pin connector can supply up to 75W and 8Pin is 150W, exactly double.
The best site on earth for PC connections, including Pinouts, electrical specification and pictures:
http://www.playtool.com/pages/psuconnectors/connectors.html#pciexpress

[saaya]

alright, cheers fox! :toast:
makes sense since the cpu pwm is fed by the 8pin 12v plug only, and we are pulling more than 150W under load for sure when going ln2

then its really confusing me though why amd is using pciE slot power to begin with...
and why do they use a 6 plus 8 pin connector? the 4890 pulls 120W under load... 120W... a single 8pin could supply that, why would they use 6+8+slot power then?
weird... very weird...

i think if we cut slot power alltogether, provided its possible, that should help a lot...
the board will have less noise from several pwms sucking and spiking and inserting emi into its power traces around the IOH, and the cards get one stable clean 12v input...

[Patch]

Got it working for now.

Multiple OS clean installs, multiple driver installs, multiple card and slot swaps, some pin taping and bridge rearranging later - finally. :up:

I can't even tell exactly what got it working, but I think it was the crossfire bridge orientation. Doesn't make a lot of sense that the bridges could be any which way when benching with tri-fire, but get more picky enough about orientation with 4 cards that it won't even get into windows.

EDIT: Notice my picture a few posts above? One bridge is "upside down" compared to the other two.

Thanks to all, especially Saaya, for assistance. Now I hope I don't blow my 24 pin connector.......

http://img141.imageshack.us/img141/7556/yayquadfire.jpg

[saaya]

great :D
please post what bridge orientation your using now in case anybody else has similar problems...
and check if your atx 12v wire gets hot or warm when you bench...

and for anybody thinking about 4870 quadxfire... not a good idea, those pull double the current through the pciE slots than 4890s, and those already blew at least 2 atx board connectors :D id like to see a picture though of a the burned atx connectors... i wonder where it exactly went bust... on the pcb, the metal connector itself, the psu cable...

[Patch]

Some preliminary benching just to see if it would complete with moderate GPU clocks on stock GPU cooling and voltage. I'll try to download and run 3D03 tomorrow. That was the hardest to complete with trifire cards - harder than 05, 06 or Vantage. Postponing the subambient runs until I can finetune quadcards.

http://img23.imageshack.us/img23/8574/30482.jpg

Here's the setup. I positioned the ram fan so it would get the 12V connector area and it didn't seem particularly warm. Don't know if the fan made a difference or not.

http://img26.imageshack.us/img26/2539/quadbridges.jpg

[saaya]

30K :D
thats about double what i get with 2 GTS250 cards in sli, impressive :D
what do 3 gtx285 cards score in 2k6 more or less?

btw, what are your water temps and ambient temps?
what settings do you use to get to 3.6?

[Mean Machine]

Come on, I did 31k with two 4890's and a QX9770 @ 5.3GHz. Up that CPU a bit. :)

[DJSUB]

that's cpu bottleneck.
i did 30274 in 3dmark06 with 2x heavily oc'ed 4770 core i7 920 @ 4.703ghz bloodrage.

you need to reach atleast 5ghz to make a goodscore with your quadfire 4890.

i also tried heavily oc'ed trifire 4770 on MSI Eclipse Plus with core i7 @ 4.5ghz i only got 289xx in 06 :(

[Paxi]

30k is not much.. You need more CPU Power. I get almost 29k with my 24/7 Setup, Xeon @ 4,4 and 4870X2 @ 825/1000. You need at least 5Ghz on CPU to get some decent scores with Quadcrossfire
Anyway nice work so far, wish you good luck :)

[saaya]

30k is not much.. You need more CPU Power. I get almost 29k with my 24/7 Setup, Xeon @ 4,4 and 4870X2 @ 825/1000. You need at least 5Ghz on CPU to get some decent scores with Quadcrossfire
Anyway nice work so far, wish you good luck :)
im sure he knows it, hes just dialing in the cards before going ln2 on the whole thing if i understood it correctly... ln2 isnt cheap and with condensation issues you can play around with settings and clocks and voltages for a long time... and when it comes to 4 cards getting them all tweaked is quite some work depending on how teaked you want everything to be :D

4.6 on water is pretty nice... i hope i can get there too :D

[Patch]

Lol, I just ran a quick bench to see if the cards would complete it. Definite CPU bottleneck.

I wanted to make sure the CPU wasn't the problem if it couldn't complete - notice the ridiculous vcore for the CPU clock.

I hope this chip does better on LN2 and DICE. Highest validation I've attained on water is 4.97 (http://valid.canardpc.com/show_oc.php?id=580139) and I haven't really tried much 3D benching with it.

Haven't even looked at temps on this setup, but with the same chip/waterblocks/mobo and similar setup upstairs (480 TFC rad instead of the 480 GTX in the garage) I didn't see temps go much higher than 80 C at load with vcore up to 1.7.

Ambients have been >90 F in the garage the last few days...

Settings-wise, I haven't found what makes me happy with the 975 yet - it's still new to me. My 920 C0 could do 3.6 all day on the bloodrage with just about everything on auto except for a little vcore bump. But getting the 920 up to 4.7 (225X21) took PCIE freq 102, latency tweaks, Vcore/VTT at 1.7 range, IOH 1.6, and ICH up about 6 notches.

LN2 is miserably expensive here. I have multiple suppliers, but the cheapest I can haggle anyone down to is $5.50/L - even bringing my own dewar to them to fill. :( I've got a few liters left right now to use, then gonna play with DICE a bit before I refill the dewar.

[saaya]

mhhh did you do the pciE clock mod to get a higher bclock?

4.97 suicide on water is nice, and with all 4 cores and ht...
LOL, module maker cannon? hahah funny cpuz bug... :D

about ln2, go find a cleanroom fab around where you live or a tech university or some physics research place... those all consume ln2 in large volumes, especially clean room fabs... the clean room fab at msc cellshock had a bi-weekly delivery of more than 1000L iirc... and the price was ridiculously low since they had a fixed supply contract for a year or maybe more... go and tab in with those guys and youll be smiling from one ear to the other about your ln2 prices ;) :D

[Patch]

Quick update.

Still wrestling with settings (and getting closer to trying the voltmod for PCIE frequency).

Just starting to do some LN2 runs with quadfire (stock Toxic cooling). The NB does not like it too cold! I was pouring every 30 seconds to micromanage temps. Finally just put some DICE pellets (no acetone) in the NB pot and it was much happier. When my second k-probe arrives I'll drill a hole in it and get some numbers it likes.

Some preliminary 3D benches at 4.91 Ghz:

33144 (http://www.hwbot.org/result.do?resultId=872317) 3dmark06

29030 (http://www.hwbot.org/result.do?resultId=872315) Vantage

A little disappointed with my 3D06 score, but pleasantly surprised with Vantage numbers. CPU numbers are low in 06, even at 4.91 Ghz. A lot of things to tweak still and I hope to bump the numbers up. I'm afraid to give the cards more volts, worrying about blowing my ATX connector. I will eventually, but would like to maximize other things first.

[dinos22]

your proxy looks bugged
rerun it and it should pickup in speed and score

[Patch]

your proxy looks bugged
rerun it and it should pickup in speed and score

Thx dinos22.

I paid more attention and it periodically gets bugged. You saved me a lot of time by noticing when frames weren't as anticipated and canceling runs early.

NCspecV81 pointed out that the bloodrage tends to downclock multi's when CPU is stressed, despite "always on" turbo. Changed max settings in BIOS and my CPU scores jumped. Though, downside is that I wasn't able to complete some benches at previous clocks. Went back and forth to try to get best overall scores.

30793 (http://www.hwbot.org/result.do?resultId=872842) Vantage
34803 (http://www.hwbot.org/result.do?resultId=872869) 3dmark06
44541 (http://www.hwbot.org/result.do?resultId=872855) 3dmark05
151413 (http://www.hwbot.org/result.do?resultId=872845) 3dmark03


This is it for a while. Been through 50+ liters of LN2 and 30 lbs of DICE. Now I'll take it down to one card and work on max clocks and some pi runs. Maybe do some soldering for another go in the future......

[The-Fox]

Patch, the option in BIOS is called "Max current override" under CPu features and it should be Enabled to prevent throtteling due to high TDP.
The "Always on" for turbo is a bit misleading :p

[Patch]

Thanks for help guys. I've been having fun with the Bloodrage and will come back to it, but I've got a Commando coming soon and some Celeron's to freeze.

Parting shot:

http://img403.imageshack.us/img403/4383/35634dm06.jpg

Toxic 4890's are stock voltage and cooling.
http://www.hwbot.org/result.do?resultId=873152

[saaya]

impressive! :toast: