How a GPU is limited by a PCI-e lane? 5870x2/395GTX in PCI-e 2.0 x8 lanes

[jfromeo]

http://img25.imageshack.us/img25/4276/39811734.png

Which are the numbers that limit first the I/O of a GPU? I'm taking for granted it is the Data rate. So, with these numbers in the hand, we get:

PCI-e 2.0 x8 : 4 GB/s
PCI-e 2.0 x16: 8 GB/s

Will an hipotetically 5870x2/395GTX be able to saturate a PCI-e 2.0 x8 lane (>4 GB/s)?

And why at Quad-SLI/Quad-CF configurations (4x4890 and 4x285GTX, mainly) a 16x/16x/16x/16x (X58 + 2xNF200) outperform by +10% (fps, synthetic benchmarks...) a 8x/8x/8x/8x layout (X58), although there is some latency added by the NF200s?

Or in other words, why do we have to go to Quad-GPU configurations to see a significant increase between the lanes capacity, when at single-GPUs, neither the 285GTX nor the 4890 (to highlight the best single-slot single-GPU solutions at the moment) are able to saturate a PCI-e 2.0 x8 lane (there is no gain, performance wise, from setting them up in a x8 than in a x16)?

So if a 4890/285GTX performs equally in a x8 and in a x16 PCI-e slot, why four of them don't? Wouldn't that be a northbridge bottleneck then (X58)?

Many thanks in advance.

[zanzabar]

the nf200 adds latency and has no added IO bandwidth just slot bandwidth so when u have a set of gpus all using data at the same time there no more bandwidth that can come from the NB so there is nothing that will improve the IO just add latency. and u have it backwards with the 4890 its better sans nf200 and there is no way to run 285 in quad without the dual nf200 so that cant be compared




as for what will saturate a buss, as of now the 4870x2 and 295 see drawbacks from and 8x slot and i would expect that 5870 and gt300 wont saturate an 8x pci-e2 buss or will not be greatly effected

[jfromeo]

Thanks.

But, in raw numbers, how much of that limited data rate from a PCI-e 2.0 x8 lane (4 GB/s) is using a 285GTX/4890?.

I can't see anywhere the PCI bus specs needed :S

[zanzabar]

im not sure how much or if they are limited, i have a quadfire board with 8x8x8x8 so i could try my 1 card in an 8x slot but im far to lazy today, maybe tomorrow affter some sleep

[tiro_uspsss]

ATi cards are less affected by less PCIE bandwidth fyi
I think Toms did a PCIE bandwidth article back in the 88GT vs 3870 days... the ATi was still doing fine in PCIEx4 1.1 - the NV just choked bad.... I've always wondered why the diff myself :shrug:

[demonkevy666]

cpu has to go to all lanes, having less lanes means a faster throughput. the 10% boost could be cause having four gpus go for ram at once might slow things down if your gpu is ram limited. could just be more efficient in electric flow of electrons lol

I believe ati cards can adjust them self's for low pci-express lanes like they can sense it while nvidia cards do not have this ability

dual cards get choked on lower lanes because they need the bandwidth on the single connection. as it's equivalent to putting to cards in one slot.

[flopper]

http://www.bit-tech.net/hardware/cpus/2009/09/14/lynnfield-pci-express-gaming-performance/1

Bit tech shows how min fps is affected between 8x and 16x using crossfire.
this is also with old cards, and it be interesting to see of this starts to be a bigger issue with 5800 series and beyond.
so, yes there is a limitation, using at least 2 cards.

If I bought a new system and planneed for crossfire, I be getting either i7 1366 plaform or an amd system.

[zanzabar]

im not sure that the bitech shows that 8x8 is causing that with the 775 and am3 minimum frames even with 1 card are highly ram sensitive

[largon]

Sometime ago, I benched my HD4890 at x16 PCIe 2.0 vs x1 PCIe 2.0 with Crysis Island timedemo:
x16 PCIe 2.0:
Avg FPS: 43.25
Min FPS: 22.83
Max FPS: 51.36

x1 PCIe 2.0:
Avg FPS: 26.67
Min FPS: 15.37
Max FPS: 37.60

Not that huge a difference, in the end. But significant, sure.
Average of 3 out of 4 runs with 1st run not included as it's always slower than others.

[SocketMan]

Or in other words, why do we have to go to Quad-GPU configurations to see a significant increase between the lanes capacity, when at single-GPUs, neither the 285GTX nor the 4890 (to highlight the best single-slot single-GPU solutions at the moment) are able to saturate a PCI-e 2.0 x8 lane (there is no gain, performance wise, from setting them up in a x8 than in a x16)?



There is another side to it as well:people with Quad-Fire/SLI don't run
1280x1024 no AA/AF,but something like 1920x1200+ | AAx8 AFx16;
in other words it's not just the number of cards that saturates the PCIe
but the high resolutions+high eye candy.


The higher the resolution and the eye candy - the more bandwidth/capacity is needed.

I also just found out recently (by accident) that my 2 8800gt's (on DFI nf4 expert) don't need the SLI bridge -to do the SLI :eek:

ATI CF does not work without the bridge - which makes me think
that Nvidia uses the PCIe rather then the SLI bridge,thus resulting in
higher PCIe utilization.Could also just be the fact that the Nforce 4 is
is getting a "little" old.

[Luka_Aveiro]

you can do sli without the bridge since ever. ;)

not a "problem" with your 8800GTs :D

[villa1n]

http://www.bit-tech.net/hardware/cpus/2009/09/14/lynnfield-pci-express-gaming-performance/1

Bit tech shows how min fps is affected between 8x and 16x using crossfire.
this is also with old cards, and it be interesting to see of this starts to be a bigger issue with 5800 series and beyond.
so, yes there is a limitation, using at least 2 cards.

If I bought a new system and planneed for crossfire, I be getting either i7 1366 plaform or an amd system.

Yes, this was the review I was trying to find in the Lynnfield thread... I dont understand why they used such a low Resolution to test... At least have 1080p, 1920x1080 with maxed out eye candy, that would put a bit of pressure on the pcie..and use 4870x2 s in xfire or 295 gtxs see if you can saturate it..

Because judging by the results it shows the advantage of the pci-e latency at lower Resolutions, but fails to address possible saturation at higher resolutions.

[villa1n]

http://www.techpowerup.com/reviews/AMD/HD_5870_PCI-Express_Scaling/25.html

This answers your question i believe :) Seems like for most of us, the 1% performance hit isnt a big deal.

[Frontl1ne]

http://www.techpowerup.com/reviews/AMD/HD_5870_PCI-Express_Scaling/25.html

This answers your question i believe :) Seems like for most of us, the 1% performance hit isnt a big deal.

That is awesome, thanks :up:

[fornowagain]

Saves me repeating it.

http://forums.overclockers.co.uk/showpost.php?p=14945968&postcount=295

Theres one significant problem with that, which is, they didn't use a P55 or the on die PCI-e controller. Meaning those results are completely irrelevant, thats probably the best case scenario which is very good, but just because one controller does very well, that same controller with 2 cards might work differently, it doesn't take into account ANY cross communication, nor the extra workload being done by the controllers when two cards are connected.

Firstly, can the P55 perform the same with no little bugs or performance wrinkles in single card mode? Secondly, does adding a second card change those not because of the speed of the slots, but because the controller suffers? Take ssd's a nice vertex will get you x speed, but put 4 on the controller and you don't see 4 times the speed the controller is overloaded.

Considering they didn't test the P55, nor the controllers ability to scale with more cards, those results are fairly meaningless. yes the card doesn't need 16x pci-e slot bandwidth to perform fully, if that was the only question here that would be fine, but theres far more than bandwidth at play here which they haven't even considered.


Haven't reviews ALREADY shown that its slower with two cards on the P55, than on 2x 16x slots on other boards, meaning, if the cards don't need the bandwidth we already know the controller , or something, isn't working as well?

[Chickenfeed]

Do keep in mind their testing was only with 1 gpu. Just because a single gpu doesn't show tangible benefits with added bandwidth does not mean this will be the case with 2 or more as there is added communication between gpus increasing the required bandwidth ( something that doesn't happen when using only 1 gpu ). I still expect that crossfire 5870s perform better with dual x16 pcie ( and by better more than a single digit percent increase )

Still good news to those who may be using a 5870 in the future and only are able to run at x8.

EDIT : Ha funny you posted that quote as I was typing this. My thoughts exactly. Hopefully some one does a proper test using P55 and Crossfire.

[W1zzard]

http://www.techpowerup.com/reviews/AMD/HD_5870_PCI-Express_Scaling/25.html

This answers your question i believe :) Seems like for most of us, the 1% performance hit isnt a big deal.

thanks to this thread for inspiring me to write the 5870 pcie scaling article

[nascasho]

I'm glad you posted that because I'll be running my second card in my 8x slot and the first in the 16x slot. One limitation of the LE...

[nascasho]

thanks to this thread for inspiring me to write the 5870 pcie scaling article

Someone give this guy an award, amazing tests!

[Computurd]

yes.but the scaling tests do not include crossfire, i am missing the relevance of these, the real question is when crossfire or SLI is in use!

[jfromeo]

Thanks Wizard :)

[Computurd]

w1zzard, can you do some crossfire tests with that setup?

[villa1n]

thanks to this thread for inspiring me to write the 5870 pcie scaling article

Yeah if you guys have the time, and the board lying around, a p55 test would be very interesting, instead of manually chocking the pci-e slots because if bandwidth isn't the issue, it would be good to see , as has been mentioned above if the pci-e controller gets overloaded.. i think I saw a test somewhere where at high res with 4870x2's xfired it saw some performance dips.. but i cant remember what site, and they attributed it to bandwidth which doesnt seem to be the case... in most games.

And it seems like COD4 and by proxy Mdrn Warfare 2 are hit by the bandwidth limitation.. :( lol only game i really play atm.

[Xoulz]

yes.but the scaling tests do not include crossfire, i am missing the relevance of these, the real question is when crossfire or SLI is in use!

Exactly. Need to know how robust the on-die PCIe controller is. Wizzard might try the same test in a multi-GPU bench.

[jaredpace]

This guy plans on doing a p55 vs x58 test with 5870 x-fire. (and possibly hydra if he gets the MSI big bang)

http://www.ocforums.com/showthread.php?t=619246

[jaredpace]

This guy plans on doing a p55 vs x58 test with 5870 x-fire. (and possibly hydra if he gets the MSI big bang)

http://www.ocforums.com/showthread.php?t=619246

[Computurd]

yea there still is much speculation on the x8 functionality of the p55 chipset.

[villa1n]

This guy plans on doing a p55 vs x58 test with 5870 x-fire. (and possibly hydra if he gets the MSI big bang)

http://www.ocforums.com/showthread.php?t=619246

He s got some results up, but i dont have 285gtx sli numbers memorized, so i can't really tell if its bottlenecking or not ^^

first numbers he has up (http://www.ocforums.com/showpost.php?p=6241419&postcount=2)

[jaredpace]

He s got some results up, but i dont have 285gtx sli numbers memorized, so i can't really tell if its bottlenecking or not ^^

first numbers he has up (http://www.ocforums.com/showpost.php?p=6241419&postcount=2)

P55
X11571
H16486
P21852
3d06-25392

X58
X11626
H16550
P21936
3d06-25758

About 98.5% for a P55 compared to X58 between PCI_E x8 and x16. matches this:

http://tpucdn.com/reviews/AMD/HD_5870_PCI-Express_Scaling/images/perfrel.gif

[Chumbucket843]

P55
X11571
H16486
P21852
3d06-25392

X58
X11626
H16550
P21936
3d06-25758

About 98.5% for a P55 compared to X58 between PCI_E x8 and x16. matches this:

http://tpucdn.com/reviews/AMD/HD_5870_PCI-Express_Scaling/images/perfrel.gif
crossfire performance is bad on p55 is bad with 4800 series. 5870 crossfire will be even worse.
http://www.bit-tech.net/hardware/cpus/2009/09/14/lynnfield-pci-express-gaming-performance/1

[jaredpace]

Thanks for that link, chumbucket. p55 is particularly slower in crysis, however faster in some benches in l4d & cod5.

[villa1n]

Thanks for that link, chumbucket. p55 is particularly slower in crysis, however faster in some benches in l4d & cod5.

Ouch, yeah overall average frames dont take that big a hit, but its the min framerates that are a sizeable piece behind x58

[bingo13]

http://www.anandtech.com/mb/showdoc.aspx?i=3649