5870 Bottleneck Investigation (CPU and/or Memory Bandwidth)

[Chickenfeed]

I don't expect to have my card until Friday at the earliest ( figures the Asus are in stock now but I'll wait for the XFX... ) but if anyone has a request of a game / test run that they feel might serve as a good comparison let me know and I'll try to set it up now ( I have the majority of the bigger games over the last 2 years so just hit me up with a PM and If it is something that is easy enough to compare, I'll do some clock scaling ) I want to do it at 1920x1200 with 8x Adaptive AA and also with no AA so there is that contrast ( obviously 8x MSAA uses more bandwidth / vram ) I don't see the point in testing at lower resolutions personally.

From what I've see so far here and elsewhere though clocks don't seem to scale any more than past designs so it is tough to say what is holding it back. I'm still a tad disappointed that the 4870x2 pulls ahead in as many things as it does. Multigpu has gotten better yes really...

[saaya]

it depends on the what it is doing .you can see here Ati is the intel of synthetic benchmarks.http://www.bit-tech.net/hardware/gra...ture-review/10. the wider you make a vector, the harder it is to keep under full load.

on average one flop takes 1 byte per second of memory performance. this translates to 2 terabytes per second of required bandwidth for rv870 so every gpu made is bottlenecked from this. the only way is to further reduce the memory operation to calculation ratio. its already 100:1 but it must go higher.

thats too much theory there if you ask me...
this bw/flop number is a guideline... just think about it for a second, according to that guideline applied as a RULE, every gpu with the same flop performance would require the same memory bandwidth... which is clearly not true... as a guideline it may work very well, but even then it depends so much on what your actually doing and how much or how little data repetition your facing and how well you can group instructions that use the same data, and how many instructions use data that a previous instruction has just created inside the gpu, in a register or in cache, so it never needs to load any data from memory etc etc etc...

as a rough guideline it works, but its not a rule

the anandtech review is very interesting...
so basically a 5850 with 2 less simd units and 15% lower clocks performs 10-15% worse than a 5870...
that means a 5850 at 5870 speeds will most likely perform so similar to it that you might not be able to tell the diference...
maybe there will be fake 5870s in china which are actually 5850s

[Kuntz]

I do not think the 5870 is bandwidth limited at all. There are many examples out there, and in this very thread, showing that 10%-12% increases in memory bandwidth is not increasing frame rates that much (1.5% - 3%). This small increase in frame rate is due to the lowering of the memory timings and not the increase in memory bandwidth. The same phenomenon happens when overclocking system memory. This benchmark shows that 10% increases in core speed is translating to a 5.5% increase in frame rate, where as the same 10% increase in memory bandwidth only increases frame rates by 1.5%.

You guys are getting marginal increases in frame rates when bumping up the memory speeds from the decrease in GDDR5 memory timings, and you're confusing that with a memory bottleneck.

[JimmyH]

5870 AF performance:

http://www.bit-tech.net/hardware/gra...re-analysis/12

Looks like there huge performance drop enabling 16xAF.

[Chickenfeed]

Interesting, so currently ATI have better anti aliasing scaling but still have inferior AF scaling. I expect GT300 to both have good AF scaling and AA performance so it shall be interesting. I just installed the card so I'll have to see what the deal is as far as AF quality ( as 5 series AF is supposed to be the highest quality or so we are told )

I hope someone makes a tool that works with the memory error correction (as in it can detect errors ) as that would make overclokng much simpler. If anyone has any clock requests ( eg at less than stock ) Please let me know and I'll do some testing over the weekend.

[saaya]

You guys are getting marginal increases in frame rates when bumping up the memory speeds from the decrease in GDDR5 memory timings, and you're confusing that with a memory bottleneck.

your saying that performance increases minimally because the mem timings get adjusted automatically to the higher memory speeds...

but what do you mean with people confuse that with a memory bottleneck?

a memory bottleneck indication would be if performance WOULD increase when ocing the memory, not when ocing memory barely does anything at all, like in this case...

and yes, the timings get adjusted automatically on ati cards, on nv cards as well afaik... but the increase in timings is not linear, so higher mem clocks still increase the bandwidth even though timings get increased.

[Chumbucket843]

your saying that performance increases minimally because the mem timings get adjusted automatically to the higher memory speeds...

but what do you mean with people confuse that with a memory bottleneck?

a memory bottleneck indication would be if performance WOULD increase when ocing the memory, not when ocing memory barely does anything at all, like in this case...

and yes, the timings get adjusted automatically on ati cards, on nv cards as well afaik... but the increase in timings is not linear, so higher mem clocks still increase the bandwidth even though timings get increased.

i do have an idealistic view on memory bandwidth but obviously you would have to consider some things require a lot of bandwidth. in games AA, AF, fillrates are bandwidth intensive. in gpgpu the bottleneck will usually be bandwidth. transcoding is a good example of this. fortunately some things dont have that problem, like perlin noise which loves vectors or texture sampling. i think it caches really well too.

[Kuntz]

your saying that performance increases minimally because the mem timings get adjusted automatically to the higher memory speeds...

No, I think the memory timings are static, they do not change with memory adjustment. So if they are at, for example, a total cycle delay of 10, that would mean:

10 cycles @ 1200MHz = 8.3ns delay
10 cycles @ 1300MHz = 7.7ns delay

Because there is no memory bottleneck on the 5870 cards, when you increase the memory speed, all you are really doing is decreasing the nanosecond delay on memory operations, which is why a 10% increase in memory speed is only getting a 1.5% increase in performance.

[jaredpace]

No, I think the memory timings are static, they do not change with memory adjustment. So if they are at, for example, a total cycle delay of 10, that would mean:

10 cycles @ 1200MHz = 8.3ns delay
10 cycles @ 1300MHz = 7.7ns delay

Because there is no memory bottleneck on the 5870 cards, when you increase the memory speed, all you are really doing is decreasing the nanosecond delay on memory operations, which is why a 10% increase in memory speed is only getting a 1.5% increase in performance.

How would you explain all these performance increases from memory overclocking that are greater than 1.5% per 100mhz?

850 / 3600 / 091.7 / 0 / 0%
850 / 4000 / 095.2 / +3.5 +3.8%
850 / 4400 / 097.9 / +2.7 +2.9%
850 / 4800 / 100.3 / +2.5 +2.5%
850 / 5200 / 102.4 / +2.1 +2.1%

850 / 4400 - 40,9 - 0 / 0%
850 / 4800 - 42,0 - +1.1 +2.7%
850 / 5200 - 43,1 - +1.1 +2.6%

785 / 4400 - 39,3 - 0 / 0%
850 / 4800 - 42,0 - +2.7 +6.9%
900 / 5200 - 44,7 - +2.7 +6.4%

850 / 4800 - 31.6 - 0 / 0%
850 / 5272 - 32.3 - +0.6 +2.2%

And this?

http://www.firingsquad.com/hardware/...ing/page10.asp

Is this because of the latency of the memory (in nanoseconds), and not due to increasing bandwidth for chip's computational throughput?

[Kuntz]

How would you explain all these performance increases from memory overclocking that are greater than 1.5% per 100mhz?

I've already explained them, they fall under the same explanation.

http://www.firingsquad.com/hardware/...ing/page10.asp

2.9% perf increase from 9.8% memory speed increase.

850 / 4800 / 100.3 / +2.5 +2.5%
850 / 5200 / 102.4 / +2.1 +2.1%

2.1% perf increase from 8.3% memory speed increase.


I'll summurize the entire article in per cent's. Each game has 3 graphs for it representing various resolutions and stuff. Each overclock was by 9.8%:

Call of Duty WaW:
-1.3%
-0.7%
-2.0%

Crysis:
-2.2%
-2.2%
-0.3%

Crysis Very High:
-2.1%
-2.2%
-1.0%

Far Cry 2:
-2.3%
-2.4%
-2.3%

Stalker Clear Sky:
-0.5%
-0.8%
-1.6%

Left 4 Dead:
-0.8%
-2.0%
-2.9%

Res Evil 5:
-1.4%
-2.0%
-2.2%

Batman:
-3.3%
-2.9%
-3.9%

HAWX:
-1.2%
-2.7%
-3.6%

The evidence is pretty clear to me, the 5870 has no memory bottleneck. There is no increase in frame rates from increasing the memory bandwidth. These marginal increases are just from the decrease in memory timings.

I am no expert, this is just my opinion & conclusion based on the evidence floating around the internet.

If ATI went with a 512-bit bus, or 384-bit like nVidia, this would have increased memory bandwidth, however memory timings would remain the same on the GDD5 IC's, thus there would be no performance increase from the increased bus width. The cost of the card would be astronomical compared to it's entry price of $379, and it would be all for a 0% performanec increase.

[Chumbucket843]

if one of you guys has a 5870 do the same thing i did.

this is furmark at 1280x1024 8xAA 16xAF on a gtx 260 192.
all clocks but memory are the same. i want to see how the regression for the 5870 compares. i know its a little ghetto but i dont care. the memory clocks are on the bottom

[Chickenfeed]

Here you go.

Furmark 1.7
1280x1024 8x AA 60000MS
HD 5870 1GB

Mem - Score

1300 - 2733

1200 - 2537

1100 - 2360

1000 - 2187

900 - 1963





I used what CCC offers for memory range.

I noticed a weird thing though. When Overdrive is enabled the gpu 2d clocks are 157 / 300 but when it is not enabled they are 400 / 1200. This is really strange. Also I suffer from the flickering display issue on the monitor connected to the second DVI port when over drive is enabled. Its fine for benching but too annoying to be usable for games. I'm sure it will be fixed soon though.

EDIT : I discovered that the correct 2d power play clocks are in fact 157 / 300. HOWEVER due to the timing differences when using more than 1 display the lowest they go is 400/1200. In other words so much for 27watt idle with more than one display Realistically I don't really care that much, I'll take a second display over a few watts any day. It is still an issue however as you are unable to properly overclock when using more than 1 display as it stands. I'm glad I've found this out as no reviews I've seen mention it ( as most reviews use 1 24-30" display ) I'd be fine with the same 400/1200 clocks at idle if it meant the display wouldn't flicker.

[jaredpace]

Nice test chickenfeed. SO the 5870 CAN use all the bandwidth you can give it. It's scaling almost linearly.

[Chickenfeed]

Yeah seems to be the case.

I tried doing a work around for the monitor flickering issue with overdrive enabled by setting custom clock states in a profile but sadly the memory still idles at 300 regardless of the setting in the profile meaning that I can't overclock with more than 1 display. I am going to use my 20" for the time being ( my 24" has image retention so I'm going to let it take a break for a week or so )

I will do some further testing with furmark with the gpu clock as well and gpu / memory combined. I am going to do the tests at 3 resolutions to give us the full picture as well.

[demowhc]

Interesting, so currently ATI have better anti aliasing scaling but still have inferior AF scaling.

This is true, but you have to take into account that AF quality is superior on the 5800 series aswell..

[gamervivek]

Nice test chickenfeed. SO the 5870 CAN use all the bandwidth you can give it. It's scaling almost linearly.

or the test is simply bandwidth starved.I used to get 100 more fps by overcloking memory on my 3870 in the ati tool window,made no difference in games.

[jaredpace]

or the test is simply bandwidth starved.I used to get 100 more fps by overcloking memory on my 3870 in the ati tool window,made no difference in games.

yeah good point. all we're really looking for is fps increase in games. firing squad already proved that for us.

[lutjens]

What may also be possible is that the card may be intentionally detuned or purposely not optimized. Given that ATI has played it's hand much earlier than NVidia, it wouldn't surprise me if ATI left some performance potential in reserve to counter NVidia. This would give Nvidia the illusion that the 5870 is weaker than it is.

Just a thought...

[Chickenfeed]

I did the same test again but this time I left memory stock and changed the core speed from 600-900 in 50Mhz increments.

Furmark 1.7
1280x1024 8x AA 60000MS
HD 5870 1GB

Core - Score

900 - 2555

850 - 2551

800 - 2539

750 - 2531

700 - 2527

650 - 2524

600 - 2115



What is interesting is that from 650-900 performance barely changes. However at 600 it takes a considerable dive.

Because this is a relatively low resolution I am going to redo do both these tests also at 1680x1050 and 1920x1200 over the weekend. Perhaps there will be more gains on core increase at those resolutions but if 1280x1024 was any indication, I'll assume Furmark likes memory bandwidth over anything.

If I get really bored I might do Vantage ( although it would take a full day to properly do ; I'll probaley just run the GPU tests on Extreme and only look at the GPU score and avgs ) I am trying to pick out some games that are easily benched to do similar tests as well. I'll most likely do Far Cry 2 at the very least as its built in bench tool makes life oh so easy and it is also a recent and stressful enough game to actually show any scaling.

By the way I made the graphs at : http://nces.ed.gov/nceskids/createAgraph/default.aspx. Google searches FTW and LOL at kids zone.

[demowhc]

the 5870 is most definitly bandwidth starved imo, I just dont see the (approx) 153GB/s (?) being enough..

I suspect the refresh, 5890?, will have much faster mem and a slight bump in core speed with a huge performance gain.

I think ATi may have slightly gimped the 5870 on purpose to get more life out of the 5800 series with future revisions..

[Chumbucket843]

i went with texture filtering on high because ATi said that the 5870 is bandwidth limited 50% of the time.

ROUND 2: crysis demo benchmark
0x AA 0x AF
all settings on very high. gtx 260

framerates were MUCH smoother with memory at full speed, something you have to see to believe. a 100% increase in bandwidth gained 30% speed and much smoother fps. this should be interesting.

[Kuntz]

i went with texture filtering on high because ATi said that the 5870 is bandwidth limited 50% of the time.

ROUND 2: crysis demo benchmark
0x AA 0x AF
all settings on very high.
http://img39.imageshack.us/img39/6995/crysisbecnh.jpg
framerates were MUCH smoother with memory at full speed, something you have to see to believe. a 100% increase in bandwidth gained 30% speed and much smoother fps. this should be interesting.

You increased memory speed by 15% (1000 -> 1150) and frame rate only went up 3.4% (29 -> 30). That is not from the increased bandwidth, that is from the decrease in memory latency.

[Chumbucket843]

You increased memory speed by 15% (1000 -> 1500) and frame rate only went up 3.4% (29 -> 30). That is not from the increased bandwidth, that is from the decrease in memory latency.

went from 550mhz to 1150 mhz. i wish my memory went to 1500mhz. the frame rate went from 23.6 to 29.7. gpu's are designed to hide latency. my card can handle 24,576 threads so any time spent accessing memory is covered up by working on other threads. this wasnt meant to be memory intensive at all. nobody buys a high end card and not use any texture filtering. it might be memory size too. double the shaders, double the memory needed for all of those alu's. its kind of like a 4870 512mb.

[Astennu]

I agree with kuntz. Furmark is a nice tool but its not a game test. You dont know what it does and why the memory scales well there. If a card really is starved and you overclock the memory 10% you would see a 9-10% boost in performance. In games thats not the case. It seems to be the case in furmark but like i said its not a game test. That kind of load will not happen in games. Its to fully stress the VGA card to see if its stable. The load si way higher then any game on the market. So yes there are situations where there is not enough bandwith vor the RV870 core. But thats not the reason why is current Game performance is a bit to low for our liking.

went from 550mhz to 1150 mhz. i wish my memory went to 1500mhz. the frame rate went from 23.6 to 29.7. gpu's are designed to hide latency. my card can handle 24,576 threads so any time spent accessing memory is covered up by working on other threads. this wasnt meant to be memory intensive at all. nobody buys a high end card and not use any texture filtering. it might be memory size too. double the shaders, double the memory needed for all of those alu's. its kind of like a 4870 512mb.

You could be right about the memory. The HD4870 gained a lot but the HD4850 did not gain much by going from 512 to 1024mb. I have also seen it in the past with the Radeon 8500. the 128 mb version was a lot faster then the 64 mb version while the clocks where the same.

It could be the same problem here but personally i dont think that is the case. If its the case we could see 10-25% performance gains from the 2 GB version. We will know it when it comes out in 3-4 weeks.

[saaya]

No, I think the memory timings are static, they do not change with memory adjustment. So if they are at, for example, a total cycle delay of 10, that would mean:

10 cycles @ 1200MHz = 8.3ns delay
10 cycles @ 1300MHz = 7.7ns delay

Because there is no memory bottleneck on the 5870 cards, when you increase the memory speed, all you are really doing is decreasing the nanosecond delay on memory operations, which is why a 10% increase in memory speed is only getting a 1.5% increase in performance.

rv600 and rv670 and rv770 all adjusted the timings when you overclocked the memory... id be surprised if rv870 doesnt do it anymore...

a drop from 8.3 to 7.7ns is a latency reduction of 7% though, then how come we still only get a 1.5% boost? and memory timings tend to not really matter on vgas, they need bw bw bw... they usually have cas15 or so...

i dont think the boost we see is only from reduced memory latency, as a matter of fact, i dont think memory latency decreases a lot, since r600 all gpus use a formula to calculate memory timings based on memory clock. the latency still decreases, and bw still increases, otherwise we wouldnt see any gains at all, but its less than we would see from a static timing config with increased memory speed.

you are probably thinking of system memory performing better with lower timings, for vgas that doesnt apply... they do score better with lower timings by the boost is tiny compared to cpu system memory...

nice tests chumbucket and chickenfield (sp?)
what it shows is that in furmark the 5870 def is bw limited, but that doesnt seem to be the case in actual games...
furmark is 99% shader heavy load only isnt it?