The funny thing is that a lot of HPC workloads are bandwidth limited and can't even make use of all the flops because they can't get data to the cores fast enough. That's why caching and the use of shared memory is so important. For example, a lot of compute workloads just don't play nice with HD4xxx cards because the LDS there doesn't really function like it should. So people resort to a lot of other trickery like using the texture units instead to pump data into the cores but that's obviously not a scalable approach. Things should be much better with HD5xxx but I haven't seen independent confirmation yet.