I could go on and on about memory use reporting in Afterburner, Precision and whatnot.

They are all accurate to a certain extent since they are actively reporting the amount of memory being allocated for usage by a given application. However, many times it is the game which is the issue.

Back then, I gave Shogun 2: Total War as an example since many were reporting that it was eating up a massive amount of memory. And, it was. However, it didn't matter at the time whether you had a 1GB card or one with 4GB; memory was always a bottleneck.

The reason for this was pretty convoluted. The game wasn't "flushing" texture information from the buffer which resulted in an exponential memory footprint growth as you played the game. For example, when you started the game, the front screen options popped up with a rendered in-game sequence playing behind. At high settings the memory footprint here was ~550MB if memory serves me correct. However, starting the actual game (campaign map) resulted in that 550MB remaining rather than being flushed and then the 250MB of campaign map information was layered on top of that. If you started a battle, that 750MB to 1GB of data was added as well for a total of ~1.5GB of memory usage or more.

Using higher settings made this issue even worse.

As a result of this compounding problem, the game's forums were filled with people saying their high end graphics cards weren't enough to handle Shogun 2. The only way around this was to save the game mid-battle, restart the application and directly load your save again, skipping the campaign map.

However, the real reason wasn't memory use but rather a lack of proper coding on Creative Assembly's part. I've seen the same issue present itself in the initial releases of Assassin's Creed III and COD: Black Ops. The only reason this didn't get noticed is these games' small initially memory footprint.

Developer optimizations play a huge part as well. If a developer doesn't implement the necessary memory optimizations, even the highest-end graphics card will struggle with bandwidth constraints.