Nvidia cynically fiddled video benchmarks [INQ]

[Lightman]

Well this title says it all...

Really nice and honest article by Charlie D.

The problem is simple, Nvidia can't do video decode well, and the current media darling benchmark, HQV - especially in the HD version of the benchmark - is eating its hardware alive. On quality, Nvidia regularly loses about 25 per cent of its score in a noise reduction test - enough, to more or less take it out of that game.

So, what do you do when you are blown out of the water? Cheat with a plausible workaround, and then spin your ass off hoping not to get caught. Well, Nvidia seems to have done both.

These pics were done on identical Dell 30-inch monitors, the ATI 2600XT based box is on the left, the NV 8600GTS is on the right. Throughout the tests, the NV machine had lighter images, a benefit in some places, a downer in others. This is not a good or bad thing, it is just a difference between the cards. This picture was taken with the test paused to show details, what you are seeing is a static image, not motion effects.

The end result is that TNR makes a single test much better and does not affect the others. You would think this qualifies as a job well done, and if the world was limited to watching HQV loop, you would be right. I don't know about you, but I watch other videos on my computer too, and therein lies the rub.

What Nvidia did with TNR washes out video and notably degrades the quality. Blacks are no longer black during transitions, things blur and lose sharpness. In general, it is a mess. It does reduce noise in those videos though, and if you have a noisy high-def video setup, this is the driver release for you. This all comes at a price, the drivers are broken elsewhere, and they are broken to game a benchmark. Reviewers were not told this, and dutifully reported that the video problems were fixed.

When asked about it, NV will probably respond that it is under user control, and you can set it however you want. That is 100 per cent true, but I don't know about you, I want my video devices to work, not need adjustment every scene change. I want my video driver to reduce noise and not make things look like Micheal Jackson's privates, a manual slider change during every scene change is unacceptable. It is under your control, but nevertheless it is still broken and ineffective.

The cynical gaming was that it was done in a beta release meant for the members of the press, and handed out to them. The next version, 163.44, vastly lowered the default value, and presumably it will vanish under the waves with successive releases until a new card comes out, rinse and repeat. If I ever get an NV card, I will keep an eye on this.

And a lot more in original article here

[Zucker2k]

I guess this is another reason to trade in my 8800 ultra for a HD2900XT which imho is only going to get better with driver releases.

[Faktor]

I guess this is another reason to trade in my 8800 ultra for a HD2900XT which imho is only going to get better with driver releases.

[hollo]

g80 for games, r600 for htpc :D?

[Caparroz]

g80 for games, RV615 for htpc ?

Fixed.

Who wants a noisy power hog that can cook your diner and doesn't have UVD in a HTPC?

[Sneil]

Jake the peg!? wth, does he really have three legs? ....

[Cybercat]

I hate Charlie Demerjian. All he does is complain one-sidedly about crap he doesn't really have a clue about. He's the typical anti-big corporation, sensationalist windbag you'd expect from a worst-of-breed tabloid site.

I like how he says NVIDIA plain can't do video decoding well, with no technical explanation or backing as to why he thinks that (is it a hardware issue, Charles? forget it, you don't know). Then he proceeds to admit in not so many words that 163.44 pretty much fixes the issue.

I once sent him a message about the article he did on avoiding Bioshock at all costs, stating that 2K answered the criticisms immediately, better than most companies, and even promised to remove the protection. He replied that it wasn't good enough. So let me get this straight, Charlie, they can fix the very thing you're complaining about but you're not satisfied?

Goes to show what an excretory opening he is.

/rant

[MuffinFlavored]

I was skeptical about TheINQ, but this just cut the grass.