-
Strange VR-Zone results
I noticed something odd with the VR-Zone multi GPU results.
Supposedly they show Ivy Bridge wiping the floor with Vishera but there is a very curious thing about them. When comparing these setups at different quality settings the results are consistently 100% in Ivy Bridge favor no matter how GPU high you put the quality settings. This is odd since normally the CPU bottleneck disappears when the GPU setup can't handle running at the framerate the CPU is capable of.
Turning our attention to the games "Sniper Elite" and "Alien Vs. Predator" In the former title we have 173fps vs. 369fps at the lowest settings and 32fps vs 64fps at highest settings. Obviously they should be 100% GPU capped at that point and should be performing equally. The same thing applies to AVP where the difference is equal, i.e. 270fps vs 554fps at lowest settings and 75fps vs 148fps at highest.
The only conclusion I can draw is that VR-Zone were pitching a Vishera with single GPU vs. Ivy Bridge with Dual GPU in at least these two games. The reason could be that the crossfire is somehow disabled in these titles but how can we know if it's working properly in other titles?
I can only shake my head in dismay over such an obvious fail for a seasoned review site 
I also want to point out a lazy habit that most review sites use when calculating overall performance and this is neither describing in detail how these averages are calculated nor weighing the benchmarks appropriately.
I took for example Techreport application results compared FX-8350 and i7-3770 on one hand and FX-8350 and i5-3570 on the other and normalized the results to the slower of the two in each case (otherwise if a processor is 3x faster in one test and 3x slower in another the results would be +200% and -66% or vice versa and the end result will be skewed) and taking average of the difference. I made several logical adjustments:
1. Omitting the single thread Cinebench 11.5 results. They are interesting on their own but nobody uses a multi-threaded program in single-thread mode so they should not be factored into any average.
2. Combining both povRay results in a single score. These tests are one of the strong points for FX-8350 but I feel they should not weigh double just because we get more detailed results from this program.
3. Only using the standard LUXmark results, the GPU results are meaningless after all and the CPU+GPU give a very similar normalized results as the standard test anyway.
4. Combining x264 tests by calculating the time it takes to complete X number of frames with both passes.
The end results are:
i7-3770K is about 9% faster than FX-8350
FX-8350 is about 14% faster than i5-3570K
Compare that to the overall score TR gives (which are normalized to A8-3850) where the difference is roughly 14% (vs. 9%) and 3% (vs. 14%) so incidentally their flat average approach favors Intel and especially the i5-3570. This is also true for other overall scores I've seen around the web. Single Threaded Cinebench results are included, pass 1 & 2 in h264 and x264 are weighted equal despite pass 2 taking significantly more time than pass 1. Sysmark and PCmark are also frequently used in these averages. But these are themselves averages that are not well described so I dislike them for that very same reason.
Last edited by wICE_man; 11-08-2012 at 07:53 AM.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
Bookmarks