Just saw your edit...Originally Posted by gosh
EDIT: Checked two tests on Lost planet
Here is one: http://techreport.com/articles.x/14424/4
Comparing QX9650 and QX6850, isn't the main difference cache size? That would mean that Lost Planet likes a big cache (is there other differences between these two processors?)
I don't understand your point here nor what it has to do with your assertion that we are working on ... of course a larger cache is going to yield better overall, because Cave in Lost Planet is CPU bound, thus it responds to CPU performance. IPC goes up with cache size (as explained above), and because the CPU is the throttle in Lost Planet for this particular scene, you would expect (and you in fact point out as an observation), that it should show a response. I have always had a bad taste in my mouth when I see someone post "games love cache" ... anything loves cache, cache will improve just about anything, some more than others, some hardly noticeable... but larger cache usually means good (until it gets so large, latency negates the advantage).
The difference in the QX9650 and a QX6850 is cache size and a few other architectural tweaks, but the influence here is completely based on the difference in cache most likely. I can link several citations for you that show how cache improves performance. As explained above, and as you obviously agreed, the miss rate in cache varies with applications and the functionality of the cache size to miss rate goes as a power of the size. For example, miss rate = k*X^(-power) where k is a proportionality constant, X is the cache size, and power is a number (typically between 1 and 1.8, cannot remember I can dig up the reference) ... EDIT: though the rule of thumb has always been 1/sqrt(size), but this is not really accurate today. It depends on the associtivity, whether it is exclusive or inclusive, yada yada. and does not always track a sqrt(2) proportionality. Just wanted to make that clear. (I think it is this one http://portal.acm.org/citation.cfm?id=327132 but am not logged into my server at work to view it).
Now, using your same link as a reference ...
I have shown (see prior posts) and I have argued vehemently that Snow is GPU bound. This data also shows that same trend. Look at how interesting this is... for changes that address CPU performance there is a FPS response in Cave, but not for Snow .... this is what makes this particular game so good to use to study difference between CPUs and GPUs, I have scripts within two different regions of the game that stresses the two computational sources in the same benchmark run.
Your edit more or less helps make my point.
A side point here is to take a careful take home message. Each script is only about 2-3 minutes of game play in two different levels/locations within the game, in one case I would see CPU's all look the same even at 1152x864 (which is what Techreport used) -- i.e. GPU limited -- yet if I concluded based on Cave then I would be able to make a statement between the relative performance of the two CPUs in question. In other words -- if I only took the Snow data, I would make a false conclusion of the relative performance of various CPUs -- I would think they all are basically the same; however, Cave shows this is not true ... I would have fooled myself.
Games and benchmarks can be selectively created to force a situation if one does not pay attention, questioning what you read is a good habit.
Ok -- so how do I evaulate the game/code/BW/etc concerns of a CPU to make an honest assessment? Well ... take the GPU completely out of the equation. Doing so allows me to see how the two CPUs behave based on the way they run the code necessary to make the game work. Is this realistic ... nope, not really ... but I do want to know this information such that, for example, if I want to upgrade to a faster video card in the future... I don't need to re-address my CPU choice, I will now what performance to expect. Ideally, a good review for a CPU will show both the ultra high res stuff -- like they way we want to play the game as well as the ultra low res stuff so I can 'future' proof my rig.
EDIT2: I am sorry, last edit and one last point. I do not have a test from my experiments that mimics the QX9650 runs that Scott did in his review, so I cannot compare directly. Nor do I mimic his graphics settings ... (if you read the link in pages earlier to my LP article, I document the exact settings). With this in mind, I am matched at stock condition for the Phenom 9850 and I did collect a screen shot many moons ago for that article I linked above, and so here is my 1152x864 run:
As I said, the Graphics settings are not the same -- so Snow will be different - again, because Snow is GPU bound. However, Scott (the TR link you provided) gets around 79.2 FPS, I get 77.7 for the Cave segment ... that is within 1.5 FPS (1.9% difference)... this is what I would consider, within the margin of error ... the same. The point is ... I can reproduce the Tech report values for both the Phenom series and the QX9650 as well as the QX6850 .. though I have not done them all, I did do a few spot checks informally (no screen dumps, just checks to see if I could match that data) which was just a few tests and his numbers are spot on to what I can get with the same HW -- I have 99% confidence that, as he reported his settings, that his numbers are accurate.
What I am demonstrating and the point I want to make with this is that unless you can match configurations, ensure everything is the same, and account for the details ... then a direct compare is not possible .... THIS is why I call your GRID data bunk. I cannot reproduce it. No matter how hard I try there is no information in that thread that allows me to take the same HW, set up the same settings, and reproduce the 'hodge-podge' of results. As such, this invalidates the data set. This is also why my posts get so long, especially the lost planet experiment I produced for you ... I want to ensure that there is more than enough detail such that anyone who wants to and had the capability can reproduce my result... this provides credibility to the data by that fact alone. I was the same level of detail and verbosity in the GRID experiment, and so yes I expect you should believe my numbers over the other guys.
Without reproducibility, there is no truth.
Jack
Bookmarks