Wonder what showstopper bugs could creep in this late in the game, especially when it comes to consistency. I'm assuming the benchmarks have been run tens of thousands of times internally so inconsistent scoring is a strange thing to find at this point. Oh well.
Yeah, it is not unlikely that they haven't released the newest version of their useless bling bling benchmark because a client paid enough to make further adjustments.Originally Posted by Borden
IHVs directly pay Futuremark for the service to "validate and optimize their software to the platform". Too bad that most people don't comprehend that Futuremark is significantly hurting the industry with that and they then even actively hype this miserable product that completely fails to mimic the workload of real (future) games.
And before some bling bling Benchmark fans go for my throat again, look - they even say this openly:http://www.futuremark.com/bdp/3dmark/BDP Members get the following privileges and benefits
* Ability to influence Futuremark’s 3DMark roadmap and implementation
* Access to specification documents, alphas, betas and release candidate code drops of benchmarks
* Frequent communication and briefing
* Scheduled conference calls
* Troubleshooting for drivers and new technology
...
So yeah, if you are smart enough to comprehend the issue: don't support them by using their useless benchmark and simply use real games instead.
Last edited by Katzenschleuder; 11-29-2010 at 02:49 PM.
Yeah but when hardware vendors sponsor games they do exactly the same thing in fact taking nvidia as a example they sometimes have more influence. AMD drivers aren't going to make much of a difference between the 580 and the 6870 anyway.
If AMD had enough influence to cause the launch to be in November for driver reasons then they'd have enough influence to force the launch to be delayed due to Cayman.
3Dmark.com = ORB (the new one)
While I cannot go to exact details or give examples, you couldn't be more wrong.Yeah, it is not unlikely that they haven't released the newest version of their useless bling bling benchmark because a client paid enough to make further adjustments.
IHVs directly pay Futuremark for the service to "validate and optimize their software to the platform". Too bad that most people don't comprehend that Futuremark is significantly hurting the industry with that and they then even actively hype this miserable product that completely fails to mimic the workload of real (future) games.
And before some bling bling Benchmark fans go for my throat again, look - they even say this openly:http://www.futuremark.com/bdp/3dmark/
So yeah, if you are smart enough to comprehend the issue: don't support them by using their useless benchmark and simply use real games instead.
The goal is to have a stable and fair benchmark. Has always been. BDP helps to ensure that - you have to understand that all major vendors are in BDP. They also watch very carefully what each other does (duh!) and get early builds for testing and comments - and trust me, they do care. They also bring a lot more eyeballs for verifying and testing the code. Who do you think is best at spotting stuff that could be deemed "unfair" to vendor X than, well, vendor X themselves?
And yes, BDP has been instrumental in helping to get that final polish done.
Wasn't really showstoppers, crashes or anything like that - just minor nagging things that presented us with an option - launch on time with known issues that would need a patch quickly or take a few extra days and fix them first. Since benchmarks are trickier to update after launch (especially if the issue affects score in any way), option 2 was, in our opinion, a better idea.
Any one tryed this yet with it coming out today!
From what i know isn't coming today, is delayed.
i5 2500K@ 4.5Ghz
Asrock P67 PRO3
P55 PRO & i5 750
http://valid.canardpc.com/show_oc.php?id=966385
239 BCKL validation on cold air
http://valid.canardpc.com/show_oc.php?id=966536
Almost 5hgz , air.
So you are saying yourself that the IHV not paying you will have a disadvantage. That you call this a process of "transparency and fairness", is just laughable!
To give you an example what your own customers have to say about your business model:http://www.opengl.org/discussion_boa...287#Post167287
So, in the end the IHVs pay Futuremark for useless benchmarks directly and they have to waste additional resources with costly API call wrappers.
You say that 3DMark isn't useless? The point of 3DMark Vantage has been to predict performance of Direct3D 10 games, right? Well look for yourself:Computerbase.de
6 out of 12 misplaced performance predictions - a perfect random mean! But what else can you expect from a benchmark that creates scenes that are totally atypical for games?!
So people: Don't support Futuremark by using or even buying their benchmarks. You are hurting the game and graphics hardware industry by doing that!
Last edited by Katzenschleuder; 11-30-2010 at 05:02 AM.
Tell me first which game is perfect ? All games have bugs, some games prefer red teams cards, some prefer the green teams cards... I don't like the way the game industry is heading with so many console games being recoded for PC use... makes them buggy, choppy and so annoying for keyboard inputs...
So what do you suggest then ? Can you create the ideal benchmark to predict similar gameplay ? It's impossible, you can just stress on new technology, it's new features..., every game code got it's own quirks, bugs, etc... If a benchmark doesn't favour their brand, then they will call it lame, if the other competitor performs better : they will accuse them of using optimised drivers etc... It's always something...
The game industry it's hurting itself by using sortlike apps as Steam. Not allowing to create dedicated servers for online/LAN party wars by users, bad patch behaviour,... etc, etc,... Not by Futuremark benchmarks... don't be silly... I don't even see how futuremark could hurt the graphics industry...
Mostly the GPU's that perform well in Vantage, will do so in games... Get your facts straight bud !
Last edited by Leeghoofd; 11-30-2010 at 04:36 AM.
Question : Why do some overclockers switch into d*ckmode when money is involved
Remark : They call me Pro AsusSaaya yupp, I agree
Your numbers suggest that the Xtreme preset is quite an accurate measurement, apart from the 4870X2 which is greatly underestimated. The other cards are in the right order and even the ratio is pretty much accurate.
Where courage, motivation and ignorance meet, a persistent idiot awakens.
The quality of your will to read and consider stated arguments is direct proportional to your use of sane punctuation.
Last edited by Katzenschleuder; 11-30-2010 at 05:06 AM.
I have accidentally posted a unreleated benchmark that includes many D3D9 titles, which I fixed.
Can you also fix the chart so it compares similar resolutions? And as for the Vantage numbers: are these overall scores or GPU scores?
//edit: I saw on your site that you actually HAVE numbers with similar resolutions. Why not show that one? Why use a different graph? ...
Where courage, motivation and ignorance meet, a persistent idiot awakens.
The point is to have a GPU-limited scenario like in 3DMark and it doesn't make much difference anyway.
That's why I asked: "Are those points overall score or GPU score?"
And, yes, it actually makes a difference. At equal resolutions, the order and ratio are again quite accurate (slightly in favor of Nvidia cards) if you bear in mind that Vantage was released back in Apr'08 and has to measure the performance of a Nov'10 product.
I sincerly hope you don't spread this hate anywhere else.
Last edited by massman; 11-30-2010 at 05:49 AM.
Where courage, motivation and ignorance meet, a persistent idiot awakens.
I replied to your statement. What is the ideal benchmark to you ? Can it be done ? Don't think so... Take eg the Hawx 2 benchmark : performs brilliant on Nvidia and AMD performance is sub par... If that benchmark was taken as a reference I would partly agree with you...
The new numbers you posted show in the top that the 5970 dual GPU is maybe not correctly supported by the driver, the game doesn't benefit from it... the lower end cards, are hit and miss : maybe not enough ram on the GTX 460 (768 version) , drivers not optimised etc... I don't see that much discrepancies...
Take into consideration that Vantage is aging. Drivers, games & hardware have been optimised,... I just want to state that creating a benchmark ( maybe sometimes even before a decent game is released ( supporting the new technology ) is never gonna be ideal...
Are you saying FM is biasing their benchmark in favour of one team ? Being it the one that pays the most :p Then the beleive in the benchmark and company would go rapidly downhill... don't think they can make that man...
Last edited by Leeghoofd; 11-30-2010 at 06:17 AM.
Question : Why do some overclockers switch into d*ckmode when money is involved
Remark : They call me Pro Asus
What the heck?! Most GPUs are still rated incorrectly: 5870/470; 6850/460; 5830/465; 4870/5770; 260/5770; 450/5750; 4870/260 ... They are all rated incorrectly.
The whole point here is that this is the only pretended intent of 3DMark!if you bear in mind that Vantage was released back in Apr'08 and has to measure the performance of a Nov'10 product.
I am sure that you are going to continue to be happy to support Futuremark with their business model that is evidently hurting the industry.I sincerly hope you don't spread this hate anywhere else.
I never said that it is realistic to think that one can create a benchmark that mimics the workload of upcoming games, early in the GPU generation.
The whole point here is that Futuremark promises to deliver this and fails for good reason. Have you read the inofficial statement made by Nvidia which also makes this clear?
The even worse issue is that the business model of Futuremark makes their benchmark content directly dependent on the money paid by particular industry vendors, which is simply outrageous!
No, it's hurting itself by not taking the PC seriously enough and positioning it as a mainstream rival to PS3/XBOX360
Intel i7 920 C0 @ 3.67GHz
ASUS 6T Deluxe
Powercolor 7970 @ 1050/1475
12GB GSkill Ripjaws
Antec 850W TruePower Quattro
50" Full HD PDP
Red Cosmos 1000
That statement Of Nvidia was that the one regarding 3Dmark2003 ? Think if the chosen option was in favour of Nvidia they wouldn't have nagged at all... Think Nvidia is one company that tends to opens a lot of cans of worms with bad expressions... There are even special created drivers by this company that allow "bugged" runs on FM benchmarks, so I wouldn't take it all too seriously... Seems you are not a bencher : we all have drivers ( beta drivers) that create "bugged" runs in sub tests of some Futuremark benchmarks and this results finally in a higher output score... in reply to the statement by the above mentioned nvidia employee, they themselves created the driver to "manipulate" the outcome...
Very good point there... hence why I don't like console converted games... usually need silly hardware requirements for choppy gameplay...
Last edited by Leeghoofd; 11-30-2010 at 06:48 AM.
Question : Why do some overclockers switch into d*ckmode when money is involved
Remark : They call me Pro Asus
Why in hell would that nonsense make a difference to the problem that Futuremark practices an outrageous business model?
What the F are you talking about? ALL IHVs have to waste driver development resources for 3DMark API call wrappers.There are even special created drivers by this company that allow "bugged" runs on FM benchmarks, so I wouldn't take it all too seriously...
No, they are rated like you'd expect. The combinations you mention are based on price/performance ratios, which is very sensitive to very small differences in both FPS and price swings.
The point of running 3DMark is that you have 1 number than estimates the rough positioning in the market. Eg: 5970 > GTX580, 480 > 5870 or 4890 >= 5750.
In a very recent interview, Futuremark has explicitly stated that end-users should run as many benchmarks as possible to get a good view on the factual performance of your product. That's quite different from your idea, which is apparently based on an interview from 2003.
Link to interview: http://hwbot.org/article/newsflash/9...mark_president
Evidently?
So far I've seen an outdated quote and a couple of charts that show 3DMark is fairly accurate in grouping similar performant products.
Last edited by massman; 11-30-2010 at 07:03 AM.
Where courage, motivation and ignorance meet, a persistent idiot awakens.
WTF?! The majority of the GPUs tested have an incorrect performance rating succession when comparing 3DMark with real games! All those combinations that I have listed are rated incorrectly by 3DMark.
Why are you making up the claim that this "based on price/performance ratios", when it clearly is not?
That even they are saying that 3DMark is not a good representation of game performance is in which way speaking for them?In a very recent interview, Futuremark has explicitly stated that end-users should run as many benchmarks as possible to get a good view on the factual performance of your product.
Outdated? They continue with exactly the same business model that has been critisized by IHVs, which is even openly advertised by Futuremark today.That's quite different from your idea, which is apparently based on an interview from 2003.
So far I've seen an outdated quote
Do you have to punch your head against the wall to make yourself believe that obvious untruth or do you just have to turn a switch for that?and a couple of charts that show 3DMark is fairly accurate in grouping similar performant products.
I don't know where to even start, so I'll just state that FM stands behind it's products and states that they produce usable results within the margin of error that is reasonable to expect from a single benchmark.
Will you get more accurate results if you mix multiple benchmarks and games when compared to a single benchmark (even one that has multiple workloads)? Sure. That's simple math.
Does that invalidate any individual benchmark? Nope - a single benchmark has higher error margin than combined score from multiple benchmarks. Duh. That doesn't mean the result is invalid.
Are the differences between 3DMark and a pool of games material? No - they are well within the error margin, especially when you consider that many games run better on cards from specific vendor (accidentally or on purpose), creating bias. Spotty multi-GPU support also creates bias (while 3DMark is always offering full multi-GPU support). You also have to consider that benchmarking a pile of games and creating an average out of that is obviously (slightly) more accurate than any single result (from a benchmark or a game).
3DMark Vantage might not be the best or the prettiest product we've ever done, but one thing it does do is to produce a valid, unbiased score and seems to be doing it even with hardware that was years away from being released when it was shipped. With 3DMark 11, we have the exact same goal.
As for the business model side - I'm not the person to discuss about it. If you want to continue that discussion, I suggest you contact us at bdp [at] futuremark.com - I'm sure our guys are happy to do an interview or provide more details.
Last edited by FM_Jarnis; 11-30-2010 at 07:44 AM.
Posting Permissions
Reply With Quote
Bookmarks