Kunaak
01-31-2007, 03:53 AM
today, I was looking into some scores and I noticed a whole lot of cheats and such that I can spot pretty damn easily.
then in other ways... I noticed just a huge flaw in the sisoft sandra part of HWBot.
sisoft sandra changes the way it calculates memory, latency, bandwidth, mips and just about everything else, with every few new releases.
so you can't maintain any sort of "side by side" comparison, when one person is using a version 2 years old, and another person is using a version from 2 days ago, and the older version will give you 20,000 extra mips because it uses a older calculation method.
this makes a very simple and easy cheat, where if you dont like like the score, simply use a older version.
right now, with version 1117b you will get roughly 38,000 mips, with a allendale at 4.1 ghz.
with the old version I deleted about a week ago, I was getting over 45,000 mips.
basically, with such a high degree of inconsistency, it defeats the purpose of even trying to use that benchmark in any valid way, cause it leaves so much open to manipulation.
if this was a solid benchmark, then the scores you see in this pic would be linear...
meaning a 3.4 ghz CPU would not be faster then 4 ghz CPU.
just like in Super Pi Mod 1.5.
assuming people use that version, then the scores mostly will be linear and predictable... minus alittle leway for the various tweaks people use.
but with sisoft, its not tweaks at all...
its simply... you dont like the score you got, download a older version and upload that old version score.
look at the results here from the E6400 and you can clearly see what I mean...
theres no consistency in the results so they are all pretty useless, since they are all being done from different versions.
I think Sisoftsandra should be removed from HWBot, unless there can be a way to verify the version used, and then sort, by version too.
otherwise, theres just no way to get any consistency from it, and makes using it pointless.
then in other ways... I noticed just a huge flaw in the sisoft sandra part of HWBot.
sisoft sandra changes the way it calculates memory, latency, bandwidth, mips and just about everything else, with every few new releases.
so you can't maintain any sort of "side by side" comparison, when one person is using a version 2 years old, and another person is using a version from 2 days ago, and the older version will give you 20,000 extra mips because it uses a older calculation method.
this makes a very simple and easy cheat, where if you dont like like the score, simply use a older version.
right now, with version 1117b you will get roughly 38,000 mips, with a allendale at 4.1 ghz.
with the old version I deleted about a week ago, I was getting over 45,000 mips.
basically, with such a high degree of inconsistency, it defeats the purpose of even trying to use that benchmark in any valid way, cause it leaves so much open to manipulation.
if this was a solid benchmark, then the scores you see in this pic would be linear...
meaning a 3.4 ghz CPU would not be faster then 4 ghz CPU.
just like in Super Pi Mod 1.5.
assuming people use that version, then the scores mostly will be linear and predictable... minus alittle leway for the various tweaks people use.
but with sisoft, its not tweaks at all...
its simply... you dont like the score you got, download a older version and upload that old version score.
look at the results here from the E6400 and you can clearly see what I mean...
theres no consistency in the results so they are all pretty useless, since they are all being done from different versions.
I think Sisoftsandra should be removed from HWBot, unless there can be a way to verify the version used, and then sort, by version too.
otherwise, theres just no way to get any consistency from it, and makes using it pointless.