Sisoft Sandra Really Should Not Be Allowed as a HWBot Program.

[Kunaak]

today, I was looking into some scores and I noticed a whole lot of cheats and such that I can spot pretty damn easily.
then in other ways... I noticed just a huge flaw in the sisoft sandra part of HWBot.

sisoft sandra changes the way it calculates memory, latency, bandwidth, mips and just about everything else, with every few new releases.

so you can't maintain any sort of "side by side" comparison, when one person is using a version 2 years old, and another person is using a version from 2 days ago, and the older version will give you 20,000 extra mips because it uses a older calculation method.

this makes a very simple and easy cheat, where if you dont like like the score, simply use a older version.

right now, with version 1117b you will get roughly 38,000 mips, with a allendale at 4.1 ghz.
with the old version I deleted about a week ago, I was getting over 45,000 mips.

basically, with such a high degree of inconsistency, it defeats the purpose of even trying to use that benchmark in any valid way, cause it leaves so much open to manipulation.

if this was a solid benchmark, then the scores you see in this pic would be linear...
meaning a 3.4 ghz CPU would not be faster then 4 ghz CPU.

just like in Super Pi Mod 1.5.
assuming people use that version, then the scores mostly will be linear and predictable... minus alittle leway for the various tweaks people use.

but with sisoft, its not tweaks at all...
its simply... you dont like the score you got, download a older version and upload that old version score.

look at the results here from the E6400 and you can clearly see what I mean...

theres no consistency in the results so they are all pretty useless, since they are all being done from different versions.

I think Sisoftsandra should be removed from HWBot, unless there can be a way to verify the version used, and then sort, by version too.
otherwise, theres just no way to get any consistency from it, and makes using it pointless.

[RichBa5tard]

All those reasons are exactly why no points are awarded for sisoft sandra. You can submit scores, but it won't help your team except for the sisoft sandra team average ranking.

We couldn't delete sisoft from our suite, because many teams still want to use it on their forum.

[Kunaak]

what about the PIfast scores then?

cause they have the same issue.
theres so many versions around, yet they all look alike... but none give the same score.
some are as much as 5-10 seconds faster...

I've always used the Hexus Pifast.
which is from version 4.1 pi fast...

but alot of the scores I am seeing arent even remotely comparable, by much slower machines, yet still beating most peoples PC.

[RichBa5tard]

The pifast info page and the info in the pifast rankings state to use hexus pifast, and not the latest (4.3?) version.

Many results have been blocked in the past because they didn't use hexus pifast.

We can't help it that some popular benchapps are easy to abuse... but if new ones are added to the suite we now have a set of rules to which it has to apply (one being the outcome of the benchmark should stay comparable over different versions).

[Kunaak]

http://pifast.hexus.net/pifast.php

the version there, is version 4.1

is there a different version somewhere else?

[RichBa5tard]

Yes, hexus pifast is just the 4.1 release of ... pifast. The latest one is 4.3, but results are, like you said, not comparable.

http://numbers.computation.free.fr/C...am/pifast.html

[Kunaak]

so whats the valid version?

cause going from 4.1 to 4.3 theres exactly 5 seconds difference in the scores, even though they were run on the same machine, back to back.

version 4.3 doesn't appear to be whats being used in the database.
from what I see, version 4.1 appears to be the standard accepted version.

[RichBa5tard]

Hexus (4.1), like the info on hwbot and pifast ranking threads state.