hwbot v2.0 public testrun

[RichBa5tard]

news release (http://www.hwbot.org/hwbot.post.do?postId=654)



The hwbot crew is almost ready to present it's new baby dubbed 'hwbot v2.0 AWESOM-O'. As there are a huge amount of features and changes, we want to give it a testrun before releasing it to the public.

Everyone is free to try out v2.0 on our testserver to comment on the new features, find bugs and discuss the new hwboints algorithm.

[kiwi]

I like it :) Jumped +32 places to 24th even though I am inactive now :D

[Stapler]

So far it doesn't make a whole lot of sense to me. Way to many points for not good global results, the 178th 1m result does not deserve 10 points. I'm not saying that it's not a good result, but it's 178th on the ranking. On the other hand, 32m doesn't give nearly enough global points. My 1m result which is 128th on HWbot now gives as many global points as my 32m result which is 40th. That doesn't make a whole lot of sense, 32m is definately a lot more work than 1m.

Too many hardware points for popular hardware as well, just look at the 7900gt SLI scores. Every single first place score is over 20 points thanks to the combination of 15 hardware points and quite a few global points for results that aside from AQ3 are well over 100 in global rank (hell 3d06 is nearly 250th global).

HWBot is a competition, and this update just takes away from the competition. Rev. 2 goes to far in giving everybody points for their results, and this ruins the competition. You should have to fight for every last point, not get points just for having a result in the top 200 points. Obviously the people benching high end hardware with ln2 or cascades are going to do better in the competiton, but that's life, I don't expect to do as well as people with better cooling and hardware than I've got.

Btw, I'm point neutral as far as rev. 2 and I actually stand to gain points as I've got a 7900 GT. Though as a whole it does hurt my team, but I think I'd be against it anyway simply because it doesn't seem like as much of a competition.

[RichBa5tard]

So far it doesn't make a whole lot of sense to me. Way to many points for not good global results, the 178th 1m result does not deserve 10 points. I'm not saying that it's not a good result, but it's 178th on the ranking. On the other hand, 32m doesn't give nearly enough global points. My 1m result which is 128th on HWbot now gives as many global points as my 32m result which is 40th. That doesn't make a whole lot of sense, 32m is definately a lot more work than 1m.
You are 128th of the +3200 people in spi 1m or 40th out of 1073 people in spi 32m, it makes a lot of sense you get approximatly awarded equally. The more people that participate, the more you will be awarded for a top 500 spot. 3000 people is the cap atm, it won't increase any further.



Too many hardware points for popular hardware as well, just look at the 7900gt SLI scores. Every single first place score is over 20 points thanks to the combination of 15 hardware points and quite a few global points for results that aside from AQ3 are well over 100 in global rank (hell 3d06 is nearly 250th global).

Are you saying that if you reach #1 out of 200 people with a 7900GT, you don't deserve 15 points? It's insanely hard to reach the top 5 with popular hardware, you should be rewarded appropriately. Personally, I think 15 points is on the low side, not the high.



You should have to fight for every last point, not get points just for having a result in the top 200 points. Obviously the people benching high end hardware with ln2 or cascades are going to do better in the competiton, but that's life, I don't expect to do as well as people with better cooling and hardware than I've got.

A problem with rev1 was that if you have a great overclock with popular hardware, you probably didn't get squat. With rev1 you need to reach 10.82s in spi 1m in order to get rewarded for it. Achieving 11s will get you nothing, while it is a great achievement (you need a C2D at least at 4.5Ghz for that).

Rev2 fixes this. If you manage to get in the top 500 (you will still need to beat a hell of a lot people), you will start to recieve points. You will also recieve more points if you have the fastest hardware of a kind, but you will recieve less points if there are few people with this hardware.

Please take your time to read through the whole algorithm (http://www.hwbot.org/forums/showthread.php?t=523) of rev2. This isn't a one night job.

I hope you agree the fiercer the competition, the better you should get rewarded? Your initial response states otherwhise.

[mukmaster]

I like the new look allot ; much more OSX like !!!

[Stapler]

Are you saying that if you reach #1 out of 200 people with a 7900GT, you don't deserve 15 points? It's insanely hard to reach the top 5 with popular hardware, you should be rewarded appropriately. Personally, I think 15 points is on the low side, not the high.



I think the flaw here is that you're assuming popular hardware scales with the difficulty of attaining a top 5 result from that hardware. With some hardware this is true, but it's not the case all the time. It is definitely extremely difficult with E6600, E6700, X6800 or 8800/2900, because with that hardware you're at the top of the global list as well and thus competing with the best, but it's not true for everything. Take a look at those 7900 GT results, how many of them are over 4ghz for the CPU? Not many right, especially on the multi card side. Seems to me that their isn't a whole lot of competition in that area, even with so many results.




You are 128th of the +3200 people in spi 1m or 40th out of 1073 people in spi 32m, it makes a lot of sense you get approximatly awarded equally. The more people that participate, the more you will be awarded for a top 500 spot. 3000 people is the cap atm, it won't increase any further.



Why not lower the cap for 1m to make it competitive with 32m. Do you really think it's twice as hard to get the 1m world record as opposed to the 32m world record, or how else can you justify 55 points for 32m and 108 points for 1m. They ought to be about equal for total points if anything. 32m is a much more difficult bench then 1m, and only giving half or less as many points for the results is pretty pitiful. Here again, pure numbers don't equate with the difficulty of attaining a result.

Moreover, my e6600 really isn't a good example as it's 1m is held back significantly by an FSB wall. Most chips run 32m around 100-200mhz lower than 1m, so if 4768mhz was really the limit of my chip for 1m, it would be closer to 52nd (supposing the same performance product at 4650mhz).


I guess the issue to me here is to what extent does number of results submitted equate to the difficulty of getting a top five result. My guess is this, the older the hardware the less relevant the number of results. The reason chips like the E6600 are hard to get into the five with is not because of the quantity of the results, but because of the people benching them and the cooling they use. On the other hand, cards like the 7800 gt, 7900 gt, 7900 gtx, etc are treasure troves of points and artificially inflate scores just like grinding did with rev. 1.

[BenchZowner]

Still giving the hardware junkies a big advantage over 'real' overclockers ( hope you understand what I mean by that ;) )

[RichBa5tard]

hwbot v2.0 is released (http://www.hwbot.org/hwbot.post.do?postId=664)

[massman]

I think the flaw here is that you're assuming popular hardware scales with the difficulty of attaining a top 5 result from that hardware. With some hardware this is true, but it's not the case all the time. It is definitely extremely difficult with E6600, E6700, X6800 or 8800/2900, because with that hardware you're at the top of the global list as well and thus competing with the best, but it's not true for everything. Take a look at those 7900 GT results, how many of them are over 4ghz for the CPU? Not many right, especially on the multi card side. Seems to me that their isn't a whole lot of competition in that area, even with so many results.


A bencher will lose his global points if he doesn't keep up with the latest hardware. Someone who benches a x6800 under cascade today will have lots of points, but eventually, he'll lose those too. In the end, the only points left for him to catch are the hardware points.

You can't work around the latest hardware, it's just out there. We don't believe we have to 'punish' people who bench with the latest hardware, just because it is/seems 'easy' to grab points.



Why not lower the cap for 1m to make it competitive with 32m. Do you really think it's twice as hard to get the 1m world record as opposed to the 32m world record, or how else can you justify 55 points for 32m and 108 points for 1m. They ought to be about equal for total points if anything. 32m is a much more difficult bench then 1m, and only giving half or less as many points for the results is pretty pitiful. Here again, pure numbers don't equate with the difficulty of attaining a result.


The 1M bench is more popular, so being n°1 has to be rewarded more. Sometimes it's harder to complete a decent 1M bench than a 32M, sometimes it's the other way. Rating effort and difficulty is hard and will, imho, never be 100% succeeded. I believe both Mtzki and RB have done a great job improving the hwboints formula to get the rewarding side of the leagues more equal.

Lowering the cap doesn't solve the problem, it just places the problem into the future. If the 32M gets more results, the points will raise too.



I guess the issue to me here is to what extent does number of results submitted equate to the difficulty of getting a top five result. My guess is this, the older the hardware the less relevant the number of results. The reason chips like the E6600 are hard to get into the five with is not because of the quantity of the results, but because of the people benching them and the cooling they use. On the other hand, cards like the 7800 gt, 7900 gt, 7900 gtx, etc are treasure troves of points and artificially inflate scores just like grinding did with rev. 1.

You're missing the point, I think. The main reason why it's difficult to reach high ranks in categories of newer hardware is the cooling issue and the massive number of results. Mainly, the top places are ln² cooled results, but in fact, they do deserve those points as they have put a lot of effort into it. Ln² is a hard way of benching, it's not just container + liquid = result. The massive points top results get is because of the amount of results.

You're refering to the 7800GTX and 7900GTX because of the conroe domination? It's true, with a conroe it's easier to get higher points when benching in such a category, because at the time of the release the cards were still bottlenecked by the cpu's at that particular moment. That will, sadly or not, always be the story: older vga's with newer cpu's will boost the scores. That's the same for the FM orb or any other 3D benchmark database, not some flaw of the hwboints ;).

[massman]

Still giving the hardware junkies a big advantage over 'real' overclockers ( hope you understand what I mean by that ;) )

You got to admit: if someone puts effort into benching up to 300 hardware points, he deservers to have those points ;).

It's, by the way, not even possible to reach top50 without global points :)

[Stapler]

A bencher will lose his global points if he doesn't keep up with the latest hardware. Someone who benches a x6800 under cascade today will have lots of points, but eventually, he'll lose those too. In the end, the only points left for him to catch are the hardware points.

You can't work around the latest hardware, it's just out there. We don't believe we have to 'punish' people who bench with the latest hardware, just because it is/seems 'easy' to grab points.




I'm not really sure what you're trying to say here, but you should have to keep up with the latest hardware in order to be competitive. I know people can say that this is unfair for those of us (myself included) who can't afford the latest and greatest. The way the current system works, you can just rely on old hardware to give you plenty of points, and as CPUs advance, in categories like 7900 gt, you're no longer competing against top benchers with that hardware, because they've moved on to 8800/2900. This isn't really an issue with processors, because a year from now it's still going to next to impossible to beat Misko's e6600 32m, but with 7900gt records going to average benchers, you don't have to be an especially good bencher to take those records. Now if the 7900gt were as popular as it is, and someone like Joe_Cool held the records, it would be extremely difficult to take them back.

The issue here, that you seem to be completely ignoring, is that popularity has no correlation to the difficulty of getting a top spot, and yet you're whole system is based on that very premise. Just look at the Xeon 3070 1m result and compare it to the results for the 7900gt, you get hardly any hardware points for getting that 3070 result, but I can gaurantee it took more effort than just about every 7900 gt result, which each get 15 hardware points. I'm comparing global points, because they have no place in this comparison, as far as global points the difference is fine. Seems to me that difficulty has no correlation with popularity.



The 1M bench is more popular, so being n°1 has to be rewarded more. Sometimes it's harder to complete a decent 1M bench than a 32M, sometimes it's the other way. Rating effort and difficulty is hard and will, imho, never be 100% succeeded. I believe both Mtzki and RB have done a great job improving the hwboints formula to get the rewarding side of the leagues more equal.

Lowering the cap doesn't solve the problem, it just places the problem into the future. If the 32M gets more results, the points will raise too.


Again, popularity has nothing to do with the difficulty of getting a top spot. I 'm going to out on a limb here and say that 1m has more results simply because it takes less time to run and people are lazy. Getting a top 1m takes just as much effort as a top 32m, but for most people it's easier to get a pretty good 1m than a pretty food 32m. I didn't have much time to tweak the 32m, so it's a really slow time, but the 1m is pretty fast, and all that took was running it over and over until I got a result that wasn't bugged low.



You're missing the point, I think. The main reason why it's difficult to reach high ranks in categories of newer hardware is the cooling issue and the massive number of results. Mainly, the top places are ln² cooled results, but in fact, they do deserve those points as they have put a lot of effort into it. Ln² is a hard way of benching, it's not just container + liquid = result. The massive points top results get is because of the amount of results.


I never said it was easy to bench with LN2, but pretty much all top results are done with LN2. In fact, I'm pretty sure I said it was difficult because of the cooling required (in other words it's hard to bench with LN2), but I would contend that it has nothing to do with the number of people benching the hardware, but the people benching the hardware. If you had tons of average benchers using LN2 and e6600, I doubt they would be able to take down Newbeatle's world record, but if you had just one other pi bencher in that league there would be some serious competition. You're approach to scoring misses an essential and intangible factor, the bencher's skill. Obviously you can't take into account a bencher's skill and once you start talking about global top twenty results it does become self evident, but just because you can't take it into account does not mean you should try to add an arbitrary measure of difficulty (popularity), which has no correlation with the difficulty of the achievement.

Do you really think the majority of the top 7900gt results deserve the same number of points as the top e6600 results?



You're refering to the 7800GTX and 7900GTX because of the conroe domination? It's true, with a conroe it's easier to get higher points when benching in such a category, because at the time of the release the cards were still bottlenecked by the cpu's at that particular moment. That will, sadly or not, always be the story: older vga's with newer cpu's will boost the scores. That's the same for the FM orb or any other 3D benchmark database, not some flaw of the hwboints .


This isn't a database issue, hwbot became much more than a database the moment you started scoring people's results.

[Bigchrome]

I agree that it is indeed annoying when a result you tried so hard for is outscored by some noob who threw his popular card into the brand new expensive setup his mommy paid for...BUT, I do like the current scoring more than I used...I actually have a decent amount of points now.