hwboints rev2

[mtzki]

I'm afraid the upcoming revision might hurt XS a bit in the overall competition.

Everyone's boints should increase in the change. And the new definition rewards for trying much better. If you had hard time getting any boints before, there might a nice surprise coming.

Details here: http://www.hwbot.org/forums/viewtopic.php?id=547


Main changes:

* Increasing the position ranges for awarding boints, especially by introducing the 'offsets' to give boints to all the user's best scores in every global and hw ranking. Users are guaranteed to get at least goffset=1 boint for every bm (if they have a submission that is). This will make the system much more rewarding and motivating especially for new users.

* Team overall score computed by summing the overall points for only the top n benchers of the team. Removes the big team bias (which would be much worse after the above change), makes the needed team size much more manageable and should result in more interesting team competition in the long run.

* Steeper global boints curve between #6 to #20 and #20 to #100. The current system doesn't discriminate these positions enough. E.g. from 6th to 20th position boints drop only by 2.3. In the new system this drop would be 13.2.


Questions:

1) The current ratio between the level of hw boints vs. global boints is 1/6. Should this be higher or lower in your opinion? Please note that there is only one global ranking per benchmark whereas every benchmark has many hw categories.

2) The new system computes the overall score for a team by adding the overall boints for the top n benchers of the team (or less if the team has smaller size than n). So what should the value of n be? This first proposition has n=40.

Rev2 is only a proposition at this point. The new system won't take place before richba5tard comes back from his tour of Australia. And there's still plenty of time to share your opinions which may well count for this.

[mtzki]

The new system won't change much the relative boints levels between benchers from team to another. So we can predict the effect of the team overall competition change by computing the sums of top40s with the current boints. So the actual boint levels in the new system would be quite different, but we get a good idea for the relative boints from team to team. Yesterday 12.2. the n=40 rule would have given:

XS: 5079.6
OCX: 4481.4

And other teams far behind and with very little change to their boints. The actual situation (summing boints for all members was) XS 6.2k, OCX 4.8k.

But if we apply the new offsets and still stay with the current team overall competition definition, the situation would be very different. Consider a team that has 1000 benchers who all send one result (any!) to each hwbot bm. Maybe they don't have SLI/CF, so let's assume they get a result to the 12 different categories that give boints atm (there are 17 with SLI/CF).
-> that's at least

1000*12*(goffset + hwoffset) = 1000 * 12 * 1.1 = 13200

boints to start with regardless of the level of the results.

So the change in team competition
1) isn't that big compared with the current situation,
2) is really a necessity together with all the other changes.

[mtzki]

Silence is good or bad??

[cadaveca]

hard to make accurate judgement when the points have not been given out yet. Sounds good tho.

[[XC] riptide]

Silence is good or bad??

boints=points????

[Vapor]

Well, if XS goes to ~5k from nearly 7k and the other forums fall only a small amount....ummmm, bad?

How is that fair? We have many of the top benchers and many, many results posted, we should have all our bases covered....

[mtzki]

boints=points????

Yah. They are half-officially called hwboints, boints for short.

[ozzimark]

How is that fair?

admit it, the current revision is overly skewed towards very large teams who can just churn out results like there's no tomorrow

[HeavyH20]

Yes, death by a thousand paper cuts. I like the new boints system since it reduces the impact of thousands and allows smaller teams to compete. So, instead of a runaway first place, or a battle between two teams, the race is truly limited to the best bench teams based on the top talent (top 40) put forth by all. I suspect that we may start to see a tighter race between the top 5 teams which will make the competition more interesting.

[Gorod]

Main changes:

* Increasing the position ranges for awarding boints, especially by introducing the 'offsets' to give boints to all the user's best scores in every global and hw ranking. Users are guaranteed to get at least goffset=1 boint for every bm (if they have a submission that is). This will make the system much more rewarding and motivating especially for new users.

What you need to score higher with that system :
Here we go ... find A LOT of old cheap/free hardware and just bench it all , no need to overclock (unless you want )
Get at least 8 points (SuperPI 1M / 32M , PiFast , CPU-Z , PCMark04/05 , WPrime32/1024) guaranteed for EVERY CPU , no matter how it scores and clocks
Get at least 5 points (3d01/03/05/06/AM3) for almost every GPU, almost because some wont run all 3dmarks , no matter how it scores ...
Get all 78 Celerons , bench them all and get 78x8=624 points guaranteed no matter how bad you scored
With old (current) system at least you have to try , even with old hardware you need to be top5 in HW class to score ... with 1 point guaranteed all you have to do is get MORE and MORE hardware , and just test it non stop lol . How does that make sense ?

[kim55]

* Increasing the position ranges for awarding boints, especially by introducing the 'offsets' to give boints to all the user's best scores in every global and hw ranking. Users are guaranteed to get at least goffset=1 boint for every bm (if they have a submission that is). This will make the system much more rewarding and motivating especially for new users.

Withis wery Incorekt point. I ok to Gorod!

[BuM!!!]

I agree with Gorod. To give everyone 1 point isn't good idea!

[eva2000]

I suspect that we may start to see a tighter race between the top 5 teams which will make the competition more interesting.

you mean top 13 teams LOL

[mtzki]

What you need to score higher with that system :
Here we go ... find A LOT of old cheap/free hardware and just bench it all , no need to overclock (unless you want )
Get at least 8 points (SuperPI 1M / 32M , PiFast , CPU-Z , PCMark04/05 , WPrime32/1024) guaranteed for EVERY CPU , no matter how it scores and clocks
Get at least 5 points (3d01/03/05/06/AM3) for almost every GPU, almost because some wont run all 3dmarks , no matter how it scores ...
Get all 78 Celerons , bench them all and get 78x8=624 points guaranteed no matter how bad you scored
With old (current) system at least you have to try , even with old hardware you need to be top5 in HW class to score ... with 1 point guaranteed all you have to do is get MORE and MORE hardware , and just test it non stop lol . How does that make sense ?

One can only have a single result (the best score across all hw setups) applicable for a single global ranking. The new system only guarantees hwoffset =0.1 boints per setup in single rankings. So 600 boints guaranteed for 6000 different benches. Good luck with that.

[Gorod]

One can only have a single result (the best score across all hw setups) applicable for a single global ranking. The new system only guarantees hwoffset =0.1 boints per setup in single rankings. So 600 boints guaranteed for 6000 different benches. Good luck with that.

this sounds much better

[DDTUNG]

Changing the rules of the game in the middle of the game is never fair, IMHO.

DDTUNG

[DDTUNG]

Yes, death by a thousand paper cuts. I like the new boints system since it reduces the impact of thousands and allows smaller teams to compete. So, instead of a runaway first place, or a battle between two teams, the race is truly limited to the best bench teams based on the top talent (top 40) put forth by all. I suspect that we may start to see a tighter race between the top 5 teams which will make the competition more interesting.

To take this line of reasoning one step further, shouldn't we limit the number of scores from each member? Where does one draw the line?

DDTUNG

[Movieman]

There are points here that either I'm missing or that don't make sense to me.
Why limit a teams totals to the top 40 people on that team?
Doesn't that destroy what "team" is all about?
That would be like one of the DC apps saying that all they will give credit to is the top 40 people who return work for them when in reality it would be thousands..
You also set yourself up to have members submit work in one of those top 40 members names and trust me, that will happen.
The size of a team should not matter at all. That's the individuals choice on what team to be on but whatever is submitted should count towards that teams score.
Just take me as an example, I have I beleive app 70+ points..I didn't do this for me, but for the team totals.
If you only use the top 40 people you will lose many people from submitting.

[eva2000]

There are points here that either I'm missing or that don't make sense to me.
Why limit a teams totals to the top 40 people on that team?
Doesn't that destroy what "team" is all about?
That would be like one of the DC apps saying that all they will give credit to is the top 40 people who return work for them when in reality it would be thousands..
You also set yourself up to have members submit work in one of those top 40 members names and trust me, that will happen.
The size of a team should not matter at all. That's the individuals choice on what team to be on but whatever is submitted should count towards that teams score.
Just take me as an example, I have I beleive app 70+ points..I didn't do this for me, but for the team totals.
If you only use the top 40 people you will lose many people from submitting.

yeah that part confuses me, but remember this is all preliminary proposals I guess ?

my team only has 27 members right now but it's holding it's own even if some benchmarks we don't have the full top 10 entries yet... don't think limiting the team totals to a certain number is that good an idea.

maybe have a team total and a team top 40 total.. so 2 team HOF rankings ??

[Movieman]

yeah that part confuses me, but remember this is all preliminary proposals I guess ?

my team only has 27 members right now but it's holding it's own even if some benchmarks we don't have the full top 10 entries yet... don't think limiting the team totals to a certain number is that good an idea.

maybe have a team total and a team top 40 total.. so 2 team HOF rankings ??

Hello my friend!
I have to disagree. It should be free to all and all entries have the same chances,etc..
The "top" people will still be the top people regardless of which team they are on but to discourage the little guys(like myself) is a mistake.
In this I am able to contribute to my team and it is just that feeling that is what is important here. The being part of the whole.
Individual honors for the top people will still exist.
In the Dc world XS has one of the smaller teams in terms of numbers but some of the strongest teams because of the quality of the machinery that we run but not all of that machinery is "state of the art" Some is 2-5 years old but those members aren't excluded and they darn well aren't looked upon differently.It is the putting of the resources from all into one pool that makes a team..
I understand the thinking that made the people from HWBot think of this idea, I just disagree with the logic behind it.

[DDTUNG]

I agree with Movieman.

XS DC has been able to take top honors in several DC apps against much larger teams. We achieved that by recruiting as well as by increasing output from existing members, not by cutting the size of our competitors.

DDTUNG

[HeavyH20]

From what I see, there are couple of main efforts.

1. Increase the availability of boints so that everyone can post a decent number. This should improve interest levels from the "small guy" as someone aptly coined. Nothing like posting a big 15 points up when the leader has 700. If that spread was narrowed, the ranking would likely remain the same, but people will likely post 24 points up if the leader is at 150. That seems a more bridgable gap.

2. Make the team competition more about skill than numbers. Limiting the team score to the Top 40 will help bring about more competition. It is more interesting to watch a close race between two or more groups than to witness a run away lead. Take the OCX versus XS battle for first back in December. That was interesting. The current state, however, is not.


And, I do not think you can compare a benchmark competition to DC. In DC, it is all about CPU cycles, the more the better. In benchmarking, it is about taking some hardware and pushing the envelope. I would rather watch the antics of OPB, OPP, Hipro or the likes help push their team into the lead instead of 2000 people benchmarking P4 2.8a's for 4 points each to take the team lead. Not that benchmarking a P4 2.8 is not without merit, but, we need to focus attention on things that will draw interest. This is more about creating some excitement and competition in the OC world with, for the first time, a carefully crafted arena.

[DDTUNG]

From what I see, there are couple of main efforts.

1. Increase the availability of boints so that everyone can post a decent number. This should improve interest levels from the "small guy" as someone aptly coined. Nothing like posting a big 15 points up when the leader has 700. If that spread was narrowed, the ranking would likely remain the same, but people will likely post 24 points up if the leader is at 150. That seems a more bridgable gap.

2. Make the team competition more about skill than numbers. Limiting the team score to the Top 40 will help bring about more competition. It is more interesting to watch a close race between two or more groups than to witness a run away lead. Take the OCX versus XS battle for first back in December. That was interesting. The current state, however, is not.


And, I do not think you can compare a benchmark competition to DC. In DC, it is all about CPU cycles, the more the better. In benchmarking, it is about taking some hardware and pushing the envelope. I would rather watch the antics of OPB, OPP, Hipro or the likes help push their team into the lead instead of 2000 people benchmarking P4 2.8a's for 4 points each to take the team lead. Not that benchmarking a P4 2.8 is not without merit, but, we need to focus attention on things that will draw interest. This is more about creating some excitement and competition in the OC world with, for the first time, a carefully crafted arena.

So you are saying that a team with more people benching their rigs on phase or DI or LN2 should win? To me there is more accomplishment in benching a pair of 1.6 LV Xeons @3.3 with IWTs on a modded PC-DL than slamming a triple cascade on a E6800. Equipment does not equal skill.

Excitement in the benching world has always been generated by individual antics, not elitist teams. Until a system of point awards exists that recognizes similar levels of achievement with different equipment, there can be no fair competition.

And just for your information there is more to DC than turning on more computers for longer periods, but I'll leave that discussion to a more suitable venue.

DDTUNG

[massman]

2. Make the team competition more about skill than numbers. Limiting the team score to the Top 40 will help bring about more competition. It is more interesting to watch a close race between two or more groups than to witness a run away lead. Take the OCX versus XS battle for first back in December. That was interesting. The current state, however, is not.

That would discourage the smaller benchers a lot. If you try your best to reach an insane score with a 6800gt and you're not in the team ranking, you won't post any more results anymore.

A team stays a team, whether your first, second or third.

[[XC] riptide]

So you are saying that a team with more people benching their rigs on phase or DI or LN2 should win? To me there is more accomplishment in benching a pair of 1.6 LV Xeons @3.3 with IWTs on a modded PC-DL than slamming a triple cascade on a E6800. Equipment does not equal skill.

.......
DDTUNG

QFT^^^^

Don't lie man. You've got swiftechs finest 607 blocks on them I'd reckon. Before i sell them I must have another go on them.