The ES Cauldron | Seeking A Resolution

[64NOMIS]

I have been asked to give my view so I will do so here. I also share everyone’s dread in polluting results threads with this discussion.

First, let me try to state the problem and my view. I believe that keeping ES off of HWBot main areas is very reasonable. That seems to be the focal point for these discussions since it is an arena where the world competes and competes primarily without vendor-provided hardware. Debate on experimental work with experimental hardware that pushes the limits may decline if we solve the primary problem: mingling results with experimental or vendor hardware with the vast majority of results which do not have this potential advantage.

However, I have some issues I struggle with as I think through this issue. I don’t have the answers but let me try to state the problems and some of the dilemmas I see.

For background, let me attempt to define ES. With today’s AMD processors, those happen to be high-leakage chips that can scale with LN2/LHe. Those overclock well under unprecedented cooling for modern processors and to everyone’s surprise performed in the WR competitive range. In general, the differences spanning any vendor product appear to be next generation design nodes, binning, and leakage. The irony is that higher leakage parts are, in general, discards, unable to run within stock power specifications, and those are precisely the ones that, either found by luck or culled out, may be the better overclockers.

Is any vendor-provided product an ES? When you get a CPU, GPU, MB or DRAM from a vendor, it is not clear if binning has occurred or process differences exist. The notion of a “cherry” is primarily binning (sometimes related to the leakage issue, primarily just the nature of modern ASICs which have slight variations that overclockers experience as higher attainable clock speed) and advance design nodes. Are we to ban any result with vendor provided CPU, GPU, MB, or DRAM from HWBot?

Advantage, deep pockets? If we were to do this, we advantage those world-class overclockers with deep pockets on HWBot. Again the irony, we solve the ES problem but create an economic one, not for everyone, but at the top. In addition, this seems difficult and frankly no fun to enforce. Overclocking in the competitive range is an expensive hobby if entirely self-funded. Again if you can afford it, then it is in your interest to ban vendor hardware. It may be a difference of interest issue, not a moral one.

Cannot win for loosing? I am confounded by people using clearly retail grade product, getting great results, and subsequently having it called a “cherry”. At some point, trust must be maintained, even when clearly retail grade product sees extraordinarily good results. Yet, if a product is from a vendor, can the vendor be trusted?

Can we look to other sports? In motorsports, robot wars, tennis, basketball, etc. sponsor gear that makes a team or individual more competitive is part of the game. The mingling and indeed symbiosis has become an accepted norm. Overclocking is a combination of “amateur” and underpaid “professional” grade competition. At the top are the pro’s often with vendor hardware, often overclocking at vendor events. Is it part of the natural balance that the pro leagues get better, often experimental gear?

Do we need a pro league? What is interesting is that most of the overclockers who post in this section forums get gear at one point or another. At what point does one graduate from the amateur league and take on the fact that in the pro’s there is bigger, advanced weaponry that may be hard to come by? And if you are a pro, if you cry foul, at what point is it just trying to get the call, and at what point is it really a moral issue?

I do not blame anyone for creating these issues, and as I have said before I respect the competition and my competitiors. Save that I see my own actions, stretching back almost 6 years, and the changes they have incited. Yet this is extreme competition using extreme systems. It is a combat. There is only so much one can do to restrain warriors.

So if we choose, lets resolve this now so that the pursuit of overclocking may continue, hopefully with less strife and more victory celebrations.

[zalbard]

I can only think about less binning... Handing out high leakage chips doesn't make much sense in terms of fair competition. You're binning chips already to find out what model it's going to belong to, for example 945BE or 955BE. Just don't bin any further, or if you want to bin super chips, just sell them as Extreme chips for a lot of $$$ like Intel does. That way everyone has more or less equal chances of getting a good or a bad CPU.
I don't mind vendors sponsoring OC teams, just make sure not to perform super binning doing so. So those should be retail products.

[SNiiPE_DoGG]

In my observations of things that I consider similar to this debacle over ES chips, the solution is never to attempt to limit or control it, but rather to embrace it.

We need a Pro division or F1 league (whatever may come of these concepts) and in this league EVERYTHING short of cheating and altering results must be allowed. The normal division of HWbot will stay the same, but competition in that field will increase too because most of the sponsored clockers with the proven skills will be in the PRO division duking it out.

[SF3D]

Hi Simon!

Nice thoughts there.

Here is my solution for these issues: http://hwbot.org/forum/showpost.php?...&postcount=174

What do you think?

[64NOMIS]

Hi Simon!

Nice thoughts there.

Here is my solution for these issues: http://hwbot.org/forum/showpost.php?...&postcount=174

What do you think?

Petri,

I think we need to resolve the definition of ES and I fear the vendor-provided hardware issue may still be problematic.

In other sports, a pro is a pro, an amateur is an amateur. You are a pro. It might diminish ambiguity if we designated people rather than individual results.

I am worried about the extra work and the burden this places on HWBot and the admins. Perhaps it is manageable.

But, yes, I think pro designation is a good idea, I just think it may correspond not to individual results, but rather, to individuals.

[Slovnaft]

I can only think about less binning... Handing out high leakage chips doesn't make much sense in terms of fair competition. You're binning chips already to find out what model it's going to belong to, for example 945BE or 955BE. Just don't bin any further, or if you want to bin super chips, just sell them as Extreme chips for a lot of $$$ like Intel does. That way everyone has more or less equal chances of getting a good or a bad CPU.
I don't mind vendors sponsoring OC teams, just make sure not to perform super binning doing so. So those should be retail products.

That may be true if you think of overclocking as a sport that should be governed by money to invoke 'fairness'. This is the means to the same end that comes hand in hand with the "hardware sharing" discussions we have on hwbot which touch on many of the same sensitive issues as the ES/vendor hardware debate.

Personally I see overclocking and its following as an activity and community that, thus far, has grown to be uniquely friendly and communal. By that I mean that in online forums we seem to see each other as like-minded individuals all working towards a more or less common goal in many of the same ways, and because of this we support each other in our endeavors in a way that, in my opinion, transcends sportsmanship in its relative selflessness. As a symptom of our media we realize that the best form of innovation in overclocking comes through sharing of information, technique, and, inevitably, equipment.

Now overclocking differs from sports in more ways than our attitude. I'm a swimmer so I'll take my analogy there. Michael Phelps may wear a slightly different or more advanced suit than Ian Crocker, but few would argue that his suit was the reason he beat out Ian for the 100m Fly WR. It is fairly well recognized that performance in swimming is a transparently supported result of hard work and skill. Conversely, in overclocking, as the current system stands the relationship between work, skill, and performance can sometimes become obscured by the nature of our medium. To make overclocking comparable to a sport in that regard we would all need to bench the EXACT same hardware in an ideal world. Do not mistake this for a suggestion.

Inevitably, the only way the overclocking world can come to terms with itself is to embrace some of its nascent characteristics of commonality and brotherhood.

I'll complete my thoughts later, but right now I have to catch a train...

[boblemagnifique]

Limited the ES chips is more simply for me.

For exemple , when a brand of Memory , graphic cards .. selected one pcs for a overclockeur , it's not ES kit but a same Kit in the shop ... how do we know?

for the cpus , you detected The ES chips only by Cpuz , it's true ? It's not real because i have a few cpus ES doesn't recognized by CPUZ. Franck can't all do, we (brand & we) must also help too (although the last few years, there's better)

Sorry for me , the requires of overclockeurs be honest and tell the truth about of materiel of overclocking it because we can never be proof if it is a cherry picked or not ...

There are a lot of work but if we work together without thinking about the competition and ranking, we can create a league honest and fairplay to all the community

Sorry for my english

[massman]

Thank you for taking the time to digg into this issue Simon. I'm actually quite happy to see that I share a couple of your thoughts regarding the true nature of this issue as well as the consequences of agreeing to take down the scores. As I've already stated in the Hwbot forums, this discussion streches far beyond the ordinary "it's not fair"-argument, but in fact should make us think about the level of fairness of having a cherry cpu and compare that to the level of other factors that play in the overclocking world, in particular the competition, be it at Hwbot or just in a few local forums.

If we ban the ES samples because of this sole "it's not fair"-argument, we should be taking into consideration all the other factors that is not fair. In the recent past, I've seen people get in touch with motherboard manufacturers and get more than one sample to test. Is that fair? I have seen people get in touch with videocard manufacturers and recieve multiple samples of high-end cards to test. Is that fair? I know that there are a lot of people spending a lot of their free time writing reviews, round-ups and in-depth technology articles and for return, they sometimes recieve hardware. Is that fair? Some people win the lottery and are capable of buying thousands of chips, just to find the golden one. Is thát fair?

Also, I would very much like to point out that this discussion probably is a mere problem of benchmarks incapable of being limited by what they're supposed to test. Please, think about the true reason why the discussion involves CPU's and not VGA's or memory or any other type of hardware. It's because not only the 2D benchmarks are cpu-bound, but the 3D benchmarks we use are also.

Again, I would like to stress the fact that I'm not leaning towards one side or the other here, just trying to get a few thoughts across like I did in the Hwbot forums. They were read incorrectly by a lot of people, so I hope it doesn't happen with this one.

[chew*]

I think what simon is getting at is a few people me included have attained extremely good results with retail review samples.....which had to meet 955 binning criteria......etc they will never perform on par with "ES high leakage"

This is overshadowed by those calling them "cherries" in just about every post with results.......

For me I can't afford one chip much less 10......I have no choice....

The Irony is when ever I have benched "high leakage ES" I have pummeled any result I have ever acchieved on review samples.......whether posted result or internal........so there is a consistency with all results I acchieve.....

Quite frankly I know how to tell them apart, most don't......however the comments are getting rather annoying......annoying as In I ask myself daily now, why do I even bother.

The sad part is if a guy with a blue or yellow name acchieves a result with the same review samples.....it's not questioned.....they are just 1337.

My 2c.

[massman]

I am worried about the extra work and the burden this places on HWBot and the admins. Perhaps it is manageable.

Technically, yes. When put in practice, I believe it will be a very difficult task, for which the difficulty level depends on the expectations of this new PRO league.

As long as we will have to rely mostly on the honesty and integrity of the overclocker, I'm afraid we'll end up with the same kind of misery as we have today. With the background I have at Hwbot in mind, and let me be so free to extend this to the history of mankind, I know that when people are not able to win by power, they'll use their brain to get ahead, this meant in a negative way. Bending rules to have incorrect scores accepted by Hwbot, it's been done many times. And for those who blaim the internet as a fairly anonymous way to interact, recent actions have shown me that the exact same behavior happens when people are in the presence of each other.

So, with that in mind, we'd have to have rules SO strict that it would take years to develop a professional league which is regulated from bottom-up; with which I mean: regulated and controlled only by hwbot. The other way would be to have a complete top-down regulation where the manufacturer is willing to regulate the 'cherry' seeding and inform hwbot which user got a sample that can be called 'cherry'. Again, possible within certain range as we can never be sure a vendor is speaking the truth and nothing but the truth; I suppose I don't have to point fingers here.

If we're looking for a quick 'n' easy solution, I'd have to say this would be the one:

If Intel and AMD know who has cherry chips and they'd be willing to share this information with us or the community in general, it's possible to moderate the results. I'd prefer to see them indicate the fact that they're special ES samples in the cpuid tag, though. If you can make it look like retail, you can also make it look like "975 (CS)" or "X4 955 (CS)".

+

... allow people to choose to have certain scores excluded from the points, much like we now have with entire profiles.

But, again, we'd be relying totally on the honesty of the manufacturers.

[Sampsa]

First of all I want to thank AMD and Simon for support they have offered for several overclockers since and before Phenom II launch. You guys have obviously worked hard to get rid of issues like cold bug and as we have seen the CPUs scaling under cold is amazing. Hopefully we will see products like TWKR available for everyone to buy and most important for reasonable price.

In my opinion we can divide ES CPUs to two categories:

1) Normal ES: Available to partners and press before the product actually launches. These are usually close to retail parts, maybe one stepping/node before retails. From my experience (I've had access to AMD & Intel ES CPUs before the official launch around 8 years) normal ES CPUs are actually usually worse than retails because manufacturing process develops all the time and retails keep getting better. Every Phenom II press sample (not even marked as ES but couple weeks earlier than retail parts) I've tested so far have been worse than normal retails.

2) Cherry-picked ES: These are the ones which cause problems in competition and give unfair advantage compared to retail parts. These are handpicked by manufacturer straight from production line because of chip's good characteristics for overclocking. Usually these parts wouldn't make it to retail because they are way beyond spec (leaking current -> too hot). These are perfect parts for overclocking under cold (LN2 & LHe) and suitable for technology experiments / demonstrations. These parts should be marked clearly different than retail parts.

---

I'm personally interested to see how far I can push the latest technology and 2) Cherry-picked ES are best for this. But I don't want to compete with these parts against someone who is using retail parts because no retail part can ever be as good as cherry-picked ES.

It's like two guys racing against each other on straight line and one has speed limiter @ 200 km/h and other one @ 300 km/h. Who will win?

I will keep testing all CPUs: press sample ES, cherry-picked ES and retail-samples. But from now on I'll only post results with retail components to hwbot.org because 99,9% of +17500 other users are using retail parts and currently in Overclockers league you are competing against them.

My suggestion to hwbot.org is to create separate "Advanced Technology" ranking where everything goes and people can post scores with special hardware like ES CPUs or components which are not publicly available.

[chew*]

In my opinion we can divide ES CPUs to two categories:

1) Normal ES: Available to partners and press before the product actually launches. These are usually close to retail parts, maybe one stepping/node before retails. From my experience (I've had access to AMD & Intel ES CPUs before the official launch around 8 years) normal ES CPUs are actually usually worse than retails because manufacturing process develops all the time and retails keep getting better. Every Phenom II press sample (not even marked as ES but couple weeks earlier than retail parts) I've tested so far have been worse than normal retails.

Thank you.

Maybe coming from you people will actually listen.

[SF3D]

My suggestion to hwbot.org is to create separate "Advanced Technology" ranking where everything goes and people can post scores with special hardware like ES CPUs or components which are not publicly available.

I hope more people would find this new NO limits league interesting.

This league wouldn't mess up the point system, but we all would be able to see and check the current WR's from one site.

I have been talking about this new league about 6 months and now it seems, that the time is ready.


But, I have to say that taking of the ES scores out of the database, just liket that, is not the right way either. People should wait and listen. HWbot.org tries to evolve and if all the users remove ES scores from the site it does not lead anywhere. I can figure only one reson to do so, but is it the right one, no one knows.

Time will tell where our discussion will lead in HWbot.org crew section, but we will keep you all updated for sure. The start of the conversation is not too promising though

[TheGreenFoX]

mpeting against them.

My suggestion to hwbot.org is to create separate "Advanced Technology" ranking where everything goes and people can post scores with special hardware like ES CPUs or components which are not publicly available.

Then we are back to how to tell a ES (cherry) from a retail.
It would give AMD and Intel an economic interest in making cherry ES look like a retails sample ...
I dont have a solution... just pointing it out

[SF3D]

Then we are back to how to tell a ES (cherry) from a retail.
It would give AMD and Intel an economic interest in making cherry ES look like a retails sample ...
I dont have a solution... just pointing it out

They would not have to do this, cause in the new league it wouldn't matter at all. If you use ES and do a WR, it is totally ok in that league. You would not get any HWpoints from it and the overclockers league would be untouched.

At the moment both vendors might have a urge to change labels, but with this new league need for doing that is minimal.

[TheGreenFoX]

How i see it, they would disguise cherry ES as retails to make people buy their product because they perform better in benchmarks... even then they dont...

Maybe im just way to suspicious...

[Movieman]

Can I toss in my thoughts on this?
I know and respect what you guys do even though my interests aren't in this end of the hobby BUT that doesn't mean I don't read your threads and sit back in wonder at what you guys accomplish.
I think if you try and eliminate ES chips from the playing field you will stifle the hobby.
They are like the prototype engines used in Formula 1 racing and what is learned from those engines trickles down to the mainstream and benefits all over time.
To me all that is needed here is already inside each and every one of you: honor.
Put the desire to be #1 on HWbot or on Futuremark where it belongs and not as the 'OMG I got to be #1 today" sort of thinking.
Stop and think for a minute: Do you think that I, Dave Average" think any less of the accomplishments of any of you whether your #1 or #21 on HWbot?
The answer is no, I don't.
I've been here since November 2005 and read from day one of the "greats" for lack of a better word in this hobby.
Sampsa,Macci,Fugger and Hipro were the names at the top when I came here and read all of what you wrote.
Any and all of you "top" guys have my respect and the titles will go back and forth between you over time just as they always have.
If you use a "cherry" or "Dear Lord, I was sent the "ONE" of a million chip" then say so and put your records and accomplishments in perspective so that not only do we see your accomplishment but see it in the proper perspective.
Just like the use of LHe vs LN gives an advantage and is seen for what it is.
Keep the ES's, push the limits because to do less changes the meaning of what you do.

[boblemagnifique]

First of all I want to thank AMD and Simon for support they have offered for several overclockers since and before Phenom II launch. You guys have obviously worked hard to get rid of issues like cold bug and as we have seen the CPUs scaling under cold is amazing. Hopefully we will see products like TWKR available for everyone to buy and most important for reasonable price.

In my opinion we can divide ES CPUs to two categories:

1) Normal ES: Available to partners and press before the product actually launches. These are usually close to retail parts, maybe one stepping/node before retails. From my experience (I've had access to AMD & Intel ES CPUs before the official launch around 8 years) normal ES CPUs are actually usually worse than retails because manufacturing process develops all the time and retails keep getting better. Every Phenom II press sample (not even marked as ES but couple weeks earlier than retail parts) I've tested so far have been worse than normal retails.

It should also be recalled that the interest of Normal ES and very often the nice price (after it's released official OEM or BOX). For exemple for the CPU intel the price is expensive for the Retail Box for the Xtreme Edition (i7 , Xeon or Cpu Mobiles) , The ES allows some people who have no bugdet test for a lesser cost ...
Only the problem the sale of ES is dark and forbidden, but who's to blame? why there are Intel 32nm sales currently in the moment on the forum Public, and Intel knows it must continue and leaves ... let me laugh, it is tolerated because it will create a great "media buzz" even if many millions of $ going in the pockets of anybody in the dark traffic ....
Then hunting ES is really hypocrisy total ...

[SF3D]

How i see it, they would disguise cherry ES as retails to make people buy their product because they perform better in benchmarks... even then they dont...

Maybe im just way to suspicious...

Well.. that is true, but if there is no need for that, it wouldn't be so important. If everyone can use ES and be proud of it, there is no need for it.

then say so and put your records and accomplishments in perspective so that not only do we see your accomplishment but see it in the proper perspective.

Nice post Dave. This was the most interesting part.

I really think that 99% of average overclocker does not care how the records are done, but to keep the game fair in HWbot, we have to figure something out.

I may have totally different point of view cause I am connected to both things (ES cpu's and HWbot) closely. So, I will let this conversation go on without preaching about this new league

[Slappa]

I agree strongly with Sampsa and Chew* on this issue...

Thing is...preview retails (review samples) etc, are designed to represent a final retail product.....and are actually worse in most cases.
People try to to call these chips out as cherry's....which is not the case at all. I find it ludicrous to see people who can do amazing things with retails or preview retails be accused of attaining cherry chips.

People need to remember that a lot of the top OCers tend to choose preview retails from trays full of them. Nothing is binned or cherry picked specifically for these trays. The only picking being done is the hand picking from the OCer/User. Even then, it would still be the chip that it was binned and specified for, nothing more.

and on the other hand an ES is an ES. Most people state they are using an ES. Most pros also do NOT post ES results on hwbot.org to keep it fair. Same goes for cherry picked chips. Most results displayed from these chips are mostly experimental for the purpose of showing technology at it's best.

[sacha35]

I think we are still looking at this the wrong way.

I have said this many times before, but for some reason it just keeps getting put down or ignored.

If we are going to try and make this a true competitive sport then there needs to be true validation, in this I mean built in software that can validate that there has been no fail play that has taken place.
Secondly there has to be a limit put on the CPU for the benchmarks very much like the low clock challenges that are around, I am not saying run our chips at stock or water cooling but something everyone can reach with extreme cooling and again this would be truly validated by the validating software that has to be in place on the operating system to make sure the clock speed is the one that has been set/ put in place by the regulation site.

I have taken the point Sampsa has made and listed it here for this reason
(It's like two guys racing against each other on straight line and one has speed limiter @ 200 km/h and other one @ 300 km/h. Who will win?) well the same goes for clock speeds on CPU too.
I am from a motor sport background with good results and many years in motorsport and there is in place limits imposed on setups and Engines to make things more competitive which brings a bigger audience and secondly it makes things even so only the best people win.

I am not saying I don't want to see the big clocks or supper results as these are always great to see but to make our sport more competitive with more members joining in and competing on an even playing field has to be the only way this sport can grow as MHZ will always win.
there is a big enough data base to be able to determine what most CPU's can do with regards MHz so this would make things more even for everyone regardless of if the CPU is retail or ES or supper picked.

So the easy way is to have proper full validating software that shows no fail play had taken place and impose a MHz limit.

But create a new category for all high MHz benchmarks to show the manufactures what their product can do and what sort of results can be had.

[gandalfone]

Thank you for taking the time to digg into this issue Simon. I'm actually quite happy to see that I share a couple of your thoughts regarding the true nature of this issue as well as the consequences of agreeing to take down the scores. As I've already stated in the Hwbot forums, this discussion streches far beyond the ordinary "it's not fair"-argument, but in fact should make us think about the level of fairness of having a cherry cpu and compare that to the level of other factors that play in the overclocking world, in particular the competition, be it at Hwbot or just in a few local forums.

If we ban the ES samples because of this sole "it's not fair"-argument, we should be taking into consideration all the other factors that is not fair. In the recent past, I've seen people get in touch with motherboard manufacturers and get more than one sample to test. Is that fair? I have seen people get in touch with videocard manufacturers and recieve multiple samples of high-end cards to test. Is that fair? I know that there are a lot of people spending a lot of their free time writing reviews, round-ups and in-depth technology articles and for return, they sometimes recieve hardware. Is that fair? Some people win the lottery and are capable of buying thousands of chips, just to find the golden one. Is thát fair?

Also, I would very much like to point out that this discussion probably is a mere problem of benchmarks incapable of being limited by what they're supposed to test. Please, think about the true reason why the discussion involves CPU's and not VGA's or memory or any other type of hardware. It's because not only the 2D benchmarks are cpu-bound, but the 3D benchmarks we use are also.

Again, I would like to stress the fact that I'm not leaning towards one side or the other here, just trying to get a few thoughts across like I did in the Hwbot forums. They were read incorrectly by a lot of people, so I hope it doesn't happen with this one.

Bingo!!

I agree

[Movieman]

Well.. that is true, but if there is no need for that, it wouldn't be so important. If everyone can use ES and be proud of it, there is no need for it.



Nice post Dave. This was the most interesting part.

I really think that 99% of average overclocker does not care how the records are done, but to keep the game fair in HWbot, we have to figure something out.

I may have totally different point of view cause I am connected to both things (ES cpu's and HWbot) closely. So, I will let this conversation go on without preaching about this new league

Ok, then I think your answer is a "pro" league.
Put it right out there for all to see that the people in that section are getting help and special parts and let it fly.
By doing so, it does keep it in perspective for all of us mere mortals to see.
It also allows the manufacturers to put into the "right" hands chips that they think are stellar and to show off what their R&D can accomplish in the hands of people that we know and trust.
The peolple in that "pro" section should also be allowed to post results in the regular section but now we're down to the honor thing.
That is something one either has or doesn't and no amount of rules will change what is inside you guys.

[elmor]

I'm thinking along the same way as Sampsa, Simon and Pieter(Massman). It's unfair that some are able to get good cpus from manufacturer but IMO it's just as bad if you need to be really rich to handpick cpus. Still as Sampsa and Movieman are on to I'm in this for pushing the limits of the hardware which means I want to see higher clock speeds and results.

I have no real solution to this issue and as other says if we start with cpus then we will have to impose the same with vgas, rams, mbs etc. The way I see it overclocking will never be 100% fair, but the again, life isn't either. IMO, currently the best way to compete is with the live overclocking competitions with pre-binned cpus.

Clock limits is NOT the way to go, it takes almost all the fun out of overclocking/benchmarking.

[massman]

impose a MHz limit.

For me, that's a no-go for several reasons:

1) It's called overclocking
2) Completely unreliable verification with current software
3) Even with future software, it's crackable (eg: cpu-z 1.43)
4) Called Bernie Ecclestone just a few minutes ago, and he said no when I suggested a 60km/h limit; even though I said it would show us the true best driver as finding the right line would become the main skill
5) It's called overclocking