To engineering samples owners: Why no real benchmarks?

[boxleitnerb]

This bothers me everytime a new CPU is in the works and people get their hands on the chips early and leak stuff.

Things tested are superpi, wprime, chess benchmark and so forth. All of these benchmarks have no practical relevance at all, superpi is completely outdated (and inefficient btw). Or the ES are benched in games with a midrange graphics card at 1080p, moving the bottleneck towards (if not completely to) the GPU. You get the gist. No offence, but sometimes I'm under the impression that only morons get engineering samples (or people who just want to antagonize the rest of us with meaningless benches).

Why is that? Why not test real stuff like video encoding, rendering, cpu-intensive games/settings (with the use of savegames, not timedemos, which most integrated benchmarks are).

[Alex-Ro]

Because you are on goddamn xtremesystems.org.Do you know how to spell it?Main focus here is benchmarks,not your encoding,rendering and other stuff.So remember,xtremesystems.org=overclocking+benchmarks mainly.That's why you don't see games and other stuff tested very often.

[LagunaX]

In other words Alex-Ro recommends you look on Anandtech for real stuff testing you seek

[boxleitnerb]

Other sites get their samples mostly when NDA falls. That's the trouble

[bamtan2]

sometimes I'm under the impression that only morons get engineering samples

maybe only the morons leak?

[Leeghoofd]

Indeed we the m*rOns test what is relevant to our community. Superpi outdated you say, maybe to you. It still gives a good indication how ram speed, timings scale on a new platform. In case of the AMD FX CPU it clearly gave away that the CPU is underperforming at single threaded apps.
Game benchmarks, how at full maxed out detail ? So you max out the GPU and the CPU nor platform has got hardly any influence in the outcome... if that is real to you... plz visit hardwareheaven or other "affiliated I test as the hardware vendor like it" sites... If you want to show the CPU influence you have to test at either lower res and then at higher one. In the end most games will be GPU bottlenecked and the outcome will be pretty much the same...

Good reliable all round site are hardwarecanucks, techspot and co... hardwareheaven and co are not on my most appreciated list...

Most of the m*r*ns here are more power users/beta testers to debug stuff and find/solve issues/workarounds...

[boxleitnerb]

Well, the famous superpi IS outdated, it's code is stoneold. Also it distorts differences between cpus quite a bit. There, a cpu may be 100% faster than it's competitor, in normal singlethreaded apps the lead will be significantly less. So it's not at all accurate between different architectures from different IHVs. Also, go with the time. Software changes - you don't bench games from 2000, do you?

With the game tests I can just repeat myself:

A cpu test where you are gpu bottlenecked is useless, because it doesn't provide any relevant information. You say "most games will be GPU bottlenecked", that may be true to a certain extent. However, look further. SLI/CF, newer GPUs, cpu intensive games (strategy, simulations), avoiding framedrops.

I didn't mean any disrespect, I'm just so frustrated at the lack of creativity out there. Every single time the same (often meaningless) benchmarks are run. How about something else for a change that actually gives a more accurate indication of what to expect from a new cpu?

[Alex-Ro]

btw leeghoofd your review of bulldozer rocked It was just perfect,everything i wanted to know about cpu-nb,memory and cpu scaling was there.Good job

[TJ TRICHEESE]

It seems like what you've said is directed at one specific user on XS as chess benchmarks rarely come up here and usually when testing es cpu's people usually only focus on how the cpu performs and don't test how it performs in a graphics intensive environment.

[boxleitnerb]

I didn't have a specific user in mind. I'm talking generally. For example the leaks at corescn.

[TJ TRICHEESE]

To me your original post looks like its very much aimed at user 'windwithme' as you mentioned chess benchmarks... as i said they literally never come up here (maybe very rarely)

[boxleitnerb]

I don't even know this user. And I'm not only talking about leaks that come up here at XS but all over the web. Cinpai or what this site is called, the asian leaks and so on.

[TJ TRICHEESE]

Other sites get their samples mostly when NDA falls. That's the trouble

[boxleitnerb]

So? I know that really reliable result come in only at launch. However, everytime I see someone has an ES I think "bench this, bench that, DO something with that thing" and then I see...superpi.
If you cannot understand that, I cannot help you

[Movieman]

So? I know that really reliable result come in only at launch. However, everytime I see someone has an ES I think "bench this, bench that, DO something with that thing" and then I see...superpi.
If you cannot understand that, I cannot help you

An easy solution to your problem:
As them "nicely" to run X test and maybe they will.
Remember the old line" You get more with honey than you do with vinegar"

[Dumo]

A reviewer and a bencher usually not the same person

[Leeghoofd]

At least you liked it :p That's the main thing that interests me on new architectures : ram choice, timings, speeds and how to get a little bit more perf out of it... I have no time to do 25 different game tests at 10 resolutions. I leave that to the real reviewers....

[CryptiK]

This bothers me everytime a new CPU is in the works and people get their hands on the chips early and leak stuff.

I suggest you build a bridge. There are sites that do this kind of review, use them.

Things tested are superpi, wprime, chess benchmark and so forth. All of these benchmarks have no practical relevance at all, superpi is completely outdated (and inefficient btw).

That's because for the most part, the people on XS with these connections able to get pre-release samples are not interested in video encoding, rendering or cpu-intensive games/settings (with the use of savegames, not timedemos). They test what they are interested in, and what the XS community is interested in.

In case you haven't realized, some of the above named 'useless benches' to you are often held in quite high esteem here. The fact that SuperPI benching has it's own section on the forum may help you understand. Young PC users that find there way here "cos it's sounds awesome and extreme yo" tend to find out pretty rapidly we aren't focused on typical computer uses here. Kingpin doesn't gloat to his pre-pubescent mates on msn about how fast his 6.5GHz gulftown is encoding the latest pirate version of Harry Potter and the this_is_not_anandtech.com. We bench, so those with the ES's bench them. Pretty straightforward.

No offence, but sometimes I'm under the impression that only morons get engineering samples (or people who just want to antagonize the rest of us with meaningless benches). Why is that?

No offense, but calling some of our most respected members who do us all the service of testing the latest and greatest morons is not going to win you any favours here. 'The rest of you' are in the minority here, so you are unlikely to be pandered to. Accept it and move on.

Why not test real stuff like video encoding, rendering, cpu-intensive games/settings (with the use of savegames, not timedemos, which most integrated benchmarks are).

XS is not generally concerned with real stuff. Real stuff is boring. I have a slow boring PC to do real stuff on, as do most of us here. When I go into a ES testing thread, I want to see as little real stuff as possible, preferably none at all.

[Ket]

I think the OP just wants tests that are more relevent to the hardware being tested. For example; if testing a new Quad Core CPU or something like that, Cinebench is likely going to be a much more acceptable test to run than SuperPI.

[Eldonko]

Sometimes it is

A reviewer and a bencher usually not the same person

ps - PI > all

[zir_blazer]

That's because for the most part, the people on XS with these connections able to get pre-release samples are not interested in video encoding, rendering or cpu-intensive games/settings (with the use of savegames, not timedemos). They test what they are interested in, and what the XS community is interested in.

You are totally wrong. The community is composed of two groups: The extreme overclocker, that most times I wouldn't call it a true enthusiast, just a kid in a toyshop that always opens a box with something new to play with, that just cares about meaningless benchmarks in an unidimensional way that would make Sledge Hammer jealous, and the all around enthusiast that cares about the pros and cons of everything that it touchs and its budged limited. The first group is always on the spotlight precisely because they always get first the expensive or unreleased toys that everyone wants to play with, but it doesn't make it the bigger one, so don't talk about the XS community as a whole because it isn't, and I don't feel represented by any of those guys. Seems that people that manage to get Engineering Samples and aren't bound to a NDA, or like to leak results anyways, are usually from the first group, because many times I saw benchmarks that are too poor to consider reaching conclusions with them. When I registered here some years ago I though that it would allow me to get closer to those guys that got the beta toys to get early data regarding if I should wait for X platform, Y architecture and Z Stepping, and my results in that regard are fairly limited. I always had more fun among regular enthusiasts that speculates that with ES users, that even if they know the truth and could leak a few more results, they wouldn't do a more in depth analysis on an actually useful fashion. And that pretty much kills the real life point of having access to ES parts without NDA if your only objetive is impressing people with SuperPI.

[Ket]

^^ People can argue with that as much as they want, but theres a fairly good point made there.