Results 1 to 22 of 22

Thread: To engineering samples owners: Why no real benchmarks?

  1. #1
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581

    To engineering samples owners: Why no real benchmarks?

    This bothers me everytime a new CPU is in the works and people get their hands on the chips early and leak stuff.

    Things tested are superpi, wprime, chess benchmark and so forth. All of these benchmarks have no practical relevance at all, superpi is completely outdated (and inefficient btw). Or the ES are benched in games with a midrange graphics card at 1080p, moving the bottleneck towards (if not completely to) the GPU. You get the gist. No offence, but sometimes I'm under the impression that only morons get engineering samples (or people who just want to antagonize the rest of us with meaningless benches).

    Why is that? Why not test real stuff like video encoding, rendering, cpu-intensive games/settings (with the use of savegames, not timedemos, which most integrated benchmarks are).

  2. #2
    Xtreme Addict
    Join Date
    May 2007
    Location
    Romania
    Posts
    1,245
    Because you are on goddamn xtremesystems.org.Do you know how to spell it?Main focus here is benchmarks,not your encoding,rendering and other stuff.So remember,xtremesystems.org=overclocking+benchmarks mainly.That's why you don't see games and other stuff tested very often.

  3. #3
    Xtreme Addict
    Join Date
    Jun 2006
    Location
    The O.C.
    Posts
    1,452
    In other words Alex-Ro recommends you look on Anandtech for real stuff testing you seek
    i7-2600k L041C108 4.8ghz 1.32v PLL off Venomous-X Push/Pull http://www.xtremesystems.org/forums/...postcount=1063
    Asus P8P67 Vanilla
    Samsung 30nm MV-3V4G3D/US 2x4GB @ 9-10-10-28 1T DDR3 2133 1.6v http://www.techpowerup.com/forums/sh...d.php?t=159320
    Diamond HD 7970
    WD 600GB Velociraptor
    Corsair TX750W
    CM 690 II Advanced

    Q822A549 E8500 @ 4.5ghz air TRUE @ http://www.xtremesystems.org/forums/...d.php?t=203762

    G.Skill F3-12800CL6D-4GBXH @ 7-10-8-27 1T DDR3 2133 1.6v http://www.xtremesystems.org/forums/...d.php?t=266839

  4. #4
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    Other sites get their samples mostly when NDA falls. That's the trouble

  5. #5
    Xtreme Addict
    Join Date
    Mar 2009
    Posts
    1,116
    Quote Originally Posted by boxleitnerb View Post
    sometimes I'm under the impression that only morons get engineering samples
    maybe only the morons leak?

  6. #6
    I am Xtreme
    Join Date
    Sep 2006
    Posts
    10,379
    Indeed we the m*rOns test what is relevant to our community. Superpi outdated you say, maybe to you. It still gives a good indication how ram speed, timings scale on a new platform. In case of the AMD FX CPU it clearly gave away that the CPU is underperforming at single threaded apps.
    Game benchmarks, how at full maxed out detail ? So you max out the GPU and the CPU nor platform has got hardly any influence in the outcome... if that is real to you... plz visit hardwareheaven or other "affiliated I test as the hardware vendor like it" sites... If you want to show the CPU influence you have to test at either lower res and then at higher one. In the end most games will be GPU bottlenecked and the outcome will be pretty much the same...

    Good reliable all round site are hardwarecanucks, techspot and co... hardwareheaven and co are not on my most appreciated list...

    Most of the m*r*ns here are more power users/beta testers to debug stuff and find/solve issues/workarounds...
    Last edited by Leeghoofd; 10-22-2011 at 03:07 AM.
    Question : Why do some overclockers switch into d*ckmode when money is involved

    Remark : They call me Pro Asus Saaya yupp, I agree

  7. #7
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    Well, the famous superpi IS outdated, it's code is stoneold. Also it distorts differences between cpus quite a bit. There, a cpu may be 100% faster than it's competitor, in normal singlethreaded apps the lead will be significantly less. So it's not at all accurate between different architectures from different IHVs. Also, go with the time. Software changes - you don't bench games from 2000, do you?

    With the game tests I can just repeat myself:

    A cpu test where you are gpu bottlenecked is useless, because it doesn't provide any relevant information. You say "most games will be GPU bottlenecked", that may be true to a certain extent. However, look further. SLI/CF, newer GPUs, cpu intensive games (strategy, simulations), avoiding framedrops.

    I didn't mean any disrespect, I'm just so frustrated at the lack of creativity out there. Every single time the same (often meaningless) benchmarks are run. How about something else for a change that actually gives a more accurate indication of what to expect from a new cpu?

  8. #8
    Xtreme Addict
    Join Date
    May 2007
    Location
    Romania
    Posts
    1,245
    btw leeghoofd your review of bulldozer rocked It was just perfect,everything i wanted to know about cpu-nb,memory and cpu scaling was there.Good job

  9. #9
    Xtreme Enthusiast TJ TRICHEESE's Avatar
    Join Date
    May 2009
    Posts
    784
    It seems like what you've said is directed at one specific user on XS as chess benchmarks rarely come up here and usually when testing es cpu's people usually only focus on how the cpu performs and don't test how it performs in a graphics intensive environment.

  10. #10
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    I didn't have a specific user in mind. I'm talking generally. For example the leaks at corescn.

  11. #11
    Xtreme Enthusiast TJ TRICHEESE's Avatar
    Join Date
    May 2009
    Posts
    784
    To me your original post looks like its very much aimed at user 'windwithme' as you mentioned chess benchmarks... as i said they literally never come up here (maybe very rarely)

  12. #12
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    I don't even know this user. And I'm not only talking about leaks that come up here at XS but all over the web. Cinpai or what this site is called, the asian leaks and so on.

  13. #13
    Xtreme Enthusiast TJ TRICHEESE's Avatar
    Join Date
    May 2009
    Posts
    784
    Quote Originally Posted by boxleitnerb View Post
    Other sites get their samples mostly when NDA falls. That's the trouble

  14. #14
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    So? I know that really reliable result come in only at launch. However, everytime I see someone has an ES I think "bench this, bench that, DO something with that thing" and then I see...superpi.
    If you cannot understand that, I cannot help you

  15. #15
    V3 Xeons coming soon!
    Join Date
    Nov 2005
    Location
    New Hampshire
    Posts
    33,457
    Quote Originally Posted by boxleitnerb View Post
    So? I know that really reliable result come in only at launch. However, everytime I see someone has an ES I think "bench this, bench that, DO something with that thing" and then I see...superpi.
    If you cannot understand that, I cannot help you
    An easy solution to your problem:
    As them "nicely" to run X test and maybe they will.
    Remember the old line" You get more with honey than you do with vinegar"
    Crunch with us, the XS WCG team
    The XS WCG team needs your support.
    A good project with good goals.
    Come join us,get that warm fuzzy feeling that you've done something good for mankind.

    Quote Originally Posted by Frisch View Post
    If you have lost faith in humanity, then hold a newborn in your hands.

  16. #16
    XIP
    Join Date
    Nov 2004
    Location
    NYC
    Posts
    5,555
    A reviewer and a bencher usually not the same person

  17. #17
    I am Xtreme
    Join Date
    Sep 2006
    Posts
    10,379
    At least you liked it :p That's the main thing that interests me on new architectures : ram choice, timings, speeds and how to get a little bit more perf out of it... I have no time to do 25 different game tests at 10 resolutions. I leave that to the real reviewers....
    Question : Why do some overclockers switch into d*ckmode when money is involved

    Remark : They call me Pro Asus Saaya yupp, I agree

  18. #18
    Xtreme Mentor
    Join Date
    Aug 2008
    Location
    Australia
    Posts
    3,000
    Quote Originally Posted by boxleitnerb View Post
    This bothers me everytime a new CPU is in the works and people get their hands on the chips early and leak stuff.
    I suggest you build a bridge. There are sites that do this kind of review, use them.

    Quote Originally Posted by boxleitnerb View Post
    Things tested are superpi, wprime, chess benchmark and so forth. All of these benchmarks have no practical relevance at all, superpi is completely outdated (and inefficient btw).
    That's because for the most part, the people on XS with these connections able to get pre-release samples are not interested in video encoding, rendering or cpu-intensive games/settings (with the use of savegames, not timedemos). They test what they are interested in, and what the XS community is interested in.

    In case you haven't realized, some of the above named 'useless benches' to you are often held in quite high esteem here. The fact that SuperPI benching has it's own section on the forum may help you understand. Young PC users that find there way here "cos it's sounds awesome and extreme yo" tend to find out pretty rapidly we aren't focused on typical computer uses here. Kingpin doesn't gloat to his pre-pubescent mates on msn about how fast his 6.5GHz gulftown is encoding the latest pirate version of Harry Potter and the this_is_not_anandtech.com. We bench, so those with the ES's bench them. Pretty straightforward.

    Quote Originally Posted by boxleitnerb View Post
    No offence, but sometimes I'm under the impression that only morons get engineering samples (or people who just want to antagonize the rest of us with meaningless benches). Why is that?
    No offense, but calling some of our most respected members who do us all the service of testing the latest and greatest morons is not going to win you any favours here. 'The rest of you' are in the minority here, so you are unlikely to be pandered to. Accept it and move on.

    Quote Originally Posted by boxleitnerb View Post
    Why not test real stuff like video encoding, rendering, cpu-intensive games/settings (with the use of savegames, not timedemos, which most integrated benchmarks are).
    XS is not generally concerned with real stuff. Real stuff is boring. I have a slow boring PC to do real stuff on, as do most of us here. When I go into a ES testing thread, I want to see as little real stuff as possible, preferably none at all.
    Last edited by CryptiK; 10-24-2011 at 12:33 AM.
    Ci7 990X::Rampage III Extreme::12GB Corsair Dominator 1866C7GT::2 x EVGA SC Titans in SLI::Corsair AX1200::TJ07::Watercooled
    Ci7 920 3849B018::Rampage II Extreme::6GB GSKILL Trident 2000C9 BBSE::EVGA GTX580::Antec Signature SG850::TJ09::Aircooled w/TRUE 120X

  19. #19
    I am Xtreme Ket's Avatar
    Join Date
    Apr 2004
    Location
    United Kingdom
    Posts
    6,806
    I think the OP just wants tests that are more relevent to the hardware being tested. For example; if testing a new Quad Core CPU or something like that, Cinebench is likely going to be a much more acceptable test to run than SuperPI.

    "Prowler"
    Asrock Z77 Extreme6 | i5 3570k @ 4.5GHz 1.18v | 2x4GB PC12800 Ripjaws @ 1866MHz 10-10-10-15 1T 1.55v | Gigabyte GTX980, custom cooler @ 1.5GHz / 7.6GHz | Xonar DX 7.1 | 2TB Barracuda | 2x Sony Optiarc | Corsair 850w HX

    Cooling:

    3x Coolermaster Sickleflow 120mm 70CFM fans | Modified CoolIT ECO sealed loop CPU cooler | Custom made "The Judge" VGA VRM heatsink

    Asrock Z77 thread! | Asrock Z77 Extreme6 Review | Asrock P67 Extreme4 Review | Asrock P67 Extreme4/6 Pro3 thread | Asrock Z68 Extreme4 thread | Asrock Z68 Extreme4 Review | Asrock Z68 Gen3 Thread | 8GB G-Skill review | TK 2.ZERO homepage | P5Q series mBIOS thread

    (\_/) This is Bunny.
    (+.+) Bunny is dead.
    (^ ^) Copy and paste Bunny into your sig to create an army of BUNNY ZOMBIE MINIONS!!!

  20. #20
    Xtreme Addict
    Join Date
    Apr 2005
    Location
    Calgary, AB
    Posts
    2,252
    Sometimes it is
    Quote Originally Posted by Dumo View Post
    A reviewer and a bencher usually not the same person
    ps - PI > all
    MB Reviewer for HWC
    Team OCX Bench Team

  21. #21
    Xtreme Enthusiast
    Join Date
    Mar 2005
    Location
    Buenos Aires, Argentina
    Posts
    645
    Quote Originally Posted by CryptiK View Post
    That's because for the most part, the people on XS with these connections able to get pre-release samples are not interested in video encoding, rendering or cpu-intensive games/settings (with the use of savegames, not timedemos). They test what they are interested in, and what the XS community is interested in.
    You are totally wrong. The community is composed of two groups: The extreme overclocker, that most times I wouldn't call it a true enthusiast, just a kid in a toyshop that always opens a box with something new to play with, that just cares about meaningless benchmarks in an unidimensional way that would make Sledge Hammer jealous, and the all around enthusiast that cares about the pros and cons of everything that it touchs and its budged limited. The first group is always on the spotlight precisely because they always get first the expensive or unreleased toys that everyone wants to play with, but it doesn't make it the bigger one, so don't talk about the XS community as a whole because it isn't, and I don't feel represented by any of those guys. Seems that people that manage to get Engineering Samples and aren't bound to a NDA, or like to leak results anyways, are usually from the first group, because many times I saw benchmarks that are too poor to consider reaching conclusions with them. When I registered here some years ago I though that it would allow me to get closer to those guys that got the beta toys to get early data regarding if I should wait for X platform, Y architecture and Z Stepping, and my results in that regard are fairly limited. I always had more fun among regular enthusiasts that speculates that with ES users, that even if they know the truth and could leak a few more results, they wouldn't do a more in depth analysis on an actually useful fashion. And that pretty much kills the real life point of having access to ES parts without NDA if your only objetive is impressing people with SuperPI.

  22. #22
    I am Xtreme Ket's Avatar
    Join Date
    Apr 2004
    Location
    United Kingdom
    Posts
    6,806
    ^^ People can argue with that as much as they want, but theres a fairly good point made there.

    "Prowler"
    Asrock Z77 Extreme6 | i5 3570k @ 4.5GHz 1.18v | 2x4GB PC12800 Ripjaws @ 1866MHz 10-10-10-15 1T 1.55v | Gigabyte GTX980, custom cooler @ 1.5GHz / 7.6GHz | Xonar DX 7.1 | 2TB Barracuda | 2x Sony Optiarc | Corsair 850w HX

    Cooling:

    3x Coolermaster Sickleflow 120mm 70CFM fans | Modified CoolIT ECO sealed loop CPU cooler | Custom made "The Judge" VGA VRM heatsink

    Asrock Z77 thread! | Asrock Z77 Extreme6 Review | Asrock P67 Extreme4 Review | Asrock P67 Extreme4/6 Pro3 thread | Asrock Z68 Extreme4 thread | Asrock Z68 Extreme4 Review | Asrock Z68 Gen3 Thread | 8GB G-Skill review | TK 2.ZERO homepage | P5Q series mBIOS thread

    (\_/) This is Bunny.
    (+.+) Bunny is dead.
    (^ ^) Copy and paste Bunny into your sig to create an army of BUNNY ZOMBIE MINIONS!!!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •