Benchmark Ideas here!

[dippyskoodlez]

Ok.. wanting to start a thread for ideas..

Several 3d runs... an opengl and direct X would be good, atleast one of each.. probably 2 would be best? maybe more? then maybe incorporate come CPU scoring things in.. lol :p:

As for the 3d runs, we gotta have a ninja... beating the crap out of people all matrixy like... :woot:

[skate2snow]

Yes, one of each is need IMO. Tweaks that works in OpenGl may not affect Dx and vis-versa.

And a database on XS would be very great with different "sections" for individual scores AND overall scores.

[Lestat HWL]

Yes, we will be writing an ORB-type comparison script.

[matt9669]

- Cross-linking the DB for processors, graphics and memory (if possible). A long-term project obviously, but ideally users would be able to quickly check if they had the highest score for a given CPU/GPU/RAM, compare with other owners of the same/similar hardware.

- Temp logging for the run, should allow users to spot quickly if the parts are air/water/extreme cooled. Possibly even sections or DB breakdown by cooling type.

- Ability to run only specific tests, batch run with resolution/AA/AF/possibly even integration with Clockgen/Coolbits/ATITool (or other GPU clock tool) for batches on different frequencies, infinite runs for stress-testing purposes.

- Some way to simulate a level load, would take into account the HD subsystem's effect on overall performance.

- Heavily multi-threaded CPU bench, for comparison of HT/multicore/SMP platforms.

- Comparison of runs using specific optimizations (i.e. SSE3 vs non-SSE3 Pi, etc.)

- Complete system config breakdown, including driver revisions, drive configurations, even possibly monitor/mouse/keyboard for the curious ;)

I live in an ideal world don't I :D

[dippyskoodlez]

1 Temp logging for the run, should allow users to spot quickly if the parts are air/water/extreme cooled. Possibly even sections or DB breakdown by cooling type

2 Comparison of runs using specific optimizations (i.e. SSE3 vs non-SSE3 Pi, etc.)


1 will probably be very hard to keep working with the everchaning hardware.. and variety..

2- I think we should have the main UI kinda like the 3dmark one, but have one button do a "scoring run" and other buttons for specific tests, without allowing a scored run to change the detail/resolution. :D

[STEvil]

What kind of tests are planned?

We will need tests such as huge overdraw, large terrain rendering (aka Farcry), a physics test (ever played "Truck Dismount" ????), tests to cover excrutiating numbers of polygons, and many smaller tests like normal maps, bump maps, texture compression speed (efficiency), etc etc etc...

Can we bring the voxel back? :D:D

[matt9669]

I think we should have the main UI kinda like the 3dmark one, but have one button do a "scoring run" and other buttons for specific tests, without allowing a scored run to change the detail/resolution. :DCertainly there will need to be something to this effect . . .

[jrw]

Possibility for differentiation between different cooling solutions in the database...

A cool, though most likely impossible thing, would be integrating the database with newegg -> point(or fps)/$$ index :D

[dippyskoodlez]

Possibility for differentiation between different cooling solutions in the database...

A cool, though most likely impossible thing, would be integrating the database with newegg -> point(or fps)/$$ index :D


Actually, I've been working ona somewhat similar creation in VB-- its a program that will allow you to create a computer using drop downs like ibuypower.com's website, but total up the prices from newegg, and possibly even a random computer generator to go with it. May talk with lestat about it for possible incorporation in some way.. I have a ton of ideas for this project. :woot:

[jrw]

Actually, I've been working ona somewhat similar creation in VB-- its a program that will allow you to create a computer using drop downs like ibuypower.com's website, but total up the prices from newegg, and possibly even a random computer generator to go with it. May talk with lestat about it for possible incorporation in some way.. I have a ton of ideas for this project. :woot:


:banana:

[dippyskoodlez]

:banana:

only problem is I suck at programming, and I lack motivation altogether :p:

[bh2k]

I got the website and it's programming down ;o

[Geforce4ti4200]

we should make a benchmark similar to 3dmark and we can charge for premium features. include a bunch of tests, perhaps 3 of each dx7,8,9. extra features could be running different resolutions, using aa/af running some optional tests and perhaps submitting large number of projects. Make the benchmark very cpu intensive, in between and very gpu intensive. balanced like 2001 except perhaps more than 1 gpu intensive test unlike 2001 where 6 of 7 is cpu intensive

[masterofpuppets]

Make both a Windows build and a Linux build for all the Tux lovers around here! The latter lacking a D3D test. Maybe make a little SuperPi like test as well. And if you can be bothered, be sure to test pixel shaders in the D3D test, that would certainly own. In the GL and D3D tests, incorporate a CPU intensive physics engine too, then we can judge gaming performance, since most games require a decent CPU on top of a graphics card.

[eshbach]

Make both a Windows build and a Linux build for all the Tux lovers around here! The latter lacking a D3D test. Maybe make a little SuperPi like test as well. And if you can be bothered, be sure to test pixel shaders in the D3D test, that would certainly own. In the GL and D3D tests, incorporate a CPU intensive physics engine too, then we can judge gaming performance, since most games require a decent CPU on top of a graphics card.

our current solution for the linux/d3d/ogl situation is as follows:

all "game" tests will be in dx9 as well as ogl. for windows users, both versisons of each test will run and either the average score or the highest score will be used. for linux users only the opengl will run and that will count as the scoring run.

[[XC] moddolicous]

I would like to see a benchmark that really stresses the memory, something like a memtest benchmark thing, but as a benchmark.

[Fzero]

maybe something that will test like prime and serve as a burnin with options to do PI and CPU mark kinda like sandra but full of graphics benchmarks super pi all the things average benchmarkers use