Crysis Benchmarking - Methodology and Mythbusting

[WeStSiDePLaYa]

Now with my system, I get 40FPS with the gpu benchmark with HP's ultra high quality config. Turns out the 168.25 drivers were the problem.

[Warboy]

Now with my system, I get 40FPS with the gpu benchmark with HP's ultra high quality config. Turns out the 168.25 drivers were the problem.

yea, to get SLi in crysis you atleast need 169.28s

[Gen]

Q6600 @3.6 2gb
8800GTX 621/1566/900 171.16

I like this benchamrk, thanks


<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/12/2008 7:12:57 PM - Vista 64

Run #1- DX10 1024x768 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 32.35





<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/12/2008 7:26:50 PM - Vista 64

Run #1- DX10 1680x1050 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 18.465

[Hockster]

Q6600 @ 3.0
XFX 8800 GTX stock clocks
2GB 1066 ram

2/13/2008 2:06:42 AM - XP
Beginning Run #1 on Map-island, Demo-benchmark_gpu
DX9 1900x1200, AA=No AA, Vsync=Disabled, 32 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: High
================================================== ============
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 111.86s)
!TimeDemo Run 0 Finished.
Play Time: 79.96s, Average FPS: 25.01
Min FPS: 14.57 at frame 149, Max FPS: 30.03 at frame 1006
Average Tri/Sec: -23326852, Tri/Frame: -932645
Recorded/Played Tris ratio: -0.98
!TimeDemo Run 1 Finished.
Play Time: 76.08s, Average FPS: 26.29
Min FPS: 14.57 at frame 149, Max FPS: 30.35 at frame 988
Average Tri/Sec: -24197782, Tri/Frame: -920491
Recorded/Played Tris ratio: -1.00
!TimeDemo Run 2 Finished.
Play Time: 76.07s, Average FPS: 26.29
Min FPS: 14.57 at frame 149, Max FPS: 30.43 at frame 993
Average Tri/Sec: -24191418, Tri/Frame: -920163
Recorded/Played Tris ratio: -1.00
TimeDemo Play Ended, (3 Runs Performed)
================================================== ============

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/13/2008 2:06:42 AM - XP

Run #1- DX9 1900x1200 AA=No AA, 32 bit test, Quality: High ~~ Overall Average FPS: 26.29

[jcool]

Could you try a run with the Sphere level and the same settings? I'd like to see my superclocked GTS vs. your GTX since we're running both DX9 - high - fullHGD res

[jas420221]

First, let me say excellent post about benchmarking Crysis!!!! Great job!

But I have a problem with this statement below.

Today on market there are NO multicore optimized games! There are only dualcore optimized titles, and Cryris is one of them! Only use of multicore CPU's with this games is that you'll be able to run some parallel tasks (preferably on another HDD), during the gaming session!

Doesnt Supreme Commander use 4 cores? Lost Planet? Also, I have seen Crysis use 4 cores as well. They were not all maxed out, but there was significant CPU usage on cores 3 and 4.

There are a few games out already that can utilize more than 2 cores.

Let me see if I can find those threads.
Some Valve games do

I guess the key word is OPTIMIZED. There are a few games that can use more than 2 cores, but none are OPTIMIZED to do so.

[Governator]

Test was done with 169.28 driver set & no graphic card oc'ing...

Beginning Run #1 on Map-island, Demo-benchmark_gpu
DX9 1680x1050, AA=16x, Vsync=Disabled, 32 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: High
================================================== ==========
==
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 111.86s)
!TimeDemo Run 0 Finished.
Play Time: 70.37s, Average FPS: 28.42
Min FPS: 18.33 at frame 1960, Max FPS: 38.77 at frame 974
Average Tri/Sec: -19611028, Tri/Frame: -690041
Recorded/Played Tris ratio: -1.33
!TimeDemo Run 1 Finished.
Play Time: 66.23s, Average FPS: 30.20
Min FPS: 17.84 at frame 1936, Max FPS: 41.83 at frame 91
Average Tri/Sec: -20788814, Tri/Frame: -688420
Recorded/Played Tris ratio: -1.33
!TimeDemo Run 2 Finished.
Play Time: 65.67s, Average FPS: 30.46
Min FPS: 17.49 at frame 1943, Max FPS: 43.32 at frame 90
Average Tri/Sec: -20983054, Tri/Frame: -688942
Recorded/Played Tris ratio: -1.33
TimeDemo Play Ended, (3 Runs Performed)
================================================== ==========
==

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/11/2008 11:37:56 AM - XP
Run #1- DX9 1680x1050 AA=16x, 32 bit test, Quality: High ~~ Overall Average FPS: 30.33

************************************************** ************************************************

The, for kicks tried the 171.16 drivers, needless to say pretty ugly...

Update: Turned out SLI was disabled, though I could've swore it was enabled but anyhow below this is the result with it enabled ...

2/13/2008 9:27:24 AM - XP
Beginning Run #1 on Map-island, Demo-benchmark_gpu
DX9 1680x1050, AA=16x, Vsync=Disabled, 32 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: High
================================================== ============
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 111.86s)
!TimeDemo Run 0 Finished.
Play Time: 101.13s, Average FPS: 19.78
Min FPS: 14.11 at frame 1952, Max FPS: 25.37 at frame 980
Average Tri/Sec: -18566552, Tri/Frame: -938784
Recorded/Played Tris ratio: -0.98
!TimeDemo Run 1 Finished.
Play Time: 99.21s, Average FPS: 20.16
Min FPS: 14.11 at frame 1952, Max FPS: 25.37 at frame 980
Average Tri/Sec: -18694888, Tri/Frame: -927371
Recorded/Played Tris ratio: -0.99
!TimeDemo Run 2 Finished.
Play Time: 99.53s, Average FPS: 20.09
Min FPS: 13.27 at frame 1958, Max FPS: 25.37 at frame 980
Average Tri/Sec: -18635028, Tri/Frame: -927349
Recorded/Played Tris ratio: -0.99
TimeDemo Play Ended, (3 Runs Performed)
================================================== ============

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/13/2008 9:27:24 AM - XP

Run #1- DX9 1680x1050 AA=16x, 32 bit test, Quality: High ~~ Overall Average FPS: 20.125

************************************************** *******************************************

SLI Enabled now...

2/13/2008 9:41:10 AM - XP
Beginning Run #1 on Map-island, Demo-benchmark_gpu
DX9 1680x1050, AA=16x, Vsync=Disabled, 32 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: High
================================================== ============
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 111.86s)
!TimeDemo Run 0 Finished.
Play Time: 69.52s, Average FPS: 28.77
Min FPS: 18.82 at frame 156, Max FPS: 40.04 at frame 976
Average Tri/Sec: -19852152, Tri/Frame: -690026
Recorded/Played Tris ratio: -1.33
!TimeDemo Run 1 Finished.
Play Time: 64.88s, Average FPS: 30.83
Min FPS: 18.82 at frame 156, Max FPS: 42.52 at frame 82
Average Tri/Sec: -21208078, Tri/Frame: -687947
Recorded/Played Tris ratio: -1.33
!TimeDemo Run 2 Finished.
Play Time: 64.26s, Average FPS: 31.13
Min FPS: 17.47 at frame 1953, Max FPS: 43.98 at frame 87
Average Tri/Sec: -21397594, Tri/Frame: -687458
Recorded/Played Tris ratio: -1.33
TimeDemo Play Ended, (3 Runs Performed)
================================================== ============

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/13/2008 9:41:10 AM - XP

Run #1- DX9 1680x1050 AA=16x, 32 bit test, Quality: High ~~ Overall Average FPS: 30.98

[Nedjo]

Governator could you run "sphere" demo? in 1024x768 Very High DX10, so we can compare among our results...

[jas420221]

Nedjo... nothing about my post?

[Nedjo]

First, let me say excellent post about benchmarking Crysis!!!! Great job!

But I have a problem with this statement below.
Doesnt Supreme Commander use 4 cores? Lost Planet? Also, I have seen Crysis use 4 cores as well. They were not all maxed out, but there was significant CPU usage on cores 3 and 4.

There are a few games out already that can utilize more than 2 cores.

Let me see if I can find those threads.
Some Valve games do

I guess the key word is OPTIMIZED. There are a few games that can use more than 2 cores, but none are OPTIMIZED to do so.

I'm glad you like this thread.
I haven't tried Superme commander, so I might be wrong on it. Will try LP. Regarding crysis, my testing with 2/3/4 cores don't show any difference, so I'm inclined to think that mulitcore support in this game is just marketing hype.
Regarding Valve's games there are no confirmation in benchmarking that they're multithreaded. hopefully I'll get Orange Box soon to check out...

[jas420221]

This is all IMHO.......

I think the key word is optimized here. There are a few games (like the ones I listed) that show more than 2 core use. If that helps FPS, I dont know, but they use it. I think that is where we differ in our opinions.

[Governator]

Governator could you run "sphere" demo? in 1024x768 Very High DX10, so we can compare among our results...

Greets Nedjo, sorry I've been out of town for a few days, I'll try and do a run this afternoon for you

[Gen]

Q6600 @3.6 2gb
8800GTX 621/1566/900 171.16

Run #1- DX10 1024x768 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 32.35

2nd run with +99mhz memory (999mhz)

Run #1- DX10 1024x768 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 32.85

only .5 of a frame better? seems rather odd, anyone have similar results?

[jas420221]

I must be an idiot tonight... what is the exact path this goes in? I keep getting application failures... here is what my path looks like:

C:\Program Files\Electronic Arts\Crytek\Crysis

under that is BIN32, GAME, LOGBACUPS, PB, SUPPORT folders etc...

[Gen]

mine works in the game folder, you have the app and the dll right?

[wallywest]

What do you think are safe levels to run it for my hardware?
I have;
e6850 @ factory
8800 320 mb
4 gigs xms2
nothing special here.
I have been running at medium with no lag. My friends have been telling me to lay off the highs.
What do you think?

[bluehaze]

This is pretty cool thanks!

IP35Pro,Q6600@3.8,3870x2 stock on the Sphere level:



2/18/2008 1:00:07 AM - Vista 64
Beginning Run #1 on Map-sphere, Demo-autotest
DX10 1024x768, AA=No AA, Vsync=Disabled, 64 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: VeryHigh
================================================== ============
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 82.89s)
!TimeDemo Run 0 Finished.
Play Time: 78.09s, Average FPS: 25.61
Min FPS: 4.07 at frame 492, Max FPS: 44.18 at frame 792
Average Tri/Sec: -20467334, Tri/Frame: -799193
Recorded/Played Tris ratio: 0.83
!TimeDemo Run 1 Finished.
Play Time: 59.76s, Average FPS: 33.47
Min FPS: 4.07 at frame 492, Max FPS: 45.61 at frame 809
Average Tri/Sec: -26796758, Tri/Frame: -800683
Recorded/Played Tris ratio: 0.82
!TimeDemo Run 2 Finished.
Play Time: 59.80s, Average FPS: 33.44
Min FPS: 4.07 at frame 492, Max FPS: 46.10 at frame 807
Average Tri/Sec: -26776612, Tri/Frame: -800639
Recorded/Played Tris ratio: 0.82
TimeDemo Play Ended, (3 Runs Performed)
================================================== ============

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/18/2008 1:00:07 AM - Vista 64

Run #1- DX10 1024x768 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 33.455

[Nedjo]

Thanks Bluehaze for contribution!

So now we have direct comparison in terms of processing needed for this game in DX10 "eye-candy" mode:

Phenom @ 2.4 GHz + HD3870 X2 = 31.08 fps
vs.
Q6600 @ 3.8 GHz + HD3870 X2 = 33.45 fps

[bluehaze]

Something very strange is going on here LOL I have been having some bad slowdowns in parts of Crysis as evidenced from what I just posted you can see the minimum FPS dropp to 4.71 on my previous test. Well I accidentally ran the Crysis benchmark in 32 bit and noticed much higher minimum FPS so I started playing Crysis in 32bit DX10 and wow what a performance boost it was no slowdowns at all and I actually was able to increase settings.

Well now here is the strange part today I tried 64bit Crysis again and low and behold I no longer have the slowdowns and everything is running the same or better than 32bit.

Here is a bench for comparison, nothing changed except i loaded up Crysis in 32 bit and played before switching back to 64 bit, look at the minimum fps now...18 versus 5 What the hell? The whole game runs amazingly better now, I am very confused.


2/19/2008 1:09:15 AM - Vista 64
Beginning Run #1 on Map-sphere, Demo-autotest
DX10 1024x768, AA=No AA, Vsync=Disabled, 64 bit test, FullScreen
Demo Loops=3, Time Of Day= 9
Global Game Quality: VeryHigh
================================================== ============
TimeDemo Play Started , (Total Frames: 2000, Recorded Time: 82.89s)
!TimeDemo Run 0 Finished.
Play Time: 65.78s, Average FPS: 30.40
Min FPS: 18.70 at frame 115, Max FPS: 42.90 at frame 778
Average Tri/Sec: -24301898, Tri/Frame: -799327
Recorded/Played Tris ratio: 0.83
!TimeDemo Run 1 Finished.
Play Time: 60.02s, Average FPS: 33.32
Min FPS: 17.92 at frame 1769, Max FPS: 43.82 at frame 784
Average Tri/Sec: -26680642, Tri/Frame: -800660
Recorded/Played Tris ratio: 0.82
!TimeDemo Run 2 Finished.
Play Time: 59.59s, Average FPS: 33.56
Min FPS: 17.92 at frame 1769, Max FPS: 43.82 at frame 784
Average Tri/Sec: -26873494, Tri/Frame: -800690
Recorded/Played Tris ratio: 0.82
TimeDemo Play Ended, (3 Runs Performed)
================================================== ============

Completed All Tests

<><><><><><><><><><><><><>>--SUMMARY--<<><><><><><><><><><><><><>

2/19/2008 1:09:15 AM - Vista 64

Run #1- DX10 1024x768 AA=No AA, 64 bit test, Quality: VeryHigh ~~ Overall Average FPS: 33.44

[msgclb]

I was using the benchmarking tool in XP but when I first tried it with Vista64 it didn't work. I don't know what I did but I got it to run the sphere TimeDemo. This thread sure has help me a lot.



I guess it's time to get another card.

[KTE]

First, let me say excellent post about benchmarking Crysis!!!! Great job!

But I have a problem with this statement below.
Doesnt Supreme Commander use 4 cores?

You're right. Some games can use more than two cores, but it has minimal benefit over a dual core at the same MHz, since the amount of the 2+ cores they use is minimal with very little optimization to effectively make use of its resources, hence dual core actually tends to lambaste quad/octa overall.

Check out this SupCom benchmark: http://techreport.com/articles.x/14052/7



The 9775 is SkullTrail with 8x 3.2GHz cores. Those results look ridiculous considering the clockspeeds and the number of cores, but I think it does show how SupCom isn't actually multi-core optimized at all, although it does sparingly use more than 2-cores.

And if you look at Crysis with Penryn dual-core (6.32GHz) Vs quad-core (10GHz): http://www.xbitlabs.com/articles/cpu...0_5.html#sect0



That shows clockspeed on two cores in games rules rather than >3 cores. Factor in that E8500 clocks much higher than Q9300 or Q9450.

I guess the key word is OPTIMIZED. There are a few games that can use more than 2 cores, but none are OPTIMIZED to do so.

Yup, I have to agree. If any software can make complete and effective use of each architecture, loading separate libraries for different processors archs, you would see the real power of each core utilized. Right now, you don't as coding is generalized.

Great work Nedjo! Now do something, turn off one core and then two cores using Windows using the same benchmark and see what results you get, if any different.

[Nedjo]

Great work Nedjo! Now do something, turn off one core and then two cores using Windows using the same benchmark and see what results you get, if any different.

You think it'll differ from core disabling with in BIOS?

[KTE]

If you've got the ASUS RD790 which can disable the core within the BIOS then use that feature. It works just fine, better than the in-windows disabling.

[Nedjo]

If you've got the ASUS RD790 which can disable the core within the BIOS then use that feature. It works just fine, better than the in-windows disabling.

that's exactly what I've done in my second post

[KTE]

that's exactly what I've done in my second post

Sorry Nedjo, when I had read the post a week back and last night, the pictures in the 2nd post hadn't turned up so I thought you may have missed it.

They show up now and show what I was expecting quite clearly.