Page 1 of 3 123 LastLast
Results 1 to 25 of 59

Thread: [PCPer] Frame Rating Part 3: First Results

  1. #1
    Xtreme Mentor
    Join Date
    May 2008
    Posts
    2,550

    [PCPer] Frame Rating Part 3: First Results

    PCPer has been looking into some new ways of testing smoothness. They just released an article on their process so far with some very interesting results.

    If you haven't been following our sequence of stories that investigates a completely new testing methodology we are calling "frame rating", then you are really missing out. (Part 1 is here, part 2 is here.) The basic premise of Frame Rating is that the performance metrics that the industry is gathering using FRAPS are inaccurate in many cases and do not properly reflect the real-world gaming experience the user has.

    Because of that, we are working on another method that uses high-end dual-link DVI capture equipment to directly record the raw output from the graphics card with an overlay technology that allows us to measure frame rates as they are presented on the screen, not as they are presented to the FRAPS software sub-system. With these tools we can measure average frame rates, frame times and stutter, all in a way that reflects exactly what the viewer sees from the game.
    Link

  2. #2
    Xtreme Mentor
    Join Date
    Feb 2008
    Posts
    2,625
    Interesting, I was doing some more reading on this issue regarding AMD crossfire, and apparently Radeon Pro can fix it:

    http://www.tomshardware.com/reviews/...2,3329-11.html
    Intel I7 980 + Phanteks PH-TC14PE Red, Asus Rampage III Extreme, 12 GB Geil Ultra Series
    SLI MSI GTX 680 power edition, Creative X-Fi Titanium HD, 2x4 Tb Seagate HDD, 128 Gb Crucial M4, 512 Gb Crucial M4
    Gigabyte M8000X, Microsoft Sidewinder X4, Antec P182, Acer G24, Windows 7 Professional, XFX Proseries 850w XXX

    Keeping my mobo + CPU till they are obsolete

    Copper loop X48 build Blue X58 Build Red X58 build

  3. #3
    Xtreme Mentor
    Join Date
    May 2008
    Posts
    2,550
    Yeah, Toms just used a framerate cap. Sort of like when people use to use fps limiter in the 4870x2 days. It works great at eliminating stutter but when the framerate dips below that cap you'll still get microstutter.

    Its a great workaround but not really a solution.

  4. #4
    c[_]
    Join Date
    Nov 2002
    Location
    Alberta, Canada
    Posts
    20,447
    Quote Originally Posted by BababooeyHTJ View Post
    Yeah, Toms just used a framerate cap. Sort of like when people use to use fps limiter in the 4870x2 days. It works great at eliminating stutter but when the framerate dips below that cap you'll still get microstutter.

    Its a great workaround but not really a solution.
    Tried dynamic vsync option?
    Heatware || 01.01.08; AND 111.2%

    Dead Z7S-WS? Click!. || XS Z7S-WS Thread || Current Dead Asus Z7S-WS count: 26+ ($15,000 in dead motherboards).
    All along the watchtower the watchmen watch the eternal return.
    Want to use my Anti-asus logo? Go ahead, but use this link please!: http://i853.photobucket.com/albums/a...sus/noasus.gif
    Bring back the game. http://reclaimyourgame.com/. EA are mean.

  5. #5
    Xtreme X.I.P. Particle's Avatar
    Join Date
    Apr 2008
    Location
    Kansas
    Posts
    3,218
    This testing method is great for testing AFR, but it will be of limited usefulness for testing scissor or tiling. Personally, I'm a tiling fan even though it has the lowest scaling.
    Particle's First Rule of Online Technical Discussion:
    As a thread about any computer related subject has its length approach infinity, the likelihood and inevitability of a poorly constructed AMD vs. Intel fight also exponentially increases.

    Rule 1A:
    Likewise, the frequency of a car pseudoanalogy to explain a technical concept increases with thread length. This will make many people chuckle, as computer people are rarely knowledgeable about vehicular mechanics.

    Rule 2:
    When confronted with a post that is contrary to what a poster likes, believes, or most often wants to be correct, the poster will pick out only minor details that are largely irrelevant in an attempt to shut out the conflicting idea. The core of the post will be left alone since it isn't easy to contradict what the person is actually saying.

    Rule 2A:
    When a poster cannot properly refute a post they do not like (as described above), the poster will most likely invent fictitious counter-points and/or begin to attack the other's credibility in feeble ways that are dramatic but irrelevant. Do not underestimate this tactic, as in the online world this will sway many observers. Do not forget: Correctness is decided only by what is said last, the most loudly, or with greatest repetition.

    Rule 3:
    When it comes to computer news, 70% of Internet rumors are outright fabricated, 20% are inaccurate enough to simply be discarded, and about 10% are based in reality. Grains of salt--become familiar with them.

    Remember: When debating online, everyone else is ALWAYS wrong if they do not agree with you!

    Random Tip o' the Whatever
    You just can't win. If your product offers feature A instead of B, people will moan how A is stupid and it didn't offer B. If your product offers B instead of A, they'll likewise complain and rant about how anyone's retarded cousin could figure out A is what the market wants.

  6. #6
    Xtreme Mentor
    Join Date
    May 2008
    Posts
    2,550
    Quote Originally Posted by STEvil View Post
    Tried dynamic vsync option?
    The Tom's article was wrong. They were using dynamic framerate control, not dynamic vsync. Look at the Radeon pro screenshots that they were using. Dynamic vsync is similar to Nvidia's adaptive vsync. It just turns vsync off when it dips under your refresh rate. Very cool but doesn't help microstutter.

    Dynamic framerate control afaik is just a hard cap like the old fpslimiter program. That I use all of the time but again its a workaround and not a real solution since if the framerate dips under that cap frame latency will become an issue again.


    On that topic. I really do think that if AMD figures out a framerate smoothing option similar to what Nvidia uses it'll fix both the single card and afr stutter. If you look at techreport's old reviews with AMD's 6xxx series and especially 5xxx series they did very poorly with frametime consistency. What? The memory management system wasn't optimized for those cards either? I'm not buying into that excuse.
    Last edited by BababooeyHTJ; 02-24-2013 at 04:51 PM.

  7. #7
    Xtreme Addict
    Join Date
    Mar 2009
    Posts
    1,116
    very damning. in bf3 at least, crossfire is garbage. the next time someone suggests crossfire, show them how it skips down to non-crossfire performance, making it pointless: http://www.pcper.com/image/view/2096...n=node%2F56621

  8. #8
    Xtreme Mentor
    Join Date
    Feb 2008
    Posts
    2,625
    Only reason I was suggesting crossfire recently was because of the prices on 7950s / 7970s compared to Titans and even GK104s.

    Seems like the only decent high end is dual GK104s or a Titan atm then.
    Intel I7 980 + Phanteks PH-TC14PE Red, Asus Rampage III Extreme, 12 GB Geil Ultra Series
    SLI MSI GTX 680 power edition, Creative X-Fi Titanium HD, 2x4 Tb Seagate HDD, 128 Gb Crucial M4, 512 Gb Crucial M4
    Gigabyte M8000X, Microsoft Sidewinder X4, Antec P182, Acer G24, Windows 7 Professional, XFX Proseries 850w XXX

    Keeping my mobo + CPU till they are obsolete

    Copper loop X48 build Blue X58 Build Red X58 build

  9. #9
    Xtreme Mentor
    Join Date
    May 2008
    Posts
    2,550
    Crossfire is still very useful. When you can buy thee 7950s for less than the price of on Titan and then pop them onto a mid range board like the UD5h you can get a lot of power for your money. As long as you cap the framerate performance is smooth. Even then microstutter isn't always consistent so you won't see it in every game or all of the time in the games that you do.

    This is definitely an issue that AMD needs to address and I'm glad that some review sites are placing more emphasis on smoothness.

  10. #10
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    Sure, but at what cost? I've been using a frame cap in the past when my experience with SLI was still very new, but the drawbacks are just too severe.

    Because you need to cap fps at or around your minimum fps, you will waste a lot of performance in scenes where those minimum fps are exceeded. So basically most of the time. For example when playing Skyrim with SGSSAA, my fps could dip to the low 40s in dense foliage and forests, which was stuttery. I capped my fps to 40 then, but that meant losing a full 20 fps (I play at 60 Hz vsync) during most of my gameplay. I found that quite unacceptable and it can utterly destroy scaling, calling going with multiple GPUs into question in the first place. Value doesn't look that good, then.
    The only thing I really like about RadeonPro and its fps cap is that you can adjust the value via hotkey while gaming. Doesn't solve the underlying problem, but it's more comfortable than having to alt-tab out of the game, changing the value and hoping the game didn't crash due to the task switch.
    Last edited by boxleitnerb; 02-25-2013 at 04:00 AM.

  11. #11
    Xtreme Mentor
    Join Date
    Feb 2008
    Posts
    2,625
    One of the main reasons I picked Nvidia this generation was adaptive Vsync and no longer having to have FPS capped to 30 when running Vsync.

    I'm not sure how frame rate limiting works on AMD cards, but if it locks the FPS to anywhere below 45, then that's not good enough for me because I see stutter up to around 40 fps. I remember back when I had SLI I thought that the stutter I was seeing around 30-40 FPS was due to microstutter, but its still the same on a single 680 when I force my FPS to 35 for video capture..

    And the prices on GTX 680s are still ridiculous, IMO if you need more than one then just get a Titan.
    Intel I7 980 + Phanteks PH-TC14PE Red, Asus Rampage III Extreme, 12 GB Geil Ultra Series
    SLI MSI GTX 680 power edition, Creative X-Fi Titanium HD, 2x4 Tb Seagate HDD, 128 Gb Crucial M4, 512 Gb Crucial M4
    Gigabyte M8000X, Microsoft Sidewinder X4, Antec P182, Acer G24, Windows 7 Professional, XFX Proseries 850w XXX

    Keeping my mobo + CPU till they are obsolete

    Copper loop X48 build Blue X58 Build Red X58 build

  12. #12
    Quote Originally Posted by bamtan2 View Post
    very damning. in bf3 at least, crossfire is garbage. the next time someone suggests crossfire, show them how it skips down to non-crossfire performance, making it pointless: http://www.pcper.com/image/view/2096...n=node%2F56621
    You are reading the chart completely wrong.

    The variation you see on the Crossfire solution is ALL below 20ms, making it literally imperceptible to the end user.

    Honestly, without perspective people can take these charts and have them completely misrepresent the information being provided. A card can do the bloody hokey pokey all over the place below 20ms and it wouldn't make one lick of difference since the actual REAL WORLD experience would still read as smooth.

  13. #13
    Xtreme Addict
    Join Date
    Sep 2010
    Location
    US, MI
    Posts
    1,672
    20ms is slightly noticeable...
    I know this from working with directsound buffers.
    But forget about all that.

    1000ms in a sec / 20ms of lag = 50fps max...
    Does this 20ms include the monitor lag?, I don't know I should probably check the link lol.
    I'm guessing it does.

    Lets pretend for a sec here.
    2x sli or cfx = 20ms of lag.
    4x sli or cfx = 40ms of lag, or 25fps max per sec...
    That's not good lol lmao...

    I notice a little bit of "input" lag on a single monitor in farcry3 and dead island (dead island is way bad sometimes when driving, probably not the video card, probably my usb or something).


    CPU: Intel Core i7-4930k [3332B054]
    CPU Heatsink: NZXT Kraken X60
    Mobo: ASUS Rampage IV Black Edition (0507 BIOS)
    MEM: 4x 4gig G.Skill [F3-2400C9Q-16GTXD] (Dual Sidded Samsung of Some Sorts)
    PSU: Antec HCP-850 Platinum
    VGA: nVidia Grid VGX K2 (WIP with 2x 680 lightnings)
    Monitor: ASUS VG278HE & 3DVision2 Kit
    Drives: 2x WD Red 2TB Drives [WDBMMA0020HNC-NRSN] (RAID 0), Corsair Force Series 3 CSSD-F60GB3A-BK 60GB SATA3 SSD, Pioneer BDR-2208 Blue Ray Burner
    Case: Rosewill Blackhawk Ultra
    OS's: Windows 2003 Enterprise x86 SP2-R2 VLM, Windows 7 Standard Embedded x86 SP1 EVL (128GB Patched), Windows 7 Standard Embedded x64 SP1 EVL
    Mouse: Using generic mouse for now...
    Arm:Samsung Galaxy S IV i9500 Octo-Core Quad Band 16gb Unlocked and 2x 64gig sd cards
    777

    Sig is slightly out of date, I'll update it later ^^


  14. #14
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    SKYMTL,
    do you have actually tried SLI/CF for gaming, not just for benchmarking? These variations can and do affect playability depending on how perceptive you are. How can you make your absolute statements for all people?

  15. #15
    Quote Originally Posted by boxleitnerb View Post
    SKYMTL,
    do you have actually tried SLI/CF for gaming, not just for benchmarking? These variations can and do affect playability depending on how perceptive you are. How can you make your absolute statements for all people?
    Again, you are making a blanket statement.

    Do rapid fluctuations affect gaming? YES. However, the microstutter you see is likely NOT below 20ms but rather higher than that. Typically when deltas of 10ms or more quickly fluctuate at 25ms and above, you'll see microstutter. Anything below 20ms and you'd either completely miss it or be VERY picky and go looking for it.

    Let me reiterate:

    - Below 20ms: unnoticeable
    - Rapid, cyclical fluctuations above 25ms: you'll notice it
    - Sudden "spikes" (non-cyclical) above 40ms: you'll notice it
    - Grey area between 21ms and 25ms: debatable. Some will notice it while others won't.

  16. #16
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    I said "depending on how perceptive you are". So much for blanket statement...

    The rest of your post is a blanket statement. You don't speak for everyone and you cannot possibly define what is generally noticeable and what is not.

  17. #17
    Xtreme Member
    Join Date
    Aug 2003
    Location
    Finland-Oulu
    Posts
    155
    If enabling normal vsync, does it totally eliminate stuttering in CF/SLI regardless of fps/hz?

  18. #18
    Xtreme Mentor
    Join Date
    Feb 2008
    Posts
    2,625
    I've tried plenty of Xfire / SLI setups before swapping to a single GK104, I noticed no difference in smoothness with the same FPS in any games.

    Any game running at 30-40 FPS was still stuttering to me with both a 560 Ti SLI and single GK104.

    Getting the FPS up to 45 made the games smooth in both setups. It terms of smoothness and stutter between SLI and single GPU setups, FPS was the only limiting factor to me.

    The only issue I had with SLI / Xfire was that none of the MMOs I played supported it. DDO maxed out with 8x AA was dipping to around 30 FPS on SLI 560 tis, and the same with a single 560 ti. I had the exact same happening currently in Path of Exile, the 560 Ti config with one or two cards could not stay above 30 FPS because SLI wasnt working (I still have my 560 Tis and was using them when my 670 broke). On the other hand the 670 / 680 cards bring me up to 60-100+ FPS in those games at maxed settings, so for this reason I'm not bothering with SLI or Xfire again.

    The way I noticed low FPS stuttering on my 680 was by using video capture locked to 35 FPS. I then tried 40 FPS and while a bit better, it was still stuttering. Locking it to 45 FPS made the stutter go away, but it was unrecordable without frame skipping too much over 35 FPS. Exactly the same results as my past SLI / Xfire setups.
    Last edited by bhavv; 02-25-2013 at 06:46 AM.
    Intel I7 980 + Phanteks PH-TC14PE Red, Asus Rampage III Extreme, 12 GB Geil Ultra Series
    SLI MSI GTX 680 power edition, Creative X-Fi Titanium HD, 2x4 Tb Seagate HDD, 128 Gb Crucial M4, 512 Gb Crucial M4
    Gigabyte M8000X, Microsoft Sidewinder X4, Antec P182, Acer G24, Windows 7 Professional, XFX Proseries 850w XXX

    Keeping my mobo + CPU till they are obsolete

    Copper loop X48 build Blue X58 Build Red X58 build

  19. #19
    Quote Originally Posted by boxleitnerb View Post
    You don't speak for everyone and you cannot possibly define what is generally noticeable and what is not.
    It isn't me who is defining "noticeable". Read any article or scientific paper about perceptive latency and you'll likely come to the same conclusion.

    When I make a statement, don't think for a second that I haven't researched it to an extensive degree.

    Links to some of said papers:

    http://www.stuartcheshire.org/papers/LatencyQuest.html

    http://www.ncbi.nlm.nih.gov/pubmed/16639613 (link to PDF @ source)

    http://www.perceptionweb.com/abstract.cgi?id=p240749

    If you have an account @ Sciverse / Science Direct, there are several other articles in their archives which discuss the same thing. Unfortunately, I can't post links to them due to copyright terms but you'll quickly get the idea after Googling a bit.

    However, I will distill it down into plain speak: In extensive lab tests with and without human subjects, a time of 48 frames per second (ie: 20.83ms) was deemed to be the threshold at which the majority of people anbd test instruments saw completely smooth display images or refreshes.

    One test actually went so far as to rapidly flash a lightbulb at variable speeds in order to demonstrate when people would see it "flicker". That threshold for 95% of the participants was at 50 flashes per second (ie: exactly 20ms).

    Not sure how much more information anyone could be looking for here....

  20. #20
    Xtreme Mentor
    Join Date
    Feb 2008
    Posts
    2,625
    That explains why anything below 45 FPS was stuttering to me.
    Intel I7 980 + Phanteks PH-TC14PE Red, Asus Rampage III Extreme, 12 GB Geil Ultra Series
    SLI MSI GTX 680 power edition, Creative X-Fi Titanium HD, 2x4 Tb Seagate HDD, 128 Gb Crucial M4, 512 Gb Crucial M4
    Gigabyte M8000X, Microsoft Sidewinder X4, Antec P182, Acer G24, Windows 7 Professional, XFX Proseries 850w XXX

    Keeping my mobo + CPU till they are obsolete

    Copper loop X48 build Blue X58 Build Red X58 build

  21. #21
    Quote Originally Posted by bhavv View Post
    That explains why anything below 45 FPS was stuttering to me.
    Precisely.

    In addition, I think there has to be a distinction made between the continual stutter typically seen in multi GPU setups and individual frame stutter.

    Continual stutter (also called micro stutter) is cyclical in nature and continues throughout a given sequence or section of a game, often repeating itself at regular intervals. For example, it would look something like this on a frame by frame basis: 10ms, 35ms, 10ms, 35ms, 10ms and so on. You would "see" the stutter every time a frame hit the 35ms mark. It would be seen like this:



    Individual frame stutter meanwhile would point to instances where certain frames take an abnormally long time to render. It would look something like this: 10ms, 15ms, 12ms, 100ms, 10ms, 15ms, 11ms, 20ms, 18ms, 70ms. Here's an example of that:



    Basically these many not be two completely separate phenomenons but they will read differently when viewed onscreen.

  22. #22
    Xtreme Mentor
    Join Date
    May 2008
    Posts
    2,550
    I don't agree with your claims SkyMTL. If microstutter below 20ms was unnoticeable as you claim how would people even benefit from a 120hz display? By those standards the difference between 60 and 90hz shouldn't be noticeable either.

  23. #23
    Quote Originally Posted by BababooeyHTJ View Post
    I don't agree with your claims SkyMTL. If microstutter below 20ms was unnoticeable as you claim how would people even benefit from a 120hz display? By those standards the difference between 60 and 90hz shouldn't be noticeable either.
    You are confusing onscreen refresh rate and response times with processed frame times.

    For example, the move from 60Hz to 120Hz isn't about eliminating flickering. Do you see less flicker on a 120Hz screen? Of course not. Rather it is about being able to drive a display ABOVE the 60 FPS mark with v-sync on. That leads to less input latency.

    In addition, people don't buy 120Hz screens to eliminate microstutter. Indeed, a 120Hz screen won't do anything to eliminate microstutter if it is already being displayed due to above-20ms frame times.

    Now, before you go and claim that's the same thing, it isn't.

    First and foremost, quicker refresh rates all but eliminate the ghosting and input latency that occurs on lower-frequency LCD displays. Neither of those things is tied to GPU microstutter.

    With that in mind, the difference between 90Hz and 60Hz will be noticeable on the INPUT side rather than on the overly VISUAL side (other than the ghosting of course).

    Finally, they aren't claims. Perceptual latency detection above and below 48 FPS has a ton of science backing it up, much of which was sponsored by major production studios.
    Last edited by SKYMTL; 02-25-2013 at 05:54 PM.

  24. #24
    Xtreme Enthusiast
    Join Date
    Dec 2010
    Posts
    581
    Quote Originally Posted by SKYMTL View Post
    It isn't me who is defining "noticeable". Read any article or scientific paper about perceptive latency and you'll likely come to the same conclusion.

    When I make a statement, don't think for a second that I haven't researched it to an extensive degree.

    Links to some of said papers:

    http://www.stuartcheshire.org/papers/LatencyQuest.html

    http://www.ncbi.nlm.nih.gov/pubmed/16639613 (link to PDF @ source)

    http://www.perceptionweb.com/abstract.cgi?id=p240749

    If you have an account @ Sciverse / Science Direct, there are several other articles in their archives which discuss the same thing. Unfortunately, I can't post links to them due to copyright terms but you'll quickly get the idea after Googling a bit.

    However, I will distill it down into plain speak: In extensive lab tests with and without human subjects, a time of 48 frames per second (ie: 20.83ms) was deemed to be the threshold at which the majority of people anbd test instruments saw completely smooth display images or refreshes.

    One test actually went so far as to rapidly flash a lightbulb at variable speeds in order to demonstrate when people would see it "flicker". That threshold for 95% of the participants was at 50 flashes per second (ie: exactly 20ms).

    Not sure how much more information anyone could be looking for here....
    First, you absolutely cannot compare gameplay with just displayed sequences. 48 fps, is way too low for many games, think input lag. In some game engines, 100+ fps are required for truly direct control. Secondly, it is unimportant what the "majority" feels as long as there are still people who have higher standards.
    So did you actually experience SLI/CF first hand or did you not? Theory has its place, but real life can be something different altogether. This smells like sugarcoating the problems that CF has. I'm always highly suspicious if some people - especially reviewers who should be objective and very careful with such stuff - tell me what I should or shouldn't be able to feel.

  25. #25
    Xtreme Enthusiast
    Join Date
    Nov 2007
    Posts
    863
    Sheesh. I spent the day with my finger hovering over the "buy" button on a second 7970 so I could look into this.

    I have my doubts that a 7970 CF truly feels no faster than a single, and I'd like to put it to the test.

    Not to mention I want Crysis 3 and Bioshock Infinite more than any games out right now. I've been having a lot of fun playing Sleeping Dogs and Far Cry3 from the first bundle, AMD is really hitting on all cylinders with their games included.
    Intel 990x/Corsair H80 /Asus Rampage III
    Coolermaster HAF932 case
    Patriot 3 X 2GB
    EVGA GTX Titan SC
    Dell 3008

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •