PDA

View Full Version : Windows 8 Benchmark Records No Longer Accepted At HWBOT



OldChap
08-18-2013, 08:54 AM
http://hwbot.org/news/9824_breaking_windows_8_benchmark_results_no_longe r_accepted_at_hwbot

Not so sure anyone would use 8 to benchmark but due to RTC issues when using 8, HWBOT have banned its use anyway

Was this a news item? :D

andressergio
08-18-2013, 09:36 AM
lol

Evantaur
08-18-2013, 11:09 AM
it just feels like win8 fails more every day :P

BeepBeep2
08-18-2013, 08:25 PM
omfg hahahahaha!

Don_Dan
08-19-2013, 03:13 AM
Since the HWBot server has trouble showing the article due to traffic:



131171

As the result of weekend-time research, the HWBOT staff has decided to invalidate all benchmark records established with the Windows8 operating system. Due to severe validity problems with the Windows8 real time clock ("RTC"), benchmarks results achieved with Windows8 cannot be trusted. The main problem lies with the RTC being affected when over- or underclocking under the operating system. The operating system uses the RTC as reference clock, and benchmarks use it to reference (benchmark) time.

Background Information - Remember Heaven?

At the moment of writing, we do not have the full technical what's and how's figured out. Since this problem affects everyone who is passionate about overclocking, it is important to provide an explanation. It is far from the complete story, but it should be enough for you to understand why we have decided to ban Windows 8 from HWBOT.

Do you remember the history of Unigine Heaven at HWBOT? About three years ago, we launched a wrapper for the back-then brand new DX11 benchmark software Unigine Heaven. The wrapper featured an easy and secure way to submit benchmark scores to HWBOT via data files. On November 2, 2010 we posted a response to an on-going discussion about downclocking in Windows affecting the benchmark score. To make a very long story short, by downclocking in Windows the Heaven benchmark time runs slower than it really is. One second in the Heaven benchmark is equal to 1,x second of real time. Because there is more time within a second, the system can render more frames. The benchmark itself is unaware of all this - for the benchmark, one second is still exactly one second. In the end, the system renders a higher amount of frames in a longer timeframe. In the result calculation for frames per second ("FPS"), the "frames" have increased but the "second" remains the same. You get a higher score.

Later that week, we published a V1.03b version of the wrapper, which fixed the downclock issue. Without going too much into detail, the wrapper uses a second source to verify the benchmark duration and takes the relative measured time difference into account when calculating the final score. Problem solved.

Windows8 - "Support for all devices"

As you know, Microsoft is trying to come up with a unified operating system and user interface for a wide range of devices, including tablets, smartphones, Xbox One, and the desktop PC. Building this unified platform is not easy. It is not just a matter of creating an interface that can be used with a multitude of input devices (finger, mouse, controller), but it also needs to support as many devices as possible. Getting the software to run out of the box on as much hardware as possible is the challenge they are facing.

Of the many aspects to fine-tune, one feature in particular is causing Windows8 to be practically useless for (competitive) overclocking: the RTC. Quoting Wikipedia, "A real-time clock (RTC) is a computer clock (most often in the form of an integrated circuit) that keeps track of the current time. Although the term often refers to the devices in personal computers, servers and embedded systems, RTCs are present in almost any electronic device which needs to keep accurate time." Sparing you the details of development process, compared to Windows7 and previous versions, Microsoft made changes to how it measures time to be compatible with embedded or low cost PCs that do not have a fixed RTC clock. After all, having a fixed RTC clock adds cost to a platform.

Your PC system uses the RTC for many things. For example, it ensures the Windows Time on your machine is accurate. For most benchmark applications, the RTC is used as reference clock when executing the benchmark code. By synchronizing with the RTC, the benchmark knows exactly how much time has passed, and takes that value into account when calculating the performance of your system.

It's all relative (part2)

In a previous editorial ("Critical thinking - Should Maximus V Extreme be ..."), we already touched the topic of the relativity of time in our overclocking universe. "The concept of ?time? on a PC configuration is, if not synced via network or internet, an arbitrarily defined constant designed to ensure that the configuration is running in sync with the real world. In other words: hardware and software engineers ensure that ?one second? on your PC equals ?one second? in real time. One of the reasons why it?s so important to have the PC?s timer line up with the real world time is to ensure that your PC can produce accurate measurements and predictions." The points we brought up in that editorial are relevant again. To ensure that the arbitrarily defined constant of 'time' is the same on everyone's benchmark system, we rely on the OS and hardware. This worked quite well, until Windows8 came around.

The problem builds on the problems we faced with Heaven. When downclocking the system under Windows8, the Windows RTC is affected as well. The biggest difference between Windows7 and Windows8 is that now all benchmarks (no exception) are affected.

Examples: Benchmarks and Windows Time.

Let us make this more practical. On our Haswell test system we downclocked the BCLK frequency by about 6% from 130 MHz to 122MHz. Using a CPU ratio of respectively 32x and 34x, the resulting CPU frequency remains 4160MHz. Then we ran comparison benchmarks. Here are a couple examples:

131172


http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/7.14.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/122.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/130.jpg

http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/cg1.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/cg2.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/fs1.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/fs2.jpg

http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/141.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/141.jpg
http://news.hwbot.org/Articles/9824_BreakingWindows8Disallowed/150.jpg

As you can see, even the latest 3DMark is affected by this. In Windows7, most benchmarks are unaffected by this kind of "overclocking".

We can also demonstrate the Windows8 RTC problem with the Windows Time. Have a look at the videos below. In one of the videos, we underclocked the base clock frequency by 6%. After 5 minutes, Windows Time was already 18(!) seconds behind real time. When overclocking the base clock frequency, we can see the opposite effect. We overclock by roughly 4%, and after two minutes, Windows Time is 3 seconds ahead of real time.

http://www.youtube.com/embed/brkDKgvXKko
http://www.youtube.com/embed/XUi0zpRrtMM

Conclusive lines.

First, we would like to apologise if this editorial seems put together in a hurry. If it seems so, it is because it is. As we hope you gather from the data, we provided in the article, it is not possible for HWBOT to accept any benchmark results or records achieved using Windows8. Simply no benchmark - not even 3DMark - is unaffected by Microsoft's RTC design decisions. As a result, it is impossible to verify the veracity of a system performance indicative in Windows8. The resulting score of any benchmark is relative to the RTC bias of that system.

In order to compare we need the RTC bias to be equal for all systems. With Windows8, we can only use question marks.

Practically speaking, it is not allowed to:

- exploit the Windows8 RTC for benchmark purposes
- submit any benchmark results with RTC bias

The HWBOT staff will:

- Block any seemingly out-of-line Windows8-based benchmark results
- Block any Windows8-based benchmark record, even if the score seems in line with the expectations

We hope that everyone is willing to adhere to these new guidelines. The HWBOT Staff is as passionate as you are about overclocking, and this kind of external problem is affecting our hobby in a way that makes we cannot easily resolve. We are friends amongst each other, and respect the effort your fellow overclockers put into overclocking results.

Thank you for your understanding.

The HWBOT Staff.

Don_Dan
09-03-2013, 11:07 AM
Futuremark Fixes Windows 8 RTC Bug - Systeminfo 4.20 Allowed For HWBOT Submissions (http://hwbot.org/newsflash/2175_futuremark_fixes_windows_8_rtc_bug___systemin fo_4.20_allowed_for_hwbot_submissions)


News from our Head of Moderating, Christian Ney, regarding Windows 8 and Futuremark benchmarks. The HWBOT moderation rules regarding Windows 8 and Futuremark benchmarks have been updated. Here is the latest version:

- All new Windows 8 Benchmark Results will be blocked, including for on-going competitions
- Existing Windows 8 based benchmark submissions will be blocked if seemingly out of line or world record or top score.
- Existing Windows 8 based benchmark submissions will not be blocked or removed if they don't meet the requirements above
- Existing Windows 8 based benchmark submissions will not be blocked or removed if they were made on an AMD system (New ones will be clocked)

Note: if you recall downclocking when using an Intel based Windows 8 system, please remove that result.

Exceptions to the Windows 8 rule:

- non-overclockable notebooks may be used in combination with Windows 8
- 3DMark and PCMark runs on Windows 8 if an ORB Link is included, and SystemInfo 4.20 (unofficial download) is used.

More information:


I have been cooperating with Futuremark, now 3DMark and PCMark runs on Windows 8 can be trusted when they meet the following requirements:

- ORB Link
- Use of SystemInfo 4.20 or newer

How does it work

At the moment it's a "dirty" fix since it requires online check and isn't built into the benchmark for offline check. The positive news is that the fix affects all Futuremark benchmarks at once. Just need to update SystemInfo ("SI"). Make sure to uninstall any previous version of SystemInfo to not create any conflict. Submit the file online via the benchmark or the upload form. The new SI will check and monitor/log three different timers. When submiting online the site engine will check the values and if one timer is off (RTC in this case) the result will not be valid and you will get the following error: "Time measuring inaccurate. Results are invalid."

Here is an example:

Windows 8 + System Info 4.19 + no downclocking = VALID
Windows 8 + System Info 4.19 + downclocking = INVALID
Windows 8 + System Info 4.20 + no downclocking = VALID
Windows 8 + System Info 4.20 + downclocking = INVALID

Futuremark will also add a warning-type notice like: Invalid, Time measuring cannot be checked, please use latest version of SystemInfo when benchmarking under Windows 8