Attachment 131171
As the result of weekend-time research, the HWBOT staff has decided to invalidate all benchmark records established with the Windows8 operating system. Due to severe validity problems with the Windows8 real time clock ("RTC"), benchmarks results achieved with Windows8 cannot be trusted. The main problem lies with the RTC being affected when over- or underclocking under the operating system. The operating system uses the RTC as reference clock, and benchmarks use it to reference (benchmark) time.
Background Information - Remember Heaven?
At the moment of writing, we do not have the full technical what's and how's figured out. Since this problem affects everyone who is passionate about overclocking, it is important to provide an explanation. It is far from the complete story, but it should be enough for you to understand why we have decided to ban Windows 8 from HWBOT.
Do you remember the history of Unigine Heaven at HWBOT? About three years ago, we launched a wrapper for the back-then brand new DX11 benchmark software Unigine Heaven. The wrapper featured an easy and secure way to submit benchmark scores to HWBOT via data files. On November 2, 2010 we posted a response to an on-going discussion about downclocking in Windows affecting the benchmark score. To make a very long story short, by downclocking in Windows the Heaven benchmark time runs slower than it really is. One second in the Heaven benchmark is equal to 1,x second of real time. Because there is more time within a second, the system can render more frames. The benchmark itself is unaware of all this - for the benchmark, one second is still exactly one second. In the end, the system renders a higher amount of frames in a longer timeframe. In the result calculation for frames per second ("FPS"), the "frames" have increased but the "second" remains the same. You get a higher score.
Later that week, we published a V1.03b version of the wrapper, which fixed the downclock issue. Without going too much into detail, the wrapper uses a second source to verify the benchmark duration and takes the relative measured time difference into account when calculating the final score. Problem solved.
Windows8 - "Support for all devices"
As you know, Microsoft is trying to come up with a unified operating system and user interface for a wide range of devices, including tablets, smartphones, Xbox One, and the desktop PC. Building this unified platform is not easy. It is not just a matter of creating an interface that can be used with a multitude of input devices (finger, mouse, controller), but it also needs to support as many devices as possible. Getting the software to run out of the box on as much hardware as possible is the challenge they are facing.
Of the many aspects to fine-tune, one feature in particular is causing Windows8 to be practically useless for (competitive) overclocking: the RTC. Quoting Wikipedia, "A real-time clock (RTC) is a computer clock (most often in the form of an integrated circuit) that keeps track of the current time. Although the term often refers to the devices in personal computers, servers and embedded systems, RTCs are present in almost any electronic device which needs to keep accurate time." Sparing you the details of development process, compared to Windows7 and previous versions, Microsoft made changes to how it measures time to be compatible with embedded or low cost PCs that do not have a fixed RTC clock. After all, having a fixed RTC clock adds cost to a platform.
Your PC system uses the RTC for many things. For example, it ensures the Windows Time on your machine is accurate. For most benchmark applications, the RTC is used as reference clock when executing the benchmark code. By synchronizing with the RTC, the benchmark knows exactly how much time has passed, and takes that value into account when calculating the performance of your system.
It's all relative (part2)
In a previous editorial ("Critical thinking - Should Maximus V Extreme be ..."), we already touched the topic of the relativity of time in our overclocking universe. "The concept of ?time? on a PC configuration is, if not synced via network or internet, an arbitrarily defined constant designed to ensure that the configuration is running in sync with the real world. In other words: hardware and software engineers ensure that ?one second? on your PC equals ?one second? in real time. One of the reasons why it?s so important to have the PC?s timer line up with the real world time is to ensure that your PC can produce accurate measurements and predictions." The points we brought up in that editorial are relevant again. To ensure that the arbitrarily defined constant of 'time' is the same on everyone's benchmark system, we rely on the OS and hardware. This worked quite well, until Windows8 came around.
The problem builds on the problems we faced with Heaven. When downclocking the system under Windows8, the Windows RTC is affected as well. The biggest difference between Windows7 and Windows8 is that now
all benchmarks (no exception) are affected.
Examples: Benchmarks and Windows Time.
Let us make this more practical. On our Haswell test system we downclocked the BCLK frequency by about 6% from 130 MHz to 122MHz. Using a CPU ratio of respectively 32x and 34x, the resulting CPU frequency remains 4160MHz. Then we ran comparison benchmarks. Here are a couple examples:
Attachment 131172
http://news.hwbot.org/Articles/9824_...lowed/7.14.jpg
http://news.hwbot.org/Articles/9824_...llowed/122.jpg
http://news.hwbot.org/Articles/9824_...llowed/130.jpg
http://news.hwbot.org/Articles/9824_...llowed/cg1.jpg
http://news.hwbot.org/Articles/9824_...llowed/cg2.jpg
http://news.hwbot.org/Articles/9824_...llowed/fs1.jpg
http://news.hwbot.org/Articles/9824_...llowed/fs2.jpg
http://news.hwbot.org/Articles/9824_...llowed/141.jpg
http://news.hwbot.org/Articles/9824_...llowed/141.jpg
http://news.hwbot.org/Articles/9824_...llowed/150.jpg
As you can see, even the latest 3DMark is affected by this. In Windows7, most benchmarks are unaffected by this kind of "overclocking".
We can also demonstrate the Windows8 RTC problem with the Windows Time. Have a look at the videos below. In one of the videos, we underclocked the base clock frequency by 6%. After 5 minutes, Windows Time was already 18(!) seconds behind real time. When overclocking the base clock frequency, we can see the opposite effect. We overclock by roughly 4%, and after two minutes, Windows Time is 3 seconds ahead of real time.
http://www.youtube.com/embed/brkDKgvXKko
http://www.youtube.com/embed/XUi0zpRrtMM
Conclusive lines.
First, we would like to apologise if this editorial seems put together in a hurry. If it seems so, it is because it is. As we hope you gather from the data, we provided in the article, it is not possible for HWBOT to accept any benchmark results or records achieved using Windows8. Simply no benchmark - not even 3DMark - is unaffected by Microsoft's RTC design decisions. As a result, it is impossible to verify the veracity of a system performance indicative in Windows8. The resulting score of any benchmark is relative to the RTC bias of that system.
In order to compare we need the RTC bias to be equal for all systems. With Windows8, we can only use question marks.
Practically speaking, it is not allowed to:
- exploit the Windows8 RTC for benchmark purposes
- submit any benchmark results with RTC bias
The HWBOT staff will:
- Block any seemingly out-of-line Windows8-based benchmark results
- Block any Windows8-based benchmark record, even if the score seems in line with the expectations
We hope that everyone is willing to adhere to these new guidelines. The HWBOT Staff is as passionate as you are about overclocking, and this kind of external problem is affecting our hobby in a way that makes we cannot easily resolve. We are friends amongst each other, and respect the effort your fellow overclockers put into overclocking results.
Thank you for your understanding.
The HWBOT Staff.
