CL|WCL|RTL performance (SB) : 32M scaling charts : PSC WCL > CL performance bug

Printable View

04-06-2012, 02:55 PM
cheapseats

CL|WCL|RTL performance (SB) : 32M scaling charts : PSC WCL > CL performance bug
---------------------------------------------------------------------------------------------------

This project is looking at CL-based performance and scaling on Sandybridge, taking into account the effects of CL/WCL/RTL.

The aim is to give meaningful context to CL performance and performance change, based on testing and observation.

For quick reference, see tables in Post 2. The tables briefly summarise the contents of Posts 3 and 4.

---------------------------------------------------------------------------------------------------

CL = CAS Latency

WCL = CAS Write Latency (also known as CWL or WL)

RTL = Round Trip Latency

---------------------------------------------------------------------------------------------------
Index
---------------------------------------------------------------------------------------------------

Post 1 :
- Index.
Post 2 :
- Hyper & PSC : CL and WCL scaling tables for RTL, vDIMM, AIDA Read/Latency, 32M ranking.
Post 3 :
- PSC WCL > CL performance bug.
Post 4 :
- Hyper & PSC : CL/WCL/RTL 32M performance scaling @ x-10-7-26 (direct comparison timing & subtiming).
- table+chart format to show setting-to-setting scaling for CL/WCL/RTL changes.
- 32M + AIDA screenshots.
Post 5 :
- Hyper : CL10 to CL6 1067/933/800 32M performance scaling @ x-7-5-20 + optimal WCL.
- charts and 32M + AIDA screenshots.
Post 6 : reserved.

---------------------------------------------------------------------------------------------------

by cheapseats 2011-2012.
04-06-2012, 02:56 PM
cheapseats

CL | WCL | RTL : Tables (Hyper & PSC)

Tables show generalised summary of found behaviour (SB 1155)

Tables assume* WCL range 8 max to 6 min
(*as found for test hardware)

RTL | vDIMM | AIDA read : scaling table for CL & WCL combinations

http://i447.photobucket.com/albums/q..._Hyper_PSC.png

---------------------------------------------------------------------------------------------------

Superpi 32M : performance ranking table for CL & WCL combinations

Ranking position based on 32M test results (see Post 4)

Note: where PSC WCL > CL, significant overall performance limitations are found in 32M.

http://i447.photobucket.com/albums/q..._Hyper_PSC.png
04-06-2012, 02:56 PM
cheapseats
PSC WCL > CL performance bug

(also see tables in Post 2 and charts in Post 4)

---------------------------------------------------------------------------------------------------

Please note:

This 'analysis' is based on my observations and test results. It is limited in depth and is non-technical.

The general principles and behaviour described here for PSC on SB also may be found with other IC-types (not tested), and/or other memory controllers, eg on SB-E, IB, AMD (not tested).

---------------------------------------------------------------------------------------------------

Overview

The issue can be shown as the key difference in CL-based scaling behaviour found between PSC and Hyper based DIMMs on SB.

With PSC, significant performance limitations apply when WCL value is greater than CL value (WCL > CL).

PSC WCL > CL conditions produce a negative offset in net read performance scaling, and restrict overall performance.

The negative effect on net read performance is due to increased min RTL per CL where PSC WCL > CL.
This is compared to min RTL per CL where PSC WCL = CL, or compared to min RTL per CL with Hyper.
(effects of RTL changes shown in AIDA Read and Latency, for example)

The PSC WCL effect on RTL is further to write-based effects of WCL changes, which are the same for PSC and Hyper.
(effects of WCL changes shown in AIDA Copy, for example)

RTL change

http://i447.photobucket.com/albums/q...yper_PSC_l.png

General behaviour:
- RTL changes directly affect net read performance (at IMC) and significantly affect min vDIMM.
- CL changes do not directly affect net read performance, and do not directly affect min vDIMM (except marginally).
- CL changes which affect net read performance do so indirectly, by allowing (forcing) RTL change.
- CL changes which do not allow RTL change, do not affect net read performance.
- CL changes either allow or do not allow RTL changes, as determined by conditions and IC-type.
- WCL changes either allow or do not allow RTL changes, as determined by conditions and IC-type.
IC-based differences in RTL behaviour:
- Hyper CL changes allow (force) RTL changes, regardless of WCL.
- Hyper WCL changes do not affect RTL.
- PSC CL changes either allow or do not allow RTL changes, as determined by (initial and final) CL and WCL conditions.
- PSC WCL changes either allow or do not allow RTL changes, as determined by (initial and final) CL and WCL conditions.
In general for PSC compared to Hyper:

if WCL > CL, then SB min RTL with PSC is higher per CL than with Hyper.
if WCL = CL, then SB min RTL with PSC is the same per CL as with Hyper.

RTL 'thresholds' with PSC CL and WCL
- Threshold values can be described in terms of CL and WCL values in DRAM clocks.
- Thresholds are crossed by CL or WCL changes between threshold values of CL-WCL or WCL-CL (DRAM clocks).
- Crossing or not crossing of thresholds by WCL or CL changes determines effect on RTL.
If a threshold is crossed:

PSC CL +/-1 change does not allow RTL change.
PSC WCL +/-1 change allows (forces) RTL change.

If a threshold is not crossed:

PSC CL +/-1 change allows (forces) RTL change.
PSC WCL +/-1 change does not allow RTL change.

PSC WCL changes which affect min RTL:

Thresholds are crossed by WCL changes between:

WCL-CL = 1 and WCL-CL = 0 (DRAM clocks)

and (@ CL10) between:

WCL-CL = -3 and WCL-CL = -4

or if described by WCL value in terms of CL value, between:

WCL = CL+1`and WCL = CL

and (@ CL10) between:

WCL = CL-3 and WCL = CL-4

http://i447.photobucket.com/albums/q..._Table_PSC.png

PSC CL changes which do not affect min RTL:

Thresholds are crossed by CL changes between:

CL-WCL = 0 and CL-WCL = -1 (DRAM clocks)

and between:

CL-WCL = 3 and CL-WCL = 4

or if described by CL value in terms of WCL value, between:

CL = WCL`and CL = WCL-1

and between:

CL = WCL+3 and CL = WCL+4

http://i447.photobucket.com/albums/q..._Table_PSC.png

Variable RTL and vDIMM per CL

Due to the restriction (per CL) of min RTL and net read performance for PSC @ WCL > CL, it follows that PSC min vDIMM (per CL) is significantly lower where WCL > CL than where WCL = CL (or WCL < CL).

This is not found with Hyper as changes to min RTL are forced by CL changes, regardless of WCL.

For PSC at CL values which are lower than max WCL value but are equal to or higher than min WCL value, it is found that min RTL (per CL) and min vDIMM (per CL) are both significantly variable with WCL.

therefore:

if WCL8 = max, then 'binning' of PSC @ CL7 or CL6 is not meaningful without consideration of WCL.
(due to variability in min RTL and min vDIMM, with WCL changes between WCL > CL and WCL = CL)

if WCL6 = min, then PSC @ CL < 6 is not meaningfully possible.
(due to restriction of min RTL, by min WCL)

PSC WCL > CL Superpi 32M performance summary

(comparisons for CL/WCL/RTL changes, all other timing and subtiming unchanged)

PSC @ CL7 WCL8 has 8% (approx) higher min vDIMM but lower 32M performance than CL9 WCL6.

PSC @ CL6 WCL8 (or WCL7) has 11% (approx) higher min vDIMM but lower 32M performance than CL8 WCL6.

General conclusions

In the context of both performance and vDIMM scaling derived from CL/WCL/RTL changes,
the conditions created by PSC WCL > CL are significantly restrictive, misleading, and are effectively bugged.

As such, the use of PSC WCL > CL on SB should be viewed only as a performance-limiting exploit to minimise vDIMM per CL.

This may be seen as useful for CPU-Z validation of PSC @ CL7 and CL6, but is shown to be counter-productive and nonsensical in terms of performance optimisation.

---------------------------------------------------------------------------------------------------

Scaling behaviour and 32M performance ranking tables in Post 2.
32M performance tests and scaling charts in Post 4.
04-06-2012, 02:57 PM
cheapseats

CL | WCL | RTL : Superpi 32M performance scaling charts

Hyper & PSC @ x-10-7-26

---------------------------------------------------------------------------------------------------

2600K / P67A-UD5 (B2) F7e / 2x2GB Corsair CMGTX2 (Hyper) / 2x2GB Exceleram EB3103A (PSC)

100.0 x 50 (5GHz)

DDR3-2133 (1067MHz) / DDR3-1866 (933MHz) / DDR3-1600 (800MHz)

BIOS manual set : x-10-7-26 / tRC 33 / tRFC 72 / tWTR 4 / tRRD 4 / tRTP 4 / tWR 6 / tFAW 16 / CR1

XP Pro sp2 + Spitweaker v1.0 (copy waza)

---------------------------------------------------------------------------------------------------

Scaling shown in 3 ways:

(1) by 'sequential' CL & WCL change :
eg from CL9 @ WCL8, 7, 6; to CL8 @ WCL8, 7, 6; etc
(zig-zag line)

(2) by fixed WCL with CL change :
eg CL9-CL6 @ WCL8; CL9-CL6 @ WCL7; CL9-CL6 @ WCL6
(blue, orange, green lines)

(3) grouped by WCL offset to CL :
eg 'WCL = CL+1' grouping; 'WCL = CL, & WCL = CL-1' grouping; 'WCL = CL-2' grouping
('parallel' lines)

---------------------------------------------------------------------------------------------------

Hyper : 1067MHz : CL9 to CL6 (x-10-7-26)

(1)http://i447.photobucket.com/albums/q...sequential.png (2)http://i447.photobucket.com/albums/q..._Hyper_WCL.png (3)http://i447.photobucket.com/albums/q...r_parallel.png

---------------------------------------------------------------------------------------------------

PSC : 1067MHz : CL9 to CL6 (x-10-7-26)

(1)http://i447.photobucket.com/albums/q...sequential.png (2)http://i447.photobucket.com/albums/q...26_PSC_WCL.png (3)http://i447.photobucket.com/albums/q...C_parallel.png

---------------------------------------------------------------------------------------------------

PSC : 933MHz : CL9 to CL6 (x-10-7-26)

(1)http://i447.photobucket.com/albums/q...sequential.png (2)http://i447.photobucket.com/albums/q...26_PSC_WCL.png (3)http://i447.photobucket.com/albums/q...C_parallel.png

---------------------------------------------------------------------------------------------------

PSC : 800MHz : CL9 to CL6 (x-10-7-26)

(1)http://i447.photobucket.com/albums/q...sequential.png (2)http://i447.photobucket.com/albums/q...26_PSC_WCL.png (3)http://i447.photobucket.com/albums/q...C_parallel.png

---------------------------------------------------------------------------------------------------

Hyper & PSC : CL9 to CL6 (x-10-7-26)

single image for each scaling type (as columns^)

(1) 'sequential'
(2) 'fixed WCL'
(3) 'parallel'

note : 1327 x 4040 px / ~300kb (per image)

(1)http://i447.photobucket.com/albums/q...uential_th.png (2)http://i447.photobucket.com/albums/q...PSC_WCL_th.png (3)http://i447.photobucket.com/albums/q...arallel_th.png

Hyper & PSC : CL10 to CL6 (x-10-7-26)

note : 1513 x 4536 px / ~380kb (per image)

(1)http://i447.photobucket.com/albums/q...uential_th.png (2)http://i447.photobucket.com/albums/q...PSC_WCL_th.png (3)http://i447.photobucket.com/albums/q...arallel_th.png

---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------

32M Screenshots

note: all screens with AIDA v1.50 1200
(approx for newer AIDA versions: Read = -100 to -150Mb/s; Copy = -1Gb/s; Latency = same)

---------------------------------------------------------------------------------------------------

Hyper : 1067MHz : CL10 to CL6 (x-10-7-26)

Code:

............................................................................................... CL10 / WCL8 / 39-40 : 397.062 .. CL10 / WCL7 / 39-40 : 396.984 .. CL10 / WCL6 / 39-40 : 397.546 CL9 / WCL8 / 37-38 : 396.609 ... CL9 / WCL7 / 37-38 : 396.171 ... CL9 / WCL6 / 37-38 : 396.093 CL8 / WCL8 / 35-36 : 395.985 ... CL8 / WCL7 / 35-36 : 395.735 ... CL8 / WCL6 / 35-36 : 395.312 CL7 / WCL8 / 33-34 : 395.547 ... CL7 / WCL7 / 33-34 : 395.078 ... CL7 / WCL6 / 33-34 : 394.891 CL6 / WCL8 / 31-32 : 395.000 ... CL6 / WCL7 / 31-32 : 394.719 ... CL6 / WCL6 / 31-32 : 394.235

---------------------------------------------------------------------------------------------------

PSC : 1067MHz : CL10 to CL6 (x-10-7-26)

Code:

............................................................................................... CL10 / WCL8 / 39-40 : 397.125 .. CL10 / WCL7 / 39-40 : 397.047 .. CL10 / WCL6 / 37-38 : 396.922 CL9 / WCL8 / 37-38 : 396.593 ... CL9 / WCL7 / 37-38 : 396.172 ... CL9 / WCL6 / 37-38 : 396.094 CL8 / WCL8 / 35-36 : 395.985 ... CL8 / WCL7 / 35-36 : 395.734 ... CL8 / WCL6 / 35-36 : 395.312 CL7 / WCL8 / 35-36 : 396.187 ... CL7 / WCL7 / 33-34 : 395.094 ... CL7 / WCL6 / 33-34 : 394.891 CL6 / WCL8 / 33-34 : 395.641 ... CL6 / WCL7 / 33-34 : 395.359

---------------------------------------------------------------------------------------------------

PSC : 933MHz : CL10 to CL6 (x-10-7-26)

Code:

............................................................................................... CL10 / WCL8 / 38-39 : 403.594 .. CL10 / WCL7 / 38-39 : 403.469 .. CL10 / WCL6 / 36-37 : 403.500 CL9 / WCL8 / 36-37 : 403.109 ... CL9 / WCL7 / 36-37 : 402.641 ... CL9 / WCL6 / 36-37 : 402.515 CL8 / WCL8 / 34-35 : 402.406 ... CL8 / WCL7 / 34-35 : 402.157 ... CL8 / WCL6 / 34-35 : 401.656 CL7 / WCL8 / 34-35 : 402.735 ... CL7 / WCL7 / 32-33 : 401.453 ... CL7 / WCL6 / 32-33 : 401.203 CL6 / WCL8 / 32-33 : 402.156 ... CL6 / WCL7 / 32-33 : 401.750 ... CL6 / WCL6 / 30-31 : 400.500

---------------------------------------------------------------------------------------------------

PSC : 800MHz : CL10 to CL6 (x-10-7-26)

Code:

............................................................................................... CL10 / WCL8 / 37-38 : 412.750 .. CL10 / WCL7 / 37-38 : 412.625 .. CL10 / WCL6 / 35-36 : 412.562 CL9 / WCL8 / 35-36 : 412.172 ... CL9 / WCL7 / 35-36 : 411.547 ... CL9 / WCL6 / 35-36 : 411.469 CL8 / WCL8 / 33-34 : 411.328 ... CL8 / WCL7 / 33-34 : 411.031 ... CL8 / WCL6 / 33-34 : 410.406 CL7 / WCL8 / 33-34 : 411.735 ... CL7 / WCL7 / 31-32 : 410.234 ... CL7 / WCL6 / 31-32 : 409.891 CL6 / WCL8 / 31-32 : 410.984 ... CL6 / WCL7 / 31-32 : 410.610 ... CL6 / WCL6 / 29-30 : 409.063
04-06-2012, 02:57 PM
cheapseats

Hyper CL | RTL : Superpi 32M performance scaling charts

(x-7-5-20 + optimal WCL)

---------------------------------------------------------------------------------------------------

2600K / P67A-UD5 (B2) F7e / 2x2GB Corsair CMGTX2 (Hyper)

100.0 x 50 (5GHz)

DDR3-2133 (1067MHz) / DDR3-1866 (933MHz) / DDR3-1600 (800MHz)

BIOS manual set : x-7-5-20 / tRC 25 / tRFC 55 / tWTR 4 / tRRD 4 / tRTP 4 / tWR 6 / tFAW 16 / CR1

XP Pro sp2 + Spitweaker v1.0 (copy waza)

---------------------------------------------------------------------------------------------------

Hyper : 1067MHz : CL10 to CL6 (x-7-5-20)

http://i447.photobucket.com/albums/q...er_WCL_opt.png

---------------------------------------------------------------------------------------------------

Hyper : 933MHz : CL10 to CL6 (x-7-5-20)

http://i447.photobucket.com/albums/q...er_WCL_opt.png

---------------------------------------------------------------------------------------------------

Hyper : 800MHz : CL10 to CL6 (x-7-5-20)

http://i447.photobucket.com/albums/q...er_WCL_opt.png

---------------------------------------------------------------------------------------------------

1414 x 3018px / ~220kb

http://i447.photobucket.com/albums/q...WCL_opt_th.png

---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------

32M Screenshots

note: all screens with AIDA v1.50 1200
(approx for newer AIDA versions: Read = -100 to -150Mb/s; Copy = -1Gb/s; Latency = same)

---------------------------------------------------------------------------------------------------

Hyper : 1067/933/800MHz : CL10 to CL6 (x-7-5-20)

Code:

............................................................................................... 1067MHz CL10 / WCL7 / 39-40 : 393.750 .CL9 / WCL6 / 37-38 : 392.828 .CL8 / WCL6 / 35-36 : 392.156 .CL7 / WCL6 / 33-34 : 391.797 .CL6 / WCL6 / 31-32 : 391.188 933MHz CL10 / WCL7 / 38-39 : 399.688 .CL9 / WCL6 / 36-37 : 398.672 .CL8 / WCL6 / 34-35 : 397.906 .CL7 / WCL6 / 32-33 : 397.547 .CL6 / WCL6 / 30-31 : 396.860 800MHz CL10 / WCL7 / 37-38 : 408.016 .CL9 / WCL6 / 35-36 : 406.828 .CL8 / WCL6 / 33-34 : 405.922 .CL7 / WCL6 / 31-32 : 405.437 .CL6 / WCL6 / 29-30 : 404.641
04-06-2012, 02:58 PM
cheapseats

reserved.
04-06-2012, 04:53 PM
M.Beier

Great job, and that is ALOT of testing indeed!
Thanks for heads up.
04-06-2012, 05:13 PM
Hazzan

Great experience this awesome really...:up::up:
04-06-2012, 06:02 PM
youngpro

seriously sick testing dude, when i get time i will read the whole thing, thanks for sharing the hard work
04-06-2012, 06:44 PM
Hondacity

Thanks ChiefSeats :D
04-07-2012, 06:46 AM
cheapseats

Thanks very much for your comments guys :up:
04-07-2012, 08:41 AM
Punisher!

Quote:

Originally Posted by Hazzan

Great experience this awesome really...:up::up:

+1!!!
04-08-2012, 07:11 AM
Don_Dan

Wow, that must have been a lot of work! Thanks for posting! :up:
04-10-2012, 04:46 AM
Hyperhorn

Awesome thread, thank you for sharing the information! :clap:
I'll take a deeper look later ...
04-10-2012, 05:53 AM
eva2000

wow, I am absent from forums for a while, and such great info gets posted!

awesome stuff cheapseats !
04-10-2012, 06:37 AM
massman

Whoah! One of the better threads I've seen here in the last year or so.

:up:
04-10-2012, 07:06 AM
Mafio

damn, that's some serious testing
thanks for sharing
04-11-2012, 10:00 PM
Calathea

Very helpful, thanks a lot! I tried 8-9-6 wcl 6 instead of 7-9-7 wcl 8(min with cl7) with ripjaws bbse and got around 1 sec in 32m :)
04-12-2012, 11:37 AM
cheapseats

Thanks to all :)

Quote:

Originally Posted by Calathea

Very helpful, thanks a lot! I tried 8-9-6 wcl 6 instead of 7-9-7 wcl 8(min with cl7) with ripjaws bbse and got around 1 sec in 32m :)

nice to hear you got a boost :D

WCL 7 or 6 should also be possible at CL7, but setting either of those WCL values
will force RTL decrease compared with RTL @ CL7 WCL8.

The RTL decrease will require a significant increase in min vdimm, eg ~11%
so CL7 WCL7 (or WCL6) will be unbootable until vdimm is raised sufficiently.
04-12-2012, 01:22 PM
Raja@ASUS

Woah, lots of data must of taken a huge chunk of time. Bottom line is Hyper ICs could handle higher IO at a given clock frequency (where signal integrity is not a limting factor) than the newer parts.

The RTL changes simply restrict the peak IO when multiple transactions are made and provide a little bit of breathing room plus are affected by any IO buffer delays and clock skew. What you are effectively playing with is a combination of setting adjustments needed to remain within the IO capabilities of the memory ICs at a given level of VDIMM.

The overall balance for scores in a benchmark as far as adjustments to WCL versus CL come from the associated read to write delays, some of which are limited due to the architecture, and electrical mechanisms.
04-12-2012, 03:05 PM
M.Beier

The thread should deff be sticky...
05-26-2013, 01:42 PM
StrategosSan

Respect for sharing the result of your hard work :) Thumbs up!

Thank you man