CL|WCL|RTL performance (SB) : 32M scaling charts : PSC WCL > CL performance bug

[cheapseats]

---------------------------------------------------------------------------------------------------

This project is looking at CL-based performance and scaling on Sandybridge, taking into account the effects of CL/WCL/RTL.

The aim is to give meaningful context to CL performance and performance change, based on testing and observation.

For quick reference, see tables in Post 2. The tables briefly summarise the contents of Posts 3 and 4.

---------------------------------------------------------------------------------------------------

CL = CAS Latency

WCL = CAS Write Latency (also known as CWL or WL)

RTL = Round Trip Latency


---------------------------------------------------------------------------------------------------
Index
---------------------------------------------------------------------------------------------------

Post 1 :

• Index.

Post 2 :

• Hyper & PSC : CL and WCL scaling tables for RTL, vDIMM, AIDA Read/Latency, 32M ranking.

Post 3 :

• PSC WCL > CL performance bug.

Post 4 :

• Hyper & PSC : CL/WCL/RTL 32M performance scaling @ x-10-7-26 (direct comparison timing & subtiming).

• table+chart format to show setting-to-setting scaling for CL/WCL/RTL changes.

• 32M + AIDA screenshots.

Post 5 :

• Hyper : CL10 to CL6 1067/933/800 32M performance scaling @ x-7-5-20 + optimal WCL.

• charts and 32M + AIDA screenshots.

Post 6 : reserved.


---------------------------------------------------------------------------------------------------


by cheapseats 2011-2012.

[cheapseats]

CL | WCL | RTL : Tables (Hyper & PSC)

Tables show generalised summary of found behaviour (SB 1155)

Tables assume* WCL range 8 max to 6 min
(*as found for test hardware)


RTL | vDIMM | AIDA read : scaling table for CL & WCL combinations





---------------------------------------------------------------------------------------------------


Superpi 32M : performance ranking table for CL & WCL combinations

Ranking position based on 32M test results (see Post 4)

Note: where PSC WCL > CL, significant overall performance limitations are found in 32M.

[cheapseats]

PSC WCL > CL performance bug

(also see tables in Post 2 and charts in Post 4)

---------------------------------------------------------------------------------------------------

Please note:

This 'analysis' is based on my observations and test results. It is limited in depth and is non-technical.

The general principles and behaviour described here for PSC on SB also may be found with other IC-types (not tested), and/or other memory controllers, eg on SB-E, IB, AMD (not tested).

---------------------------------------------------------------------------------------------------


Overview

The issue can be shown as the key difference in CL-based scaling behaviour found between PSC and Hyper based DIMMs on SB.

With PSC, significant performance limitations apply when WCL value is greater than CL value (WCL > CL).

PSC WCL > CL conditions produce a negative offset in net read performance scaling, and restrict overall performance.

The negative effect on net read performance is due to increased min RTL per CL where PSC WCL > CL.
This is compared to min RTL per CL where PSC WCL = CL, or compared to min RTL per CL with Hyper.
(effects of RTL changes shown in AIDA Read and Latency, for example)

The PSC WCL effect on RTL is further to write-based effects of WCL changes, which are the same for PSC and Hyper.
(effects of WCL changes shown in AIDA Copy, for example)


RTL change





General behaviour:

• RTL changes directly affect net read performance (at IMC) and significantly affect min vDIMM.

• CL changes do not directly affect net read performance, and do not directly affect min vDIMM (except marginally).

• CL changes which affect net read performance do so indirectly, by allowing (forcing) RTL change.

• CL changes which do not allow RTL change, do not affect net read performance.

• CL changes either allow or do not allow RTL changes, as determined by conditions and IC-type.

• WCL changes either allow or do not allow RTL changes, as determined by conditions and IC-type.

IC-based differences in RTL behaviour:

• Hyper CL changes allow (force) RTL changes, regardless of WCL.

• Hyper WCL changes do not affect RTL.

• PSC CL changes either allow or do not allow RTL changes, as determined by (initial and final) CL and WCL conditions.

• PSC WCL changes either allow or do not allow RTL changes, as determined by (initial and final) CL and WCL conditions.

In general for PSC compared to Hyper:

if WCL > CL, then SB min RTL with PSC is higher per CL than with Hyper.
if WCL = CL, then SB min RTL with PSC is the same per CL as with Hyper.


RTL 'thresholds' with PSC CL and WCL

• Threshold values can be described in terms of CL and WCL values in DRAM clocks.

• Thresholds are crossed by CL or WCL changes between threshold values of CL-WCL or WCL-CL (DRAM clocks).

• Crossing or not crossing of thresholds by WCL or CL changes determines effect on RTL.

If a threshold is crossed:

PSC CL +/-1 change does not allow RTL change.
PSC WCL +/-1 change allows (forces) RTL change.

If a threshold is not crossed:

PSC CL +/-1 change allows (forces) RTL change.
PSC WCL +/-1 change does not allow RTL change.


PSC WCL changes which affect min RTL:

Thresholds are crossed by WCL changes between:

WCL-CL = 1 and WCL-CL = 0 (DRAM clocks)

and (@ CL10) between:

WCL-CL = -3 and WCL-CL = -4

or if described by WCL value in terms of CL value, between:

WCL = CL+1`and WCL = CL

and (@ CL10) between:

WCL = CL-3 and WCL = CL-4





PSC CL changes which do not affect min RTL:

Thresholds are crossed by CL changes between:

CL-WCL = 0 and CL-WCL = -1 (DRAM clocks)

and between:

CL-WCL = 3 and CL-WCL = 4

or if described by CL value in terms of WCL value, between:

CL = WCL`and CL = WCL-1

and between:

CL = WCL+3 and CL = WCL+4





Variable RTL and vDIMM per CL

Due to the restriction (per CL) of min RTL and net read performance for PSC @ WCL > CL, it follows that PSC min vDIMM (per CL) is significantly lower where WCL > CL than where WCL = CL (or WCL < CL).

This is not found with Hyper as changes to min RTL are forced by CL changes, regardless of WCL.

For PSC at CL values which are lower than max WCL value but are equal to or higher than min WCL value, it is found that min RTL (per CL) and min vDIMM (per CL) are both significantly variable with WCL.

therefore:

if WCL8 = max, then 'binning' of PSC @ CL7 or CL6 is not meaningful without consideration of WCL.
(due to variability in min RTL and min vDIMM, with WCL changes between WCL > CL and WCL = CL)

if WCL6 = min, then PSC @ CL < 6 is not meaningfully possible.
(due to restriction of min RTL, by min WCL)


PSC WCL > CL Superpi 32M performance summary

(comparisons for CL/WCL/RTL changes, all other timing and subtiming unchanged)

• PSC @ CL7 WCL8 has 8% (approx) higher min vDIMM but lower 32M performance than CL9 WCL6.

• PSC @ CL6 WCL8 (or WCL7) has 11% (approx) higher min vDIMM but lower 32M performance than CL8 WCL6.

General conclusions

In the context of both performance and vDIMM scaling derived from CL/WCL/RTL changes,
the conditions created by PSC WCL > CL are significantly restrictive, misleading, and are effectively bugged.

As such, the use of PSC WCL > CL on SB should be viewed only as a performance-limiting exploit to minimise vDIMM per CL.

This may be seen as useful for CPU-Z validation of PSC @ CL7 and CL6, but is shown to be counter-productive and nonsensical in terms of performance optimisation.


---------------------------------------------------------------------------------------------------

Scaling behaviour and 32M performance ranking tables in Post 2.
32M performance tests and scaling charts in Post 4.

[cheapseats]

CL | WCL | RTL : Superpi 32M performance scaling charts

Hyper & PSC @ x-10-7-26

---------------------------------------------------------------------------------------------------

2600K / P67A-UD5 (B2) F7e / 2x2GB Corsair CMGTX2 (Hyper) / 2x2GB Exceleram EB3103A (PSC)

100.0 x 50 (5GHz)

DDR3-2133 (1067MHz) / DDR3-1866 (933MHz) / DDR3-1600 (800MHz)

BIOS manual set : x-10-7-26 / tRC 33 / tRFC 72 / tWTR 4 / tRRD 4 / tRTP 4 / tWR 6 / tFAW 16 / CR1

XP Pro sp2 + Spitweaker v1.0 (copy waza)

---------------------------------------------------------------------------------------------------

Scaling shown in 3 ways:

(1) by 'sequential' CL & WCL change :
eg from CL9 @ WCL8, 7, 6; to CL8 @ WCL8, 7, 6; etc
(zig-zag line)

(2) by fixed WCL with CL change :
eg CL9-CL6 @ WCL8; CL9-CL6 @ WCL7; CL9-CL6 @ WCL6
(blue, orange, green lines)

(3) grouped by WCL offset to CL :
eg 'WCL = CL+1' grouping; 'WCL = CL, & WCL = CL-1' grouping; 'WCL = CL-2' grouping
('parallel' lines)

---------------------------------------------------------------------------------------------------

Hyper : 1067MHz : CL9 to CL6 (x-10-7-26)

(1) (2) (3)

---------------------------------------------------------------------------------------------------

PSC : 1067MHz : CL9 to CL6 (x-10-7-26)

(1) (2) (3)

---------------------------------------------------------------------------------------------------

PSC : 933MHz : CL9 to CL6 (x-10-7-26)

(1) (2) (3)

---------------------------------------------------------------------------------------------------

PSC : 800MHz : CL9 to CL6 (x-10-7-26)

(1) (2) (3)


---------------------------------------------------------------------------------------------------

Hyper & PSC : CL9 to CL6 (x-10-7-26)

single image for each scaling type (as columns^)

(1) 'sequential'
(2) 'fixed WCL'
(3) 'parallel'

note : 1327 x 4040 px / ~300kb (per image)


(1) (2) (3)


Hyper & PSC : CL10 to CL6 (x-10-7-26)

note : 1513 x 4536 px / ~380kb (per image)


(1) (2) (3)


---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------


32M Screenshots

note: all screens with AIDA v1.50 1200
(approx for newer AIDA versions: Read = -100 to -150Mb/s; Copy = -1Gb/s; Latency = same)

---------------------------------------------------------------------------------------------------


Hyper : 1067MHz : CL10 to CL6 (x-10-7-26)

...............................................................................................





CL10 / WCL8 / 39-40 : 397.062 .. CL10 / WCL7 / 39-40 : 396.984 .. CL10 / WCL6 / 39-40 : 397.546

 CL9 / WCL8 / 37-38 : 396.609 ... CL9 / WCL7 / 37-38 : 396.171 ... CL9 / WCL6 / 37-38 : 396.093

 CL8 / WCL8 / 35-36 : 395.985 ... CL8 / WCL7 / 35-36 : 395.735 ... CL8 / WCL6 / 35-36 : 395.312

 CL7 / WCL8 / 33-34 : 395.547 ... CL7 / WCL7 / 33-34 : 395.078 ... CL7 / WCL6 / 33-34 : 394.891

 CL6 / WCL8 / 31-32 : 395.000 ... CL6 / WCL7 / 31-32 : 394.719 ... CL6 / WCL6 / 31-32 : 394.235

---------------------------------------------------------------------------------------------------


PSC : 1067MHz : CL10 to CL6 (x-10-7-26)

...............................................................................................





CL10 / WCL8 / 39-40 : 397.125 .. CL10 / WCL7 / 39-40 : 397.047 .. CL10 / WCL6 / 37-38 : 396.922

 CL9 / WCL8 / 37-38 : 396.593 ... CL9 / WCL7 / 37-38 : 396.172 ... CL9 / WCL6 / 37-38 : 396.094

 CL8 / WCL8 / 35-36 : 395.985 ... CL8 / WCL7 / 35-36 : 395.734 ... CL8 / WCL6 / 35-36 : 395.312

 CL7 / WCL8 / 35-36 : 396.187 ... CL7 / WCL7 / 33-34 : 395.094 ... CL7 / WCL6 / 33-34 : 394.891

 CL6 / WCL8 / 33-34 : 395.641 ... CL6 / WCL7 / 33-34 : 395.359

---------------------------------------------------------------------------------------------------


PSC : 933MHz : CL10 to CL6 (x-10-7-26)

...............................................................................................





CL10 / WCL8 / 38-39 : 403.594 .. CL10 / WCL7 / 38-39 : 403.469 .. CL10 / WCL6 / 36-37 : 403.500

 CL9 / WCL8 / 36-37 : 403.109 ... CL9 / WCL7 / 36-37 : 402.641 ... CL9 / WCL6 / 36-37 : 402.515

 CL8 / WCL8 / 34-35 : 402.406 ... CL8 / WCL7 / 34-35 : 402.157 ... CL8 / WCL6 / 34-35 : 401.656

 CL7 / WCL8 / 34-35 : 402.735 ... CL7 / WCL7 / 32-33 : 401.453 ... CL7 / WCL6 / 32-33 : 401.203

 CL6 / WCL8 / 32-33 : 402.156 ... CL6 / WCL7 / 32-33 : 401.750 ... CL6 / WCL6 / 30-31 : 400.500

---------------------------------------------------------------------------------------------------


PSC : 800MHz : CL10 to CL6 (x-10-7-26)

...............................................................................................





CL10 / WCL8 / 37-38 : 412.750 .. CL10 / WCL7 / 37-38 : 412.625 .. CL10 / WCL6 / 35-36 : 412.562

 CL9 / WCL8 / 35-36 : 412.172 ... CL9 / WCL7 / 35-36 : 411.547 ... CL9 / WCL6 / 35-36 : 411.469

 CL8 / WCL8 / 33-34 : 411.328 ... CL8 / WCL7 / 33-34 : 411.031 ... CL8 / WCL6 / 33-34 : 410.406

 CL7 / WCL8 / 33-34 : 411.735 ... CL7 / WCL7 / 31-32 : 410.234 ... CL7 / WCL6 / 31-32 : 409.891

 CL6 / WCL8 / 31-32 : 410.984 ... CL6 / WCL7 / 31-32 : 410.610 ... CL6 / WCL6 / 29-30 : 409.063

[cheapseats]

Hyper CL | RTL : Superpi 32M performance scaling charts

(x-7-5-20 + optimal WCL)

---------------------------------------------------------------------------------------------------

2600K / P67A-UD5 (B2) F7e / 2x2GB Corsair CMGTX2 (Hyper)

100.0 x 50 (5GHz)

DDR3-2133 (1067MHz) / DDR3-1866 (933MHz) / DDR3-1600 (800MHz)

BIOS manual set : x-7-5-20 / tRC 25 / tRFC 55 / tWTR 4 / tRRD 4 / tRTP 4 / tWR 6 / tFAW 16 / CR1

XP Pro sp2 + Spitweaker v1.0 (copy waza)

---------------------------------------------------------------------------------------------------

Hyper : 1067MHz : CL10 to CL6 (x-7-5-20)



---------------------------------------------------------------------------------------------------

Hyper : 933MHz : CL10 to CL6 (x-7-5-20)



---------------------------------------------------------------------------------------------------

Hyper : 800MHz : CL10 to CL6 (x-7-5-20)



---------------------------------------------------------------------------------------------------

1414 x 3018px / ~220kb



---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------

32M Screenshots

note: all screens with AIDA v1.50 1200
(approx for newer AIDA versions: Read = -100 to -150Mb/s; Copy = -1Gb/s; Latency = same)

---------------------------------------------------------------------------------------------------

Hyper : 1067/933/800MHz : CL10 to CL6 (x-7-5-20)

...............................................................................................


1067MHz

CL10 / WCL7 / 39-40 : 393.750
.CL9 / WCL6 / 37-38 : 392.828
.CL8 / WCL6 / 35-36 : 392.156
.CL7 / WCL6 / 33-34 : 391.797
.CL6 / WCL6 / 31-32 : 391.188


933MHz

CL10 / WCL7 / 38-39 : 399.688
.CL9 / WCL6 / 36-37 : 398.672
.CL8 / WCL6 / 34-35 : 397.906
.CL7 / WCL6 / 32-33 : 397.547
.CL6 / WCL6 / 30-31 : 396.860


800MHz

CL10 / WCL7 / 37-38 : 408.016
.CL9 / WCL6 / 35-36 : 406.828
.CL8 / WCL6 / 33-34 : 405.922
.CL7 / WCL6 / 31-32 : 405.437
.CL6 / WCL6 / 29-30 : 404.641

[cheapseats]

reserved.

[M.Beier]

Great job, and that is ALOT of testing indeed!
Thanks for heads up.

[Hazzan]

Great experience this awesome really...

[Punisher!]

Great experience this awesome really...

+1!!!

[youngpro]

seriously sick testing dude, when i get time i will read the whole thing, thanks for sharing the hard work

[Hondacity]

Thanks ChiefSeats

[cheapseats]

Thanks very much for your comments guys

[Don_Dan]

Wow, that must have been a lot of work! Thanks for posting!

[Hyperhorn]

Awesome thread, thank you for sharing the information!
I'll take a deeper look later ...

[eva2000]

wow, I am absent from forums for a while, and such great info gets posted!

awesome stuff cheapseats !

[massman]

Whoah! One of the better threads I've seen here in the last year or so.

[Mafio]

damn, that's some serious testing
thanks for sharing

[Calathea]

Very helpful, thanks a lot! I tried 8-9-6 wcl 6 instead of 7-9-7 wcl 8(min with cl7) with ripjaws bbse and got around 1 sec in 32m

[cheapseats]

Thanks to all

Very helpful, thanks a lot! I tried 8-9-6 wcl 6 instead of 7-9-7 wcl 8(min with cl7) with ripjaws bbse and got around 1 sec in 32m

nice to hear you got a boost

WCL 7 or 6 should also be possible at CL7, but setting either of those WCL values
will force RTL decrease compared with RTL @ CL7 WCL8.

The RTL decrease will require a significant increase in min vdimm, eg ~11%
so CL7 WCL7 (or WCL6) will be unbootable until vdimm is raised sufficiently.

[Raja@ASUS]

Woah, lots of data must of taken a huge chunk of time. Bottom line is Hyper ICs could handle higher IO at a given clock frequency (where signal integrity is not a limting factor) than the newer parts.

The RTL changes simply restrict the peak IO when multiple transactions are made and provide a little bit of breathing room plus are affected by any IO buffer delays and clock skew. What you are effectively playing with is a combination of setting adjustments needed to remain within the IO capabilities of the memory ICs at a given level of VDIMM.

The overall balance for scores in a benchmark as far as adjustments to WCL versus CL come from the associated read to write delays, some of which are limited due to the architecture, and electrical mechanisms.

[M.Beier]

The thread should deff be sticky...

[StrategosSan]

Respect for sharing the result of your hard work Thumbs up!

Thank you man