Sandforce Life Time Throttling

[sergiu]

It is not so wild I proved the hang was TRIM related over in the Endurance thread.

I have not expressed my thoughts well, my mistake. What I would suggest would be disabling trim (+OS restart to make sure everything is applied), do a secure erase with all methods you know and start the endurance again. Maybe writing at a lower speed due to lack of trim will trigger the Duraclass much later.

[Ao1]

I have not expressed my thoughts well, my mistake. What I would suggest would be disabling trim (+OS restart to make sure everything is applied), do a secure erase with all methods you know and start the endurance again. Maybe writing at a lower speed due to lack of trim will trigger the Duraclass much later.

Will do.

[Ao1]

I have not expressed my thoughts well, my mistake. What I would suggest would be disabling trim (+OS restart to make sure everything is applied), do a secure erase with all methods you know and start the endurance again. Maybe writing at a lower speed due to lack of trim will trigger the Duraclass much later.

Sergiu, it's probably best to leave the V2 to idle for a week or so. Anything I do now will most likely be cut quite short.

I can do it now if you think that would be better. Your shot

[sergiu]

Sergiu, it's probably best to leave the V2 to idle for a week or so. Anything I do now will most likely me quite short.

I can do it now if you think that would be better. Your shot

I believe a few hours of idle would be enough, let's say 5-8h.
I also have another theory based on behavior. Assuming that lifetime throttle is indeed around 50% then what we see is a completely abnormal throttling which might be related to some partial hardware failure. Another guess (this time completely wild) is that there is some electronic component that, due to extensive stress generated by writing, is heating up and no longer behaves normally thus inducing some kind of errors internally. These errors are either corrected successfully in a longer time, either, controller itself is switching to a lower internal speed. This unfortunately could be confirmed by running same tests on a second identical drive, preferably from another batch.

[Ao1]

I believe a few hours of idle would be enough, let's say 5-8h.
I also have another theory based on behavior. Assuming that lifetime throttle is indeed around 50% then what we see is a completely abnormal throttling which might be related to some partial hardware failure. Another guess (this time completely wild) is that there is some electronic component that, due to extensive stress generated by writing, is heating up and no longer behaves normally thus inducing some kind of errors internally. These errors are either corrected successfully in a longer time, either, controller itself is switching to a lower internal speed. This unfortunately could be confirmed by running same tests on a second identical drive, preferably from another batch.

It would be great if someone else could duplicate what I have done. Then its not me saying it again. I really doubt however that the drive is performing in any way different to how it was designed to behave.

Anyway, I'll start a new run tomorrow.

From the 1TB of 0fill experiment there were 128GB of writes

131,072MB in 2.85hours

45,990MB per hour

Avg write speed to NAND = 12.77MB/s

Anvils app was showing Avg ~100MB/s whilst it was running during this period.

[gojirasan]

It seems groberts101 is simply denying the existence of life/warranty throttling completely despite the repeated admissions of OCZ and boasts of this "performance throttling" right on Sandforce's web site. OCZ, Sandorce, and Ao1's testing are all obviously wrong. Ao1's data must be some kind of drive malfunction that strangely seems to mimic a data/time curve .

He seems to be asserting the existence of 3 and only 3 states:

1. Fresh. Drive is brand new or has been secure erased. No block assignments in the FTL and all blocks = 1.

2. Garbage collection begins. All blocks have been written too. So Sandforce must reluctantly start recycling used blocks with GC thus theoretically slowing down the drive if they ignored GC initially. This is what he is calling "settled". Why does the controller wait until there is absolutely no choice to start GC? Possibly in order to get higher benchmarks in drive reviews from reviewers who didn't write to every block on their drive before benchmarking. Or it could just be the way the GC algorithms were written. The algorithms may just ignore the "fresh" state because it is so brief.

Ao1 hasn't seen any difference between state 1 and 2. Maybe because most reviewers aren't being sent those small drives for review. So there is no advantage in delaying the normal GC operation. Or maybe at such slow right speeds there isn't any initial speed advantage to temporarily disabling GC.

3. Garbage collection failure. Inadequate GC cannot keep up with sustained writes and the drive has run out of clean blocks to write to. In order to prevent the kind of application interrupting stuttering that Ao1 has been seeing after a TRIM, you throttle the drive down to a speed which the GC process can keep up with. This is a good sort of throttling. Just enough to smooth out the curve. Ao1 hasn't seen this behavior. Probably due to a firmware bug. With sufficiently aggressive GC this short of throttling may not be necessary at all, but such aggressive GC may not be optimal for typical SSD usage patterns in terms of both peak speed and write amplification. So this sort of GC failure is not necessarily a bad thing.

[Ao1]

^ The workload from Anvil's app is not very aggresive. Mostly sequential. If it was 4K random full span, it would no doubt have seen performance degradation.

When I run the app with TRIM deactivated I expect to see performance degradation.

There is also one other form of throttling (and possibly others). Burst throttling.

[zads]

sigh.. I'm trying very hard to stay with you here but you seriously need to stop thinking that you have this controller figured out here.
...
Lets just drop the ego and pride out of it and use some facts that have been proven by many.
...

Hm, I wonder if you could you do the same..?

...
#1
"settled state" throttles(which is my preferenced term over the "lifetime thottle" being used for these drives.
...

I think this is why you are a little bit mistaken.
Settled/steady state performance is much different from the Lifetime/Warranty throttle.
Their result might seem similar on the surface (much reduced write performance), but their causes are different. The settled performance degradation can be reset by SE, the Lifetime/Warranty throttle can not.
Settled/steady state performance is a consequence of the pages being filled with random incompressible data, rather than an actual firmware feature/built-in "throttle".
If you don't believe me, ask your trusted source Tony@OCZ.

...
Hundreds upon hundreds have cleared their drives with an SE and is SOP when issues or heavy throttles(hammered state) arise. And to be clear here.. I have had my drives(singles and arrays) throttled to all known stages(which I've also explained previously, as well). SE recovers/restores factory fresh speeds each and EVERY time.
...

Again, SE is clearing a steady state performance degradation.
How could you tell that the "hundreds upon hundreds" were experiencing a lifetime/warranty/hammered throttle state and not a steady state performance degradation?
If the SE restored performance; either the lifetime throttle released after sitting for some idle time,
or they are NOT experiencing a lifetime/warranty throttle!

...
I know these drives internal algorithms like a bad dream and even know EXACTLY when settled states hit my array. Not to be confused with hammered states.
...

Well, knowing test data in and out of a black box doesn't mean you know the internal algorithms.

What is your personal definition differentiation between settled and hammered state?
I assume your hammered state is referring to lifetime/warranty throttling..

...
#3 This is where you are having difficulty getting your head wrapped around the stages of throttle on this controller. That massive slowdown would be considered a "hammered state throttle" and NOT a settled state. A settled state throttle will limit incompressible write speeds to about 70% of fresh speed. Hammered states would be about 50% of fresh when writing incompressible data.
...

Please don't call it a "settled state throttle".
Please call it steady state/settled performance degradation.
Throttle implies that there is something in the firmware causing a reduction in performance on purpose.
This is not such a case.
Be careful with those numbers too. You aren't stating specific conditions for those performance numbers (70%), which will make it vary widely.
This is not absolute for steady state performance degradation.
It depends on existing data and transaction history, compression, etc.
I can make run steady state performance on any SF-1200/1500/2100/2200/2500 all over the board.
The 2500 starts off at 90k random 4k write IOPS, and I can run it steady state (>24 hours) from 8k to 60k IOPS.

Would need a lot more trust and opening of the mind before that would be possible, though.
...
It's far more to do with what I think of myself and the need to share.. than what anyone else thinks of me as I;m not nearly that shallow. If you and everyone else keep putting pride and ego in the way then I'll just sit back and watch you all flounder around and develop even more false pride in "what you know" about these controllers. Again.. I'm trying to be patient with some here(you mostly) and will take some time to help you along as best I can but constantly revisiting things that have already been explained will surely just dry up my unthusiasm to do so. Sounds harsh and ultimatum like, I know.. but again with the "from the hip" attitude.
....
lalalala
...

I understand you are trying to help here, and you have many of the basic concepts correct.
But there is some confusion/mixup in your definitions and understanding of what's going on in the controller.

Then you clearly have NOT had your drive in the state where it is throttled from writing too many GBs in too few power-on hours. Ao1 has. You are wrong, Ao1 is correct. You might try writing less and reading (and thinking) more.

I think you are correct that his drives never hit the lifetime/warranty throttle.
It can be hard to hit the lifetime throttle barrier if you have significant idle time on the drive.
For example, assuming steady state uncompressible sequential performance of 100MB/sec (just a made up number) on a 120GB SF-1200 drive with 34nm flash (5k PE cycles) with the life curve set at 1 year:
If you let it idle for a week before trying to hammer the drive, you have to hit it for more than 38 hours nonstop before it hits the theoretical limit line.
And then the controller will allow a little bit longer than that before it seriously cuts your writes.