Raid 0 - Impact on NAND Writes

[Ao1]

New results based on ASU Client added to first post.

Write distribution between the drives is a lot better, but both drives are still incurring writes in excess of the amount of data written.

Also it seems that after reaching a steady state there is a further degraded state down the line – Avg 66.34MB/s = write speeds 50% below steady state after writing ~300GiB.

[Ao1]

With static data around, ratio between spare area and usable area for 4K full span is higher which would lead also to better performance. Trimming might have helped because of the file size (4MiB which is equal to 512 or 1024 pages) possibly at the cost of higher WA. But for 100% 4K writes full span where drive would be seen as a single file in which pages are constantly overwritten, I cannot see how trim will help, as each overwritten page would free only one. But who knows, maybe with trim help, wear leveling algorithm would choose better candidates for erasing.

I'm going to re-test a single drive later. I believe it might have had 50% static data on it.

[Anvil]

I'll send you a link to a slightly adjusted app.
(some minor adjustments)

[Ao1]

With static data around, ratio between spare area and usable area for 4K full span is higher which would lead also to better performance. Trimming might have helped because of the file size (4MiB which is equal to 512 or 1024 pages) possibly at the cost of higher WA. But for 100% 4K writes full span where drive would be seen as a single file in which pages are constantly overwritten, I cannot see how trim will help, as each overwritten page would free only one. But who knows, maybe with trim help, wear leveling algorithm would choose better candidates for erasing.

You are right. I'm running the enterprise app (4K random) on the V2 & V3 as single drives with no static data and the drives are degrading just as badly as the R0 array. I'm down to 13MB/s on both drives after only 10GB of data

edit. lol, after 16GB I'm down to 10MB/s on both drives. Worse that R0. Maybe with R0 the writes are getting cached. I'm going to create a R0 array but this time I'll set aside 50% for OP.

[sergiu]

Let it run for a few hundred's of GB, as I have the feeling the speed will stabilize to an even lower value. Also, what is WA value? For the WA, please take the readings after write speed stabilization, as I believe instant WA increases as write speed decreases. With 50% OP you should see a huge difference in write speed, but what would be more interesting to follow is average/max write latency between those states.

[Ao1]

Stroll on. R0 128K stripe 50% OP 4K random "full" span

[Ao1]

^ I'll go back to the single drives later as they will take forever at that speed. (and I'll monitor avg/ max latency as well. I already know its going to get very high).

R0 128K stripe 50% OP 4K random "full" span





Uploaded with ImageShack.us

[sergiu]

What QD is used for generating 4K writes? If it is 1, then it would be also interesting to see what happens with higher QD values, although I suspect aggregated speed will not increase

[Ao1]

First post updated. Using 50% OP transformed the outcome. Hard to believe it had that much impact.




@ Sergiu I believe it is all qd1

[Ao1]

Vertex 2 - 4K random - 14.65GB user space, 22.62GB OP



Meanwhile the Vertex 3 is struggling. 4K random – no OP

[sergiu]

At least is still much faster than a RAID array based on normal HDDs. Yet the difference is impressive. Luckly there is no "normal" usage pattern that would lead to something like that

[alfaunits]

OT: I'd be quite worried about "technological progress" if a newer (Vertex 3) SSD of the same company (OCZ) is outperformed heavily by an earlier generation SSD (Vertex 2).

[Ao1]

OT: I'd be quite worried about "technological progress" if a newer (Vertex 3) SSD of the same company (OCZ) is outperformed heavily by an earlier generation SSD (Vertex 2).

It’s not. The difference is over provisioning. I can't believe how much difference it makes. It’s as if the work load was converted to sequential. Now I can see why TRIM for raid is not on the top of developers list. With a random work load it makes no difference. With raid you get cache to help alleviate random writes, plus with OP it doesn’t matter anyway.

I’m finishing off the 4K random on the V3 and then I will run again but with static data. Let’s see how good the wear levelling is. From initial testing it’s not as good as OP, but it’s not far behind.

[Ao1]

I had to reboot before I got to 200GB First post updated.

[sergiu]

Wow... so speed dropped even further. Could you look for WA for next 50-100GB? Now write speed should have been stabilized. Also, there might be a difference in speed in favour of Vertex 2 because of NAND geometry in worst case scenario. Having 512KB blocks instead of 2MB ones (or 1MB) might help a little when recycling. However these might be negated by advancements in controller and better NAND write speed.

[Ao1]

V2 with ~50% static data. So far holding out better when compared to no static data, but not as well as OP.

[Ao1]

The graph is lacking in granularity, but it was the only way I could find to show the stepped drop in performance that occurs with 4k random (Vertex 3, no OP, full span on a SE’d drive). Write speeds are consistent for the first 2 minutes and then they drop by around 50% but fluctuate below. After 6 minutes write speeds fluctuate considerably. On avg I’d say another 50% drop.

[Ao1]

Here is a direct comparison between a drive with no static data and an empty drive. (tests started around the same time) Having static data certainly helps, but it’s not as good as OP.



Here are the response times of the V3 taken as the drive started to degrade. 83.44% of writes below 10MB/s and 41.28% of (4K) xfers between 500us and 1ms. I’ll check if I get time but I think the V3 is much better with response times compared to the V2.

[Anvil]

Getting really interesting

You should be able to read SMART even if the drives are in raid, I can read my m4's using the Intel SSD Toolbox, will try a small array of SF drives later.
(the question is will all attributes display or only the ones that Intel uses?)

[Ao1]

So with 50% static data long term performance is double and writes to nand are reduced. I'm wondering how this fits into what could be observed with the B1 values on SymbiosVyse's drive (that had no static data).

Getting really interesting

Thanks Anvil, this has been quiet hard work. (Time intensive) It's been a bit of an eye opener for me though so I'm glad I did it.

[Ao1]

Wow... so speed dropped even further. Could you look for WA for next 50-100GB? Now write speed should have been stabilized. Also, there might be a difference in speed in favour of Vertex 2 because of NAND geometry in worst case scenario. Having 512KB blocks instead of 2MB ones (or 1MB) might help a little when recycling. However these might be negated by advancements in controller and better NAND write speed.

There you go. (Ignore the elapsed time as I had to pause briefly a few times.).
Ratio between nand writes and host writes = 1:0.148 (nasty)

[Ao1]

The reads are just as important (if not more so) as the writes and the test shows that it's highly degraded, I'd say 134MB/s is as expected from such an exercise. (unfortunately)

HDTune won't let you run a write test with an active partition but the read test should work, otherwise there is something seriously wrong.

edit:
Next time you should try deleting the file from the Endurance app and then Fill the drive it using 4MB blocks and then do a HDTune read test. It should show how quickly it restores performance, reads in particular.

This is on a single V3. When I stopped the 4K writting I deleted the test file and started up HD Tune. Looks like a TRIM operation cleared up drive to allow read speeds to return.





(Edit: Note only one IOP for the TRIM command. That is why the drive appears to hang. The hole drive is cleaned up in one go)

[Anvil]

It cleans up nicely, 20-21 seconds isn't that bad for a full TRIM.

I expect this is on a 3Gb/s controller?
6Gb/s will probably show marginally better results.

[Ao1]

Yep it’s a 3Gb/s controller. Still for 4k, bandwidth limits are not such a problem (Who knows for how much longer. Maybe its not so far away).
I was going to update to the X79, but I’m not so sure now. Might wait another gen.

[sergiu]

There you go. (Ignore the elapsed time as I had to pause briefly a few times.).
Ratio between nand writes and host writes = 1:0.148 (nasty)

That's quite a good ratio for such a test, much better than what I expected! I would expected something like 1:0.1-1:0.03. But, most probably you need to know fine details of the wear leveling algorithm to design something against it.