Raid 0 - Impact on NAND Writes

[sergiu]

I have watched all HDTune screenshots with a grain of salt, as results seems to be quite odd. I have run the benchmark on a Vertex 2 and 3 (both OS drives) and results were quite interesting:
Vertex 2 120GB (@SATA1, no TRIM, no OP): access time = 0.220 ms average speed: 102MB/s
Vertex 3 120GB (@SATA2, TRIM, no OP): access time = 0.362 ms average speed: 137MB/s

I took a look at how HDTune does the requests using diskmon and I found that is doing the read test using sequential 64KB blocks, while access time is done using single 512byte sectors, with unaligned addresses. Similar read speeds were easily reproduced with ASU by setting random read block to 64KB and thread number to 1. Given the conditions, non Sandforce drives might have similar results.
From my point of view, HDTune tests are not relevant as for sequential access, block size used by real world applications is in MB range. 64KB means 8 pages. Depending on how these were written, it might be spread in 1 to 8 dies, so the controller could or could not reach all of them concurrently. This explains why after trim you see max speed, as there is nothing to read, you just get automatic generated zeroes.

[Anvil]

You need to change the block size in HDTune, 64KB is default.

I never paid much attention to the other "benchmarks" in HDTune as they were made for HDD's and not SSD's.
(it still works pretty well for HDD's)

[sergiu]

Not even noticed it has an option for block size. I took some tests at various block size and I believe is seriously flawed. Here are the first read commands captured from diskmon:
Sector - Length
0 1
1 1024
1025 1024
2049 1024

During the benchmark, from time to time, it is reading just one sector, and sector address is shifted accordingly. Interesting part is that I really needed the maximum selectable block size (8MB) to achieve most stable result.

[johnw]

Not even noticed it has an option for block size. I took some tests at various block size and I believe is seriously flawed. Here are the first read commands captured from diskmon:
Sector - Length
0 1
1 1024
1025 1024
2049 1024

During the benchmark, from time to time, it is reading just one sector, and sector address is shifted accordingly. Interesting part is that I really needed the maximum selectable block size (8MB) to achieve most stable result.

Is it actually reading every LBA on the device? I had thought it would read a chunk, then skip a large chunk then read, skip, etc. Because reading every LBA on, say a 2TB HDD, would take a LONG time.

[sergiu]

Normally it uses chunks. I selected full drive to see what is the first LBA read. However, both chunk/full drive result in shifted LBAs.

[johnw]

Normally it uses chunks. I selected full drive to see what is the first LBA read. However, both chunk/full drive result in shifted LBAs.

I'm not sure 4KiB alignment makes a big difference when reading large chunks. If it reads 1024 consecutive 512B sectors, then if aligned on a 4KiB boundary the SSD would need to read 128 pages (4KiB each). If it is not aligned, then the SSD has to read 129 pages (one additional page). That is only 0.8% overhead.

[some_one]

That's a very astute observation sergiu, but I'd be inclined to agree with johnw here. Perhaps though we're looking at it too simply. What happens when you request a sector of 512bytes, does windows read a full cluster?

[Ao1]

With HD Tune you can set the block size in the options tab. You can also set a partial or full test. I have used the 4MB block size throughout.



Currently my V2 is set as a single drive. I used ASU to run a read benchmark with a 1GB test file on the V2. Read speed = 143.46MB/s. I then ran a HD Tune benchmark. Avg read speed = 262.4MB/s, however the read speed at the location of the ASU test file came out at 135.6MB/s.



Here I run ASU with a 32GB test file (4MB blocks) Read speed = 142.09MB/s. I then ran a HD Tune benchmark. Avg read speed = 155.5MB/s, the read speed at the location of the ASU test file came out at 134.3MB/s.

[Ao1]

That's a very astute observation sergiu, but I'd be inclined to agree with johnw here. Perhaps though we're looking at it too simply. What happens when you request a sector of 512bytes, does windows read a full cluster?

For reads a page is the minimum amount that can be read. For an erase operation it is a block, so a read of 512b will equal a 4K or 8K read depending on nand geometry.

[Ao1]

Same run now for the V3. Read speeds are not impacted in the same way as the V2.

[Ao1]

V3. 4MB block read speeds after 6 minutes/ 10GB of 4K random writes. It looks like the V3 is much more resilient with 4MB sequential xfers, but 4K xfers still have a big impact on read speeds.

Min read speed = 53.8MB/s
Read speed at location of 4K write = ~150MB/s
Max read speed 253.2MB/s



For client work loads, which cannot be compressed that well by the SF controller, read/ write performance sucks.

[sergiu]

I'm not sure 4KiB alignment makes a big difference when reading large chunks. If it reads 1024 consecutive 512B sectors, then if aligned on a 4KiB boundary the SSD would need to read 128 pages (4KiB each). If it is not aligned, then the SSD has to read 129 pages (one additional page). That is only 0.8% overhead.

For large blocks (>256K) indeed does not matter, but for small one, like the default 64K value, it counts for sure.

@Ao1
Filling the drive with random 4K data will force the drive to optimize writes in such a way that reads would be better also random. Have you tried to fill the drive with movies and then read it with HDTune (4MB blocks) ? This should give you the maximum read speed that drive can sustain for it's configuration. And for a small drive, like 60GB models, it would make sense to have smaller values. The question is if this is significantly lower than what competition can do with same NAND.

[Ao1]

V2. Drive SE’d and then filled with 4MB blocks using ASU. Run read with HD Tune:



Delete test file. Run read with HD Tune. It can be seen that the TRIM command executed just after the read operation started. As soon it was complete read speeds were restored. In comparison to the R0 array reads speeds did not restore, even after 4MB blocks were written several times.



It would be good to see how non SF drives performed in comparison. I can only conclude that running SF drives in R0 is not a good move.

[Ao1]

For large blocks (>256K) indeed does not matter, but for small one, like the default 64K value, it counts for sure.

@Ao1
Filling the drive with random 4K data will force the drive to optimize writes in such a way that reads would be better also random. Have you tried to fill the drive with movies and then read it with HDTune (4MB blocks) ? This should give you the maximum read speed that drive can sustain for it's configuration. And for a small drive, like 60GB models, it would make sense to have smaller values. The question is if this is significantly lower than what competition can do with same NAND.

Let me test with real data. I will use a mixture of data to try and get a balance between compressible and non-compressible.

[Anvil]

I've spent some time testing the array of Intel G1's I use for booting on one of the X58's

I was surprised at the outcome, will post a little later.

[Ao1]

V2 Drive SE’d.

Copy Windows folder over = 19.8GB

It looks like the Windows folder can be compressed by around 50%.

[Ao1]

Add My Videos folder = 11.1GB

It looks like the Videos folder could not be compressed.

[Ao1]

Add My Documents folder = 4.91GB

It doesn’t look like it could be compressed by much.

[Ao1]

For comparison this is from an X25-MG 2 160GB drive. It’s the same drive that I used to copy over the My Documents and My Videos folders. The drive has 50GB free of a formatted space of 149GB. This drive contains all my static data, which for the most part is not compressible. This drive has not been SE’d for a very long time.

[sergiu]

Well... mistery solved: 130-140MB/s read speed for incompressible data for 60GB model. This is all you can get from drive, nomatter if it is in RAID or not. Now we only need to find out the values for other models (120, 240 and 480GB), both V2 and V3.

[Ao1]

And here is my other X25-M 160GB that I use for the OS. This drive was SE’d around a month ago. 95.4GB free of 148GB

[Ao1]

Double post

[Anvil]

This is before and after a Fill drive (using 46%) on the F3 used in the Endurance test

(before)


(filled drive)


(after deleting file, performance restored)


The degrading performance seen here is due to the random writes which are part of the loop.
All random IO is performed on a single file and deleting that file does not clean up the drive, this type of IO is hard to clean.

edit:
This kind of degrading performance can be seen on other drives as well, e.g. the Intel controller.

[Ao1]

It would be great to see a wider range of results; however from post # 118 the V2 came up with an Avg read speed of 192.7MB/s. The same (real life) data in post #119 on the X25-M returned an Avg read speed of 251.5MB/s.
The X25-M > V2hands down when it comes to read speeds.

[Ao1]

As can be seen in post #92, 4MB block writes using 0 fill does the same job as a TRIM operation with regards to restoring read performance.

As could be seen from the compression tests (in other threads), SF compression is not that good for typical client workloads. The compression algorithm was most likely designed and optimised for enterprise workloads. Bottom line, as far as I can see, is that client compression is nothing like 50%. On average I’d guess it would be single digit compression at best.

The Windows folder was compressed by 50% but even here, when the SF drive was written to only once straight after a SE the read speeds fluctuated wildly between 140MB/s & 260MB/s. I’d guess the fluctuation was due the fact that not all of the files within the folder could be compressed by the same amount.