Anvil's Storage Utilities

[Christopher]

I'm no Shill for SF and SF's marketing, but I'd rather have a universally accepted SNIA spec than a fractured comittee. If you're looking at writes by volume, most are small compressible writes anyway. If you look at it by size, the larger the write, the less likely it's compressible. On average, I think the 46 to 67 percent encapsulates an average client workload. The host system is always writing small bits of data, while larger accesses are usually initiated by the user.

I'd like to see more transfer size and access patterns vs. data compressibility. The larger the transfer, the less random and less compressible it becomes.

If SNIA has any chance of getting the acceptance it deserves on client side storage, some concessions to SF will have to be made. But the more time I spend trying to understand SF, the more I come to terms with that. The truth is, the best SF drives are extremely competitive on speed. Latency will be a problem in some cases, but you do get some advantage even with 80% compressible data. After 46% SF really plateaus, but there is still an advantage there. Now, my time futzing with SF leads me to believe that there is more overhead than is user visible, perhaps enough to overcome the compression endurance advantage, but it could also be a case where more over provisioning would pay dividends as I've maintained for some time.

Now, this is entirely separate from steady state performance, but the more compressible the data, the longer the time to achieve steady state. I'll have to play around some more when I get home, but I also believe some of the housekeeping algorithms in 5.0 reference FW are different, but steady state performance will continue to be an area where improvement is needed. But some SF do have some generally desirable attributes above and beyond the obvious, like stellar 4k qd1 performance. That's not a SF exclusive trait, but it's one area of performance I prize.

[johnw]

You can babble all you want, but the fact is that the only objective test possible is using random data, as the SNIA SSS documents specify for the mandatory test. SNIA allows testers to use non-random data streams (as long as the data stream is specified) in addition to the mandatory random data stream test, but the random data stream is mandatory.

As for you're other vague claims, they are highly debatable (even evidence posted in these forums contradicts them), and have no place in an objective test specification such as SNIA SSS testing protocols.

[Christopher]

Those are merely my own observations, but SNIA is an industry group which needs unity from its members.

Let me know when you get elected king of SNIA.

[Computurd]

It is easier for some to set back and criticize others efforts, than it is for them to actually do anything to contribute.

I guess that could be some sort of nice saying;

It is easier to criticize than it is to do.

[flamenko]

I just think that when benchmarking, it's ridiculous to just show zero fill. It's principle more than anything.

That's why I like ASU -- I can just bench a SF with every compressibility level and then weight the results as I please. 47% to 67% are far more realistic an average than 0/100%, but 67% on SF is pretty much incompressible I think.

I'd have to double check, but I did break out a SF2281 the other day to upgrade some FW and do a SE. I was pleased with it's incompressible performance.

Happened to come across this and thought I would throw in my two cents. I don't agree at all with this statement and think, quite honestly, that such negates a very important piece of the pie when we speak of benchmark testing and it's relationship to computer use. In simple terms, the importance of testing in 0Fill, or highly compressible data, cannot be understated for the consumer side of things, just as testing in incompressible data (or random data samples) holds a more specific value for the enterprise side of things.

I can go back to the beginning of testing with this same argument and, quite honestly, would have believed the naysayers of testing in highly compressible data (OFill) would have seen the light by now. WE went head to head for years now with many berating the ideal that reviewers, myself included, would test in highly compressible data and show it's meaning and value on a review.

Imagine if you would how confused the consumer would be if we had never shown that side of things and explained the difference between the two.

Moving on, PCMark Vantage is recognized by all reviewers as being the 'industry standard' of consumer SSD testing and, well the simple facts show that the scoring realized through Vantage follows that of testing highly compressible data (OFill) much more accurately than testing that of incompressible data. Actually the new Vertex 4 is pretty much the icing on the cake with respect to an example on this one.

I know I may be going against the views of two very good friends on this one but, the truth is that, for the typical consumer, oFill (or testing with highly compressible data) is just as important as testing with incompressible data for more specific needs such as video and photography and even reaching right into the business and enterprise side of things.

To make a statement that says that you believe that testing with oFill data should be outlawed (colorful term) shows a very close minded attitude and really negates the entire side of the debate. Quite frankly, it goes so far as to even put the credibility of the person making the statement into jeopardy.

Just my thoughts! It is kind of amusing actually because I can probably pull up threads on this forum just over two years ago where I stood strong on this exact subject. To my advantage, test results show a very clear picture.

As for SandForce and their marketing, as much as many might not like hearing it, it borders on brilliance. There have been very few in technology (much less the storage industry) to make the steps that they did over the course of just under three years. They have become a part of every consumer SSD manufacturer today EXCEPT for Samsung and Crucial. Yup, that includes Intel. They were then purchase by LSI to top things off. Just how big would the line have been if they went public before the LSI purchase?

[canthearu]

I can go back to the beginning of testing with this same argument and, quite honestly, would have believed the naysayers of testing in highly compressible data (OFill) would have seen the light by now. WE went head to head for years now with many berating the ideal that reviewers, myself included, would test in highly compressible data and show it's meaning and value on a review.

While I don't think 0-fill testing is completely pointless, I have the following points to make:

a) Sandforce drives do far too well on 0-fill testing .... almost no real world load has the same results. A sandforce with 64gig Async NAND does just as well as a sandforce drive with 64gig Toggle NAND in 0-fills. Real world situations start at 47% compressible and really just get less compressible from there.
b) Running a benchmark in 0-fill mode on a sandforce controller really reveals very little since you are really just speaking to the controller and barely touching the NAND behind it. It really isn't a very useful benchmark for exploring a sandforce SSD. If you want to see any differences, you need to look at less compressible datasets.
c) In my real world monitoring of sandforce drives, the average write amplification tends to suggest loads of between 47% and 67% compression. (Of course, during long periods of idling, I see quite a lot of NAND activity, which increases NAND writes but doesn't increase Host Writes much)

However this makes those cheap Sandforce drives with Async NAND viable. Most people can buy a dirt cheap async NAND 120gig, and it will actually perform very well for real world tasks in their system. Drives without compression using Async NAND can have pretty ugly performance stats.

[Ao1]

0 fill is a worthless benchmark statistic unless you happen to be in marketing and would like to completely misrepresent the performance of your product.

The only benefit of running a 0 fill benchmark is that it enables an end user to mimic how marketing people came up with misleading performance statistics.

SNIA is the only benchmark that properly tests a SSD’s performance.

Edit: Disagree? Name one application that uses 0 fill data and then identify how it benefits from 60,000 IOPS.

[canthearu]

SNIA is the only benchmark that properly tests a SSD’s performance.

Do SNIA publish the details of this benchmark?

[Ao1]

http://www.snia.org/forums/sssi/pts

http://www.snia.org/forums/sssi/knowledge/education

[canthearu]

http://www.snia.org/forums/sssi/pts

http://www.snia.org/forums/sssi/knowledge/education

Type have nice ideas there .... but unfortunately there is no downloadable benchmark or results from any drives for comparison (the drives they have done are anonymous ... I can stab in the dark at what they are, but cannot easily confirm)

[XavierMaxx]

Results from recently purchased 2x120 GB SanDisk Extreme SSD at 46%. I used that setting because I'm using them for OS/apps and when I apply the lightest of compression to a full backup, it comes out to 44.9% (and I pre-archive a lot of my stuff), so it is "realistic" for at least my usage. When the drives have some wear and tear, they score closer to 5400. Sorry for leaving out the drives on the image, I hadn't planned on posting this but thought what the heck. Still experimenting with these, the read scores are low on this particular run (usually score 100 points higher on read); is the OS drive after 500GB of writes in a few days, 64k stripe.

anvil_5800.jpg

[johnw]

0 fill is a worthless benchmark statistic unless you happen to be in marketing and would like to completely misrepresent the performance of your product.

The only benefit of running a 0 fill benchmark is that it enables an end user to mimic how marketing people came up with misleading performance statistics.

SNIA is the only benchmark that properly tests a SSD’s performance.

With SNIA, a random data stream is a mandatory part of the test, which makes a lot of sense. SNIA also allows additional tests with non-random data streams (which must be reported by the tester), but the random stream is still mandatory.

Clearly, random data streams should be the default for any test. If someone is testing an SSD that has the ability to do compression, they may want to add some non-random data streams (and report in detail what they used), but they should ALWAYS include a random data stream.

SNIA got this exactly right with their SSS enterprise AND consumer ("client") tests -- a random data stream is mandatory for both enterprise AND consumer tests.

http://www.snia.org/tech_activities/..._standards/pts

[flamenko]

0 fill is a worthless benchmark statistic unless you happen to be in marketing and would like to completely misrepresent the performance of your product.

And I think exactly this thought process would negate the purpose of testing which is to explore the performance in all typical environments. Of course enthusiasts such as yourself want the 'incompressible data' testing first and foremost but the truth is that compressible data is utilized just as incompressible data is and, in fact, many would say more so in the typical user experience. To state on one side that 0 Fill (or testing with highly compressible data) is a worthless benchmark is paramount to stating that testing in 100% incompressible data is just as useless.

Effective testing explores all the variables.

[canthearu]

SNIA got this exactly right with their SSS enterprise AND consumer ("client") tests -- a random data stream is mandatory for both enterprise AND consumer tests.

http://www.snia.org/tech_activities/..._standards/pts

So how does one go about testing an SSD to SNIA standards. I mean, it is great to have standard and the SNIA tests look fairly comprehensive ... but without a means of carrying out the tests, and with nobody doing these tests, it is all a bit academic at the moment.

But I do feel that for at least sandforce drives, all compression levels are worth looking at, because all will appear in typical workloads. Few workloads will persistantly and consistantly present an incompressable or 0-fill load, so it is good to see how performance graduates between fully compressable and fully incompressible.

[johnw]

but the truth is that compressible data is utilized just as incompressible data is and, in fact, many would say more so in the typical user experience.

If "many" would say that, then "many" would be wrong. You are arguing with Ao1 who looked in depth at that very question.

Data compressible by Sandforce controllers is relatively rare for most users day-to-day SSD writes. About the only commonly compressible data is OS and program installs (not day-to-day things for most users, just very occasional), and database and VM applications, which if those are in use, they are usually power users and are well aware of the compressibility of their data. Most users do not run large databases or VMs.

[johnw]

But I do feel that for at least sandforce drives,

And why should anyone care about your vague "feel"?

[flamenko]

If "many" would say that, then "many" would be wrong. You are arguing with Ao1 who looked in depth at that very question.

Data compressible by Sandforce controllers is relatively rare for most users day-to-day SSD writes. About the only commonly compressible data is OS and program installs (not day-to-day things for most users, just very occasional), and database and VM applications, which if those are in use, they are usually power users and are well aware of the compressibility of their data. Most users do not run large databases or VMs.

John W. my old friend... So what you are saying is that, according to Ao1, the typical user utilizes incompressible data more often in typical things such as ...ohhhh I don't know.... system starting, system software such as explorer and e-mail use and even MS Word file creation?

[Ao1]

John W. my old friend... So what you are saying is that, according to Ao1, the typical user utilizes incompressible data more often in typical things such as ...ohhhh I don't know.... system starting, system software such as explorer and e-mail use and even MS Word file creation?

Whilst I believe compressiblity for client applications is limited my statement was based on 0 fill. I am always prepared to be enlightened if you can tell me how an end user benefits from 0 fill. Name one application in which it is relevant and in which the IOPS are utilised and I will change my view.

[Ao1]

First, a sincere apology to Anvil for derailing his thread and detracting from ASU, which is a great benchmark for end users, providing flexibility and ease of use.

The SNIA tests are beyond an end users ability to undertake, but I believe this is the benchmark that vendors should use for their specifications. The benchmark is something that all major SSD vendors have contributed towards and it provides granularity and comparative performance assessments that are beyond any other method of testing.

Here is a shot of 17 drives that were tested with the SNIA specification using 65% reads/ 35% writes. (17 SSD’s and one Enterprise HDD [edit: in yellow]) It is clear to see that there is a significant difference in performance between SSD’s.


http://www.brighttalk.com/webcast/23848%20

Here is a shot of a Sandforce drive. Blue is incompressible. Red is a data base pattern and the green line is 0 fill. Interestingly 0 fill is close to the data base load, but the max IOPS come out at ~35K. Sandforce specify 60,000 burst/20,000 sustained (@4K blocks) for the SF2x drives and 30,000 burst/10,000 sustained (@4K blocks) for SF1x drives.



Sandforce don’t state how they arrived at their specification figures, but presumably they were obtained on a FOB drive using 0 fill. The SNIA test is based on steady state, which is the representative condition of a drive in use.

To prevent Anvil’s thread from being further derailed there should be a separate thread to discuss SNIA. There is already a thread to discuss SF compression.

[Computurd]

here are results with incompressible and compressible data with three different drive states, FOB, Steady State, and Overprovisioned with SandForce enterprise class drives:

http://thessdreview.com/our-reviews/...-ssd-review/4/

http://thessdreview.com/our-reviews/...-ssd-review/4/

[johnw]

John W. my old friend... So what you are saying is that, according to Ao1, the typical user utilizes incompressible data more often in typical things such as ...ohhhh I don't know.... system starting, system software such as explorer and e-mail use and even MS Word file creation?

System starting is almost entirely reads. MS Word files are already compressed. Email, without attachments, is a VERY small amount of writes, and the attachments are usually incompressible.

[kaktus1907]

Corsair P256 Pro on Sata2



http://www.abload.de/img/mfauuiro.png

[Anvil]

Corsair P256 Pro on Sata2

That looks very good for a socket 775 based system

[Anvil]

First, a sincere apology to Anvil for derailing his thread and detracting from ASU, which is a great benchmark for end users, providing flexibility and ease of use.

The SNIA tests are beyond an end users ability to undertake, but I believe this is the benchmark that vendors should use for their specifications. The benchmark is something that all major SSD vendors have contributed towards and it provides granularity and comparative performance assessments that are beyond any other method of testing.

...
To prevent Anvil’s thread from being further derailed there should be a separate thread to discuss SNIA. There is already a thread to discuss SF compression.

It is not a problem at all, as long as it's conducted in a civilized manner

The next or subsequent beta will include an option to use real-life data when testing, One_Hertz gave me the idea some time ago and I've been testing a lot of configurations (drives/controllers) the last few weeks, looking good so far.

[Rubycon]

Here's my results with 8 Intel 520 180GB SSD in RAID0 with 64KB stripe size. Areca ARC 1880ix-24, 4GB Unigen DIMM, Battery Backup.

First is zero fill, second is 100% incompressible. The hit isn't too bad considering it's a Sandforce controller. This is a 32GB test as well since the 1GB tests are heavily influenced by the controller's cache. 1GB scores are over 14,000!