New LSI 9200 series controllers: 6Gb/s, 2.88 GB/s seq. reads, 1.87 GB/s seq. writes

[GullLars]

If i'm not mistaken, you need Write-Back Cache to get OK write speeds in RAID-5, write-through give the horrible write speed you see.

[Mastakilla]

Right before going to work this morning, I quickly tried this out and you are absolutely right!!

With Write back enable I get write speeds of around 300MB/sec...

Now I knew that disabling Write back means less performance, but I thought it was something like 60-70% of the performance, not 10%...
Actually that is no longer a loss in performance, but a total destruction of performance

Well... I guess disabling Write back is not really an option in this case, so I need to "go back to the drawing board" and figure out if I now need to buy this overly expensive BBU anyway

[Mastakilla]

www.span.com
there 2 ways that the power losses.. 1 way i dont know how to say it in english... but i will try to express it in my own way... the power goes up and then dives down like eg if ur power socket ensure in stady way 220volts for miliseconds goes to 260volts then to 175Volts and then 220volts that power goes in to ur psu and after psu inside ur pc. the second way is when the power goes from 220 stady volts to 175volts and then goes up to the 260 volts. the second way for me is more more more dangerous couse the loss of the power reduse from minimum to max and that gap is very painfull to electronics.
ALL serius RAID controllers have bbu's. when u work ur data for seconds are on the cache of the controller those 512mb or more that the controller have. if the power loss hit u in that moment ur data stays in the controllers cache for 72 hours top. thats what u pay for the bbu. For a home user maybe this bbu is useless but for enteprises is gold.
Now for home users the only benefit that the bbu stands is for Write Cache Back. if u dont have the bbu u cant enable it.

was that a one time offer at span.com when you got the controller and battery for only 412 euro?
Cause now I see the LSI 9260-8I for 421,42 euro and the battery for 120,90...

I have 2 APC SurgeArrests for 8 outlets each to protect all my electronic devices, so power spikes shouldnt be a problem.
Power drops and complete power loss however can still occur.

In case of power loss I wouldnt mind very much to loose what i was working on. However the chance of loosing all data because of power loss causing a corrupt RAID 5 is unacceptable (that is the reason why I'm investing in this expensive RAID controller, I do not want the chance anymore to loose all data. And I'm tired of putting everything on DVDs :p)

So can anyone confirm that I don't need a BBU to ensure that I will not loose all data?

About the write back cache: Interesting... I didnt know you cant enable it without a BBU... Does it matter much in performance? Or where lies the difference?

Thanks!

It seems Tiltevros statement about "not being able to enable write back without a BBU" was incorrect.

I just enabled write back cache without having a BBU and got almost 1000% write speed increase...

But my old question about the BBU still wasn't answered...

So I'll ask it again:
Is it possible to loose the complete RAID 5 IF
* write back is enabled
* there is no BBU
* a powerloss occurs or a system / application crash occurs

OR will this only cause me to loose the data in the cache (+ a bit of corresponding data that was already on the disc)?

[mbreslin]

I can only tell you I have crashed my system hundreds of times while overclocking and never dropped my array. I have no bbu and enable write back. Tilt has recommended write back with and without bbu in many other threads so I'm sure he knows you can write back without bbu I'm not quite sure what happened in the post you're quoting.

So I'll ask it again:
Is it possible to loose the complete RAID 5 IF
* write back is enabled
* there is no BBU
* a powerloss occurs or a system / application crash occurs

OR will this only cause me to loose the data in the cache (+ a bit of corresponding data that was already on the disc)?

The answer to this is: both.

It's really quite simple: If you have data you can not lose buy a bbu and backup off site.

You can lose data at any time for more reasons than I could even mention. Your array can fail at any moment. bbu or no bbu, ups or no ups. Ask large datacenter operators, they will tell you 10k$ raid cards fail all the time.

Last but not least your computer could catch on fire and melt all your 010101010's! Life is crazy!

[Mastakilla]

thanks for the answer...

I guess your crashing experience tells me enough to know I don't need the absurdly expensive BBU (price has even risen to 163 euro minimum now????!!!!)

I know I can still loose all data at all moments because of many reasons and I've looked in to tape streamers for off site backups, but they are waaaaay to expensive for the amount of data I'll have
I now have a RAID 5 to have a more secure data system than the RAID 1 (120GB) + RAID 0 (500GB) that I had before (even without a BBU and write back enabled this should certainly be the case...)

btw: I've read somewhere that if the LSI controller dies, you usually can still replace it with the same controller and keep your array

[Mastakilla]

for those interested, below are some screenshots showing the insane performance difference between Write Back and Write Trough on normal HDs:

Write Through:

vs Write Back:


Write Through:

vs Write Back:


Write Through:

vs Write Back:


and also a screenshot of CrystalDiskMark set to 4GB with Write Back enabled (to minimize the cache advantage)


as you can see, not only the write speeds increase by almost 1000%, but also both read and write speed stabilize a lot

[trans am]

ive never had a crash on win7 with 9260 in raid0 with wb enabled without a bbu. battery is totally unnecessary. then again im not using raid 5 just 2 sandforce ssd in raid0

I ordered a fastpath key and it comes tomorrow. I cant wait to post results. before and after.

[Computurd]

I cant wait to see your results either TA!

[AceNZ]

for those interested, below are some screenshots showing the insane performance difference between Write Back and Write Trough on normal HDs:

The performance difference is because in write back mode, the controller indicates that the write operation is complete after the data has been transferred to cache. (So you're really measuring how fast you can write to cache, not to the drives).

In write through mode, the operation isn't complete until the data has been written onto the drive(s).

This, of course, means that you should have a battery backup unit for the controller if you use write back mode and care about the integrity of your data.

This also means that the perf increase is "temporary" -- it only lasts until the write cache is full.

[Computurd]

yup, and there is also a cache flushing interval that will clear your cache every four seconds by default. i set mine to 2 seconds to kinda hedge my bets as i roll without BBU. also adds about 1k points in vantage for some weird reason.

[Tutto]

I'm currently running 8x seagate 320GB drives on a 1231ML. I really would like to get into SSD's but won't until trim support is available for a Raid 5 setup so I plan to upgrade to Savvio 15k.2 drives. Would I be better of getting 12 drives and keeping my 1231 or getting a 9260 and using 8 drives?

[AceNZ]

I'm currently running 8x seagate 320GB drives on a 1231ML. I really would like to get into SSD's but won't until trim support is available for a Raid 5 setup so I plan to upgrade to Savvio 15k.2 drives. Would I be better of getting 12 drives and keeping my 1231 or getting a 9260 and using 8 drives?

From a maximum sequential transfer rate perspective, I think you'd probably do better with the 9260. The numbers I've seen are that the 1231ML peaks at about 800 MB/s sustained.

If you're going for random IOPS, then with 6 Mbps SAS drives like the Savvio, given that the 1231ML is optimized for 3 Mbps SATA, I'd still go with the 9260.

Do you need 2.5-inch drives? If not, the Cheetah 15K.7 is faster than the Savvio 15K.2 for sequential access: 204 vs. 160 MB/s (the Savvio is slightly faster for random), and you get about twice the capacity for the same price. Eight 15K.7 drives in R0 should be able to get close to 1600 MB/s on a short-stroked partition.

FWIW, your CDM numbers would be more meaningful if you set the size to be at least twice the size of the cache on your 1231ML board.

[SteveRo]

I have a 1231 I am waiting to see what the areca 1880 looks like before deciding on a replacement.

[Mastakilla]

The performance difference is because in write back mode, the controller indicates that the write operation is complete after the data has been transferred to cache. (So you're really measuring how fast you can write to cache, not to the drives).

In write through mode, the operation isn't complete until the data has been written onto the drive(s).

This, of course, means that you should have a battery backup unit for the controller if you use write back mode and care about the integrity of your data.

This also means that the perf increase is "temporary" -- it only lasts until the write cache is full.

I dont know why, but when I copy a 10GB file from my SSD to the RAID5 I get a constant speed of 250-290MB/sec and not a constant speed of 15-19MB/sec (the cache is only 512MB)

not that I'm complaining

[AceNZ]

I dont know why, but when I copy a 10GB file from my SSD to the RAID5 I get a constant speed of 250-290MB/sec and not a constant speed of 15-19MB/sec (the cache is only 512MB)

With RAID-5, there's an additional benefit of write back caching, which is that the controller can buffer enough data to write full stripes at a time. The alternative is partial stripes, which require reading-then-writing the parity disk. With a full-stripe write, the parity drive is just written, and not read first. With rotating media, there is a huge performance difference, since you don't have to wait for the drive to make a full rotation before writing.

[Mastakilla]

With RAID-5, there's an additional benefit of write through caching, which is that the controller can buffer enough data to write full stripes at a time. The alternative is partial stripes, which require reading-then-writing the parity disk. With a full-stripe write, the parity drive is just written, and not read first. With rotating media, there is a huge performance difference, since you don't have to wait for the drive to make a full rotation before writing.

I suppose you mean write back instead of write through?

[TuKo]

Hi guys,

I'm following this thread with attention, as I'm the new owner of 9260-4i and 4x X25-M.

Here is the performance shot I came to :

-----------------------------------------------------------------------
CrystalDiskMark 3.0 x64 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

Sequential Read : 906.028 MB/s
Sequential Write : 347.498 MB/s
Random Read 512KB : 675.881 MB/s
Random Write 512KB : 421.801 MB/s
Random Read 4KB (QD=1) : 21.067 MB/s [ 5143.2 IOPS]
Random Write 4KB (QD=1) : 47.653 MB/s [ 11633.9 IOPS]
Random Read 4KB (QD=32) : 166.830 MB/s [ 40729.9 IOPS]
Random Write 4KB (QD=32) : 119.813 MB/s [ 29251.1 IOPS]

Test : 1000 MB [C: 12.9% (38.1/295.9 GB)] (x5)
Date : 2010/06/15 20:04:34
OS : Windows 7 Ultimate Edition [6.1 Build 7600] (x64)

RAID0
64k Stripe Size
Read Adaptive
R/W
Direct I/O
Drive Cache Enabled
BBU Off
NCQ Disabled
Write Back

Do you have any tips to improve this score ?

[GullLars]

RAID0
64k Stripe Size
Read Adaptive
R/W
Direct I/O
Drive Cache Enabled
BBU Off
NCQ Disabled
Write Back

Do you have any tips to improve this score ?

Enable NCQ dude. You should see a boost in 4KB random @ QD 32.
Also try running AS SSD.

BTW, have you got the newest firmware on both the drives and the controller? And have you got the latest LSI drivers?

If you want more performance, the next step is a FastPath key, it will boost your IOPS and give better accesstimes, even at fairly low QDs, and will allow you to get close to 120K 4KB random read IOPS max from your 4 M's.

[Tutto]

I got a 9260 with cables for $400 including shipping. Now I have to decide between the 15k.2 and 15k.7 drives. I'm leaning toward the 15K.7 but have to wait for a check to arrive before I make the purchase.

This is my first time using sas drives and want to make sure these are the correct cables.

http://www.scsi4me.com/lsi-07-00021-...pin-power.html

[AceNZ]

I suppose you mean write back instead of write through?

Yes, sorry; I edited my post.

[Mastakilla]

Yes, sorry; I edited my post.

no problemo

thanks for the explanation

still a bit weird that the difference is so enormous

the benchmarks of other controllers I found show a 40-60% difference between write through and write back, not a difference of 1000% percent :p

[0xdeadbeef]

I got a 9260 with cables for $400 including shipping.

It's a great price, where did you buy your card? The best price I find here in Sweden is $890 for a 9260-8i without shipping.

[Tutto]

It's a great price, where did you buy your card? The best price I find here in Sweden is $890 for a 9260-8i without shipping.

I won it on ebay. He had it as a buy-it-now or Best Offer. I offered $400 and he accepted. Could you verify if I am getting the right cables for the Seagate 15k.7 SAS drives.

[AceNZ]

This is my first time using sas drives and want to make sure these are the correct cables.

http://www.scsi4me.com/lsi-07-00021-...pin-power.html

Those cables should work. However, they require connecting to molex-style power cables. With a modern power supplies that tend to have lots of SAS/SATA-style power cables, I prefer the SFF-8087 to mini-SAS style. For example:

http://www.newegg.com/Product/Produc...82E16816116100

Also, it's possible to connect the 9260 to both ports on dual-ported SAS drives like the 15K.7; your performance will improve if you do.

FWIW, some of the 9260s come with cables (the "kit" SKU).

[Mastakilla]

I just encountered something weird when setting up my RAID5:

I set up my RAID5 in the WebBios using the default settings (except for forcing to always use Write Back).
At the end of the wizzard WebBios asks me if I want to do a fast initialization, so I choose yes.
Then almost immediately I come to a screen where I can again choose to initialize and also do other things (such as a consistency check).
Since I just choose to fast initialize already, I just exited WebBios and booted into Windows.

When I open Disk Management in Windows, it asks which type of partition I want for the RAID5.
I choose GPT and create on big NTFS partition with a custom Stripe size of 64K (same as configured in WebBios)

Then I went to LSI MSM and tried to do a consistency check.
This gave a warning that the disk wasn't initialized yet??

So I thought: perhaps it is still busy in the background... So I left my PC running for 24hours...
Then I tried again, it still sais the disk isn't initialized??

So I try to initialize and it warns I will loose all data. To make it a bit easier for MSM, I then removed the partition and did an fast initialize.
After the initialize, I rebooted (else Disk Manager was flipping), I again created a GPT partition type with NTFS partition (again 64k stripe).

Then I retried to do a consistency check, but it warns the disk isn't initialized??

So I again remove the partition, do a fast initialize and tried to do a consistency check before created a partition.
This time it started, but immediately found inconsistent parities.

So I stopped and decided to a full initialize (not fast), which is running now...

Can anyone explain me what I'm doing wrong? What is the correct way to do it?
I want to be able to do consistency checks while keeping the partitions of course

And I know I can do a consistency check on a "non-initialized" disk, but I still prefer my initialized disk to be recognized as initialized as well