LSI 9260 vs. AMD SB850

[F@32]

Hi all,

Today I embark on a quest to figure out if RAID controller which is three times as expensive as the motherboard it will be plugged into, can outperform said motherboard in simple 2-SSD drive RAID-0 configuration. I solicit advice from all the storage gurus on how to tweak LSI controller for the optimal performance. As you can see LSI is falling short in AS SSD benchmark. Any thoughts?

The set-up:
Drives: Intel X18-M Gen1 50nm 80Gb, 4K NTFS allocation unit
Controller: LSI 9260-4i, FW 12.11.0-0016, Driver 4.32.0.64, no FastPath
Mobo: Gigabyte GA-890GPA-UD3H, bios F8, 8.71 storage driver
CPU: Phenom II 965, TRUE-120 default clocks
RAM: 2x2Gb DDR3 G-Skill Ripjaws F3-12800cl9d
Video: Asus HD5850, Cat 10.6
Sound: M-Audio Audiophile Delta 2496
NIC: Intel Gigabit CT PCIe
PSU: Antec Quattro 850Wt
Case: Antec 1200
OS: Windows 7 Ultimate 64bit

LSI 9260 settings will be listed in the following order:

Stripe - 128K
Read Policy - ARA
Write Policy - AWB
IO Policy - Cached IO
Access Policy - Read Write
Disk Cache Policy - Enabled
BAckground Initialization - Enabled

Windows Write-caching policy
Write caching - Enable
Write-cache buffer flushing - Disable

1. SB850 Windows 7 soft raid


2. SB850 128K stripe, NCQ on, write-back on, flushing off


3. LSI 128K stripe (128 ARA AWB CIO RW E E. E D)


4. LSI 16K stripe (16 ARA AWB CIO RW E E. E D)


5. LSI max throughput check

[alfaunits]

Since when is 9260 that cheap to cost only 2 AMD mobos?

[F@32]

Since when is 9260 that cheap to cost only 2 AMD mobos?

Lol, good catch, fixed it ;-)

[Computurd]

do you have fastpath?

[Anvil]

I just performed a few tests on my 2R0 G2 160GB on my 9260 w/FP key and the latest LSI firmware..
(the drives are in steady-state)

Strip size 32KB, NRA (no read ahead), DIO (direct IO), WB (write back)

I was a bit surprised to see that Seq. read on AS SSD is quite low, have to check that one with previos tests.

Your results do look low, do not use Read Ahead on SSDs using the 9260, I would also try changing from cached IO to Direct IO.

[F@32]

I just performed a few tests on my 2R0 G2 160GB on my 9260 w/FP key and the latest LSI firmware..
(the drives are in steady-state)

Strip size 32KB, NRA (no read ahead), DIO (direct IO), WB (write back)

I was a bit surprised to see that Seq. read on AS SSD is quite low, have to check that one with previos tests.

Your results do look low, do not use Read Ahead on SSDs using the 9260, I would also try changing from cached IO to Direct IO.

Thanks Anvil, will fire her up Yep, something is fishy with AS SSD. Works fine with onboard...
CT, no FP at the moment.

[odditory]

I don't trust any results from AS SSD. From the first time I ran it, I knew it was screwy.

[F@32]

Anvil, note mine are 80Gb G1, so they are 70Mb/s on writes... I knew that Id need bunch of drives to push 9260, and it looks nice with 4, but not sure I like $/performance especially with added read latency. Note the latency with re-badged A-Data SF1222 drives. Wow...

OK, here's 2R0 32K, DIO, NRA, AWB


2R0, 128K, DIO, NRA, AWB


3R0 with SF1222 drives (not impressed with them), 128K, ARA, AWB, CIO


4R0 16K (2 X18M, 2 SF 1222)


4R0 128K (2 X18M, 2 SF 1222)


May be AMD mobo doesn't like it?

[Anvil]

F@32,
Could be the AMD/LSI combo, I think it might be worth getting the FP key, as you can see from my tests it scales fairly OK.

4R0 Vertex 2 on the 9260 w/FP key.

Personally I don't have any issues with the SF based SSDs

AS SSD
as-ssd-bench LSI MR9260-8i SC 12.25.2010 7-31-50 PM_DIO_WT_NRA_DCE.png

CDM using 0-Fill option (best case)
4R0_V2_9260_fp_dio_nra_wt_dcd.PNG

random data (worst case)
4R0_V2_9260_fp_dio_nra_wt_dce_random.PNG

[F@32]

Thanks Anvil. FP would be sweet, but that would mean another $150. Thing is SB850 with one worker @ QD16 gives me 50K IOPS with 4K random. 9260 without FP at least on my PC is 35K... That's not right

[Computurd]

FastPath will do 160,000 IOPS @4k random.

[Anvil]

I'll see if I can do a test of the 9260 on my AMD rig sometime soon, the results should compare to my latest results on the Intel setup.
(might be able to squeeze it in tomorrow)

The 9260 has always been a screamer at high QD, the FP key somewhat changed that.

Do you need >50K iops or is it just for the comparison?

[Anvil]

FastPath will do 160,000 IOPS @4k random.

Somewhat close to 150-160' iops is correct, depends on the SSDs in question.

This is the 4R0 Vertex 2 using a 5GB testfile.

4r0_io_4K_random_read_FP_QD64.PNG

[F@32]

Do you need >50K iops or is it just for the comparison?

Just for comparison, "free" fake raid shouldn't be outperforming 9260

[Computurd]

"free" fake raid shouldn't be outperforming 9260

with only two devices it might outperform it in some scenarios. but not in stability of course. a few bluescreens and your array is a goner on most fakeraids.
however, you shouldnt be buying a 500 raid card for two devices anyway. this is for larger arrays, where the ICH or SB850 couldn't even begin to keep up.
also, i couldnt help but notice that you are comparing a soft raid to a real raid. any type of software raid will be faster, while also being inherently unstable, and chew your cpu resources up as well.
that doesnt seem to be a fair representation. the 9260 is designed for uber throughput, with large arrays for specific use in enterprise environments, and supercomputing. it is tuned for large arrays and scales beautifully with them.
for low latency usage they have the fastpath key, which will tear up anything that you put it up against.

Note the latency with re-badged A-Data SF1222 drives.

havent heard good things about that set of drives. seems to be knock-offish with poor firmware support.

[Anvil]

There is more to it than just the iops.

@50K iops you are consuming > 1/3 of the CPU on the AMD rig, the 9260 has it's own "cpu" and shouldn't be consuming > 10% in most cases.
As the QD increases, CPU utilization on the AMD (and Intel's as well) steadily increases and in the end they'll consume most of the processing power available on the computer, this comes at a cost. (other tasks suffer)

OTOH, there was never any doubt about the onboard controller being competitive for most desktop tasks, just look at the ICH, a race-horse at heart and the SB850 is indeed a great step up from previous chipsets.

[F@32]

with only two devices it might outperform it in some scenarios. but not in stability of course. a few bluescreens and your array is a goner on most fakeraids.
however, you shouldnt be buying a 500 raid card for two devices anyway. this is for larger arrays, where the ICH or SB850 couldn't even begin to keep up.

Oh, I totally agree. I wouldn't have paid full retail for it, temptation was great to mess with it. I will be frank, I feel that FP idea is a rip off. Should have been enabled by default.

havent heard good things about that set of drives. seems to be knock-offish with poor firmware support.

They are back to where they came from. But FW seemed to be 3.1.2 SF1222...

@50K iops you are consuming > 1/3 of the CPU on the AMD rig, the 9260 has it's own "cpu" and shouldn't be consuming > 10% in most cases.

Yep, you're right there. Noe that in the test only 1 worker was running. For desktop/gaming use I sure hope 4K >QD16 is not a very common scenario. In any case, at least there is some utilization for multiple cores.

6%


And yes, at QD32 with 4 workers AMD is tapped out. You have to agree I hope, that 40% CPU sacrifice for 60K is worthy.







4R0 LSI no FP (again, something is borked on that mobo)


4R0 AMD

[Computurd]

those are some nice numbers, however, the fact that you cant boot a soft array just totally ruins it for me. i just have no use for a non-boot array personally. even though i have toyed with some passthrough soft array on the 9211 with five devices and five on a 9260 as the boot. uber fast to say the least

tbh that cpu usage you are showing with that is shocking! wow 78 percent! ive never seen ICH kick off utilization like that with a soft array, however, i havent toyed with it too much....but that number seems abnormally high to me.

does your mobo have the NF200? seems like some motherboards and the nf200 do not mix AT ALL with the 9260...like kryptonite on the classified e759

[Anvil]

F@32

I had a shot at the 9260 on my Asus CHIV, it didn't work as my boot drive was configured for RAID and and as soon as I installed the 9260 it just wouldn't let me select my boot drive again.
All that was left was the 9260 and I can't reconfigure the drives right now.

Will give it a go at some later point in time.

The performance you saw might just be a result of mixing drives, sometimes it works OK but not all of the time.
After all the X25 and the SF are different beasts.

[alfaunits]

And yes, at QD32 with 4 workers AMD is tapped out. You have to agree I hope, that 40% CPU sacrifice for 60K is worthy.

Not really, but I am quite happy to see AMD's drivers actuall use multiple cores! LSI 9211 would only use ONE core on my C2Q build, and it was obvious it is exactly CPU limited. (it gave awesome numbers though :P)
I think that also Intel's ICHxR will not use more than one core. Windows "RAID" won't use a full core even on a large RAID5, I don't know why (neither will it use it for EFS!). I am at a loss why.

[Anvil]

To achieve ~100' IOPS on the SB850 I had to use 4 or 6 workers (iometer), CPU utilization >90%

Jasper Forest might change raid-5 performance on the Intel platform

[F@32]

those are some nice numbers, however, the fact that you cant boot a soft array just totally ruins it for me. i just have no use for a non-boot array personally.

Hey CT, sorry if I was not clear: those numbers are from SB850 RAID which is bootable. I only did pass through mode once to see if there was anything funny going on.

tbh that cpu usage you are showing with that is shocking! wow 78 percent! ive never seen ICH kick off utilization like that with a soft array, however, i havent toyed with it too much....but that number seems abnormally high to me.

Yep, I'm glad AMD drivers utilize more than one core even with 1 worker.

does your mobo have the NF200? seems like some motherboards and the nf200 do not mix AT ALL with the 9260...like kryptonite on the classified e759

Nope, that's a 890GX/SB850 mobo.

F@32

I had a shot at the 9260 on my Asus CHIV, it didn't work as my boot drive was configured for RAID and and as soon as I installed the 9260 it just wouldn't let me select my boot drive again.
All that was left was the 9260 and I can't reconfigure the drives right now.

Will give it a go at some later point in time.

Appreciate your help. I would like to see someone else's AMD+LSI numbers. I experience problem installing Win7 on LSI array. Every other or so re-boot it drops one or both drives. No problems otherwise... Right now I'm not too happy with LSI 4K IOPS without FP. 40% CPU utilization with AMD QD16 and 50K IOPS vs 6% utilization with LSI and 35K IOPS... For $350 card I would have expected it to whipe the floor with SB850 even with 2R0, but it seems another $150 needed to match and beat it. That's $500 vs something that's already there. This build is for gaming and 24/7 use. I don't think I will see benefits going 4R0, bottleneck is be back on CPU anyway.

Not really, but I am quite happy to see AMD's drivers actuall use multiple cores! LSI 9211 would only use ONE core on my C2Q build, and it was obvious it is exactly CPU limited. (it gave awesome numbers though :P)
I think that also Intel's ICHxR will not use more than one core. Windows "RAID" won't use a full core even on a large RAID5, I don't know why (neither will it use it for EFS!). I am at a loss why.

Yeah, 9211 is a darling but LSI shows no love with gimped firmware and drivers. I could imagine with proper multi-core support and above 128K sequential scaling it could fly... I'm still thinking about picking up IBM M1015 re-badged LSI 9240 for the heck of it... Funny thing is, it seems 9240 is not gimped like 9211. And it's only $20 or so more expensive.

To achieve ~100' IOPS on the SB850 I had to use 4 or 6 workers (iometer), CPU utilization >90%
Jasper Forest might change raid-5 performance on the Intel platform

Nice, was it in 4R0 SF?

[Anvil]

Nice, was it in 4R0 SF?

Not sure, could have been the Intel G1's, I'll check.

[F@32]

OK, 9240-8i aka 9220-8i aka M1015 is on its way. Will see what it can do.

[mattkosem]

Those numbers on SB850 look a bit low. I don't know if it's the drives not playing nice, or some other factor. But I generally see >1GB/sec reads from SB850. http://www.xtremesystems.org/forums/...6&postcount=93

the fact that you cant boot a soft array just totally ruins it for me.

Seriously. Microsoft needs to step up with that. OSX can boot from soft arrays without issues, even on a hackintosh.

--Matt