Fastpath..

[GullLars]

Here's updated graphs for tiltevros with 4W and 8W numbers. It seems the 4W numbers got better. The 8W numbers are still compared to the rest. For some reason, it seems like the setup doesn't like 8 threads...

I'm hoping mr mbreslin will do a 2/4/8/16 worker run of the C300 array. If there is any scaling at all, we'll be looking at even more extreme numbers.

BTW, i would LOVE results for this config on ioXtreme to compare with these extreme fastpath arrays, and steve's Acards on ML1231.

EDIT: forgot the graphs XD



EDIT2: would you consider a mild OC to see if there is any difference tiltevros?

[Tiltevros]

i cant overclock. im waiting for the SR-2 and then

[Computurd]

umm on the iops/access time issue, if you have double the iops, wouldnt your access time go higher, regardless?
i dont understand the iops/averageaccess time chart. what are you deriving numbers of 500000 and 800000 from? how are you calculating the numbers?

tilt cant oc his processors, or he would have a long time ago. dual socket motherboard.

[Computurd]

excellent results btw mbreslin, very impressive, that answers my question on scaling and the performance of my drives being the limiting factor.
hwo is your pcmark quest going? also i am getting best results at 128 stripe, and lsi is officially recommending 256

[mbreslin]

So I was out to dinner just got back, I was running iometer and noticing lower than expected results and realized I hadn't stuck the fp key back on! I think it will be a great comparison so I'm upping it anyway. Gull if you could overlay this with my coming fp results that would be great.

Edit: Pcmv probably tomorrow ct, still got better hdd suite test then I've ever gotten with fp (and without really tweaking yet). Should be fun.

[mbreslin]

Ok I need help, I ran the test but I have a 150k iop wall. I know it is 100% not drive related because I can reach this wall with the 4 worker test and an array of only 3 drives.

My ideas are:
1) Something is wrong with my controller.
2) The cables I'm using suck.

I'm not using the kit cables I bought some other ones that were black sleeved. (I sleeved every cable in my pc with black mdpc-x sleeving)

#2 will be cleared up tomorrow because I have the kit cables just need to search for them.

Any other ideas please?

[SteveRo]

Good morning, for comparison here are the results of old storage h/w - file attached.
I am curious how it compares to these fastpath uber systems.

[Computurd]

if you read the fastpath documentation on LSI web site i believe that 150k iop is around the limits with the current FW. here is the PDF....

http://www.lsi.com/DistributionSyste..._PB_042910.pdf

also bear in mind that the speed with which your array hits that wall (the amount of QD to reach it) is tremendous. you are hitting some very impressive low QD numbers, which translates to real world speed!!
ich or anything out there isnt close to those numbers.

[Computurd]

@steve o....those numbers are very nice for low QD....scales up extremely fast, i wonder if that is just the brute force of the acards, and whether or not they would scale that high at low qd with fastpath. also, are those speeds with cache or without? amazing how fast it gets going, just hits an iop wall very quickly. your access times at the low QD are much better as well.

[SteveRo]

Paul - probabaly the acards since we see the same behavior on ich10.
edit - the runs above are with full 4GB cache

[mbreslin]

@Computurd: Explain tilt's 250k+ then?

[mbreslin]

Here is mine.. I also threw in 6/12 workers at the end..

[GullLars]

Here's mbreslins new 1-16W results, the first one he posted was just 1W.
It somehow seems he was not limited by single thread performance. The only difference seen is at QD 32, where the difference between any number of workers is max 3-4%.

For comparative charts' i'll stick to just 1W for mbreslin untill there are new numbers that show scaling with workers for his setup.
The numbers whitout FP show the same tendency, graphs comming shortly.
I'll also post graphs for steve and a bigger lineup later.

@computurd, the IOPS/average accesstime numbers are derived from the formula {IOPS}/{average responsetime (ms)}. 500.000 could be from 50K IOPS @ 0,1ms average responsetime. I'll see if i can dig up some numbers for single x25-M and barefoot drive later to show some context later. I'll make a new benchmark thread later where we can go in-depth in this stuff.

[mbreslin]

With the new firmware the ceiling is now 160k.

[GullLars]

OK, is the scaling still the same?

BTW, since you included 6 and 12W, i made an average of the 2^n Workers and average of 6+12W IOPS, and made a graph with more points on the x axis

[Anvil]

With the new firmware the ceiling is now 160k.

I noticed the increase as well, 2R0 C300 @4KB are now at 91-92' iops.

As for the scaling with workers, it could be that Tilt is using a server MB with a completly different architecture or that his MB allows for "special" tuning.

[mbreslin]

Yeah scaling is the same.

I'm still wondering how tilt's g1/g2 m's get 100k more iops than lsi did with e's.

Edit: Special tuning? Isn't it controller bound? LSI Wouldn't have tested on server boards?

[GullLars]

Here's mbreslins C300 array whitout the FP key:



Since there's virtually no deviance between # of workers here, i'll just use 1W for the comparison graphs comming up in a couple of minutes.

[Anvil]

mbreslin,

He's been mentioning PCIE packet size amongst others, not sure what else it could be. (PCIE packet size is not possible to change on most 1366 MBs)

Your setup scales just like mine, theres no increase in iops but for each added worker QD is shifted 1 level.

edit:

If your'e willing to try, do the following.
Compress the iometer test file and do a run
I've tried it once on my system and the iops just sky-rocketed , not very useful though. (it does show how far you can take the iops)

[mbreslin]

I guess Anvil. It just seems to me they would have been happy to claim 250k+ random read iops "under the right conditions". And certainly they tested on all manner of server hardware.

[GullLars]

Here's graphs for comparing mbreslins array with fastpath vs no fastpath:


Notice especially the second graph, the point of "diminishing returns" have shifted. Basically, the peak of the graph is where IOPS increases less than accesstime.

[mbreslin]

Pretty awesome, you were right before I bought the key, nearly 100% increase in some spots.

[Anvil]

We'll just have to wait for some input from Tilt.

No need to speculate as to how he got that performance, your drives should do better, no doubt about that.

[mbreslin]

Yeah I'm not suggesting anything or doubting tilt. I've seen what he did with the 9211 no doubt he can work some magic with the 9260.

I guess I just got excited after my 1 worker performance and expected more.

[Anvil]

mbreslin,

The performance using the Key is excellent.
QD for desktop tasks are seldom above 8-16 so you might not "feel" the difference for everyday tasks.
I'm sure there are other adVantages to the Key, when do you plan on doing some more pcmv.