Fastpath..

[GullLars]

Oki.
This will be a little OT here, but i thought i'd share
ATM i'm working on 3D models in sketchup. I've just completed a 32 TSOP DIMM (240pin). Basically identical PCB and socket to DDR2 with 8 dual-stacked NAND packages on each side. If you use quad die packages with 25nm 2-bit MLC NAND for such a DIMM, you'd get 1TB pr DIMM
Using numbers from another thread, you could get 11GB/s sequential read and 1GB/s write using 64 ONFI 2.1/2.2 (200MBps) channels. If you settle for 1GB/s R/W pr DIMM you could get away with 5 channels (wich coinsidentally is the number of physical channels you could get in pass-through from 48-pin TSOP to 240-pin DIMM.
I took a ss of the model:

Height-wise you can fit it in a 1U rack, and it's only 7,5mm wide (socket. 5,3mm from NAND to NAND). The lenght is 144,2mm, like a 240-pin DIMM. If you space DIMM sockets 0,5mm from eachother, you could fit 50+ in the width of a 1U rack.

[Computurd]

enjoying the holiday weekend today, but i will be putting up some numerous results soon!

[SteveRo]

Again for comparison - 8xR0 acard 9010 on LSI 9211 no oc pcie 100 9211 F/W -2118ir - detailed iometer run -

[[XC] Oj101]

ok dynamic disk softraid 12xR0 acard 9010 with 1231ML4g as pass through no oc pcie 100, atto kept hanging on the last test, don't know why -

Can someone please explain why I'm seeing the same drive listed ten times? Also, what is softraid?

[Anvil]

Again for comparison - 8xR0 acard 9010 on LSI 9211 no oc pcie 100 9211 F/W -2118ir - detailed iometer run -

From the looks of it, the 9211 performs just as good as the Areca.
(the 9211 is better at high QDs)

@oj101
Software RAID is what you are seeing.
Software RAID (in general) means that it is defined on the OS level rather than on the controller.

[[XC] Oj101]

I don't understand why he'd want that with such a high end controller?

[Anvil]

Oj101,
Software RAID can in some cases perform better than VDs defined on the controller.
Software RAID is not bootable, that's one of the downsides.

[SteveRo]

Good evening,
8x_Acard2GB_dynamic_disk_softraid_LSI 9211_980x_no_oc
I think I saw a max of 142K iops - very nice! - too bad not bootable.
Detailed iometer run -

[Computurd]

oh yes...do not underestimate the 9211....i think in time (with fastpath) it will be the best price v performance hands down

[GullLars]

I was busy yesterday, i'm sorry for the wait, I'll post some graphs for steve's results, and when Computurd uploads his "detailed" results i'll have a nice dataset for more finely granulated comparisons at realistic QD ranges.

Here's Steve's integrated 8R0 acard on 9211.

8R0 9211 Softraid and comparisons to 1231ML comming up.

[GullLars]

Here's the 8R0 Acard 9211 pass-through softRAID.


I've got two things to say about these graphs:
#1: holy bananas
#2: This is beyond any doubt the best numbers i've seen for QD<16, for QD <8 nothing comes close.

[Anvil]

About the same # iops as I got using the 9211 + software raid. (not sure about at what QD, I'll check)

A real pity one can't boot using that setup.

[GullLars]

Well, at least you can use it for your VMs, and databases and stuff.

Here's comparative numbers for steve's Acard arrays on 1231ML 4GB vs 9211.




I think i'll make a composite of the 1W and 4W numbers here, and take out the 2^n QD for comparing with the numbers i have for tilt, mbreslin, and computurd for QD 1-128. I'm wondering if i should include steve's 4R0 x25-V frankenstein RAID numbers also just for reference to how insane these setups are.

EDIT:
I'm waiting in anticipation for Computurd and mbreslin's "detailed" numbers

I'll be happy to, unfortunately I'm very busy today, I can likely do it tomorrow and if not it will be monday for sure.

[Tiltevros]

i will try to get a second 9260 and anothere 8 intels to make a dual 9211 with 2x8 intels to make a funny Dynamic drive

[Tiltevros]

gulllars can u make a test file with 8 and 16 thread QD 1-8 to test me and SteveRO? i have some update to test it with Acards and LSI's controllers.
SteveRO with the 9211 with dynamic and me with 9260 on raid with FP

[GullLars]

You can just copy the workers in the "detailed 4W" config and set the QD range 1-8 (wich will be PR WORKER). If you want i can do it for you. For 8 workers it will give you QD 8-64, and for 16 workers QD 16-128.
Total QD < # of workers is not possible :P

Here's the results for steve's softRAID inserted into the lineup i had earlier.
QD 2^n, n=0-7 (QD 1-128).


Below QD 32, steve's acard softRAID has the clear lead. After that Tiltevros' array with 16 workers (also with 2 workers but i didn't care to include because of clutter) takes over. mbreslins C300 array is the best bootable over-all array.

[GullLars]

This is sort of redundant, but since the images are <100kb, 2 more won't hurt much :P

I added numbers from steves frankenstien RAID. 4R0 x25-V from ICH10R (WBC off). It's the yellow line. I thought it would give a nice comparison for how extreme these arrays are.
The x25-v array tops out around 115K IOPS, and for 100% read, 100% random @ 4KB can be considered a powerfull array, total overkill for most people.

[mbreslin]

I was able to get ~265k random read iops with the 9211 in softraid. I believe it was 8 workers/qd16.

I would like to see someone with 4 or more x25e's run the test with 9260+fp, seems like they would do even better than what we've seen so far, with the 'official support' and everything.

http://www.xtremesystems.org/forums/...3&postcount=61

[SteveRo]

So why is it that 9211/8xAcards does so great in iometer but I can't break 30k in pcmv (suite score) with it - and this when 5x acard on the 1231(with same oc) does over 34K?
4gb cache I guess?

[GullLars]

Nice.

While i'm waiting for numbers from Computurd and mbreslin, i'm wokring on som real in-depth stuff for a storage benchmark thread. This is slightly off topic here, but if you want a sneak peek at what i'm messing around with, have a look at this excell file:
31-05-10 different ways of showing scaling.zip
It's the 3 last pages (the illustration ones) that show off what i'm thinking and messing around with. Basically it's different ways of showing scaling, well illustrated by HDDs (and HDD RAID), different SSDs, and SSD RAID.
This may help shed some light on what exactly the IOPS/accesstime graph is, and why i use it to show array scaling.

[SteveRo]

GullLars - also - thanks for the graphs! much thanks, very interesting! looking forward to learning more.

[mbreslin]

I should have some time tomorrow to finish up benches and then maybe have a go at pcmv. I've never got over 100k hdd suite score before now so we'll see how well that translates into overall score.

[SteveRo]

@ Mr GullLars - what is the best way to open/view xml files?

[GullLars]

I wouldn't know, i haven't worked with xml files.

mbreslin, will you have the opportunity to 15 minutes worth of benches on 9211 softRAID (dynamic disc) before being done with benchmarking and settleing for a 24/7 setup? I would love to have detailed IOmeter results for the setup you linked to above hitting 265K+ IOPS. The good part with 9211 and softRAID for benching is it uses no cache, so a 1GB testfile and 10 sec pr run will be fairly accurate.

[AceNZ]

So why is it that 9211/8xAcards does so great in iometer but I can't break 30k in pcmv (suite score) with it - and this when 5x acard on the 1231(with same oc) does over 34K?
4gb cache I guess?

My guess is that the 9211 does poorly with pcmv because of its relatively large strip size. The 64K strip doesn't hurt when accesses are fully random, because all drives will still be busy. However, as accesses become more serialized and more local -- as with regular file access -- then fewer and fewer drives are active at the same time, and aggregate throughput declines.