Originally Posted by
alfaunits
@OP: I think you won't be able to physically achieve more than that, and it has little to do with the card, but the Mtrons.
In best case scenario, the ranom I/O would be split across each drive in order, but in reality it will never happen ;(
That's true, but logic tells me that eventually on average both devices would get a similar amount of random I/O. I might be able to monitor somewhere how much I/O each device would be getting during a load test, but I would be very surprised really if it appears that one device had to process 80% of the I/O while the other had to do only 20%.
IIRC, a single Mtron does ~17000 random IOPs, so that makes perfect sense for 4x17000 = ~65000 IOPs.
Well, we were using a total of 16 drives, not 8 
Each controller had 8 Mtron's. We striped two controllers, for a total number of 16 disks.
Originally Posted by
stevecs
Very interesting study though a couple comments and observations:
First per your article you were using two ARC-1680IX-12's and using LVM2 to stripe at the OS level. What was your stripe size both for the LVM2 and for your array.
The stripe size for the array (set when creating the array in the areca bios) was 128k. For LVM we used the following command:
Code:
pvcreate --metadatasize 250 /dev/sdb /dev/sdc - vgcreate ssd /dev/sdb /dev/sdc - lvcreate -i2 -I128 -L447G -n ssd-striped ssd
Getting all the right alignment settings right proved to be rather tricky. I'm not 100% sure that we got everything absolutely right to be honest.
In your write up you mentioned that you got ~3200MB/s out of two cards? That to me screams ram testing not disk testing. I could see with 2 controllers around 1800MB but not double that, unless I've mis-read what you were doing there.
The 3.3GB/s was reported by bm-flash, but only at very large block sizes (>256K) and with multiple threads (>10). This is most likely indeed the cache, although with smaller block sizes we weren't getting this speed.
- create a ram disk (system ram) and run your tests against that (with lvm2 & put a file system on it).
Okay, what would you suggest as the best way to create a ram disk? I created one using GRUB (5GB), and got a lot more IOPS:
Code:
Filling 4G before testing ... 4096 MB done in 2 seconds (2048 MB/sec).
Read Tests:
Block | 1 thread | 10 threads | 40 threads
Size | IOPS BW | IOPS BW | IOPS BW
| | |
512B |207091 101.1M |202807 99.0M |204409 99.8M
1K |197413 192.7M |194549 189.9M |195600 191.0M
2K |184750 360.8M |180739 353.0M ^C
I have to find out how to create two RAM disks so LVM can be used on those.
- if possible, try cards that DO NOT have a built-in expander (the -IX line from areca does, so you're going through that chip each time). The ARC-1680 doesn't have the expander which may improve performance a bit (also may help resolving some incompatibility issues you mentioned in the article)
That quite interesting to hear indeed. We just received the 1231ML and are going to test with that tomorrow. Currently it's well passed midnight here in Amsterdam, so I have to call it a day
Reply With Quote
Bookmarks