Fastpath..

[GullLars]

Here's a finely granulated linear QD stepping comparison for low, medium, and high QD for the arrays listed in my post above.
This time i listed IOPS and IOPS/accesstime side by side. I want your feedback on using this as a sort of IOPS = throughput and IOPS/accesstime = quality of service.







My reasoning is looking at each IO as a product, and the delivery of IOs as the service. The accesstime would then be delivery time for each IO, and count as the quality of each IO.
By this reasoning, if you double the number of IOs, but trippel the delivery time, you get lower quality of service (66% of previous).
If you double the number of IOs and increase delivery time with 50%, you get better quality of service (33% higher).
As a note, (tired, possibly temporarily flawed) logic dictates you'll always peak QOS at QD =< #channels (or # of drives in the case of HDDs) unless* there's something funky going on.
(*EDIT: wich there is for steve's 8R0 softraid from 9211, wich peaks at QD 10...)
(*EDIT2: i found a plausable explanation. It could be due to channel saturation and statistical distribution, if the IOPS gained in % is higher than the added accesstime in %, the QOS increases. With QD 8 for 8 channels, you'd have something like SUM(8-(8/n)), n {1, 8}. I'm a bit to tired to do it acurately now, but i'd guess around 65-70% saturation, so if increasing QD by 1 gives max 1/(previous QD) higher accesstime, and that 1 QD increase gives higher IOPS gain % from higher saturation increase than accesstime increase from the queue, you could increase QD > #channels a few steps and still get higher QOS.)

For reference, the total number of channels for the setups above are:
Stevero 8R0 acards: 8 channels
Stevero 12R0 acards: 12 channels
Anvil 2R0 x25-E: 20 channels
Computurd 8R0 vertex: 32 channels
Mbreslin 8R0 C300: 64 channels
Tiltevros 8R0 x25-M: 80 channels
This correlates nicely with the shape of the graphs, but not neccesarily with the score, since steve's acards have around or more than double the IOPS pr channel than some of the other setups.
If i included my own 2R0 Mtron Pro (4 internal channels, but no NCQ = acting as 1 channel each), it would start out right between the C300's and acards, and peak at QD 2, and be a line at the bottom of the graphs beyond QD 8.

Now i got to get some sleep, i'll have a look at the 1-8R0 detailed results when i get up mbreslin
You should have enough graphs and numbers in the meantime, and if you get bored, there's the zip with the xlsx file i posted a couple of pages back