Here is something I found interesting. I reran the XFS iozone benchmark with the extra iozone flag "-I" for Direct-IO. The sequential write and the random read/write was improved while the sequential reads were not as good for smaller record sizes.

Here are the iozone results:


Iozone: Performance Test of File I/O
Version $Revision: 3.394 $
Compiled for 64 bit mode.
Build: linux

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.

Run began: Fri Feb 1 13:30:57 2013

O_DIRECT feature enabled
Record Size 1024 KB
Record Size 2048 KB
Record Size 4096 KB
Record Size 8192 KB
Record Size 16384 KB
File size set to 100663296 KB
Command line used: /usr/bin/iozone -I -i 0 -i 1 -i 2 -r 1m -r 2m -r 4m -r 8m -r 16m -s 96g -f /DataRAID/testfile
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
------------------------------------------------------random random bkwd record stride
----KB----reclen---write---rewrite----read----reread---read----write-----read rewrite read fwrite frewrite fread freread
100663296 1024 2891087 3034149 1347364 1350111 1184200 2972416
100663296 2048 3285530 3356490 2581875 2593956 2319664 3260048
100663296 4096 3509246 3506871 2868134 2879221 2902174 3482614
100663296 8192 3455400 3484193 3261035 3274733 3243127 3444259
100663296 16384 3438511 3496536 3528632 3548018 3508518 3444243

iozone test complete.