A question came up recently with a network file server disk I/O load and performances under different raid types, I didn't have anything saved previously so I ran some simple tests with their request size (this was for large files shared over a SMB v1 network, so block sizes being limited to 64KiB) to illustrate some raid points to them. Figured it may be of interest here as it shows nicely the performance of the raids w/ random workloads and how the raid levels handle them everything else being equal.
This is not intended to be an absolute (depends on raid implementation, and other items) this is mainly good though for illustration purposes to help people get their minds around concepts like data instance redundancy (ie raid-1 or 10 is has more iops than any other raid for reads), where the write penalties show for the parity raids et al.
all OS & drive write caching is disabled only the areca card has a write cache with the BBU & normal read-ahead but data size is 24GiB and card has just 512MiB to mitigate that. This is more akin to show how it would perform under proper deployment.
RAID 0:
RAID 1+0 (10):Code:Areca 1680ix-24 512MB Cache; HDD Read Ahead Cache: Enabled Volume Data Read Ahead: Normal HDD Queue Depth: 32 Disk Write Cache Mode: Disabled RAID 0; 8KiB Stripe Size; 8 x ST973451SS (73GB 2.5" SFF SAS 15K Savvio) Windows XP x64 SP2, NTFS, 32GB Partition; no allignment (file system start sector 63) ===================================================================================================== H:\xdd\bin>xdd.exe -verbose -op read -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek ra nge 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Mon Dec 22 11:23:07 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 2376 Thread ID, 2376 Processor, all/any Read/write ratio, 100.00, 0.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 166.256 51.667 788.38 0.0013 2.36 read 65536 TARGET PASS0002 0 128 8589934592 131072 166.541 51.578 787.02 0.0013 2.33 read 65536 TARGET PASS0003 0 128 8589934592 131072 166.186 51.689 788.71 0.0013 2.72 read 65536 TARGET Average 0 128 25769803776 393216 498.751 51.669 788.40 0.0013 2.47 read 65536 Combined 1 128 25769803776 393216 498.751 51.669 788.40 0.0013 2.43 read 65536 Ending time for this run, Mon Dec 22 11:31:28 2008 H:\xdd\bin>xdd.exe -verbose -op write -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek r ange 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Mon Dec 22 11:35:06 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 2464 Thread ID, 2464 Processor, all/any Read/write ratio, 0.00, 100.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 240.651 35.695 544.66 0.0018 2.10 write 65536 TARGET PASS0002 0 128 8589934592 131072 246.432 34.857 531.88 0.0019 2.27 write 65536 TARGET PASS0003 0 128 8589934592 131072 246.922 34.788 530.82 0.0019 1.97 write 65536 TARGET Average 0 128 25769803776 393216 734.004 35.109 535.71 0.0019 2.12 write 65536 Combined 1 128 25769803776 393216 734.004 35.109 535.71 0.0019 2.12 write 65536 Ending time for this run, Mon Dec 22 11:47:21 2008
RAID5:Code:Areca 1680ix-24 512MB Cache; HDD Read Ahead Cache: Enabled Volume Data Read Ahead: Normal HDD Queue Depth: 32 Disk Write Cache Mode: Disabled RAID 1+0; 8KiB Stripe Size; 8 x ST973451SS (73GB 2.5" SFF SAS 15K Savvio) Windows XP x64 SP2, NTFS, 32GB Partition; no allignment (file system start sector 63) ===================================================================================================== H:\xdd\bin>xdd.exe -verbose -op read -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek nge 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 15:37:11 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 3952 Thread ID, 3952 Processor, all/any Read/write ratio, 100.00, 0.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 100.944 85.096 1298.46 0.0008 3.46 read 65536 TARGET PASS0002 0 128 8589934592 131072 101.129 84.941 1296.09 0.0008 3.55 read 65536 TARGET PASS0003 0 128 8589934592 131072 100.580 85.404 1303.16 0.0008 3.61 read 65536 TARGET Average 0 128 25769803776 393216 302.036 85.320 1301.88 0.0008 3.54 read 65536 Combined 1 128 25769803776 393216 302.036 85.320 1301.88 0.0008 3.49 read 65536 Ending time for this run, Sun Dec 21 15:42:15 2008 H:\xdd\bin>xdd.exe -verbose -op write -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek r ange 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 15:44:02 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 1600 Thread ID, 1600 Processor, all/any Read/write ratio, 0.00, 100.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 267.955 32.057 489.16 0.0020 1.79 write 65536 TARGET PASS0002 0 128 8589934592 131072 269.735 31.846 485.93 0.0021 1.77 write 65536 TARGET PASS0003 0 128 8589934592 131072 268.538 31.988 488.09 0.0020 1.80 write 65536 TARGET Average 0 128 25769803776 393216 806.226 31.964 487.72 0.0021 1.79 write 65536 Combined 1 128 25769803776 393216 806.226 31.964 487.72 0.0021 1.78 write 65536 Ending time for this run, Sun Dec 21 15:57:30 2008
RAID 6:Code:Areca 1680ix-24 512MB Cache; HDD Read Ahead Cache: Enabled Volume Data Read Ahead: Normal HDD Queue Depth: 32 Disk Write Cache Mode: Disabled RAID 5; 8KiB Stripe Size; 8 x ST973451SS (73GB 2.5" SFF SAS 15K Savvio) Windows XP x64 SP2, NTFS, 32GB Partition; no allignment (file system start sector 63) ===================================================================================================== H:\xdd\bin>xdd.exe -verbose -op read -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek ra nge 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 22:25:06 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 3892 Thread ID, 3892 Processor, all/any Read/write ratio, 100.00, 0.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 171.366 50.126 764.87 0.0013 2.41 read 65536 TARGET PASS0002 0 128 8589934592 131072 170.789 50.296 767.45 0.0013 2.49 read 65536 TARGET PASS0003 0 128 8589934592 131072 171.240 50.163 765.43 0.0013 2.43 read 65536 TARGET Average 0 128 25769803776 393216 512.806 50.253 766.79 0.0013 2.44 read 65536 Combined 1 128 25769803776 393216 513.000 50.234 766.50 0.0013 2.40 read 65536 Ending time for this run, Sun Dec 21 22:33:40 2008 H:\xdd\bin>xdd.exe -verbose -op write -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek r ange 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 22:33:59 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 2364 Thread ID, 2364 Processor, all/any Read/write ratio, 0.00, 100.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 572.414 15.007 228.98 0.0044 0.89 write 65536 TARGET PASS0002 0 128 8589934592 131072 578.774 14.842 226.46 0.0044 0.87 write 65536 TARGET PASS0003 0 128 8589934592 131072 576.499 14.900 227.36 0.0044 0.89 write 65536 TARGET Average 0 128 25769803776 393216 1727.685 14.916 227.60 0.0044 0.88 write 65536 Combined 1 128 25769803776 393216 1727.685 14.916 227.60 0.0044 0.88 write 65536 Ending time for this run, Sun Dec 21 23:02:49 2008
Code:Areca 1680ix-24 512MB Cache; HDD Read Ahead Cache: Enabled Volume Data Read Ahead: Normal HDD Queue Depth: 32 Disk Write Cache Mode: Disabled RAID 6; 8KiB Stripe Size; 8 x ST973451SS (73GB 2.5" SFF SAS 15K Savvio) Windows XP x64 SP2, NTFS, 32GB Partition; no allignment (file system start sector 63) ===================================================================================================== H:\xdd\bin>xdd.exe -verbose -op read -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek ra nge 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 19:30:13 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 3160 Thread ID, 3160 Processor, all/any Read/write ratio, 100.00, 0.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 173.154 49.609 756.97 0.0013 2.45 read 65536 TARGET PASS0002 0 128 8589934592 131072 171.793 50.002 762.97 0.0013 2.33 read 65536 TARGET PASS0003 0 128 8589934592 131072 174.457 49.238 751.31 0.0013 1.97 read 65536 TARGET Average 0 128 25769803776 393216 518.649 49.686 758.15 0.0013 2.25 read 65536 Combined 1 128 25769803776 393216 519.000 49.653 757.64 0.0013 2.22 read 65536 Ending time for this run, Sun Dec 21 19:38:54 2008 H:\xdd\bin>xdd.exe -verbose -op write -targets 1 S24GiB -dio -blocksize 512 -reqsize 128 -mbytes 8192 -passes 3 -seek random -seek r ange 50000000 -queuedepth 128 IOIOIOIOIOIOIOIOIOIOI XDD version 6.5.013007.0001 IOIOIOIOIOIOIOIOIOIOIOI xdd - I/O Performance Inc. Copyright 1992-2007 Starting time for this run, Sun Dec 21 19:45:01 2008 ID for this run, 'No ID Specified' Maximum Process Priority, disabled Passes, 3 Pass Delay in seconds, 0 Maximum Error Threshold, 0 Target Offset, 0 I/O Synchronization, 0 Total run-time limit in seconds, 0 Output file name, stdout CSV output file name, Error output file name, stderr Pass seek randomization, disabled File write synchronization, disabled Pass synchronization barriers, enabled Number of Targets, 1 Number of I/O Threads, 128 Computer Name, ANALYTICAL, User Name, stcost Operating System Info: NT 5.2 Build 3790 Service Pack 2 Page size in bytes, 4096 Number of processors on this system, 4 Megabytes of physical memory, 4095 Seconds before starting, 0 Target[0] Q[0], S24GiB Target directory, "./" Process ID, 2192 Thread ID, 2192 Processor, all/any Read/write ratio, 0.00, 100.00 Throttle in MB/sec, 0.00 Per-pass time limit in seconds, 0 Blocksize in bytes, 512 Request size, 128, blocks, 65536, bytes Number of Requests, 1024 Start offset, 0 Number of MegaBytes, 8192 Pass Offset in blocks, 0 I/O memory buffer is a normal memory buffer I/O memory buffer alignment in bytes, 4096 Data pattern in buffer, '0x00' Data buffer verification is disabled. Direct I/O, enabled Seek pattern, queued_interleaved Seek range, 50000000 Preallocation, 0 Queue Depth, 128 Timestamping, disabled Delete file, disabled T Q Bytes Ops Time Rate IOPS Latency %CPU OP_Type ReqSize TARGET PASS0001 0 128 8589934592 131072 702.788 12.223 186.50 0.0054 0.63 write 65536 TARGET PASS0002 0 128 8589934592 131072 708.464 12.125 185.01 0.0054 0.64 write 65536 TARGET PASS0003 0 128 8589934592 131072 712.876 12.050 183.86 0.0054 0.72 write 65536 TARGET Average 0 128 25769803776 393216 2124.127 12.132 185.12 0.0054 0.66 write 65536 Combined 1 128 25769803776 393216 2124.127 12.132 185.12 0.0054 0.66 write 65536 Ending time for this run, Sun Dec 21 20:20:28 2008
Reply With Quote
Bookmarks