hIOmon SSD Performance Monitor - Understanding desktop usage patterns.

[Ao1]

Hi overthere,

I have a few questions and apologies if they are a bit dumb.

Read Data Xfer = the amount of data transferred?

Read Xfer Max size = the maximum block size transfer?

Here I ran a winsat disk command. The QD's went through the roof!

[Ao1]

Here I copy the Adobe folder from Program Files to the desktop:

2502 Items in the directory with a combined size of 710MB



Here I copy an AVI file from desktop to the C drive.
File size 635MB

[overthere]

Hi overthere,

I have a few questions and apologies if they are a bit dumb.

Read Data Xfer = the amount of data transferred?

Read Xfer Max size = the maximum block size transfer?

You are on the right track. The values as exported by the hIOmon software are:

Read Data Xfer = The accumulated total (in bytes) of the data transferred by the read I/O operations as observed by the hIOmon I/O Monitor.

Read Xfer Max Size = The largest data transfer size (in bytes) associated with an observed read I/O operation.

A couple of other further explanations of the metrics shown that might be of interest to folks following this thread:

1) The "FastIOPcounts" (e.g., the "ReadFastIOPcount"); these counts (for either read or write I/O operations) reflect the accumulated number of I/O operations respectively that were successfully completed in less than one millisecond as observed by the hIOmon I/O Monitor.

2) The "Read Time Total" and the "Write Time Total" reflect the accumulated total (in seconds) of the time durations associated with the respective I/O operations.

Consequently, you can derive the "average" response time by dividing the "time total" by the respective "IOP Count". So, for example, a "Read Time Total" of 11.9739501 seconds for an associated "Read IOP Count" of 37322 indicates an average Read I/O operation response time of .3208 milliseconds.

3) Similarly, the "Read IOP Max Time" and the "Write I/O Max Time" reflect the maximum time duration (in seconds) for a read or write I/O operation respectively as observed by the hIOmon I/O Monitor. So a "Read IOP Max Time" of 0.1502034 reflects a maximum read I/O operation response time of 150.2034 milliseconds.

Finally, there are many more I/O operation metric types that can be collected/exported by the hIOmon software.

For example, there is the "ReadDataXferRateMax" metric, which reflects the maximum data transfer rate, i.e. the maximum amount (in bytes) of data transferred by I/O operations of the respective type (i.e., either read or write) as detected by the hIOmon I/O Monitor during a one-second interval.

The various hIOmon clients can be used to add additional metrics to an export file. The hIOmon Add-On scripts can also be modified with a simple change so as to include additional metric types for export.

[Ao1]

Thanks for the clarification.

So, outside of the monitoring results that included benchmarks the vast majority of read/ write IOPS are concluded in less than one millisecond.

When I copied a single 635MB file the max largest transfer size of 1.02MB is what occurred with a single read I/O operation? Now I'm struggling. The 635MB file is split into small "chunks" each with its own I/O operation? The larger the file the larger the "chunk" per I/O and the faster the write? Why are my Read/ Write Xfer Max Size typically comming in at around 1MB? Is that the optimum size for an I/O operation? (Sorry for being dumb again).

[overthere]

When I copied a single 635MB file the max largest transfer size of 1.02MB is what occurred with a single read I/O operation? Now I'm struggling. The 635MB file is split into small "chunks" each with its own I/O operation? The larger the file the larger the "chunk" per I/O and the faster the write? Why are my Read/ Write Xfer Max Size typically comming in at around 1MB? Is that the optimum size for an I/O operation? (Sorry for being dumb again).

There is not necessarily a one-to-one correspondence between a given file I/O operation and any actual I/O operation to the disk drive itself.

As simple examples:

1) A single file I/O operation can incur two or more I/O operations to the disk drive (e.g., file fragmentation). BTW, the hIOmon software can capture I/O operation metrics that help empirically identify the performance impact of file fragmentation:

http://www.hyperIO.com/hIOmon/Screen...WMIbrowser.htm

2) Two or more file I/O operations can be satified by a single device I/O operation. This is largely the result of the "system file cache" usage.

For example, some data is read from the device into the system file cache (which is maintained by the operating system) and multiple file I/O operations are subsequently satisfied from the system file cache (rather than directly from the device).

See, for instance, the second screenshot on the following web page:

http://www.hyperIO.com/hIOmon/hIOmon...fQuestions.htm

Overall, the interactions between the I/O operation activity at the "file level" within the I/O stack within the operating system and the lower physical volume and physical device levels within the I/O stack can be complex.

This is a major reason why the hIOmon software provides the ability to capture/collect I/O operation performance metrics at all three levels (and, moreover, optionally do so concurrently so that you get a much clearer picture of what exactly is going on).

But back to your questions, there are a number of factors that can be involved in determining the particular types of I/O operations that will actually result from a "simple" copy operation (e.g., the nature of the copy program itself, buffering nuances, system resources available/used at the time of the copying, any other concurrently running processes, etc.).

So unfortunately there are no simple answers to your questions, but indeed the copying of a "large" file will likely entail multiple write I/O operations to the actual disk device.

[Ao1]

Boy storage is complicated stuff. The more I think I've learnt the more I realise I don't know.

HIOmon is really helping me though and I really appreciate your insights.
I've noticed that the ReadSystemCacheIOPcount and ReadSystemCacheIOPmissCount (same with writes) are always 0. I think the only exception was when I ran the winsat disk benchmark.

What does that tell me?

EDIT:

Also is there a way to monitor what happens when the PC boots?

[overthere]

Boy storage is complicated stuff. The more I think I've learnt the more I realise I don't know.

HIOmon is really helping me though and I really appreciate your insights.
I've noticed that the ReadSystemCacheIOPcount and ReadSystemCacheIOPmissCount (same with writes) are always 0. I think the only exception was when I ran the winsat disk benchmark.

What does that tell me?

I certainly agree: storage in general is inherently pretty complex.

The wide range of summary metrics made available by the hIOmon software provides an explicit reflection of this complexity in large measure.

Of course, the complexity involved in actually capturing/collecting/exporting/etc. these summary metrics is yet another matter!

Anyway, regarding the ReadSystemCacheIOPcount and the ReadSystemCacheIOPmissCount metrics, these metrics reflect read I/O operations that, as observed by the hIOmon I/O Monitor, were directed towards a file and which explicitly requested the use of the system file cache. The same applies to the write variants of these metrics as regards write I/O operations.

As such, they only pertain to file-level I/O operations. Accordingly, these metrics are not applicable to I/O operations that are observed by the hIOmon I/O Monitor at either the physical volume or the physical device level within the operating system (and so, of course, their value will be zero in these cases).

[overthere]

Also is there a way to monitor what happens when the PC boots?

The hIOmon software does provide an option to collect I/O trace and/or summary metrics during the system boot/start process.

This can get a bit tricky given, for instance, the volume of I/O operation activity that generally occurs during the system boot/start - along with the mechanics required to collect the requested metrics while the system itself is not fully operational yet.

If you would like to pursue this, please see the "Boot Logging" section (starting on page 68) and section "7.5 Boot Logging" within the "hIOmon User Guide" document.

[Ao1]

Here I use the WMI browser to give me a live feed of the top 10 processes, whilst running all the typical programs that I use, namely: Power DVD, WMP, Photoshop, Modern Warfare 2 SP, Tracktor, Office, IE9, Messenger & MS Essentials (Quick scan).

Bandwidth
Interestingly MW2 SP only loaded at a max read transfer rate of 81.78MB/s

If I add up all the top ten maximum max read transfer rates I get a total of 327.12MB/s, so if I wanted to run all of the them at the same time I would need a bandwidth of 327.12MB/s, although that would not be physically possible to do. If I took out MW2 SP I would need a bandwidth of 245.35MB/s, which is possible with the X25-M, but again it would still be physically impossible to run them all at the same time. (Without running a batch file).

The fastest write speed I obtained was 42MB/s using Photoshop. If I add up all the top ten maximum max write transfer rates I get a total of 98.88MB/s, which is within the specs of an X25-M, but again impossible to implement together at the same time.

IOPS
The IOPS constantly change when being monitored so between selecting specific filters it keeps increasing, however it is clear to see that virtually 100% of IOPS are done in less than one millisecond.

Queue depths
The maximum QD's for writes surprised me but the averages (not shown below) are always below 2. The same with reads. I ran more proceses togther than what I would do normally to try and push the QD up.

Now I see exactly why a SSD raid 0 array was not any faster for my particular usage patterns in comparison to a single SSD. I didn't need the bandwidth or an increase in IOPS.