hIOmon SSD Performance Monitor - Understanding desktop usage patterns.

[Ao1]

Cleaned up, as superseded in post #88.

[Anvil]

Give it a shot using the summary from the Presentation Client.

There are of course other metrics that are of interest

Do the test as you've planned, the outcome might need second runs and then you can include other metrics.

[Anvil]

Ok, so I tried the hIOmonBenchmarkExport batch file and captured an AS SSD run.

Is there some way to display the csv file using hiomon or should I just open it in Excel?

(this output is generated using the wmi browser,
it's not accurate as there were activities recorded while generating the output)
Capture_as_ssed.PNG

corresponding AS SSD screenshot
as-ssd-bench Volume0 27.10.2010 23-58-46.png

[Ao1]

You can open it with excel. It will only open as a read only file but you can still work with it and "save it as" if you want to keep the changes.

You know the other thing I'm really intereted to see is if the read/ write speeds for a single IOP go up when I get the raid 0 array going.


Edit Anvil is that with 3xRO

[overthere]

Ok, so I tried the hIOmonBenchmarkExport batch file and captured an AS SSD run.

Is there some way to display the csv file using hiomon or should I just open it in Excel?

Hi Anvil,

I gather that you used the "Save" option with the hIOmonBenchmarkExport batch file, which resulted in the current accumulated summary metrics being written to the CSV file.

The short answer is that you will need to use some other program (e.g., Excel as you mentioned) to display this CSV file.

Depending upon how the hIOmon software was configured to collect the I/O operation performance metrics, the hIOmon Presentation Client can be used to display the collected metrics from the hIOmon "File I/O Log" file - but this is another story!

In any case, none of the hIOmon clients can be currently used to directly view a CSV file.

The basic idea has been that once the metrics were exported/contained within a CSV file, then a variety of other existing tools (e.g., Excel) could be used to display, process, etc. these metrics in a manner familiar to the user and leveraging the features of such tools.

[Anvil]

Hi overthere

I used the Save option and I expected Excel was the answer

I've included the result file if anyone would like to have a look at it.

It looks like the collected metrics are really close to what one would expect from an AS SSD run, looking great so far

[Anvil]

Ao1

Yes, that using 3R0, strip size 16KB, RST 10.0.0.1026

I've downloaded the latest RST, I'll install it this weekend.

[Ao1]

Anvil, have you tried using the default SSDPerfAnalysisFS script? (post #5) It has quite a few good performance metrics on it as standard. (It's also possible to add more).

If you didn't select the automated monitoring when you installed hIOmon its easy to do it afterwards.

Start>hIOmon>Getting started help>hIOmon automated monitoring configuration set up. Then select option 3

When I compare results from my As Benchmark run I see from your cvs file that your avg and max read QD were much higher than mine but conversely my IOP count was much higher.

Edit: What strip size shall I use? I was going to use 128k, but I think a much lower strip will be more interesting (?)

[Anvil]

Ao1,

I'm currently using the SSDPerfAnalysisFS script and I've been playing a bit with the automated configs.

I havent loaded the csv file yet, my bench-pc is for benchmarks only , I'll load it on my laptop shortly.

Most of the time I've been using 8-16KB, for the Intels 16KB is looking really good, especially using RST 9.5 or later.

[Computurd]

very interesting stuff here. interested in your high QD numbers, thinking here of reasons... 10 channels per device, three devices, total of 30 channels. that would be a QD of 50 per-channel roughly...might be explainable?? but doubt it. i think the system sees the devices as one drive.

NCQ allows the system to ask for I/O in a different manner, optimized and designed for HDD, it still works phenomenal wonders with SSD.

The shopping analogy applies very well here, for those who aren't sure of its functions:

Imagine you have a shopping list of 50 items, and you go to the grocery store. if you would go and find the first item, then go to the checkout lane and pay for it, then go back, get another item, then go pay for it, repeat repeat repeat 50 times, that is in essence how I/O operations work without NCQ. the system waits for the I/O request to 'return' before it asks for another. it has its list, but it wants it one at a time.
NCQ gives you the list, then tells the device to go get it, in any order, all at once. it doesn't care what order the "items" are obtained, it just wants them all to return at once. it does not wait for return verification.
so....much like taking the 50 item list shopping, go get all 50 items in the most efficient order, THEN go check out.
much faster, and more efficient.
NCQ works exceedingly well with SSD. So...maybe the higher QD are the result of these 'lists' being served so quickly, or being dumped on to the controller of the SSD so quickly. what if it is counting each request on the list as a QD? so one list of 50 would be a 50 QD spike? maybe longer 'lists' are served to devices that can return the 'lists' faster....maybe these higher QD numbers are the result of VERY efficient NCQ being run by the system.

god who knows. interesting any way you slice it
how would you prove such a theory though?

i guess that has become my side-mission here. maybe my main mission! to find how NCQ 'lists' are monitored as QD would be very interesting. what if there is a curveball here though...what if one 50 item list, only counts as ONE QD???

[overthere]

It looks like the collected metrics are really close to what one would expect from an AS SSD run, looking great so far

Certainly good to hear!

[overthere]

i guess that has become my side-mission here. maybe my main mission! to find how NCQ 'lists' are monitored as QD would be very interesting. what if there is a curveball here though...what if one 50 item list, only counts as ONE QD???

CT,

The basic mechanism behind the "queue depth (QD)" metric within the hIOmon software is fairly straightforward:

When the hIOmon I/O Monitor observes the start of an individual monitored I/O operation, then the "queue depth" counter is incremented.

And when the hIOmon I/O Monitor observes the completion of a monitored I/O operation, then it decrements the respective "queue depth" counter.

There is a separate "queue depth" counter maintained for each monitored device, file, and process, along with associated "subset" QD counters (e.g., separate "Read" and "Write" QD counters, again upon an individual monitored device, file, and process basis).

Also, the above applies separately at each level of the I/O stack being monitored by the hIOmon software, i.e., at the file (logical drive/disk), physical volume, and physical device levels.

[Ao1]

Here I compare raid 0 against a single drive.

The monitoring period started up after boot up. I ran the following one after the other with only IE & MSM left running throughout.

I tried to replicate the use of apps as close as possible on each run.

• Play MW2 SP (Intro and then a couple of level loads).
• IE
• MS Messenger
• Play DVD
• Play WMC
• Photoshop - Open 4 jpeg files from the desktop simultaneously. File sizes: 1MB, 2.14MB, 22.3MB, 127MB. Run a "save as" on each file back to the desktop. (CPU maxed out a bit on the larger files sizes).
• Tracktor Scratch Pro
• MS Essentials quick scan

I avoided multi taking because I didn't want the CPU to get maxed out and possibly skew the results, but I will try it later. The strip was set at 128K. I might try a really small strip to see what that does.

General observation.

Unlike the big difference between ssd and hdd I did not notice any difference in boot time or game loading. Photoshop did seem to save the files quicker, but the CPU struggled a bit on larger files sizes, so file saves weren't that fast anyway.

[Computurd]

when you ran these did you get any plain ol C:\ drive stats? as opposed to the app monitoring?

[Ao1]

Hi Comp,

The 1st image shows drive summaries. The other images drill down into the top ten processes. As Overthere has stated an exe file is not necessarily the only process associated with an application and to get that I would need to monitor the specific folder of interest.

iw4sp.exe is Modern Warfare 2 SP. The max MB/s read was actually marginally slower on the raid 0 configuration!

The big surprise for me (based on my usage patterns) so far is that MB/s read speeds don't come in anywhere close to the capability of a single drive, never mind raid 0, but conversely faster MB/s writes speeds are actually quite useful.

[Ao1]

Here is a comparision of what happens during boot up.

[Ao1]

From now on: page file off\

[Computurd]

ok so i want to start with the results on the post 88 and tell you what i see.....

RAID.............................................. ................................................Si ngle drive



the most glaring here...latency is much faster for reading. .0058 v .0263 (22 percent faster) and .3925 v .8114 (48 percent faster)



Fast write IOPS around twenty percent better with raid. this is just the fast writing, not the overall btw,.



onto overall writing, damn near twice as fast. 478 raid v 253 single device




mixed read/write fast iops are much better,, around ten percent, with the raid




overall response times, again a big difference here... .0065 v .0256 (25 percent faster) and also .6828 v 1.1426 (60 percent faster)

R/W Fast IOPS 93.7 v 84.4 (10 percent better)

---------------------------------------------------------------------------
had to steal this from the SSD raid o "bootlog" vs single device "bootlog" section since the data was missing from post 88...
so the format is different....

single device............................................ ..........................................raid 0



there is a tremendous difference here....
l
--------------------------------------------------------------------------------------------



the other metrics are quantitative...I.E. they aren't good for comparisons sake...should be close to same amount of data transferred (x amount) etc...really the rest of them are not going to help with comparing.

the other testing with the ssd raid 0 vs single device boot logging mirror these results pretty clearly.

Whether or not your system is fast enough as a whole to capitalize on these differences is another thing entirely. imagine when programs are coded for SSD...jesus the difference would be phenomenal.

[Ao1]

Here I monitor statistics, post boot, for 30 minutes with a single SSD. I multitasked as I would normally do and opened and used the typical applications/ games that I use.



Now I try to consider the IOPS that occurred in use vs the IOPS capability of my drive. To do this I consider average queue depth and average latency readings to get some ball park figures using the following formula:

Queue depth*(1/.latency in ms) = IOPS (in seconds)

Based on the average latency of 0.49ms

• Queue depth 1 x (1/0.00049) = 2,040 IOPS
• Queue depth 64 x (1/0.00049) = 130,612 IOPS

My average queue depth is 1.57

• 1.57 x (1/0.0049) = 3,204 IOPS

My maximum IOP count occurred during boot @ 190 IOPS, but my SSD is capable of 3,204 IOPS based on average latency and queue depth.

I know there could be a lot of variables to the averages but have I got that correct as a ball park figure?

[Ao1]

Now I consider the average response times (latency) between a single SSD and Raid 0.



What I discover when I turn off the page file is that the average response time for a single SSD drops to the average level of an SSD Raid 0 array.

Turning off the page file in the Raid 0 array had no impact on average response times.

The average response times between a single SSD with no page file and an SSD raid 0 array are therefore of no significant difference.

My average queue depths remain in the same ball park figure regardless if I use a single SSD or a Raid 0 array.

Consequently there is no significant average variable to generate more IOPS between the two configurations, although as per post #94 I can't even use anywhere close to the IOPS I get from a single drive.

So no improvement on average latency or average queue depth with Raid 0 based on my storage demands.

Looking at the individual processes I ran none of them could utilise the max sequential read speed capability of my SSD. When running multiple processes together to try and reach the maximum sequential read speed capability of my SSD the CPU would max out.

Using Raid 0 did not improve the maximum sequential read speed of games or most other processes.

Using Raid 0 did however improve the maximum sequential write speeds for applications such as Photoshop.

Does all this sound right or am I making a mistake to focus on averages?

[IanB]

Looking at the individual processes I ran none of them could utilise the max sequential read speed capability of my SSD. When running multiple processes together to try and reach the maximum sequential read speed capability of my SSD the CPU would max out.

This does underscore the things we need to think about when looking at SSD marketing numbers for our relatively undemanding desktop systems.

10,000 IOPs or more may scream "better!" but if the highest IOPs actually demanded by the system is in the hundreds there's no practical gain from that. Sucking in huge quantities of data faster (higher streaming speed) from disks is better, surely? But the processor then has to do something with that vast quantity of data, which places the bottleneck back on processing speed beyond a certain point.

For server situations, high IOPs are significant since you have multiple users making multiple requests and higher streaming speeds may mean the individual requests can complete faster. That's obviously an advantage.

But the bottom line you've demonstrated, if my skim of the results and your analysis is correct, is that the read/write latency is the only significant factor affecting speed for most single-user desktop systems that do not have specialised requirements. That should be the obvious metric we need to compare between disks. And if you think about it... that's what we've always done with storage! The mechanical disk with a lower track seek has always had the overall speed advantage.

And yes, the pagefile result is fascinating! I don't think anyone would have expected that turning it off competely affects speed. As a guess, maybe Windows is dumping some reads speculatively into pagefile simply because it's there, wasting time if the physical memory in the system is enough to handle the total VM requests.

[Ao1]

Here is an AS SSD benchmark of an OCZ Core on the left and an X25-M on the right.



The OCZ Core has no NCQ support, so IOPS can not increase with queue depth.

• Both drives have fast sequential read/ write speeds. (Compared to HDD).
• Read response times are low on both drives. (Compared to HDD).

The write response time is however very high on the OCZ Core when you start writing lots of small files randomly.

This is due to the write penalty. The OCZ Core had a 4KB page file. The 4KB page files sit in a 512KB block. Once a 4KB page has been written, it can’t be overwritten, it must be erased first before you can write to it again. So, a 4KB write can require a 512KB block to be erased before it can be written. A delay is therefore caused by the deletion of a 512KB block before the 4KB file can be written, which is why the write access time becomes very high.

From what I have monitored large [single files like avi's] are undertaken with only a few IOPS, hence there should be no problem copying large [single] files with the OCZ Core. Lots of small writes are however a problem.

When the OCZ Core is hammered with 4K writes the access time reached 2.5 seconds.

• Queue depth 1 x (1/0.002496) = Write 400 IOPs

When the X25-M is hammered with 4K writes the access time reached 0.091ms.

• Queue depth 1 x (1/0.000091) = 10,989 IOPS (Which is above "up to" max specs of 8,600 IOPS)

I hope I have understood this correctly. Please jump in if I am making incorrect assumptions.

[Ao1]

This does underscore the things we need to think about when looking at SSD marketing numbers for our relatively undemanding desktop systems.

10,000 IOPs or more may scream "better!" but if the highest IOPs actually demanded by the system is in the hundreds there's no practical gain from that. Sucking in huge quantities of data faster (higher streaming speed) from disks is better, surely? But the processor then has to do something with that vast quantity of data, which places the bottleneck back on processing speed beyond a certain point.

For server situations, high IOPs are significant since you have multiple users making multiple requests and higher streaming speeds may mean the individual requests can complete faster. That's obviously an advantage.

But the bottom line you've demonstrated, if my skim of the results and your analysis is correct, is that the read/write latency is the only significant factor affecting speed for most single-user desktop systems that do not have specialised requirements. That should be the obvious metric we need to compare between disks. And if you think about it... that's what we've always done with storage! The mechanical disk with a lower track seek has always had the overall speed advantage.

And yes, the pagefile result is fascinating! I don't think anyone would have expected that turning it off competely affects speed. As a guess, maybe Windows is dumping some reads speculatively into pagefile simply because it's there, wasting time if the physical memory in the system is enough to handle the total VM requests.

I am trying to muddle through this with a very limited understanding. The monitoring results are what they are. I really hope the conclusions I try to draw are not misleading.

It would be great to get some feedback if I am looking at it incorrectly.

[Computurd]

Now I try to consider the IOPS that occurred in use vs the IOPS capability of my drive. To do this I consider average queue depth and average latency readings to get some ball park figures using the following formula:

Queue depth*(1/.latency in ms) = IOPS (in seconds)

Based on the average latency of 0.49ms

• Queue depth 1 x (1/0.00049) = 2,040 IOPS
• Queue depth 64 x (1/0.00049) = 130,612 IOPS

My average queue depth is 1.57

• 1.57 x (1/0.0049) = 3,204 IOPS

that cannot be correct, because you are assuming that the latency is always the same here. it isnt. it changes with QD, and also with the type of access that you are doing. mixed read/write kills latency

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

What I discover when I turn off the page file is that the average response time for a single SSD drops to the average level of an SSD Raid 0 array.

So no improvement on average latency or average queue depth with Raid 0 based on my storage demands.

what are you talking about? look at the numbers here...how can you say that it is the same level? they arent even close!! this is from the graphic above where you posted this, if you look at the numbers...



you have .0239 v .0051 that is over twice as fast! also, you have 474.8354ms v 132.0012ms that is a 3x difference!

can you highlight the numbers that show where average response time for a single SSD drops to the average level of an SSD Raid 0 array?? i mean not to be incredulous or anything, but i am

Consequently there is no significant average variable to generate more IOPS between the two configurations, although as per post #94 I can't even use anywhere close to the IOPS I get from a single drive.

well you have two programs requesting the same amount of information, correct? it is the latency of the IOPS that counts.

Looking at the individual processes I ran none of them could utilize the max sequential read speed capability of my SSD.

of course not. do the processes even max out the sequential read capability of your HDD? nope. again, not the amount, the latency is why they are faster. there are many programs that do not max out the speed of your hdd which are way faster with SSD. latency.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

It would be great to get some feedback if I am looking at it incorrectly.

your thoughts on post 93? and this post? they both cover much the same info. maybe we are looking at different numbers.

[Ao1]

Hi Comp, I'll get back to you on your posts soon, but 1st I thought some more info on the device summary statistics would be useful.