IO Sizes and Alignments with Dtrace

In some previous postings I've talked a bit about the impact of IO size and alignments on the performance of flash devices. These parameters are easily controlled in a microbenchmarking environment, but can be quite difficult to determine in more complex application environments.

A lot of folks have asked me about using iostat in this context. The iostat utility is great for telling you the total number of reads and writes, but it doesn't convey information about individual sizes and alignments (start addresses) of those IOs. You can easily figure out the \*average\* IO size from iostat, but that may or may not correlate well to actual size of individual reads and writes depending on the workload.

One way of getting more visibility into IO sizes and alignments is to use dtrace to gather the information of interest. In this post, I'll share with you a script that has been used internally for this purpose. This should be viewed as a "unsupported, evolving" script, and I will post updates as we tweak it. Nonetheless, it seems to be a fairly useful bit of code and worth sharing. A link to the full script is provided at the bottom of this post.

A few bits of info about this script...

The script instruments a function in the sd driver, and basically just histograms the sizes of IO packets. It turns out that one of the difficult parts about doing this is not getting the packet size, but gathering the information for only the devices you are interested in.

One option is to restrict by using major/minor device numbers, but that is not the most intuitive thing from a user standpoint. These also tend to vary quite a bit, so one would always be hacking up the script. We needed a better way...

Paul Riethmuller came up with the swell idea of using the product id string that is stored when a inquiry command is done. For example:

# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t1d0 
          /pci@0,0/pci108e,5354@1f,2/disk@1,0
       1. c0t2d0 
          /pci@0,0/pci108e,5354@1f,2/disk@2,0
       2. c0t3d0 
          /pci@0,0/pci108e,5354@1f,2/disk@3,0
       3. c1t0d0 
          /pci@0,0/pci8086,3408@1/pci1000,1000@0/sd@0,0
       4. c1t1d0 
          /pci@0,0/pci8086,3408@1/pci1000,1000@0/sd@1,0
       5. c1t2d0 
          /pci@0,0/pci8086,3408@1/pci1000,1000@0/sd@2,0
       6. c1t3d0 
          /pci@0,0/pci8086,3408@1/pci1000,1000@0/sd@3,0
Specify disk (enter its number): 6
selecting c1t3d0
[disk formatted]


FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        fdisk      - run the fdisk program
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !     - execute , then return
        quit

format> inquiry
Vendor:   ATA     
Product:  MARVELL SD88SA02
Revision: D20R

The script as currently presented below tracks the number of total IOs issued for all devices, but only keeps detailed statistics for those devices that have a Product ID starting with "MARVEL". (This happens to be the Product ID returned if you run the inquiry command on a Sun F5100 or F20 flash device. ;-)).

Also note that currently the script produces limited output every 5 seconds, but the full histogram of IO sizes is only printed when one hits \^C. Obviously this can be changed to suit. Also note I'm not much of a dtrace hacker- you can find much better in other places at Sun. ;-)

Acknowledgements: Thank you, Paul Riethmuller, for all the good bits and ideas. Any broken bits are due solely to me.

Some sample output:

# ./align.d

4.966478729 seconds elapsed
       2 IOs issued

5.000003686 seconds elapsed
    2465 IOs issued
    2437 Marvell IOs issued
               5 RDs        2432 WRs
       0 Marvell IOs misaligned start
       5 Marvell IOs non-multiple of 4KB size
       0 Percent non-4k Marvell IOs

\^C
11.461851773 seconds elapsed

    2774  Tot IOs issued
    2740 Marvell IOs issued
       0 Marvell IOs misaligned start addr
       5 Marvell IOs non-multiple of 4KB
       0 Percent non-4k Marvell IOs

 MARVELL WRITE IO 
 IO_size Count

               36                0
              512                0
                0                2
           131072             2733

 MARVELL READ IO 
 IO_size Count

                0                0
           131072                0
               36                2
              512                3

 MARVELL ALL IO 
 IO_size Count

                0                2
               36                2
              512                3
           131072             2733

align.d script

Updated 2/14/2010, minor bugfixes (errors in comments):
align-v2.d script

Comments:

The script is great. Could it be improved to be used on a given disk?

Posted by larry liu on January 29, 2010 at 05:33 AM EST #

Post a Comment:
Comments are closed for this entry.
About

Lisa Noordergraaf

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Feeds