Thursday Mar 08, 2012

Creating Vdbench503 Swat charts using Swat302

A few weeks ago I was reminded of an incompatibility between Vdbench503and Swat302 that I introduced quite a while back. The file format when using Swat302 'File' 'Import Vdbench data' for Vdbench503 data has changed. Since at this time no new version of Swat is available outside of Oracle that of course is a problem.

To help you convert the new 503 'swat_mon.txt' file contents so that it can be used by Swat302, run the  AWK program below.

Henk



#
# Conversion of Vdbench503 swat_mon.txt file to a format that can be read
# by Swat302 using 'Import Vdbench data'.
#
# Note that this AWK program reads file swat_mon.txt (or a copy of it) and
# then replaces swat_mon.txt again.
# I suggest therefore that you first make a copy of swat_mon.txt and use that
# as input, because if you would accidentally run TWICE using swat_mon.txt as
# both input AND output you will have destroyed the original content of the file.
#
#
# To run:
# - cd /vdbench503
# - awk -f convert.awk output/swat_mon.txt.copy > output/swat_mon.txt
#
# Henk.
#

BEGIN {

}


{
  if ($1 == ":vdbench503_vdbench_data_for_swat303")
    print ":vdbench_data";
  else if (NF != 14)
    print($0);
  else
  {
    printf("%d %d %d %d %d %d %d %d %d %d %d %d %d %d\n",
            $1, $2, $3, $4, $5/1000, $6, $7, $8, $9/1000, 0, 0, 0, 0, 0);
  }
}



Wednesday Oct 12, 2011

Shared libraries for zLinux

IBM (thanks David) just sent me two shared libraries for 64bit zLinux, one for vdbench502, and one for vdbench503rc11.

Just click on the link above and store the file in your /vdbench/linux/ directory.

Henk.

Friday Jul 15, 2011

SlaveList.waitForSlaveWorkCompletion()

After elapsed=nn seconds, Vdbench tells its slaves to terminate the current run, and waits for them to do so. If that takes more than 30 seconds Vdbench displays this message: "SlaveList.waitForSlaveWorkCompletion()".

Why could it take more than 30 seconds? At the end of a run no new i/o is generated by Vdbench, but it still has to wait for i/o that is already outstanding to complete. But why can that take more than 30 seconds? There can be multiple reasons for that.

  • One reason can be that the storage device is dead or dying, something I hope you won't see out there in the real user world.
  • An other reason is that we specified such a huge amount of Vdbench thread count that the amount of i/o outstanding at run termination time is so overwhelming that indeed it can't finish within 30 seconds.
  • One more reason: you are running against a file system (or a cached volume, e.g. /dev/dsk instead of /dev/rdsk) and flushing all the remaining dirty blocks from cache takes very long.

Thursday Jun 09, 2011

Vdbench, Linux, and openflags=x_SYNC

It was just brought to my attention that there are some problems with the way that I implemented the O_SYNC, O_DSYNC, and O_RSYNC flags for Linux.

First of all, I misinterpreted what I found in /usr/include/bits/fcntl.h The flags are octal, not hex as I mistakenly read, meaning that what I coded as 0x010000 should have been 0x1000.

Second, I learned that the flags themselves have changed since I last looked at it, with the three flags in some versions of Linux no longer having the same value (octal 010000), but instead now have different values.

Since I can't begin to identify and check all the different flavors and versions of Linux and their flag settings I highly suggest before using these flags to check the contents of above header file and code openflags=0x…… with the proper HEX value as needed.

The next 503rcx version will have the 'proper' flags set to 0x1000, (whatever 'proper' means today).

Thursday May 26, 2011

Installing Java so that you can run Swat and Vdbench

Just received an email:

"Vdbench and Swat require JAVA to run.  Is there any way around this?  They would not let me load Java for fear it would change how the system was running."

There is no need to install Java in system directories and/or replace an existing version of Java. You can install Java in, and then run it from, your own private directories.



Henk.


Monday Dec 20, 2010

Swat 3.02 has been moved to My Oracle Support (MOS)

As of February 15 the link to MOS in this blog entry has changed!

Swat 3.02 has been moved to My Oracle Support (MOS) and is available for download for properly licensed Oracle customers from here. Swat is now only distributed as a zip file but contains everything needed for both Unix or Windows systems.

You will need to have a valid MOS logon ID. You can also just go to MOS and request patch 10350687.

Henk.

November 2013: An update: when I made Swat 302 available I put a five year expiration date on, fully expecting a new version to be available by then. And then Sun became Oracle, and a decision was made to no longer offer external support for Swat or make new versions available.

This now will cause Swat302 to expire on 'Thu Feb 13 14:45:12 MST 2014', warning messages will show up three months before that.

To work around this, download https://blogs.oracle.com/henk/resource/fixes/dont_expire_swat302.zip,  place it in the swat302 directory, and unzip. You should now have a new /swat302/classes/Swt/ directory. This will eliminate the expiration check.



Thursday Dec 16, 2010

Follow up on Swat download from sun.com failed: "User Not Authorized"

Though my discussions with Oracle legal have not completed yet it looks as if the limitation that only customers that have 'some' Oracle contract can download Swat will stay. I understand that the contract itself will be some minimal contract (you don't have to spend $100 million to get it), so loads of customers therefore should be able to get access to Swat. There is a Swat 3.03 coming; the legal status is pending.

Henk.

Tuesday Sep 21, 2010

Swat download from sun.com failed: "User Not Authorized"

Swat has been generally available for free now for several years, and that will continue to be so.
The transition from Sun to Oracle however is causing some problems.
Masses of websites are being moved, lots of changes are made to policies related to Solaris and other licenses.
Since Swat alas resides on one of the websites whose policies have changed it fell through the cracks.
I have been working with legal to get this resolved, but legal is now 100% busy with Oracle's annual Oracle Open World (OOW) conference and, sad to say, Larry Ellison has a higher priority than poor lowly me :-)
Until then, if your company is a licensed Oracle Sun customer you may have to find a valid contract number to add to your signon. If you don't have a license you'll have to hold off until I get this resolved.
I'll keep working on it from my side here.

Henk.

Thursday Aug 05, 2010

FSD size= mismatch

The followings message shows up after a format has completed successfully: The FWD parameters defined for 'fwd=xxxxx' do not match the parameters used in the previous run.

This message is there to tell you that a newly defined file structure does not match the existing structure and therefore can not be reused without the old structure first being deleted using the 'format=yes' parameter.

However, in this case a size=6.6m was used, which translates to 1024\*1024\*6.6=6920601.6 bytes. In the code where I compare the sizes I compare the old integer value of 6920601.0 with 6920601.6 and therefore report a mismatch. In this example changing size=6.6m to 6600k works around this problem. I will fix this in the next release.

Henk

Wednesday Jul 21, 2010

Abort in Vdbench503 due to long SD name.

Vdbench aborts after displaying this message on localhost-0.stdout.htm:

Vdbench JNI abort: generate lfsr_data(): String passed must be 8 bytes long:  sd3456789


It's clear that I never test with long SD names, it's usually sd1,sd2, etc.
503 introduced a dependency to an SD name not being longer than 8 characters. The SD name is used as input to the Logical Feedback Shift Register (LFSR) logic used to create the default data pattern. Until I fix this, just keep the SD name no longer than 8 characters (it may be shorter than 8).

Henk.



Monday Jun 28, 2010

Vdbench full screen editor

Some times you put little things in your code to make your life easier, and then you realize, "Maybe someone else can use this also". When back in 1978 I changed to using a full screen editor I thought "Wow, I'll never go back to a line editor.". And then of course when I arrived in the Unix world I had to take one gigantic step back and had to use a Very Inefficient line editor. All the primitive tricks needed to insert, delete, scroll, etc, just became too much for me to deal with so I spent less than an hour and wrote a primitive full screen editor in Java. No need to memorize ctrl/alt/del/:/q/r/left/right/up/down/a/b/c/d/e/...../z/zzzz (I am falling asleep just typing these zzzzzzzs).

So when I quickly need to create or edit a Vdbench parameter file I just enter (vdbench 502) './vdbench edit file.name' and I am back in the 21st century.

Henk.


Friday Jun 18, 2010

Vdbench and Swat: how to identify what is what when using 'ps -ef'

For obvious reasons I frequently have multiple Swat or Vdbench processes running, and sometimes get confused as to what is what. 'ps' output is not very helpful. I can maybe compare the heap size values, but I don't always remember them:

hvxxxx 21027 21008   0 09:02:01 pts/12      0:06 java -Xmx512m -Xms128m -cp ./:./classes:./swat.jar:./javachart.jar:./swing-layo
hvxxxx 21060 21041   0 09:02:04 pts/6       0:03 java -Xmx1024m -Xms512m -cp ./:./classes:./swat.jar:./javachart.jar:./swing-lay

A primitive little trick now is making my life easier by using the -D java parameter. It shows me that I am running one background data collector and one local real time monitor (swat -c and swat -l)

hvxxxx 21102 21083   0 09:04:49 pts/12      0:07 java -Dreq=-c -Xmx512m -Xms128m -cp ./:./classes:./swat.jar:./javachart.jar:./s
hvxxxx 21240 21221   0 09:06:01 pts/6       0:34 java -Dreq=-l -Xmx1024m -Xms512m -cp ./:./classes:./swat.jar:./javachart.jar:./

Update to the swat script, you can make a similar change to the Vdbench script

if ("$1" == "-t" || "$1" == "-p" || "$1" == "-l") then
  $java -Dreq=$1 -Xmx1024m -Xms512m -cp $cp Swt.swat $\*
else
  $java -Dreq=$1 -Xmx512m  -Xms128m -cp $cp Swt.swat $\*
endif



 Henk.

Wednesday Jun 09, 2010

Swat trace and prtvtoc

For Swat to be able to create a proper Replay parameter file it needs to know how large the luns are. For that the prtvtoc command is run during trace creation for each lun found in /dev/rdsk/. That can get awfully annoyingly slow when there are a lot of luns. To make this much faster replace the 'prtvtoc' lines in tnfe.sh with:

# Generate prtvtoc data (Is needed to create Replay parameter file)
# (devfsadm -C will clean up old garbage in /dev/rdsk)
printf "Running prtvtoc command"
ls /dev/rdsk/\* > /tmp/tnfe1
nawk '{disk = substr($1,1,length($1)-2); if (disk != last) print $1; last = disk}' \\
      /tmp/tnfe1 > /tmp/tnfe2
rm /tmp/tnf_prtvtoc.txt 2> /dev/null
while read disk;do
   echo Running prtvtoc $disk
   echo $disk       >> /tmp/tnf_prtvtoc.txt
   prtvtoc -h $disk >> /tmp/tnf_prtvtoc.txt 2>/dev/null
done  < /tmp/tnfe2
cp /tmp/tnf_prtvtoc.txt tnf_prtvtoc.txt


 Henk

Thursday May 13, 2010

Updates to Vdbench503 beta code

In this blog I'll try to keep you informed about the problems that have been found. Updates can be found on http://vdbench.org under 'View all files'

  • rc2: All platforms: an abort in JNI (C code) when trying to display an error or warning message BEFORE the first workload is started. Message in file localhost-0.html: "A fatal error has been detected by the Java Runtime Environment". Fixed in rc3.
  • rc2+rc3: All platforms: For raw I/O functionality (using SDs) all writes done using the new default random data pattern are reported as reads, though a write was correctly done. Fixed in rc4
  • rc2-rc4: Similar problems, not only with the random data pattern, but also with dedup and compression. Fixed in rc5

Wednesday May 12, 2010

First beta version of Vdbench 5.03, including Dedup and Compression

I placed vdbench503rcxxx on vdbench.org to give my users the opportunity to verify that there are no hidden problems in the code.Once I feel confident enough to make this code ready for GA I’ll also distribute the source code.

Would you like to measure the performance differences when running:
- without dedup and compression
- with only dedup and no compression
- with only compression and no dedup
- with dedup and compression
- for dedupratio=1,2,3,4,5,...
- for compratio=1,2,3,4,5,...?

Then pick up a copy of vdbench503rcxxx from vdbench.org.

For release notes see vdbench503_notes.html

For a tar or zip file, go to http://vdbench.org, select 'View all files', and look for vdbench503beta.

Any problems? Contact me at vdbench@sun.com

Henk.

About

Blog for Henk Vandenbergh, author of Vdbench, and Sun StorageTek Workload Analysis Tool (Swat). This blog is used to keep you up to date about anything revolving around Swat and Vdbench.

Search

Categories
Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today