Thursday Apr 22, 2010

Pkgapp 3.4 Released

Yes, I've released it!

Photo © Lee TrujilloI've updated the Licensing agreements etc., packaged the files and released internally (only).  If you need it, you can call Sun Oracle Support and have them use the following Internal Only url (do not click this link, it is not available outside of the Sun Oracle network).  I have submitted two new packages (tars) to BigAdmin and we hope they will make it on the page sometime next week.  Keep checking the BigAdmin link.

Pkgapp 3.4 - http://surl/pkgapp

Changes in Pkgapp 3.4

Improvements

  1. On all platforms Pkgapp now utilizes the /proc filesystem to avoid locking issues (as seen on  Solaris) of the process instead of using pldd against the live process.  See “Ptools / Pstack Issues” of the Server Impact section.
  2. Removed old 3.2 code and reworked how the -a parameter checks are displayed (see the pdf guide for more detail).
  3. When -i (include core) is not used, renames the core/gcore file to - pkgapp-<hostid>-<corefile name>.  This will help Sun Oracle employees easily locate the core file based on the name.
  4. Enhanced the error checking with regard to usage options.
  5. Set umask on the tmp bin and databin locations to enhance security.

Fixes

  1. Fixed bug 6918386 pkgapp 3.3 fails when processing argument for parameter -a
  2. Fixed bug 6919793 Pkgapp 3.3: awk fails on excessive string length from a localized binary.
  3. Fixed bug 6920593 Pkgapp 3.3: missed capturing sparcv9 \*_db.so\* on Solaris
  4. Fixed an issue where Pkgapp would ignore the core (when provided via -c) when the -j parameter was given.

Example:

Solaris
root[/opt/pkgapp]#./pkgapp -itr -p 18101 -c /var/tmp/cores/core.18101 -a /opt/dsee7/lib/64/ns-slapd -s /var/tmp/storage
\* ------------------------------------------------------------------------------
\* Oracle Corporation CSD pkgapp 3.4 Solaris                         [04/22/2010]
\* ------------------------------------------------------------------------------
\* Checking install epoch/delta          [good, 772538 seconds]
\* Checking pkgapp path                  [using /opt/304-solaris]
\* Checking for locale en_US.UTF-8       [failed]
\* Checking for locale en_US.ISO8859-1   [failed]
\* Checking for locale en_US             [failed]
\* Checking for locale C                 [success]
\* Checking usage history                [not recently run]
\* OS release [Solaris]                  [5.10]
\* Platform/Type                         [SUNW,Sun-Blade-T6300 sparcv9]
\* Checking [-p] pid.                    [using pid 18101]
\* Checking [-a] path                    [using /opt/dsee7/lib/64/ns-slapd]
\* Process root                          [/opt/dsee7/lib/64/ns-slapd]
\* Checking [-c] core                    [using core /var/tmp/cores/core.18101]
\* Databin parameter [-s] checks         [reset to /var/tmp/storage]
\* Creating Databin directories          [/var/tmp/storage]
\* Databin used/created is               [/var/tmp/storage/pkgapp-042210-01]
\* Creating temp area                    [/tmp/pkgapp.15891/]
\* Checking corefile for a valid pldd    [pldd is good with 80 elements]
\* Checking if corefile is truncated     [CORE core.18101 is GOOD. Core contains 320 segments ]
\* Continuing...                         
\* Process binary                        [ns-slapd]
\* ns-slapd binary bit version           [64]
\* core.18101 bit version                [64]
\* Checking path [-a] to binary name     [success, path includes binary name]
\* Checking path [-a] exists             [success]
\* Locating ns-slapd                     [success]
\* Checking located ns-slapd is 64 bits..[success]
\* Binary located                        [/opt/dsee7/lib/64/ns-slapd]
\* Adding binary to pkgapp.pldd          [success]
\* Grabbing libs from /proc              [success]
\* Grabbing pstack from core             [success]
\* Grabbing pmap from core               [success]
\* Grabbing pcred from core              [success]
\* Grabbing pflags from core             [success]
\* Grabbing pargs from core              [success]
\* Grabbing [-i] core/gcore              [success]
\* Grabbing core/gcore cksum             [success]
\* Javatools [-j] not set                [skipped]
\* Grabbing /var/adm/messages            [success]
\* Grabbing uname -a                     [success]
\* Grabbing date/time                    [success]
\* Grabbing showrev -p                   [success]
\* Grabbing pkginfo -l                   [success]
\* Grabbing /etc/release                 [success]
\* Grabbing coreadm                      [success]
\* Grabbing ulimit -a                    [success]
\* Gathering 64 bit libraries            
\* Grabbing libs                         [success]
\* Making lib paths app/                 [success]
\* Making lib paths libs/                [success]
\* Linking libraries                     [success]
\* Libraries linked                      [109 ttl]
\*                                       
\* Using hostid for naming .tar.gz       [84aaaac8]
\* Writing file                          [pkgapp-84aaaac8-s-6000a-t6300i-zone8-sca11-100422-092800.tar.gz]
\*                                       
\* Done gathering files                  
\* Writing dbxrc & opencore.sh files     [success]
\* Writing coreinfo.sh file              [success]
\* Writing validate-libs.sh file         [success]
\* Writing manifest-100422-092800.log    [success]
\* Writing pkgapp-args-100422-092800     [success]
\* Creating final tarfile                [success]
\* Compressing tarfile                   [success]
\* End of runtime logging                
\* Saving history info                   [success]
\* Saving runtime log                    [success]
\* Saving Library list                   [success]
\* Saving Library list README            [success]
\* Removing [-r] temp area/files         [removed]
\*                                       
\* Operations Complete                   
\* Total runtime                         [1 minutes]
\* ------------------------------------------------------------------------------
\* Upload the following file(s) to http://supportfiles.sun.com Cores Directory at
Oracle
                                        
1) File(s) located in directory /var/tmp/storage/pkgapp-042210-01
                                        
                [ pkgapp-84aaaac8-s-6000a-t6300i-zone8-sca11-100422-092800.tar.gz ]
                                        


                                Thank you.
                                Customer Service Delivery
                                Global Systems Support


NOTES:
1) You can check for updates to this script here:
        BigAdmin - http://www.sun.com/bigadmin/scripts/indexSjs.html
        Blog - http://blogs.sun.com/Dirtracer

2) GDD information located here:
        Rel Notes - http://docs.sun.com/app/docs/doc/820-0437
        Docs - http://www.sun.com/service/gdd/index.xml
        Video - http://mediacast.sun.com/search?query=gdd
        Video - http://mediacast.sun.com/search?query=dirtracer

3) Please send all Bugs and RFE\\'s to the following address:
        Subject "pkgapp bug/rfe" - pkgapp-feedback_ww@oracle.com

4) See the following reference files.
        /var/tmp/pkgapp-history/runtime-100422-092800.log
        /var/tmp/pkgapp-history/history-100422-092800.log
        /var/tmp/pkgapp-history/library.list-100422-092800
        /var/tmp/pkgapp-history/library.list-100422-092800.README 
Linux
root [/opt/pkgapp]# ./pkgapp -itr -p 11716 -c /var/tmp/cores/core.11716 -a /opt/dsee7/lib/64/ns-slapd -s /var/tmp/storage
\* ------------------------------------------------------------------------------
\* Oracle Corporation CSD pkgapp 3.4 Linux			    [04/22/2010]
\* ------------------------------------------------------------------------------
\* Checking install epoch/delta		 [0 seconds]
\* Checking pkgapp path			 [using /opt/pkgapp]
\* Checking for locale en_US.utf8	 [success]
\* Checking usage history		 [not recently run]
\* OS Release [Linux]			 [2.6.9-5.EL]
\* Platform/Type				 [x86_64 x86_64]
\* Checking [-p] pid			 [using pid 11716]
\* Checking [-a] path			 [using /opt/dsee7/lib64/ns-slapd]
\* Process root				 [/opt/dsee7/lib64/ns-slapd]
\* Checking [-c] core			 [using core /var/tmp/cores/core.11716]
\* Databin parameter [-s] checks		 [reset to /var/tmp/storage]
\* Creating Databin directories		 [/var/tmp/storage]
\* Databin used/created is		 [/var/tmp/storage/pkgapp-042210-01]
\* Creating temp area			 [/tmp/pkgapp.11771/]
\* Process binary			 [ns-slapd]
\* ns-slapd binary bit version		 [64]
\* core.11716 bit version		 [64]
\* Checking path [-a] to binary name	 [success, path includes binary name]
\* Checking path [-a] exists		 [success]
\* Locating ns-slapd			 [success]
\* Checking located ns-slapd is 64 bits.	 [success]
\* Binary located			 [/opt/dsee7/lib64/ns-slapd]
\* Adding binary to pkgapp.pldd		 [success]
\* Grabbing libs from /proc		 [success]
\* Grabbing gdb backtrace		 [success]
\* Grabbing [-i] core/gcore		 [success]
\* Grabbing core/gcore cksum		 [success]
\* Javatools [-j] not available		 [skipped]
\* Grabbing /var/log/messages		 [success]
\* Grabbing uname -a			 [success]
\* Grabbing date/time			 [success]
\* Grabbing rpm -qa			 [success]
\* Grabbing /etc/redhat-release		 [success]
\* Grabbing ulimit -a			 [success]
\* Grabbing libs			 [success]
\* Making lib paths app/		 	 [success]
\* Making lib paths libs/		 [success]
\* Linking libraries1 of 71      	 [success]
\* Libraries linked			 [71 ttl]
\* 					
\* Using hostid for naming .tar.gz	 [120ad308]
\* Writing file				 [pkgapp-120ad308-s-6048a-x6250d-vbox1-gmp03.sun.com-100422-182724.tar.gz]
\* 					
\* Done gathering files			
\* Writing dbxrc & opencore.sh files	 [success]
\* Writing coreinfo.sh file		 [success]
\* Writing validate-libs.sh file		 [success]
\* Writing manifest-100422-182724.log	 [success]
\* Writing pkgapp-args-100422-182724	 [success]
\* Creating final tarfile		 [success]
\* Compressing tarfile			 [success]
\* End of runtime logging		
\* Saving history info			 [success]
\* Saving runtime log			 [success]
\* Saving Library list			 [success]
\* Saving Library list README		 [success]
\* Removing [-r] temp area/files		 [removed]
\* 					
\* Operations Complete			
\* Total runtime				 [0 minutes]
\* ------------------------------------------------------------------------------
\* Upload the following file(s) to http://supportfiles.sun.com Cores Directory at
Oracle
					
1) File(s) located in directory /var/tmp/storage/pkgapp-042210-01
					
		[ pkgapp-120ad308-s-6048a-x6250d-vbox1-gmp03.sun.com-100422-182724.tar.gz ]
					


				Thank you.
				Customer Service Delivery
				Global Systems Support


NOTES:
1) You can check for updates to this script here:
	BigAdmin - http://www.sun.com/bigadmin/scripts/indexSjs.html
	Blog - http://blogs.sun.com/Dirtracer

2) GDD information located here:
	Rel Notes - http://docs.sun.com/app/docs/doc/820-0437
	Docs - http://www.sun.com/service/gdd/index.xml
	Video - http://mediacast.sun.com/search?query=gdd
	Video - http://mediacast.sun.com/search?query=dirtracer

3) Please send all Bugs and RFE's to the following address:
	Subject "pkgapp bug/rfe" - pkgapp-feedback_ww@oracle.com

4) See the following reference files.
	/var/tmp/pkgapp-history/runtime-100422-182724.log
	/var/tmp/pkgapp-history/history-100422-182724.log
	/var/tmp/pkgapp-history/library.list-100422-182724
	/var/tmp/pkgapp-history/library.list-100422-182724.README

Enjoy,

Lee

Monday Apr 12, 2010

Pkgapp 3.4 in the hopper!

Releasing soon

It has been a long time since my last update but a lot has transpired in the past few months.

Sun was bought by Oracle...in case you've been hiding in a cave waiting for the latest :)  As you can see I am still employed with Oracle. Although my company has changed, my day to day duties have remained the same: Sun Directory Server support, as well as DirTracer and Pkgapp maintenance.

With version 3.3, I changed how I gathered libraries from cores and processes. As a result, some international users had issues with localized versions of Sun web server.  Because of this bug and a few others, as well as needing a re-branding of the Pkgapp product from Sun to Oracle, I've been working these past few months on Pkgapp 3.4.

The code looks tight and the Pkgapp Reference Guide (pdf) is complete.  Once I get the okay from Oracle Legal on licensing, I can release this new version.  I'm hoping this will happen in the next few days, so stay tuned.

Lee

Wednesday Dec 16, 2009

The New DirTracer (DT7) and Pkgapp (3.4) are here!

Just in time for the holidays!

I am pleased to finally announce new versions of not only Pkgapp, but DirTracer as well.Photo © Lee Trujillo

Updated: March 03, 2011 with the Oracle Support Doc ID's for these.

  • DirTracer7 (DT7) - Oracle Support Doc ID 1291443.1
  • Pkgapp 3.4 - Oracle Support Doc ID 1274584.1

Both products have gone through some serious revisions to make them not only easier on the processes being tracked, but also to make their captured data slimmer. Here is a complete list of the changes to each product.  Please see the included Reference Guide for more detail.

DT7 (DirTracer 7.0)

Improvements

  1. Directory Server 7 compatible. DT7 can work with directory server versions 5.1, 5.2.x, 6.x as well as 7.0.
  2. Data capture size reduced by ½. Disk usage with a Basic Capture in DT6.0.7 was 4169 kb.  The same capture size is now 2191 kb.
  3. Internal tar files removed.
  4. Safer when used against ds processes...removed pflags, pldd, pmap, pfiles as they can create locking issues with the ds process on some os versions.  Currently the only use of these proctools are against gcores or cores if provided.
  5. Checks available databin space when a gcore is requested.  Will disable GRAB_GCORE if space is needed.  Helps customers from inadvertently filling disk space to 100% with a gcore.
  6. New validate data script provided.  Checks the manifest against the captured data.
  7. Now uses Pkgapp 3.3.
  8. Pkgapp captures are now included with final tar.  Customers no longer have to upload a second file for data analysis. 
  9. DT7 now has an epoch check feature alerting the user if the DT7 version being used is more than 120 days old.   A simple alert to have the customer check Big Admin for a newer version.
  10. Renamed the databin from a numbered path to include the name dirtracer.
  11. Changed the name of some config parameters.
  12. Changed how the high cpu tracking is seen.
  13. New data captured.
  14. Added an overall timer which is displayed at the end of the DT7 run.
  15. Renamed the README.dirtracer to README.changelog
  16. Created a text based README.FIRST.  This is a Quickstart Guide if you don't have time to read the full Reference Guide.
  17. Renamed the command file from "dirtracer" to dt7.  Also renamed the config file from "dirtracer.config" to "dt.conf" to make commands easier to type.

Fixes

  1. Fixed the nsds50ruv search to be a scope sub search as opposed to one.
  2. Fixed (enhanced) the Control C feature so users can Control C once to exit the Main Loop; this advances DT7 to the Post Loop data gathering phase.  Control C a second time allows the user to exit even the Post Loop.
  3. Fixed a minor issue with how the values were store in the dt.conf.last file for the GATHER_N_XXXXX_LOGS parameters.
  4. Fixed the paths to check “carpet” type for Directory Server 7.
  5. Fixed an issue in Configurator which wouldn't return the stty echo if you used Control C.
  6. Fixed an issue where DT7 couldn't derive the correct bit mode (32 or 64bits) when SERVER_DOWN=1.  Both DT7 and Configurator ask the user for this information if it is not available.
  7. Fixed an issue where Pkgapp isn't gathered if DT7 is set to capture 2 gcores and is exited before the second gcore is gathered.

Pkgapp 3.3

Improvements

  1. Added a new parameter -l.  This takes a known good pldd text file to be used in place of either a pldd against the pid or the core.  This allows admins to use a previously generated pldd text file, instead of attaching to the live pid or a possibly bad core file.
  2. Changed the overall usage. Both can be used on Solaris but the -p <pid> is now required on Linux.  This is due to a limitation of the ldd command and lack of pldd on Redhat and SuSE linux.  ldd misses most libs that a real process runs with. -p is now used for the <pid> and -c is now sed for the <core>.
  3. Added an Epoch (date of the first run) check to alert users when their pkgapp version is 120 days old.   Just a simple reminder to check if a newer version exists.
  4. Uses the /proc file system to quickly gather library names instead of using pldd etc (when a pid is used).  Solaris - uses pldd on the core if available. Linux - requires a pid to use the /proc fs as ldd is not accurate.
  5. Created a validate-libs.sh file which can be used to validate all the libraries gathered (as seen in the manifest) in the capture paths.
  6. Now copies the library-list and a README to the /var/tmp/pkgapp-history/
  7. Updated the post-run NOTES: section to include new links and information.

Fixes

  1. Fixed bug 6881865: 6881865 Pkgapp 3.2 fails to exec gb when pkgapp is run from outside its install path
  2. Fixed bug 6882174: 6882174 Pkgapp 3.2 fails to gather jstack/jinfo

What is next?

Due to time constraints (my regular job, etc.) I had to cut development of DT7 off as DS7 was released.  I had a bigger list of the enhancements I wanted to add but ran out of time.  Here are a few of them I want to explore for DT7.1.

  1. Anonymizer. Data anonymizer for sensitive customer data.
  2. ReptopCk inclusion
  3. Search Debug Feature. Set debug: 4 on, play SRCH/MOD etc, set debug off. Great for debugging problem searches/mods.
  4. Add precise timing for all DT executions. Would get the date before each command ever done as well as adding it to a log...so you can see each command executed in a linear list.
  5. For DS6 and 7 do an ls -laR on the CARPETPATH and INSTANCEPATH
  6. Rewrite the config file processor
  7. Add a new command processor. This would allow for more flexibility to run a specific search or use a Dtrace script.

More on these later.

I hope the new versions help you out, and remember you can always send me feedback for new improvements or for issues encountered.

 Happy Holidays and Regards,

 Lee

Friday Nov 13, 2009

Pkgapp 3.3 Soon to be released!

Wow I've been busy but all for the good.  Photo © Lee Trujillo

I've been in development with SLX, internal Engineer Videos for DirTracer as well as DT7 (DirTracer 7) and now Pkgapp 3.3 development.

I've fixed a couple bugs, added a few enhancements and...changed its usage a bit... 

More on these changes later but here are some example run's in the mean time.

Pkgapp 3.3 Beta 5 Solaris

\* ----------------------------------------------------------------------------------
\* Sun Microsystems TSC pkgapp 3.3b5 Solaris                             [11/13/2009]
\* ----------------------------------------------------------------------------------
\* Checking install epoch/delta          [0 seconds]
\* Checking pkgapp path                  [using /opt/pkgapp]
\* Checking usage history                [not recently run]
\* OS release [Solaris]                  [5.10]
\* Platform/Type                         [SUNW,Sun-Blade-T6300 sparcv9]
\* Checking [-p] pid                     [using pid 20578]
\* Checking [-a] path                    [reset to /usr/lib]
\* Process root                          [/usr/lib]
\* Checking [-c] core                    [using core /var/tmp/cores/core.20578]
\* Databin parameter [-s] checks         [reset to /var/tmp/storage]
\* Creating Databin directories          [/var/tmp/storage]
\* Databin used/created is               [/var/tmp/storage/pkgapp-111309-01]
\* Creating temp area                    [/tmp/pkgapp.27404/]
\* Checking corefile for a valid pldd    [pldd is good with 36 elements]
\* Checking if corefile is truncated     [CORE core.20578 is GOOD. Core contains 106 segments ]
\* Continuing...                         
\* Process binary                        [sendmail]
\* sendmail binary bit version           [32]
\* Checking path [-a] to binary name     [success, path != binary name]
\* Checking path [-a] is a directory     [success]
\* Locating sendmail                     [success]
\* Checking located sendmail is 32 bit   [success]
\* Binary located                        [/usr/lib/sendmail]
\* Adding binary to pkgapp.pldd          [success]
\* Grabbing libs from /proc              [success]
\* Grabbing pstack from core             [success]
\* Grabbing pmap from core               [success]
\* Grabbing pcred from core              [success]
\* Grabbing pflags from core             [success]
\* Grabbing pargs from core              [success]
\* Grabbing [-i] core/gcore              [success]
\* Grabbing core/gcore cksum             [success]
\* Javatools [-j] not set                [skipped]
\* Grabbing /var/adm/messages            [success]
\* Grabbing uname -a                     [success]
\* Grabbing date/time                    [success]
\* Grabbing showrev -p                   [success]
\* Grabbing pkginfo -l                   [success]
\* Grabbing /etc/release                 [success]
\* Grabbing coreadm                      [success]
\* Grabbing ulimit                       [success]
\* Grabbing libs                         [success]
\* Making lib paths app/                 [success]
\* Making lib paths libs/                [success]
\* Linking libraries                     [success]
\* Libraries linked                      [80 ttl]
\*                                       
\* Using hostid for naming .tar.gz       [84aaaac8]
\* Writing file                          [pkgapp-84aaaac8-s-6000a-t6300i-zone8-sca11-091113-125258.tar.gz]
\*                                       
\* Done gathering files                  
\* Writing dbxrc & opencore.sh files     [success]
\* Writing validate-libs.sh file         [success]
\* Writing manifest-091113-125258.log    [success]
\* Writing pkgapp-args-091113-125258     [success]
\* Creating final tarfile                [success]
\* Compressing tarfile                   [success]
\* End of runtime logging                
\* Saving history info                   [success]
\* Saving runtime log                    [success]
\* Saving Library list                   [success]
\* Saving Library list README            [success]
\* Removing [-r] temp area/files         [left alone]
\*                                       
\* Operations Complete                   
\* Total runtime                         [0 minutes]
----------------------------------------------------------------------------------
Upload the following file(s) to your supportfiles.sun.com Cores Directory at Sun
                                        
1) File(s) located in directory /var/tmp/storage/pkgapp-111309-01
                                        
                [ pkgapp-84aaaac8-s-6000a-t6300i-zone8-sca11-091113-125258.tar.gz ]
                                        


                                Thank you.
                                Sun Software Technology Service Center (STSC)


NOTES:
1) You can check for updates to this script here:
        BigAdmin - http://www.sun.com/bigadmin/scripts/indexSjs.html
        Blog - http://blogs.sun.com/Dirtracer
        Sun Learning Exchange Video's - https://slx.sun.com/search/node/dirtracer

2) Release Notes and Guides located here:
        Docs - http://docs.sun.com/app/docs/doc/820-0437

3) GDD information located here:
        Docs - http://www.sun.com/service/gdd/index.xml

4) Please send all Bugs and RFE\\'s to the following address:
        Subject "pkgapp bug/rfe" - gdd-issue-tracker@sun.com

5) Please send all other questions etc to:
        Subject "pkgapp feedback" - gdd-feedback@sun.com

6) See the following reference files.
        /var/tmp/pkgapp-history/runtime-091113-125258.log
        /var/tmp/pkgapp-history/history-091113-125258.log
        /var/tmp/pkgapp-history/library.list-091113-125258
        /var/tmp/pkgapp-history/library.list-091113-125258.README


Pkgapp 3.3 Beta 5 Redhat (same as SuSe)

\* ----------------------------------------------------------------------------------
\* Sun Microsystems TSC pkgapp 3.3.b5 Redhat				 [11/13/2009]
\* ----------------------------------------------------------------------------------
\* Checking install epoch/delta		 [0 seconds]
\* Checking pkgapp path			 [using /opt/SUNWpkgapp]
\* Checking usage history		 [not recently run]
\* OS Release [Redhat]			 [2.6.9-5.EL]
\* Platform/Type				 [x86_64 x86_64]
\* Checking [-p] pid			 [using pid 686]
\* Checking [-a] path			 [reset to /opt/sun/dir/ds6/lib]
\* Process root				 [/opt/sun/dir/ds6/lib]
\* Checking [-c] core			 [using core /var/cores/core.686]
\* Databin parameter [-s] checks		 [reset to /var/tmp/storage]
\* Creating Databin directories		 [/var/tmp/storage]
\* Databin used/created is		 [/var/tmp/storage/pkgapp-111309-01]
\* Creating temp area			 [/tmp/pkgapp.11773/]
\* Process binary			 [ns-slapd]
\* ns-slapd binary bit version		 [32]
\* Checking path [-a] to binary name	 [success, path != binary name]
\* Checking path [-a] is a directory	 [success]
\* Locating ns-slapd			 [success]
\* Checking located ns-slapd is 32 bit	 [success]
\* Binary located			 [/opt/sun/dir/ds6/lib/ns-slapd]
\* Adding binary to pkgapp.pldd		 [success]
\* Grabbing libs from /proc		 [success]
\* Grabbing [-i] core/gcore		 [success]
\* Grabbing core/gcore cksum		 [success]
\* Javatools [-j] not set		 [skipped]
\* Grabbing /var/log/messages		 [success]
\* Grabbing uname -a			 [success]
\* Grabbing date/time			 [success]
\* Grabbing rpm -qa			 [success]
\* Grabbing /etc/redhat-release		 [success]
\* Grabbing ulimit			 [success]
\* Grabbing libs				 [success]
\* Making lib paths app/			 [success]
\* Making lib paths libs/		 [success]
\* Linking libraries8 of 68      	 [success]
\* Libraries linked			 [68 ttl]
\* 					
\* Using hostid for naming .tar.gz	 [120ad308]
\* Writing file				 [pkgapp-120ad308-s-6048a-x6250d-vbox1-gmp03.uk.sun.com-091113-205150.tar.gz]
\* 					
\* Done gathering files			
\* Writing dbxrc & opencore.sh files	 [success]
\* Writing coreinfo.sh file		 [success]
\* Writing validate-libs.sh file		 [success]
\* Writing manifest-091113-205150.log	 [success]
\* Writing pkgapp-args-091113-205150	 [success]
\* Creating final tarfile		 [success]
\* Compressing tarfile			 [success]
\* End of runtime logging		
\* Saving history info			 [success]
\* Saving runtime log			 [success]
\* Saving Library list			 [success]
\* Saving Library list README		 [success]
\* Removing [-r] temp area/files		 [left alone]
\* 					
\* Operations Complete			
\* Total runtime				 [0 minutes]
\* ----------------------------------------------------------------------------------
Upload the following file(s) to your supportfiles.sun.com Cores Directory at Sun
					
1) File(s) located in directory /var/tmp/storage/pkgapp-111309-01
					
		[ pkgapp-120ad308-s-6048a-x6250d-vbox1-gmp03.uk.sun.com-091113-205150.tar.gz ]
					


				Thank you.
				Sun Software Technology Service Center (STSC)


NOTES:
1) You can check for updates to this script here:
	BigAdmin - http://www.sun.com/bigadmin/scripts/indexSjs.html
	Blog - http://blogs.sun.com/Dirtracer
	Sun Learning Exchange Video's - https://slx.sun.com/search/node/dirtracer

2) Release Notes and Guides located here:
	Docs - http://docs.sun.com/app/docs/doc/820-0437

3) GDD information located here:
	Docs - http://www.sun.com/service/gdd/index.xml
	Video - https://slx.sun.com/search/node/GDD
	Video - https://slx.sun.com/search/node/DIRTRACER

4) Please send all Bugs and RFE\\'s to the following address:
	Subject "pkgapp bug/rfe" - gdd-issue-tracker@sun.com

5) Please send all other questions etc to:
	Subject "pkgapp feedback" - gdd-feedback@sun.com

6) See the following reference files.
	/var/tmp/pkgapp-history/runtime-091113-205150.log
	/var/tmp/pkgapp-history/history-091113-205150.log
	/var/tmp/pkgapp-history/library.list-091113-205150
	/var/tmp/pkgapp-history/library.list-091113-205150.README


Lee

Thursday Jul 30, 2009

DirTracer, Pkgapp and SLX

Alo all,

I received good news from the Big Admin admins today, the new DirTracer and Pkgapp versions are online!

The following versions are now available here on Big Admin. For information on the latest changes in this update, please see my previous blog - DT 6.0.7 Coming soon!

DirTracer 6.0.7 Photo © Lee Trujillo

Pkgapp 3.2

I have also released the SLX 02 DirTracer Quickstart on SLX (Sun Learning Exchange) - "This video is a Quickstart guide to getting DirTracer6 running in a short amount of time using Configurator 6. I also step through a real example of how DirTracer runs using the config file created by Configurator."

Finally, I am in production of the latest DirTracer SLX video "Server Impact".  Using Slamd, Cacti and DirTracer I plan on showing exactly what impact a Directory Server will see from DirTracer runs based on different run/problem types.  More on this later.

Cheers!

Lee

Thursday Jul 23, 2009

Multiple DirTracer iterations - What not to do.

Hey all,

Quick tech tip here. 

We recently came across a situation where a Customer ran DirTracer 3 separate times in the same second; one for each of the three instances on a single system.  Each config file had the correct configuration for each instance but all three were used at the same time.

Example: Using a wrapper script to launch all 3 in the same second.

#!/bin/sh

nohup ./dirtracer -f ./dirtracer.config.1 &

nohup ./dirtracer -f ./dirtracer.config.2 &

nohup ./dirtracer -f ./dirtracer.config.3 &

As a result, DirTracer is launched 3 times in the same second and each see's that there is no existing sub-databin inside /var and only 1 sub-databin is created; i.e. /var/072309-01. 

<snip>

\* Using config file                     [./dirtracer.config.1]
\* Using config file                     [./dirtracer.config.2]
\* Using config file                     [./dirtracer.config.3]
\* Dirtracer mainloop will run for     \* Dirtracer mainloop will run for      \* Dirtracer mainloop will run for   

[100 sec. Between 1 & 2 min.]
[100 sec. Between 1 & 2 min.]
[100 sec. Between 1 & 2 min.]
\* Databin parameter check 1             \* Databin parameter check 1             \* Databin parameter check 1         

[success]
[success]
[success]
\* Databin parameter check 2             \* Databin parameter check 2             \* Databin parameter check 2         

[success]
[success]
[success]
\* Databin Found                         [/var]
\* Databin Found                         [/var]
\* Databin Found                         [/var]
\* Databin used is                       \* Databin used is                       \* Databin used is                   

[/var/072309-01]
[/var/072309-01]

[/var/072309-01]

<snip>

To ensure DirTracer runs properly and uses a separate sub databin (as intended) the following workaround can be used.

#!/bin/sh

nohup ./dirtracer -f ./dirtracer.config &

sleep 1

nohup ./dirtracer -f ./dirtracer.config &

sleep 1

nohup ./dirtracer -f ./dirtracer.config &

By giving DirTracer a 1 second delay between executions gives it enough time to notice there is an exiting sub-databin in the intended path and increment the sub-databin number.

Example:

myzone root[/var/tmp/data]# ls -l
drwxr-xr-x  11 root     root          30 Jul 23 14:48 072309-01
drwxr-xr-x  11 root     root          30 Jul 23 14:48 072309-02
drwxr-xr-x  11 root     root          30 Jul 23 14:48 072309-03

Hope this helps...

Cheers,

Lee


Tuesday Jul 21, 2009

DT 6.0.7 and Pkgapp 3.2 (Linux) Available!

Hi all,

Good news is always welcome :D

I have finalized testing of not only DirTracer (DT) 6.0.7 but have also finished coding and testing of Pkgapp 3.2 for both Redhat and SuSE Linux. Photo © Lee Trujillo

See my previous blog DT 6.0.7 Coming soon! or the README.dirtracer for details on what has changed.

Available DirTracer and Pkgapp versions:

  • Solaris Sparc 8/9/10
  • Solaris x86 9/10
  • SuSE Linux 9 U4/10 U1
  • Redhat Linux 3/4/5

Remember: DirTracer comes with Pkgapp (see the dirtracertools path) but for non Sun Directory Server customers, Pkgapp 3.2 can be downloaded separately.

Currently these versions are only available internally until BigAdmin can update their copies.  Until that happens, contact your Sun Customer Service Representative and have them send you the latest versions.

Enjoy!

Lee

Thursday Jul 16, 2009

DT 6.0.7 Coming soon!

Hi all,

Well, I hopped on the fix for the last updates bug so fast I decided to just release a new version of DirTracer (DT 6.0.7).

 Here is a list of the current change's...

DirTracer:
  1. Fixed bug 6861010 DirTracer 6.0.6 incorrectly identifies the DS as 64bit if running in as 32bit on AMD64 (Solaris Only).
  2. Removed post loop (non needed) pflag, pmap, pfiles gathering (Solaris Only).
  3. Replaced the older Pkg_app 2.7 with the new Pkgapp 3.2 (Solaris Sparc, Solaris x86 and Linux only at this time).
  4. Pkgapp lib tar file now included with the final DirTracer tar.gz
  5. Also added a check to skip grabbing /etc/system on a Solaris 10 Zone when not available.
  6. Added links to my Sun Blog and Sun Learning Exchange Video's for DirTracer to the displayed links when DT completes.

Configurator
  1. Fixed an issue when long path names are used for the server root.  Configuator would fail to derive the correct server root to locate pwdhash and would fail.  See bug 6511933 Dirtracer (GDD): greps on dse.ldif parameters fail when ldif wrapping is used.

As the changes only affected a couple of bugs I will not be reworking the pdf doc giude to reflect the new version information.  This also allowes me to quickly generae and release new fixes for current issues.

I have made the above changes to the Solaris Sparc, Solaris x86, Linux and HP-UX versions.  I have not coded a new version of Pkgapp 3.2 yet for HP-UX so I will release this version a bit later.  I will update soon when I have released DirTracer internally for download.  Once I do and you wish to get a copy, please contact your Sun Frontline Engineer.

Regards,

Lee

Wednesday Jul 15, 2009

Alert: DirTracer 6.0.6 incorrectly identifies the DS as 64bit if running in as 32bit on AMD64

Hi all,

Quick update on DirTracer.  I knew about this issue for a couple of months but haven't had time to deal with it.  I have created a bug (see below) and a fix.  I will try to get this out soon as DirTracer 6.0.7.  The fix is already in DT7 (DirTracer 7)

CR 6861010 DirTracer 6.0.6 incorrectly identifies the DS as 64bit if running in as 32bit on AMD64

Cheers all

Lee

Tuesday Jul 07, 2009

DirTracer Overview on SLX (Sun Learning Exchange)

Great news all,

I have released my first vidcast on Sun's Learning Exchange (SLX) site! Dream Sequence

SLX 01 - DirTracer Overview (runtime 7min 32sec)

This is the first in what I hope to be a long series discussing DirTracer, Configurator and Pkgapp.  If you are unfamiliar with DirTracer, this Overview is an easy and fast way to get acquainted with Sun's #1 Directory support tool.

On July 24th I will be releasing the second vidcasts titled "DirTracer Quickstart".  This will walk you through getting DirTracer up and running in the shortest amount of time using Configurator.

I am currently in production on the third vidcast titled "Server Impact".  This is a hot topic when it comes to running DirTracer on production systems.

That's all for now folks,

Enjoy

Lee

Friday Jun 26, 2009

Running DirTracer in the Background

Hello all,

I was approached today by a Partner about running DirTracer in the background.  I believe I have been asked about this once before.

Currently I have not tested nor can DirTracer be run in the background.  I just tried this in fact and saw nothing in the databin path, nor in the /tmp area for the runtime log.  I have found a way however.

If I background or no hup it I get no output -> (nohup ./dirtracer -f ./dirtracer.config &).

But if I create a stub script that launches DirTracer with nohup and & and do the same with the stub...it works.

myzone root[/opt/dirtracer]#cat backgrounder
#!/bin/sh

nohup ./dirtracer -f ./dirtracer.config &
myzone root[/opt/dirtracer]#nohup ./backgrounder &
[1] 9159
myzone root[/opt/dirtracer]#Sending output to nohup.out

[1]    Done                          ./backgrounder

In most circumstances, (99% of the time) you should be running DirTracer in the foreground because the runtime is generally short.  I can only see backgrounding the process when you use the MEM_LEAK_TRACKING option or if you have such a long DirTracerrun that you will have to close down your terminals and leave.

Thanks to Brian for this great question.


Lee

Tuesday Jun 02, 2009

Pkgapp 3.2 for Solaris available on BigAdmin

 

Hey all,

Quick update to say Pkgapp 3.2 is now available externally on the Big Admin scripts site.  You can download it here.

Currently I am working on Pkgapp 3.2 ports to Linux.

Big Admin Scripts: Sun Java System Category

Big Admin MOTD

Cheers!

Lee

Tuesday May 26, 2009

DirTracer now DS7 compatible!

 Yes it's true!  DirTracer now works with DS7!It's the Tubes

I was fortunate to get access to DS7 during the Directory Masters event in New Jersey.  While there I was able to quickly install DS7 and migrate an old dead DS6.2 instance.  I ran DT 6.0.6 against this new instance and was ecstatic to see it ran with 0 errors.  In reality DS Development had moved two commands but DT had used its defaults to guess things like install type (zip/native) and where the dsrk's ldapsearch and ldapmodify were located.

Today I decided to revisit these two items and found it was easy as pie to change; only took a few minutes.  I had to copy/paste and change approximately seven lines of code (mostly path based) and that was it. 

To mirror the DT version I used when DS6 was released, I will now re-version this as DirTracer 7.0.0 or DT7.  While DT7 works perfectly with DS7,  I'm not saying DT7 won't change.  In fact I'm planning many new features for DT7 for its first

--------------------------------------------------------------------------------
Sun Microsystems dirtracer 7.0.0 Solaris Sparc                        05/26/2009
--------------------------------------------------------------------------------
\* Checking Config File                  [success 1211182548 != 3006111464]
\* Using config file                     [./dirtracer.config]
\* OS Type and Version                   [SunOS 10 sparc]
\* Dirtracer mainloop will run for       [1 sec.]
\* Databin parameter check 1             [success]
\* Databin parameter check 2             [success]
\* Databin Found                         [/var/tmp/dtcapture]
\* Databin used is                       [/var/tmp/dtcapture/052609-02]
\* Relocating runtime log                [success]
\* Case Number                           [NOT AVAILABLE]
\* User is root. Continuing...          
\* PID Check     (18925)                 [success]
\* Process State                         [not hung]
\* High CPU tracking                     [off]
\* Mem Leak tracking                     [off]
\* Config Only Capture                   [on]
\* Pstack Force (-F)                     [off]
\* Instance Name is                      [/opt/dsee62/ldapinst3]
\* Using host name                       [s-6000a-t6300i-zone8-sca11]
\* Using host id                         [84aaaac8]
\* Ldap Tools Path                       [/opt/dsee7/dsrk/bin]
\* DS Version                            [7.0 - 64 bit - zip install]
\* Backends Found                        [sun]
\* Transaction log location              [/opt/dsee62/ldapinst3/db]
\*   Solaris tar found                   [/usr/sbin/tar]
\*   gzip found                          [/bin/gzip]
\*   elfdump found                       [/usr/ccs/bin/elfdump]
\*   pstack found                        [/bin/pstack]
\*   prstat found                        [/bin/prstat]
\*   netstat found                       [/bin/netstat]
\*   iostat found                        [/bin/iostat]
\*   vmstat found                        [/bin/vmstat]
\*   mpstat found                        [/bin/mpstat]
\*   ldapmodify found                    [/opt/dsee7/dsrk/bin/ldapmodify]
\*   ldapsearch found                    [/opt/dsee7/dsrk/bin/ldapsearch]
\*   pms.sh found                        [/opt/dirtracer/dirtracertools/pms.sh]
\* Access/Error Logging                  [left as is]
\* Audit Logging                         [left as is]
\* Iostat available.     Executing...    [success]
\* Vmstat available.     Executing...    [success]
\* Mpstat available.     Executing...    [success]
\* pms.sh (pmonitor) executed            [success]
\*   pms.sh interval(1) x checks(1)      [pms.sh run time (1 sec.)]
\*                                      
\* Entering Main Performance Gathering Loop
\*                                      
\* Loop 0 - 090526-132724                [pstack/prstat override on]
  togo[1sec]-timer[1]
\*
\*                                      
\* Exiting Main Performance Gathering Loop
\*                                      
\* Done gathering stats                  [pms.sh iostats vmstats etc]
\* Access/Error Logging                  [left as is]
\* Audit Logging                         [left as is]
\* Logging level                         [left as is]
\*                                      
\* Gathering needed customer defined configuration
\*   ls -laR of slapd Instance           [success]
\*   nsslapd-db-home-directory           [success]
\*   nsslapd-db-logdirectory             [success]
\*   nsslapd-directory db                [success]
\*   nsslapd-directory sun               [success]
\*                                      
\* Gathering Other Environment Information
\*   ns-slapd version                    [success]
\*   nsds50ruv dc=sun,dc=com             [success]
\*   cn=config search                    [success]
\*   rootdse search                      [success]
\*   idsktune                            [not found]
\*   dirtracer7.0.0                      [success]
\*   /etc/release                        [success]
\*   /etc/system                         cp: cannot access /etc/system
[failed]
\*   /var/adm/messages                   [success]
\*   uname -a                            [success]
\*   uptime -a                           [success]
\*   date/time                           [success]
\*   ifconfig -a                         [success]
\*   mount -p                            [success]
\*   ps -aef                             [success]
\*   pflags                              [failed]
\*   pldd                                [failed]
\*   pmap                                [failed]
\*   pfiles                              [failed]
\*   prtconf -v                          prtconf: devinfo facility not available
[success]
\*   showrev -p                          [success]
\*   swap -s                             [success]
\*   pkginfo -l                          [success]
\*   df -k                               [success]
\*   logs - access: gathering 1 files   
\*       access                          [success]
\*       access rotation                 [success]
\*   logs - error: gathering 1 files    
\*       errors                          [success]
\*       error rotation                  [success]
\*   logs - audit: gathering 1 files    
\*       audit                           [success]
\*       audit rotation                  [success]
\*   config dse.ldif's                   [success]
\*   schema                              [success]
\*   shared mem files                    [skipped]
\*                                      
\* [ALERT 1] Shared Mem files were not included in the dirtracer tar file and were NOT copied/saved to the databin.
\*                                      
\*   ds lib files cksum                  [skipped]
\*   ds lib files elfdump                [skipped]
\* Writing used dirtracer.config.last    [success]
\* Copying dirtracer.config.last         [success]
\* Writing manifest-090526-132724.log    [success]
\*                                      
\* Packaging files                      
\*   Preparing files - pstack            [0 files]
\*   Preparing files - prstat            [0 files]
\*   Preparing files - netstat           [success]
\*   Preparing files - ds-config-data    [success]
\*   Preparing files - ds-logs           [success]
\*   Preparing files - ds-mmap-files     [success]
\*   Preparing files - ds-srch-monitor   [success]
\*   Preparing files - ds-srch-cache     [success]
\*   Preparing files - ds-txnlogs-data   [success]
\*   Preparing files - os-info           [success]
\*   Preparing files - gcore             [0 files]
\*   Preparing files - crash             [0 files]
\*   Preparing Main file                 [success]
\* Leaving my temp files intact         

Operations Complete
--------------------------------------------------------------------------------
1) Dirtracer capture data located in directory [ /var/tmp/dtcapture/052609-02 ]

Upload "only" this file to your supportfiles.sun.com cores directory at Sun

        [ dirtracer-84aaaac8-s-6000a-t6300i-zone8-sca11-090526-132724.tar.gz ]


                                Thank you.
                                Sun Software Technology Service Center (STSC)


NOTES:
1) You can check for updates to this script here:
        BigAdmin - http://www.sun.com/bigadmin/scripts/indexSjs.html

2) Release Notes and Guides located here:
        Docs - http://docs.sun.com/app/docs/doc/820-0437

3) GDD information located here:
        Docs - http://www.sun.com/service/gdd/index.xml

4) Please send all Bugs and RFE's to the following address:
        Subject "dirtracer bug/rfe" - gdd-issue-tracker@sun.com

5) Please send all other questions etc to:
        Subject "dirtracer feedback" - gdd-feedback@sun.com

--------------------------------------------------------------------------------

ALERTS ENCOUNTERED: See /var/tmp/dtcapture/052609-02/ALERTS-090526-132718.log


1) Shared Mem files were not included in the dirtracer tar file and were NOT copied/saved to the databin.

As I mentioned two blogs ago I was preparing to start releasing DT Video (pocasts?) on the Sun Learning Exchange (SLX).  The first draft is complete for my first video titled "DirTracer Overview".  I will start production of the video later today or tomorrow and hope to release it to SLX quite soon.

Cheers!

Lee

Thursday May 21, 2009

Pkgapp 3.2 for Solaris released!

Hi all!

Pkgapp 3.2 is available internally...yeah!

Artistic Bridge

I finally had time to update the Pkgapp documentation and retest it on various application cores/gcores and it is now ready for prime time.  I have also submitted it to BigAdmin and it should be available in the next few days.  If you need it now, please contact your favorite Sun Support representative and request he send it from the internal download site.  I will also update "this" post once Big Admin has uploaded the new version.

Pkgapp 3.2 has 2 big new features...

  1. It includes adm64 libraries that were previously not included making debugging easier on that platform for java issues.
  2. It includes a new core truncation feature developed by a member of the Directory Server Sustaining team.  The great news is, the feature not only works on Directory Server cores but any other core/gcore you may be dealing with!

If the core/gcore is good, the user is given the following message:

\* Checking if corefile is truncated     [CORE gcore.cshttpd.28734 GOOD Core contains 293 segments ]

If the core is truncated, the user is alerted and required to answer yes/no as to whether they understand the data may be totally unusable.

\* Checking if corefile is truncated     [WARNING! core gcore.cshttpd.28734.truncated is truncated (40396864d bytes instead of 40397397d bytes)]
\*                                     
\* ALERT! The core was found to be truncated, it therefore may not be usable
\* pkgapp will continue to gather data and you can still upload the result to Sun
\*                                     
\* Do you understand the core may not be usable? [yes|no] yes
                                        [Answer was -> yes]
\* Continuing...  

If you use/have an RSS Reader such as NetNewsWire or  Google Reader you can see the latest updates to Big Admin using the following links.

Big Admin Scripts: Sun Java System Category

Big Admin MOTD

Next up...I plan to finalize and release DTR and begin production of my first SLX video for Dirtracer users!

Take care,

Lee

Thursday May 07, 2009

DTR (Dirtracer Reporter) is coming and other news!

Hi all,

Thought I would drop in a quick blog and let you know what I've been up to.

Directory Masters Event in Somerset New Jersey Lee Trujillo talking DirTracer at the Directory Master event in Somerset NJ

As mentioned in my last blog I was asked to present Dirtracer at the DM event.  The event, hosted by Michael Melore, Wajih Ahmed and Ludovic Poitou was great!  I was able to meet people I've only talked to on the phone or emailed over the past 5 years.  I was able to meet with Sun Partners and Sun Development and Support Engineers...

I presented an overview of Dirtracer and its components as well as my latest project, DTR (Dirtracer Reporter); more on this below.  I have to admit I was very nervous.  In past lives I was a pre-sales engineer and presentations like this were easy.  Needless to say I haven't presented to a large group since joining Sun; just not something I do in my Support role.

In the end I forgot about being nervous and soon was on a roll.

It was great meting you all, I had a blast.  See Ludo's Sketches blog for more info on this event.

Dirtracer Reporter (DTR)

Over the past couple of months I have been ratifying a few data reporting tools I and others have created into one nice package called DTR.  Dirtracer Reporter is a bundle of existing Directory Server data reporting scripts rolled into a single usable interface.  DTR shines when run against Dirtracer captured data, it however can run against basic Directory Server data such as dse.ldif and log files.

I am in the final test phase and making sure a slight glitch here and there are fixed and hope to release to Big Admin in the next coming weeks.  More to come on DTR soon.

Sun Learning Exchange (SLX)

One of my other sub projects which will kick off soon is the creation of SLX videos for Dirtracer.  These will be video podcast in nature and will show how Dirtracer is used for various problem types.

 Here is a list of topics I hope to highlight.

  • Dirtracer overview
  • Process Hangs
  • High CPU
  • Replication
  • Crashes
  • Memory Leaks
  • Server Down issues
  • Basic Captures
  • Configuration only captures
  • Configurator for Dirtracer usage

These videos will supplement my blog content as well as giving you audio and visual queues of how Dirtracer looks when running for a specific problem type.

Until my next blog...ciao!

Lee

About

A Tech Blog about the Sun Java Systems Dirtracer Toolkit. Dirtracer and this blog written and maintained by Lee Trujillo an Oracle Senior Principal Support Engineer.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today