Montag Jun 07, 2010

prstat and microstate accounting

You never stop learning.  As a reply to my last blog entry, it was pointed out to me that with Solaris 10, microstate accounting is always enabled, and prstat supports this with the option "-m".  This option removes the moving average lags from the values displayed, and is much more accurate.  I wanted to know more about the background.  Eric Schrock was kind enough to provide it on his blog.  Here's a short summary.

The legacy output of prstat (and some of the other monitoring commands) represents moving averages based on regular samples.  With higher CPU frequencies, it is more and more likely that some scheduling events will be missed completely by these samples.  This makes the reports more and more unreliable.  Microstate accounting collects event statistics for every event, when the event happens.  Thanks to some implementation tricks introduced with Solaris 10, this is now efficient enough to be turned on all the time.  If you use this more precise data with prstat, a CPU hog will show up immediately, showing 100% CPU on all threads involved.  In this way, you're much more precise, and you need'nt convert from the number of CPUs in the system to the corresponding %-age as in the example in my blog entry.  A singlethreaded process will be visible instantly. This is easier do to, easier to understand, less error prone and more exact.

I've also updated the presentation to represent this.

Thanks for the hint - you know who you are!  It's from things like this that I notice that I've been using prstat and the likes (successfully) for too long .  It's just like Eric mentioned in his blog: This great feature slipped past me, with all the more prominent stuff like containers, zfs, smf etc.  Thanks again!

Dienstag Jun 01, 2010

Is my application a good fit for CMT ?

CMT-CPUs have been around for quite a while now.  That they were developed for parallel, throughput oriented loads is a well known fact.  However, finding out if a specific application is a good fit for these CPUs seems to remain a challenge, and is one of my personal FAQs.  I'll try to write down a few helpful hints that might help you answer this question yourself.

The first and most important criterion for suitability is always the service time of your application.  If this is sufficient, then the application is OK on CMT. If it is not, and the reason is actually the CPU and not some other high-latency component (like a remote database), you will need to test on other CPU architectures.

It is of course desirable that the application is multi-threaded,  and the individual threads actually perform useful work in parallel.  Only then will the application be able to make use of all the CPU resources available.  It is important to understand that (high) server utilization can never be a criterion for good application performance.  Performance always needs to be measured using application metrics like throughput (transactions per minute) or service time, or both.  Server or CPU utilization only indicate whether the application actually makes good use of the available resources.

Use threadbar to establish if an application is multi threaded and if these threads are actually doing something.  Or, if you're more the commandline type, use prstat -L.  There, the individual threads of your processes are listed, sorted by CPU usage.  What you want to see here is many threads with CPU usage higher than 0%.  What's also important is to check if any thread is limited by the single thread performance of the CPU or strand.  A first estimate of this can also be obtained from prstat.  The column "%CPU" relates to all active CPU (strands) in your system.  For example, a T5120 would show you 64 CPUs.  One strand is therefore equivalent to 1/64 or about 1.5% of the overall system.  What you want is that no one thread of your application consumes 1.5% CPU permanently.  This would be a hint that this thread is limited in performance by the single thread performance of the CPU strand.

I've put these hints and some examples into a small presentation.  Perhaps it's helpful for some of you.


Neuigkeiten, Tipps und Wissenswertes rund um SPARC, CMT, Performance und ihre Analyse sowie Erfahrungen mit Solaris auf dem Server und dem Laptop.

This is a bilingual blog (most of the time). Please select your prefered language:
The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.


« June 2016