What can you expect from Package Manger for 2009.06
By migi on Mar 26, 2009
For the past few months Package Manager was under heavy development. Even if you will not notice many of the changes and bug fixes trust me - it's faster, more stable and way more polished then it was ever!
There are few things which you will notice for sure. John already wrote in his blog about Start Page and Web Installer, so I will write about remote search and startup performance improvements. Of course there are plenty of other things worth mentioning, but I want to focus only on those two, so the blog entry will be relatively short.
What does it mean to perform remote search and why it was introduced?
We do have per repository view. To find a packages based on their names and descriptions, which aren't in the currently visible repository we had to switch to another repository, perform the search and so on if the package was not found. That was fine, because the number of repositories was limited. Currently more and more repositories are out there, so some mechanism of finding packages across all repositories had to be introduced and this is how remote search came to the live (with a little more time spent by few people which were designing and developing this feature. From the GUI side Padraig O'Briain wrote most of the things - great work!). So switching to the "All Repositories Search" mode allows to find a package and install this package, even if you were in different repository view. Going out from the "All Repositories Search" is very simple, you will find a way... this will restore your previous view.
Startup performance improvements
We did use few dtrace scripts to find out what is happening, and how to improve startup performance, but the major change is the local cache.
Using python with DTrace probes really helped us to identify bottlenecks and functions on which we should focus more. For example to see what is happening when the package is being installed, we were starting the dtrace script entry_entry.d as follows:
pfexec ./entry_entry.d installupdate.py __init__ installupdate.py __operations_done
Starting Package Manager and installing some package after running above command will produce in nice format the number of calls to the functions and time spent in those, once again the power of DTrace is amazing! I will write about this a little bit more, because it's interesting (at least for me) how we were able to connect all parts of automated tests with DTrace performance scripts.
Dumping the data models using cPickle and allowing to use this data together with per repository data model reduced Package Manager startup to around 3 seconds for http://pkg.opensolaris.org/release! I won't write how long did it take before if someone had few repositories configured... At the moment from time to time, when the Package Manager will discover that the cache is out of date, then you will see loading packages progress, but it should be way faster then it was in 2008.11.
ENJOY installing your packages!