Publishing your own packages with IPS - getting started.

It's been a while since I blogged on packaging... and we've been busy. I've had several people ask for our developers documentation, but that's still being written, so I thought I'd blog about how to publish a simple package with IPS.

I've been having mysterious networking issues (somewhere, a router doesn't like my system), so I decided I'd package a tool I use often to diagnose flaky networks - mtr. Like many open source programs, it needed some work to get it to compile properly on Solaris. The primary issue is that we've currently hidden libncurses off in /usr/gnu/lib, which is silly - there is no "Solaris" version in /usr/lib, so it should have gone there. Secondly, the configure.in file insists on bringing in -ltermcap - but this causes problems since it exports data that differs in size from the definitions in libncurses. So, after commenting out the offending line:

# AC_CHECK_LIB(termcap, tgetent)

things worked:

export CC=cc
export LDFLAGS="-zguidance -zdirect -zlazyload -zignore -R/usr/gnu/lib -L/usr/gnu/lib -lncurses"
export DESTDIR=/home/barts/publish/proto
./configure --exec-prefix=/usr --prefix=/usr
make install

The -zignore option tells the linker to not include the libraries that aren't actually needed; for some reason configure insists on placing the transitive closure of the library dependencies on the link line rather than  just the ones needed by mtr itself.

So, after all this fiddling, we end up w/ two files installed into our proto area: a man page: usr/share/man/man8/mtr.8 and usr/sbin/mtr, the binary itself.  The installation of the man page into section 8 isn't right for Solaris, but we can fix that in packaging.  We should patch the man page to correct its section number, but this blog entry is supposed to be about packaging, not coercing OSS into the right form for Solaris.

Well, to package the software the first thing we need is a manifest, which contains actions - files, directories, etc.  The pkgsend command will generate a manifest for us from a variety of different inputs, including SVR4 packages, tar files, and directory hierarchies. The latter will be useful here:

: barts@cyber[186]; pkgsend generate /home/barts/publish/proto
dir group=bin mode=0755 owner=root path=usr
dir group=bin mode=0755 owner=root path=usr/share
dir group=bin mode=0755 owner=root path=usr/sbin
dir group=bin mode=0755 owner=root path=usr/share/man
dir group=bin mode=0755 owner=root path=usr/share/man/man8
file usr/share/man/man8/mtr.8 group=bin mode=0644 owner=root path=usr/share/man/man8/mtr.8
file usr/sbin/mtr group=bin mode=04755 owner=root path=usr/sbin/mtr
: barts@cyber[187]; 

Here we can see the directories with default owner & group permissions, and our two files.  Note that the files have a path to the content in the proto area, and a path attribute that indicates where to install them.  They match by default, but we can edit the installation directory in the manifest; this is often easier than getting the configure script to move things around for us.  We also want to delete the directory entries; they're not needed for this package as they already exist, and we would need to make them match exactly.

So, let's dump that in a file:

: barts@cyber[187]; pkgsend generate /home/barts/publish/proto > mtr.p5m

and use pkgmogrify(1) to edit the manifest.  Of course we could do this by hand - but automating the transformations needs to produce a finished manifest from the raw compilation is an important part of making software changes easy, and reducing mistakes.

If you read the man page for pkgmogrify, you'll find several examples; here we simply modify the manifests with the following transforms:

<transform -> edit path man8/mtr.8 man1m/mtr.1m>
<transform dir -> drop>

So, with the above lines in a file named section8, we can invoke pkgmogrify:

: barts@cyber[189]; pkgmogrify mtr.p5m section8 
file usr/share/man/man8/mtr.8 group=bin mode=0644 owner=root path=usr/share/man/man1m/mtr.1m
file usr/sbin/mtr group=bin mode=04755 owner=root path=usr/sbin/mtr
: barts@cyber[190]; 

Ok, we now dump this into another file and run pkgdepend generate on the manifest to discover our dependencies:

: barts@cyber[195]; pkgmogrify mtr.p5m section8 > mtr.p5m.1 
: barts@cyber[196]; pkgdepend generate -md /home/barts/publish/proto mtr.p5m.1 > mtr.p5m.2
: barts@cyber[197]; cat mtr.p5m.2
file usr/share/man/man8/mtr.8 group=bin mode=0644 owner=root path=usr/share/man/man1m/mtr.1m
file usr/sbin/mtr group=bin mode=04755 owner=root path=usr/sbin/mtr
depend fmri=__TBD pkg.debug.depend.file=libresolv.so.2 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libnsl.so.1 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libsocket.so.1 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libgdk_pixbuf-2.0.so.0 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libgtk-x11-2.0.so.0 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libm.so.2 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libncurses.so.5 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libgdk-x11-2.0.so.0 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libc.so.1 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libglib-2.0.so.0 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libpthread.so.1 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
depend fmri=__TBD pkg.debug.depend.file=libgobject-2.0.so.0 pkg.debug.depend.path=lib pkg.debug.depend.path=usr/gnu/lib pkg.debug.depend.path=usr/lib pkg.debug.depend.reason=usr/sbin/mtr pkg.debug.depend.type=elf type=require
: barts@cyber[198]; 

As you can see, the manifest now contains dependency prototypes specifying which files are needed. We can now run the resolve step to examine the packaging manifests on the system (or in a repository) to determine on which packages we depend. This takes a bit of time to run, since it loads in all the manifests first; this still needs some performance work. It dumps its output in a file with the same name as the input but with ".res" appended; this is so multiple packages can be resolved at once:

barts@cyber[200]; pkgdepend resolve -m mtr.p5m.2          
: barts@cyber[201]; cat mtr.p5m.2.res 
file usr/share/man/man8/mtr.8 group=bin mode=0644 owner=root path=usr/share/man/man1m/mtr.1m
file usr/sbin/mtr group=bin mode=04755 owner=root path=usr/sbin/mtr
depend fmri=pkg:/library/desktop/gtk2@0.5.11-0.161 type=require
depend fmri=pkg:/library/glib2@0.5.11-0.161 type=require
depend fmri=pkg:/library/ncurses@0.5.11-0.161 type=require
depend fmri=pkg:/system/library/math@0.5.11-0.161 type=require
depend fmri=pkg:/system/library@0.5.11-0.163 type=require
: barts@cyber[202]; 

The dependencies have all been resolved... the version numbers match whatever packages I have installed on my system. Now let's create a file based repository, set the publisher name and publish the package:

: barts@cyber[231]; pkgrepo create /home/barts/publish/repo
: barts@cyber[232]; pkgrepo add-publisher -s /home/barts/publish/repo bart
: barts@cyber[233]; pkgsend -s /home/barts/publish/repo publish -d /home/barts/publish/proto mtr@0.80,5.11-0.1 mtr.p5m.2.res
pkg://bart/mtr@0.80,5.11-0.1:20110328T202205Z
PUBLISHED
: barts@cyber[234]


We can now install this package; let's do a dry run with lots of output so we can see what will happen:

: barts@cyber[234]; sudo pkg install -nvvg /home/barts/publish/repo mtr
Password: 
               Packages to install:     1
           Create boot environment:    No
              Rebuild boot archive:    No
Changed fmris:
  None -> pkg://bart/mtr@0.80,5.11-0.1:20110328T202205Z
Services:
  None
Actions
  None -> pkg://bart/mtr@0.80,5.11-0.1:20110328T202205Z
  None -> set name=pkg.fmri value=pkg://bart/mtr@0.80,5.11-0.1:20110328T202205Z
  None -> file e76b633a531d5b1081d13e449efe7534df25105f chash=16844052aeb4716f4b7dec96d04d76e709c1885f group=bin mode=0644 owner=root path=usr/share/man/man1m/mtr.1m pkg.csize=2047 pkg.size=4939
  None -> file f71673bbaa607a5aeb24f73849154b7034ed6026 chash=88d4f4183ab9ff6866c5a699a04254aa27f6b260 elfarch=i386 elfbits=32 elfhash=735cf51f4cbaadb2c1fc9f935b59343354a53b56 group=bin mode=04755 owner=root path=usr/sbin/mtr pkg.csize=48466 pkg.size=131228
  None -> depend fmri=pkg:/library/desktop/gtk2@0.5.11-0.161 type=require
  None -> depend fmri=pkg:/library/glib2@0.5.11-0.161 type=require
  None -> depend fmri=pkg:/library/ncurses@0.5.11-0.161 type=require
  None -> depend fmri=pkg:/system/library/math@0.5.11-0.161 type=require
  None -> depend fmri=pkg:/system/library@0.5.11-0.163 type=require

: barts@cyber[235]; 


Looks plausible; let's install and test it:

: barts@cyber[235]; sudo pkg install -g /home/barts/publish/repo mtr
               Packages to install:     1
           Create boot environment:    No
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         2/2      0.0/0.0

PHASE                                        ACTIONS
Install Phase                                    8/8 

PHASE                                          ITEMS
Package State Update Phase                       1/1 
Image State Update Phase                         2/2 

PHASE                                          ITEMS
Reading Existing Index                           8/8 
Indexing Packages                                1/1
: barts@cyber[236]; 

Seems to work; user interface is a little ... clunky... when updating the host to be tracerouted....

Ok, this is a good starting point. Some more things to do before declaring success include:

  • Fix the man page as noted earlier – some sed script seems appropriate.

  • Add a facet to the man page so that it's not installed if the user doesn't want documentation. This is easily done w/ pkgmogrify.

  • Adding a file containing exec-attr info for this binary to match what Solaris does w/ /usr/sbin/traceroute.

  • Add gnome-menu times, icons, etc. and add appropriate restart_fmri tags (see pkg(5)) so that gnome caches are properly refreshed.


A quick recap:


We used pkgsend generate to generate a manifest... and modified the generated manifest with pkgmogrify. We then used pkgdepend to generate and resolve package dependences... and then published with pkgsend again.


There are four steps to publishing your own packages: generate, transmogrify, determine dependencies, publish.

More on this topic later on... hope to get some of the commonly used pkgmogrify transforms into /usr/share/ips or similar; this will make things easier yet.



Comments:

Post a Comment:
Comments are closed for this entry.
About

An engineer's viewpoint on Solaris...

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today