filesystem-based repository access
By swalker on May 12, 2010
With the putback of changeset 1895 to the pkg(5) gate, pkg(1) now supports filesystem-based repository access. Please note that this functionality is not expected to be available in an OS build until after b139.
What changed?There are no user-visible changes to command syntax or client operation. The only noticeable change is that any command that accepted a repository URI such as 'http://localhost:10000' now also accepts URIs for filesystem locations such as 'file:/path/to/my/repository'.
For example, if you were creating a new image and had a repository located at '/export/myrepo', you could do this:
pkg image-create -p file:/export/myrepo /path/to/image
See the pkg(1) man page for more examples.
What does this mean?This change means that a pkg.depotd server is no longer required for serving package data if a client can access the package repository via the filesystem (e.g. NFS mounts, etc.).
What are the related bug entries?The following RFEs/Bugs were resolved by changeset 1895:
15762 client support for filesystem-based repository access
10244 caching dictionaries as a class variable prevents multi-image and repo search
14802 ability to have separate read / write download caches
15763 test suite should not start depot server unless necessary
15764 file_manager insert needs to stat() less and return more
What are the potential issues?Currently, the client will not cache data from a repository that is accessed via the filesystem in the image's download cache (e.g. /var/pkg/download), with the assumption that access to the repository via filesystem is reliable and/or reasonably fast.
In some use case scenarios (such as NFS, SMB, etc.) filesystem-based access may be slow, unreliable, or otherwise unsuitable for client transport. In those cases, it is advised that http-based access to the repository via pkg.depotd(1m) is used instead. It is hoped that the client can account for the performance of available filesystem-based resources automatically in the future when retrieving package content.
Search operations performed using a file-based repository can use substantially more memory when a repository contains a large number of packages (such as /dev). This is because the client is essentially acting as the search server in addition to the client.
To avoid this overhead, the related publisher can be configured with both an http(s) origin and the file-based origin. When performing search operations, the client will use the http(s) based origin(s) first to perform searches, but check the file-based origin first for all package file retrieval operations. For example:
pkg set-publisher -g file:/export/myrepo \\
-g http://localhost:10000 example.com