"Direct" package publishing
By swalker on Feb 13, 2009
Who says good things don't happen on Friday the 13th?
For several weeks now, I've been working on improvements to our publishing process. The primary one, among those, is the ability to publish using pkgsend or solaris.py directly to disk without running the pkg.depotd server. Here's a summary of the major changes:
Revision 873 / b95d76c53b5e
This revision integrated enhancements and fixes for the following issues:
- 2691 ability to publish packages to local disk repository
- 850 pkgsend traceback if invalid or unavailable repository url specified
- 894 pkgsend should do more action verification
- 1236 pkgsend open should not allow both -e and -n
- 2430 pkgsend add of non-existent file causes traceback
- 5088 server can still create a repo when in —readonly mode
- 5294 pkgsend doesn't provide server response when operations fail
- 5944 pkgsend traceback if invalid or no action type specified
- 6566 catalog is always rebuilt when server config is not readonly
As a result of these changes, the following changes in behaviour should be noted:
- pkg.depotd will no longer automatically create or populate the inst_root specified using pkg/inst_root or the -d option if started with —readonly. Instead, it will exit with an error message. Starting the server in a mode other than readonly will create the specified directory and any necessary subdirectories or files.
- pkg.depotd will no longer automatically rebuild the repository catalog if not started with —readonly. However, search indices will continue to be refreshed automatically if needed. If, for some reason, the package catalog does need to be rebuilt on pkg.depotd startup or the old behaviour is desired, pkg.depotd can be started with the —rebuild option. This change should result in significantly faster startup (and less disk I/O) for large repositories served by pkg.depotd in publishing mode.
- pkgsend and the importer now support file:// URLs for publishing. In the case of pkgsend:
PKG_REPO=file:///path/to/repo pkgsend open firstname.lastname@example.org
pkgsend -s file:///path/to/repo open email@example.com
In the case of solaris.py (the importer), you will need to set the REPO variable in the makefile or override it another way. Be aware that if you specify a directory that is not a valid pkg(5) repository, publishing will fail with an error indicating that. You must create a repository first by starting the pkg.depotd server with the desired path and then exiting. A subcommand to create repositories will be added to pkgsend in the near future (bug 6574).
- It is important to note that only one user at a time should publish directly to a file:// repository URL; the result of more than one user publishing to a file:// repository URL at a time is completely untested and is likely to fail. Bug 2696 will help enforce this by ensuring that a repository lock is obtained during publishing mode. If you need the ability to allow multiple users to simultaneously publish to a repository, you must use pkg.depotd and publish to a http:// repository URL. The tradeoff is that file:// repository URL publishing can be significantly faster and will result in significantly reduced memory usage and disk I/O.
- Significantly more verification is now done during the publishing process. Expect things that may have appeared to work before to no longer be accepted by the server or by the pkgsend client. Additionally, pkgsend once more produces useful error information when operations fail.