Getting Started With The Vagrant Libvirt Provider For Oracle Linux

Introduction

As recently announced by Sergio we now support the libvirt provider for our Oracle Linux Vagrant Boxes.

The libvirt provider is a good alternative to the virtualbox one when you already use KVM on your host, as KVM and VirtualBox virtualization are mutually exclusive. It is also a good choice when running Vagrant on Oracle Cloud Infrastructure.

This blog post will guide you through the simple steps needed to use these new boxes on your Oracle Linux host (Release 7 or 8).

Virtualization

Virtualization is easily installed using the Virtualization Host package group.

On Oracle Linux 7, first enable the ol7_kvm_utils channel to get recent version of the packages:

sudo yum-config-manager --enable ol7_kvm_utils

After installing the packages, start the libvirtd service and add you user to the libvirt group:

sudo yum group install "Virtualization Host"
sudo systemctl enable --now libvirtd
sudo usermod -a -G libvirt opc

Do not forget to re-login to activate the group change for your user!

Vagrant

We need to install HashiCorp Vagrant as well as the Vagrant Libvirt Provider contributed plugin:

# Vagrant itself:
sudo yum install https://releases.hashicorp.com/vagrant/2.2.9/vagrant_2.2.9_x86_64.rpm
# Libraries needed for the plugin:
sudo yum install libxslt-devel libxml2-devel libvirt-devel \
  libguestfs-tools-c ruby-devel gcc make

Oracle Linux 8: at the time of this writing there is a compatibility issue between system libraries and the ones embedded with Vagrant. Run the following script as root to update the Vagrant libraries:

#!/usr/bin/env bash
# Description: override krb5/libssh libraries in Vagrant embedded libraries
set -e

# Get pre-requisites
dnf -y install \
  libxslt-devel libxml2-devel libvirt-devel \
  libguestfs-tools-c ruby-devel \
  gcc byacc make cmake gcc-c++

mkdir -p vagrant-build
cd vagrant-build
dnf download --source krb5-libs libssh
# krb5
rpm2cpio krb5-1.17-*.src.rpm | cpio -idmv krb5-1.17.tar.gz
tar xzf krb5-1.17.tar.gz
pushd krb5-1.17/src
./configure
make
cp -a lib/crypto/libk5crypto.so.3* /opt/vagrant/embedded/lib64/
popd

# libssh
rpm2cpio libssh-0.9.0-*.src.rpm | cpio -imdv  libssh-0.9.0.tar.xz
tar xJf libssh-0.9.0.tar.xz
mkdir build
pushd build
cmake ../libssh-0.9.0  -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded
make
cp lib/libssh* /opt/vagrant/embedded/lib64/
popd

We are now ready to install the plugin (as your non-privileged user):

vagrant plugin install vagrant-libvirt

Firewall

The libvirt provider uses NFS to mount the /vagrant shared folder in the guest. Your firewall must be configured to allow the NFS traffic between the host and the guest.

Oracle Linux 7

You can allow NFS traffic in your default zone:

sudo firewall-cmd --permanent --add-service=nfs3
sudo firewall-cmd --permanent --add-service=mountd
sudo firewall-cmd --permanent --add-service=rpc-bind
sudo systemctl restart firewalld

Alternatively you can add the libvirt bridge to your trusted zone:

sudo firewall-cmd --zone=trusted --add-interface=virbr1
sudo systemctl restart firewalld

Oracle Linux 8

With Oracle Linux 8, the libvirt bridge is automatically added to the libvirt zone. Traffic must be allowed in that zone:

sudo firewall-cmd --permanent --zone libvirt --add-service=nfs3
sudo firewall-cmd --permanent --zone libvirt --add-service=mountd
sudo firewall-cmd --permanent --zone libvirt --add-service=rpc-bind
sudo systemctl restart firewalld

Privileges considerations

To configure NFS, Vagrant will require root privilege when you start/stop guest instances. Unless you are happy to enter your password on every vagrant up you should consider enabling password-less sudo for your user.

Alternatively you can enable fine grained sudoers access as described in Root Privilege Requirement section of the Vagrant documentation.

Using libvirt boxes

Your first libvirt guest

You are now ready to use livirt enabled boxes!

mkdir ol7
cd ol7
vagrant init oraclelinux/7 https://oracle.github.io/vagrant-boxes/boxes/oraclelinux/7.json
vagrant up

Libvirt configuration

While the libvirt provider exposes quite a lot of configuration parameters, most Vagrantfiles will run with no or little modification.

Typically when you have for VirtualBox:

config.vm.provider "virtualbox" do |vb|
  vb.cpus = 4
  vb.memory = 4096
end

You will need for libvirt:

config.vm.provider :libvirt do |libvirt|
  libvirt.cpus = 4
  libvirt.memory = 4096
end

The Oracle vagrant-boxes repository is being updated to support the new libvirt boxes.

Tips and tricks

Virsh

The virsh command can be used to monitor the libvirt resources. By default vagrant-libvirt uses the qemu:///system URI to connect to the KVM hypervisor and images are stored in the default storage pool.

Example:

[opc@bommel ~]$ vagrant global-status
id       name               provider   state    directory
--------------------------------------------------------------------------------------------------
7ec55b3  ol7-vagrant        libvirt    shutoff  /home/opc/src/vagrant-boxes/OracleLinux/7
3fd9dd9  registry           libvirt    shutoff  /home/opc/src/vagrant-boxes/ContainerRegistry
c716711  ol7-docker-engine  libvirt    running  /home/opc/src/vagrant-boxes/DockerEngine
6a0cb46  worker1            libvirt    running  /home/opc/src/vagrant-boxes/OLCNE
a262a29  worker2            libvirt    running  /home/opc/src/vagrant-boxes/OLCNE
538e659  master1            libvirt    running  /home/opc/src/vagrant-boxes/OLCNE
b6d2661  ol6-vagrant        libvirt    running  /home/opc/src/vagrant-boxes/OracleLinux/6
41aaa7e  oracle-19c-vagrant libvirt    running  /home/opc/src/vagrant-boxes/OracleDatabase/19.3.0

[opc@bommel ~]$ virsh -c qemu:///system list --all
 Id   Name                             State
-------------------------------------------------
 23   DockerEngine_ol7-docker-engine   running
 24   OLCNE_worker1                    running
 25   OLCNE_worker2                    running
 26   OLCNE_master1                    running
 30   6_ol6-vagrant                    running
 31   19.3.0_oracle-19c-vagrant        running
 -    7_ol7-vagrant                    shut off
 -    ContainerRegistry_registry       shut off

[opc@bommel ~]$ virsh -c qemu:///system vol-list --pool default
 Name                                                        Path
-----------------------------------------------------------------------------------------------------------------------------------------------
 19.3.0_oracle-19c-vagrant.img                               /var/lib/libvirt/images/19.3.0_oracle-19c-vagrant.img
 6_ol6-vagrant.img                                           /var/lib/libvirt/images/6_ol6-vagrant.img
 7_ol7-vagrant.img                                           /var/lib/libvirt/images/7_ol7-vagrant.img
 ContainerRegistry_registry-vdb.qcow2                        /var/lib/libvirt/images/ContainerRegistry_registry-vdb.qcow2
 ContainerRegistry_registry.img                              /var/lib/libvirt/images/ContainerRegistry_registry.img
 DockerEngine_ol7-docker-engine-vdb.qcow2                    /var/lib/libvirt/images/DockerEngine_ol7-docker-engine-vdb.qcow2
 DockerEngine_ol7-docker-engine.img                          /var/lib/libvirt/images/DockerEngine_ol7-docker-engine.img
 ol7-latest_vagrant_box_image_0.img                          /var/lib/libvirt/images/ol7-latest_vagrant_box_image_0.img
 OLCNE_master1.img                                           /var/lib/libvirt/images/OLCNE_master1.img
 OLCNE_worker1.img                                           /var/lib/libvirt/images/OLCNE_worker1.img
 OLCNE_worker2.img                                           /var/lib/libvirt/images/OLCNE_worker2.img
 oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.130.img   /var/lib/libvirt/images/oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.130.img
 oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.132.img   /var/lib/libvirt/images/oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.132.img
 oraclelinux-VAGRANTSLASH-7_vagrant_box_image_7.7.17.img     /var/lib/libvirt/images/oraclelinux-VAGRANTSLASH-7_vagrant_box_image_7.7.17.img
 oraclelinux-VAGRANTSLASH-7_vagrant_box_image_7.8.135.img    /var/lib/libvirt/images/oraclelinux-VAGRANTSLASH-7_vagrant_box_image_7.8.135.img

Removing box image

The vagrant box remove command removes the box from the user .vagrant directory, but not from the storage pool. Use virsh to cleanup the pool:

[opc@bommel ~]$ vagrant box list
oraclelinux/6 (libvirt, 6.10.130)
oraclelinux/6 (libvirt, 6.10.132)
oraclelinux/7 (libvirt, 7.8.131)
oraclelinux/7 (libvirt, 7.8.135)
[opc@bommel ~]$ vagrant box remove oraclelinux/6 --provider libvirt --box-version 6.10.130
Removing box 'oraclelinux/6' (v6.10.130) with provider 'libvirt'...
Vagrant-libvirt plugin removed box only from your LOCAL ~/.vagrant/boxes directory
From Libvirt storage pool you have to delete image manually(virsh, virt-manager or by any other tool)
[opc@bommel ~]$ virsh -c qemu:///system vol-delete --pool default oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.130.img
Vol oraclelinux-VAGRANTSLASH-6_vagrant_box_image_6.10.130.img deleted

Libvirt CPU emulation mode

The default libvirt CPU emulation mode is host-model, that is: the guest inherits capabilities from the host.

Should the guest not start in this mode, you can override it using the custom mode – e.g.:

config.vm.provider :libvirt do |libvirt|
  libvirt.cpu_mode = 'custom'
  libvirt.cpu_model = 'Skylake-Server-IBRS'
  libvirt.cpu_fallback = 'allow'
end

You can list the available CPU models with virsh cpu-models x86_64.

Storage

By default, the Vagrant Libvirt provider will use the default libvirt storage pool which stores images in /var/lib/libvirt/images. The storage_pool_name option allows you to use any other pool/location. Example:

On the libvirt side, create a pool:

[opc@bommel ~]$ virsh -c qemu:///system
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # pool-define-as vagrant dir --target /data/vagrant
Pool vagrant defined

virsh # pool-start vagrant
Pool vagrant started

virsh # pool-autostart vagrant
Pool vagrant marked as autostarted

In your Vagrantfile, set the storage_pool_name option:

config.vm.provider :libvirt do |libvirt|
  libvirt.storage_pool_name = 'vagrant'
end

Vagrant Libvirt defaults

If you have site specific options, instead of modifying all your Vagrantfiles, you can define them globally in ~/.vagrant.d/Vagrantfile (see Load Order and Merging). E.g:

# Vagrant local defaults

Vagrant.configure(2) do |config|
  config.vm.provider :libvirt do |libvirt|
    libvirt.cpu_mode = 'custom'
    libvirt.cpu_model = 'Skylake-Server-IBRS'
    libvirt.cpu_fallback = 'allow'
    libvirt.storage_pool_name = 'vagrant'
  end
end

VirtualBox and libvirt on the same host

You cannot run VirtualBox and libvirt guests at the same time, but you still can have both installed and switch from the one to the other providing there is no guest VM running when you switch.

The only thing you have to do is to stop/start their respective services – e.g. to switch from VirtualBox to libvirt:

systemctl stop vboxdrv.service
systemctl start libvirtd.service

Getting Started With The Vagrant Libvirt Provider For Oracle Linux

Introduction

Virtualization

Vagrant

Firewall

Oracle Linux 7

Oracle Linux 8

Privileges considerations

Using libvirt boxes

Your first libvirt guest

Libvirt configuration

Tips and tricks

Virsh

Removing box image

Libvirt CPU emulation mode

Storage

Vagrant Libvirt defaults

VirtualBox and libvirt on the same host

Screencast

Philippe Vanhaesendonck

Oracle Linux Vagrant Boxes Now Include Catalog Data and Add Support for libvirt Provider

Btrfs on the Unbreakable Enterprise Kernel 6

Getting Started With The Vagrant Libvirt Provider For Oracle Linux

Introduction

Virtualization

Vagrant

Firewall

Oracle Linux 7

Oracle Linux 8

Privileges considerations

Using libvirt boxes

Your first libvirt guest

Libvirt configuration

Tips and tricks

Virsh

Removing box image

Libvirt CPU emulation mode

Storage

Vagrant Libvirt defaults

VirtualBox and libvirt on the same host

Screencast

Authors

Philippe Vanhaesendonck

Oracle Linux Vagrant Boxes Now Include Catalog Data and Add Support for libvirt Provider

Btrfs on the Unbreakable Enterprise Kernel 6