How to Install and Create a Verrazzano Cluster

Remember to take note about the `MountPoint`. We will need this information later when we create the cluster.

Another thing that we will need is the image that we should use when creating the cluster. Because KIND images are prebuilt for each release, we need to execute `$ kind version` and look at https://github.com/kubernetes-sigs/kind/releases for the image that we will use. For this blog post, we will use KIND version 0.11.1 and the Kubernetes version 1.20.

Execute the following command to create a Kuberntes cluster, replacing the `hostPath` with the `MountPoint` from `docker volume inspect` command, and the image with one suitable for the KIND release you are using. Example:

kind create cluster --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    image: kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9
    kubeadmConfigPatches:
      - |
        kind: ClusterConfiguration
        apiServer:
          extraArgs:
            "service-account-issuer": "kubernetes.default.svc"
            "service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"
    extraMounts:
      - hostPath: /var/lib/docker/volumes/containerd/_data
        containerPath: /var/lib/containerd 
EOF

With the Kubernetes cluster running, we need to instal and configure MetalLB. This step is needed because KIND doesn’t come with any implementation of network load balancers (Services of type LoadBalancer).

For use by MetalLB, assign a range of IP addresses at the end of the kind network’s subnet CIDR range:

kubectl apply -f - <<-EOF
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 172.18.0.230-172.18.0.250
EOF

3 – Accessing Verrazzano

Verrazzano installs a lot of consoles, and you can be aware of each console URL by running the following command: 

$ kubectl get vz -o jsonpath="{.items[].status.instance}" | jq .
{
  "consoleUrl": "https://verrazzano.default.172.18.0.23x.nip.io",
  "elasticUrl": "https://elasticsearch.vmi.system.default.172.18.0.23x.nip.io",
  "grafanaUrl": "https://grafana.vmi.system.default.172.18.0.23x.nip.io",
  "keyCloakUrl": "https://keycloak.default.172.18.0.23x.nip.io",
  "kibanaUrl": "https://kibana.vmi.system.default.172.18.0.23x.nip.io",
  "prometheusUrl": "https://prometheus.vmi.system.default.172.18.0.23x.nip.io",
  "rancherUrl": "https://rancher.default.172.18.0.23x.nip.io"
}

Note that the URLs use the format 1.2.3.4.nip.io. Nip.io is a DNS service that easily provides host names that point to IP addresses without needing to edit your `/etc/hosts` file. In this way, the host verrazzano.default.172.18.0.231.nip.io will resolve to the IP 172.18.0.231. This IP is the internal KIND network.

Just open any of the URLs and you will see the respective console.

3.1 – Accessing Verrazzano from MacOS computers

Because MacOS uses Hyperkit to create a host running the docker daemon, the interfaces for docker’s bridge network aren’t actually routable interfaces. This means that you won’t be able to open your browser to make requests to any IPs in the network 172.18.0.0/16. 

The solution for this problem is to use the project `docker-tuntap-osx` that will attach a TunTap interface into MacOS, creating a bridge interface between the docker daemon and MacOS.

The first thing that we need is clone this project, and install the TAP interface using the following commands:

git clone https://github.com/AlmirKadric-Published/docker-tuntap-osx

cd docker-tuntap-osx/

docker-tuntap-os$ ./sbin/docker_tap_install.sh
Installation complete
Restarting Docker
Process restarting, ready to go
 

With the interface already installed, it’s time now to bring it up, and run the command `ifconfig` to get the details of this TAP interface:

docker-tuntap-os$ ./sbin/docker_tap_up.sh
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
a0d0a0d46f8b: Pull complete
Digest: sha256:e1c082e3d3c45cccac829840a25941e679c25d438cc8412c2fa221cf1a824e6a
Status: Downloaded newer image for alpine:latest

$ ifconfig
...
tap1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 56:d6:3d:13:99:d3
inet 10.0.75.1 netmask 0xfffffffc broadcast 10.0.75.3
media: autoselect
status: active

Now that we have this tap interface up and running, we should redirect all the traffic to the network 172.18.0.0/16 using this interface’s gateway (10.0.75.2) through the following command:

sudo route add -net 172.18.0.0 -netmask 255.255.0.0 10.0.75.2

To access Verrazzano from a MacOS computer, extract the Server SSL certificates and import them into MacOS Keychain as trusted certificates. So let’s extract the server certificates and store them in  the user’s home folder using the following commands. Remember to replace the IP addresses in the -connect and -servername parameters: 

openssl s_client -connect 172.18.0.23x:443 -servername verrazzano.default.172.18.0.23x.nip.io -showcerts 2>/dev/null </dev/null |  sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/verrazzano.pem

sudo security add-trusted-cert -d -r trustAsRoot -k /Library/Keychains/System.keychain ~/verrazzano.pem

openssl s_client -connect 172.18.0.23x:443 -servername keycloak.default.172.18.0.23x.nip.io -showcerts 2>/dev/null </dev/null |  sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/keycloak.pem

sudo security add-trusted-cert -d -r trustAsRoot -k /Library/Keychains/System.keychain ~/keycloak.pem

openssl s_client -connect 172.18.0.23x:443 -servername elasticsearch.vmi.system.default.172.18.0.23x.nip.io -showcerts 2>/dev/null </dev/null |  sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/istio.pem

sudo security add-trusted-cert -d -r trustAsRoot -k /Library/Keychains/System.keychain ~/istio.pem