Over the past couple of months I’ve been working on getting some infrastructure up for running a bunch of services. For the most part I’ve been in my comfort zone standing up Oracle Solaris IPS repositories and Automated Installer network boot services. I’ve also been standing up local Docker registries on Oracle Linux, and figured I’d share my experiences.
Unfortunately there’s not wonderful tooling around Docker registries that I’ve seen for cloning a local registry, so decided to write my own tooling that relies a little on the Docker Registry HTTP v2 API and a bit of simple Python scripting. My use case was to pull all images with a specific tag and create a local Docker registry. Here’s an overview of the workflow that you can follow.
1. Install Docker engine
Pretty simple really, a yum command to install the docker-engine command:
# yum install docker-engine
Given I was writing this in Python, a simple check_call would have been sufficient, but decided to use the private CLI API so could quickly use the following snippet:
ybc = cli.YumBaseCli() ybc.conf.assumeyes = True ybc.conf.downloadonly = False ybc.installPkgs(packages) ybc.buildTransaction() ybc.doTransaction()
2. Configure Docker to pull from an internal registry over HTTP
I was pulling from an internal Docker registry, so needed to add it as an insecure registry prior to enabling the Docker service. You can do this by modifying /etc/sysconfig/docker and adding the following:
INSECURE_REGISTRY="--insecure-registry internal-docker-reg.us.oracle.com:5000"
3. Enable the Docker service
Another simple command to enable and start the Docker service:
# systemctl enable docker # systemctl start docker
4. Get a list of images using REST
I wanted to pull all images of a specific tag. Docker has the concept of the ‘latest’ tag, which is the last version of an image that’s published. Unfortunately this doesn’t mean that it’s necessarily the latest version so I’ve avoided that like the plague in terms of consistency. We can get a list of all images contained within a repository (essentially a group of images) in a registry using a simple web request using the following bit of Python:
docker_url = "http://internal-docker-reg.us.oracle.com:5000/v2/_catalog" r = requests.get(docker_url, verify=False) json_data = r.json()
From there you can simply parse through the resulting dict and find the list of Docker images. You get something of the following form (in my case a bunch of OpenStack based Docker images):
{u'repositories': [u'openstack/ol-openstack-aodh-api', u'openstack/ol-openstack-aodh-base', ... ]}
And parsing this is easy:
for c in json_data['repositories']:
print("Image: %s" % c)
5. Get a list of tags for an image using REST
We can now recurse and get the list of tags for each image once we know the image name. The following snippet achieves this in Python:
docker_image_tag_url = "http://internal-docker-reg.us.oracle.com:5000/v2" + image_name + "/tags/list" r = requests.get(docker_image_tag_url, verify=False) image_tags = r.json()
It will result in something like the following:
{u'name': u'openstack/ol-openstack-aodh-api', u'tags': [u'3.0.1.latest', u'3.0.1.90', ... ]}
And again, something very easy to parse through.
6. Pull the image to local registry
The next step is to pull the image down based on the tag that you’re interested in:
# docker pull http://internal-docker-reg.us.oracle.com:5000/v2/openstack/ol-openstack-aodh-api:3.0.1.153
I also optioned to make a local copy of the image as a tarball using the docker save command:
# docker save internal-docker-reg.us.oracle.com:5000/openstack/ol-openstack-aodh-api:3.0.1.153 | gzip -c > openstack/ol-openstack-aodh-api.tar.gz
7. Running a Docker registry instance
If you want to make your Docker images externally accessible, you’ll also need to pull the registry Docker image from docker.io and run it:
# docker pull registry:2 # docker run -d -p 5000:5000 --restart=always --name=registry registry:2
You will see that you’ll have a running Docker container using the docker ps command:
# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3e03011bb4a5 registry:2 "/entrypoint.sh /etc/" 5 weeks ago Up 5 weeks 0.0.0.0:5000->5000/tcp registry
8. Republish your Docker images
For the final step we need to tag the images in our local registry and push them into our registry. For this step, we need the image ID so I pulled this from the following command:
# docker images --format="{{.ID}},{{.Repository}}"
This gets you something looking like the following that can be easily parsed:
5d289275c280,ca-openstack-docker-reg.us.oracle.com:5000/openstack/ol-openstack-neutron-base 393d7c702a00,ca-openstack-docker-reg.us.oracle.com:5000/openstack/ol-openstack-ironic-base cc52d8d76395,ca-openstack-docker-reg.us.oracle.com:5000/openstack/ol-openstack-ceilometer-base ...
We can proceed to tag the image and push to our registry instance:
# docker tag 5d289275c280 http://mynewhost.us.oracle.com:5000/openstack/ol-openstack-neutron-base:3.0.1.153 # docker push http://mynewhost.us.oracle.com:5000/openstack/ol-openstack-neutron-base:3.0.1.153
And I did this at scale with Python for all the images with a simple for loop.
What we’ll end up is a list such as this:
# docker images REPOSITORY TAG IMAGE ID CREATED SIZE mynewhost.us.oracle.com:5000/openstack/ol-openstack-neutron-base 3.0.1.113 751f286bc25e 4 months ago 958 MB ...
Once we have reached this stage, all our Docker images have been republished for this new host mynewhost.us.oracle.com. This means that other clients can now pull from my internal server easily. Enjoy!