X

News, tips, partners, and perspectives for the Oracle Linux operating system and upstream Linux kernel work

Getting Started With The Oracle Cloud Infrastructure Python SDK

In a recent blog post I illustrated how to use the OCI Command Line Interface (CLI) in shell scripts.

While the OCI CLI is comprehensive and powerful, it may not be the best solution when you need to handle a lot of data in shell scripts. In such cases using a programming language such as Python and the Oracle Cloud Infrastructure Python SDK makes more sense. Data manipulation is much easier, and the API is —as expected— more complex.

In an attempt to demystify the use of the OCI Python SDK, I have re-written and improved the sample oci-provision.sh shell script in Python. This sample project is named oci-compute and is published on GitHub.

This blog post highlights the key concepts of the OCI Python SDK, and together with the oci-compute sample code it should help you to get started easily.

About oci-compute

The oci-compute tool does everything oci-provision.sh does; better, faster and with some additional capabilities:

  • List available Platform, Custom and Marketplace images
  • Create Compute Instances from a Platform, Custom or Marketplace image
    A cloud-init file can be specified to run custom scripts during instance configuration
  • List, start, stop and terminate Compute Instances

Command line syntax and parameters naming are similar to the OCI CLI tool.

See the project README for more information on usage and configuration.

I am using this tool on a daily basis to easily manage OCI Compute instances from the command line.

OCI Python SDK installation

At the time of this writing, the SDK supports Python version 3.5 or 3.6 and can be easily installed using pip, preferably in a Python virtual environment.

Installation and required dependencies are described in detail in the documentation.

oci-compute installation

The oci-compute utility is distributed as a Python package. The setup.py file lists the SDK as dependency; installing the tool will automatically pull the SDK if not already installed.

See the README file for detailed installation steps, but in short it is as simple as creating a virtual environment and running:

$ pip3 install .

The package is split in two main parts:

  • cli.py: handles the command line parsing using the Click package.
    It defines all the commands, sub-commands and their parameters; instantiate the OciCompute class and invoke its methods.
  • oci_compute.py: defines the OciCompute class which interacts with the OCI SDK.
    This is the most interesting part of this project.

OCI SDK Key concepts

This section describes the key concepts used by the OCI SDK.

Configuration

The first step for using the OCI SDK is to create a configuration dictionary (Python dict). While you can build it manually, you will typically use the oci.config.from_file API call to load it from a configuration file.

The default configuration file is ~/.oci/config. It is worth noticing that the OCI CLI uses the same configuration file and provides a command to create it:

$ oci setup config

For oci-compute, the configuration file is loaded during the class initialization:

        self._config = oci.config.from_file(config_file, profile)

API Service Clients

The OCI API is organized in Services, and for each Service you will have to instantiate a Service Client.

For example, our oci-compute package uses the following Services:

  • Compute Service (part of Core Services): to manage the Compute Services (provision and manage compute hosts).
  • Virtual Network Service (part of Core Services): to manage the Networking Components (virtual cloud network, Subnet, …)
  • Identity Service: to manage users, groups, compartments, and policies.
  • Marketplace Service: to manage applications in Oracle Cloud Infrastructure Marketplace

We instantiate the Service Clients in the class initialization:

        # Instantiate clients
        self._compute_client = oci.core.ComputeClient(self._config)
        self._identity_client = oci.identity.IdentityClient(self._config)
        self._virtual_network_client = oci.core.VirtualNetworkClient(self._config)
        self._marketplace_client = oci.marketplace.MarketplaceClient(self._config)

Models

Models allows you to create objects needed by the API calls.

Example: to use an image from the Marketplace, we need to subscribe to the Application Catalog. This is done with the ComputeClient create_app_catalog_subscription method.
This method needs an CreateAppCatalogSubscriptionDetails object as parameter. We will use the corresponding model to create such object: oci.core.models.CreateAppCatalogSubscriptionDetails.

In oci-compute:

            app_catalog_subscription_detail = oci.core.models.CreateAppCatalogSubscriptionDetails(
                compartment_id=compartment_id,
                listing_id=app_catalog_listing_agreements.listing_id,
                listing_resource_version=app_catalog_listing_agreements.listing_resource_version,
                oracle_terms_of_use_link=app_catalog_listing_agreements.oracle_terms_of_use_link,
                eula_link=app_catalog_listing_agreements.eula_link,
                signature=app_catalog_listing_agreements.signature,
                time_retrieved=app_catalog_listing_agreements.time_retrieved
            )
            self._compute_client.create_app_catalog_subscription(app_catalog_subscription_detail).data

Pagination

All list operations are paginated; that is: they will return a single page of data and you will need to call the method again to get additional pages.

The pagination module allows you, amongst other, to retrieve all data in a single API call.

Example: to list the available images in a compartment we could do:

    response = self._compute_client.list_images(compartment_id)

which will only return the first page of data.

To get get all images at once we will do instead:

    response = oci.pagination.list_call_get_all_results(self._compute_client.list_images, compartment_id)

The first parameter to list_call_get_all_results is the paginated list method, subsequent parameters are the ones of the list method itself.

Waiters and Composite operations

To wait for an operation to complete (e.g.: wait until an instance is started), you can use the wait_until function.

Alternatively, there are convenience classes in the SDK which will perform an action on a resource and wait for it to enter a particular state: the CompositeOperation classes. Example: start an instance and wait until it is started.

The following code snippet shows how to start an instance and wait until it is up and running:

            compute_client_composite_operations = oci.core.ComputeClientCompositeOperations(self._compute_client)
            compute_client_composite_operations.instance_action_and_wait_for_state(
                instance_id=instance_id,
                action='START',
                wait_for_states=[oci.core.models.Instance.LIFECYCLE_STATE_RUNNING])

Error handling

A complete list of exceptions raised by the SDK is available in the exception handling section of the documentation. In short, if your API calls are valid (correct parameters, …) the main exception you should care about is the ServiceError one which is raised when a service returns an error response; that is: a non-2xx HTTP status.

For the sake of simplicity and clarity in the sample code, oci-compute does not capture most exceptions. Service Errors will result in a Python stack traceback.

A simple piece of code where we have to consider the Service Error exception is illustrated here:

        for vnic_attachment in vnic_attachments:
            try:
                vnic = self._virtual_network_client.get_vnic(vnic_attachment.vnic_id).data
            except oci.exceptions.ServiceError:
                vnic = None
            if vnic and vnic.is_primary:
                break

Putting it all together

The oci-compute sample code should be self explanatory, but let’s walk through what happens when e.g. oci-compute provision platform --operating-system "Oracle Linux" --operating-system-version 7.8 --display-name ol78 is invoked.

First of all, the CLI parser will instantiate an OciCompute object. This is done once at the top level, for any oci-compute command:

        ctx.obj['oci'] = OciCompute(config_file=config_file,
                                    profile=profile,
                                    verbose=verbose

The OciCompute class initialization will:

  • Load the OCI configuration from file
  • Instantiate the Service Clients

The Click package will then invoke provision_platform function which in turn will call the OciCompute.provision_platform method.

We use the oci.core.ComputeClient.list_images to retrieve the most recent Platform Image matching the given Operating System and its version:

        images = self._compute_client.list_images(
            compartment_id,
            operating_system=operating_system,
            operating_system_version=operating_system_version,
            shape=shape,
            sort_by='TIMECREATED',
            sort_order='DESC').data
        if not images:
            self._echo_error("No image found")
            return None
        image = images[0]

We then call OciCompute._provision_image for the actual provisioning. This method uses all of the key concepts explained earlier.

Pagination is used to retrieve the Availability Domains using the Identity Client list_availability_domains method:

        availability_domains = oci.pagination.list_call_get_all_results(
            self._identity_client.list_availability_domains,
            compartment_id
        ).data

VCN and subnet are retrieved using the Virtual Network Client (list_vcns and list_subnets methods)

Metadata is populated with the SSH public key and a cloud-init file if provided:

        # Metadata with the ssh keys and the cloud-init file
        metadata = {}
        with open(ssh_authorized_keys_file) as ssh_authorized_keys:
            metadata['ssh_authorized_keys'] = ssh_authorized_keys.read()

        if cloud_init_file:
            metadata['user_data'] = oci.util.file_content_as_launch_instance_user_data(cloud_init_file)

Models are used to create an instance launch details (oci.core.models.InstanceSourceViaImageDetails, oci.core.models.CreateVnicDetails and oci.core.models.LaunchInstanceDetails methods):

        instance_source_via_image_details = oci.core.models.InstanceSourceViaImageDetails(image_id=image.id)
        create_vnic_details = oci.core.models.CreateVnicDetails(subnet_id=subnet.id)
        launch_instance_details = oci.core.models.LaunchInstanceDetails(
            display_name=display_name,
            compartment_id=compartment_id,
            availability_domain=availability_domain.name,
            shape=shape,
            metadata=metadata,
            source_details=instance_source_via_image_details,
            create_vnic_details=create_vnic_details)

Last step is to use the launch_instance_and_wait_for_state Composite Operation to actually provision the instance and wait until it is available:

        compute_client_composite_operations = oci.core.ComputeClientCompositeOperations(self._compute_client)
        response = compute_client_composite_operations.launch_instance_and_wait_for_state(
            launch_instance_details,
            wait_for_states=[oci.core.models.Instance.LIFECYCLE_STATE_RUNNING],
            waiter_kwargs={'wait_callback': self._wait_callback})

We use the optional waiter callback to display a simple progress indicator

oci-compute demo

Short demo of the oci-compute tool:

Conclusion

In this post, I’ve shown how to use oci-compute to easily provision and manage your OCI Compute Instances from the command line as well as how to create your own Python scripts using the Oracle Cloud Infrastructure Python SDK.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.