Of course, one of the most important tasks in providing optimal performance and security of any software system is to make sure that the latest software updates are installed, tested, and rolled out promptly and efficiently with minimal disruption to system availability. Oracle provides different types of patches for WebLogic Server, such as Patch Set Updates, Security Patch Updates, and One-Off patches. The patches you install, and the way in which you install them, depends upon your custom needs and environment.
For WebLogic Server running on Kubernetes, we recently shared in Github the steps for creating a WebLogic Server instance with a shared domain home directory that is mapped to a Kubernetes volume. In Kubernetes, Docker, and on-premises environments, we use the same OPatch tool to patch WebLogic Server. However, with Kubernetes orchestrating the cluster, we can leverage the update strategy options in the StatefulSet controller to roll out the patch from an updated WebLogic Server image. In this blog, I explain how.
- Create the WebLogic Server environment on Kubernetes based on the instructions provided at https://github.com/oracle/docker-images/tree/master/OracleWebLogic/samples/wls-k8s-domain. The patching processes described below will be based on the environment created.
- Make sure that the WebLogic Server One-Off patches or Patch Set Updates are accessible from the environment created in the preceding step.
Patch Set Updates and One-Off Patches
- Patch Set Updates are cumulative patches that include security fixes and critical fixes. They are used to patch Oracle WebLogic Server only and released in a regular basis. For additional details related to Patch Set Updates, see Fusion Middleware Patching with OPatch.
- One-Off patches are targeted to solve some known issues or to add feature enhancements. For information about how to download patches, see My Oracle Support.
Kubernetes Update Strategies for StatefulSets
There are three different update strategy options available for StatefulSets that you can use for the following tasks:
- To configure and disable automated rolling updates for container images
- To configure resource requests or limits, or both
- To configure labels and annotations of the pods
For details about the Kubernetes StatefulSets update strategies, see Update StatefulSets at the following location:
These update strategies are as follows:
- On Delete
- Manually delete the pod in any random sequence based on your environment configuration. When Kubernetes detects that a pod is deleted, it creates a new pod that is based on the specification defined for that StatefulSet. This is the default update strategy when StatefulSet is created.
- Rolling Updates
- Perform a rolling update for all the pods in the cluster. Kubernetes will delete and re-create all the Pods defined in the StatefulSet controller one at a time, but in reverse ordinal order.
- Rolling Update + Partition
- A rolling update can also be partitioned, which is determined by the partition value in the specification defined for the StatefulSet. For example, if there are four pods in the cluster and the partition value is 2, only two pods are updated. The other two pods are updated only if partition value is set to 4 or if the partition attribute is removed from the specification.
Methods for Updating the StatefulSet Controller
In a StatefulSet, there are three attributes that need to be verified prior to roll out: image, imagePullPolicy, and updateStrategy. Any one of these attributes can be updated by using the in-place update approach. These in-place options are:
- kubectl apply
- Update the yaml file that is used for creating the StatefulSet, and execute kubectl apply -f <statefulset yml file> to roll out the new configuration to the Kubernetes cluster.
- Sample updated yaml file:
- kubectl edit
- Directly update the attribute value using kubectl edit statefulset <statefulset name>
- Sample edit of a StatefulSet
- kubectl patch
- Directly update the attribute using kubectl patch.
- An example command that updates updateStrategy to RollingUpdate:
- An example command that updates updateStrategy to RollingUpdate with the partition option:
- An example command to use JSON format to update the image attribute to wls-k8s-domain-v1:
- Kubernetes Dashboard
Drill down to the specific StatefulSet from the menu path and update the value of image, imagePullPolicy, and updateStrategy.
Steps to Apply One-Off Patches and Patch Set Updates with an External Domain Home
To create a patched WebLogic image with a new One-Off patch and apply it to all the pods:
- Complete the steps in Example of Image with WLS Domain in Github to create a patched WebLogic image.
- If the Kubernetes cluster is configured on multiple nodes, and the newly created image is not available in the Docker registry, complete the steps provided in docker save and docker load to copy the image to all the nodes in the cluster.
- Update the controller definition using one of the 3 methods described in Methods for Updating the StatefulSet Controller. If you want Kubernetes to automatically apply the new image on all the pods for the StatefulSet, you can set the updateStrategy value to RollingUpdate.
- Apply the new image on the admin-server pod. Because there is only one Administration Server in the cluster, the preferred option is to use the RollingUpdate update strategy option. After the change is committed to the StatefulSet controller, Kubernetes will delete and re-create the Administration Server pod automatically.
- Apply the new image to all the pods defined in the Managed Server StatefulSet:
a) For the OnDelete option, get the list of the pods in the cluster and change the updateStrategy value to OnDelete. You need to manually delete all the pods in the cluster to roll out the new image, using the following commands:
b) For the RollingUpdate option, after you change the updateStrategy value to RollingUpdate, Kubernetes will delete and re-create the pods created for the Managed Server instances in a rolling fashion, but in reverse ordinal order, as shown in Figure 1 below.
c) If the Partition attribute is added to the RollingUpdate value, the rolling update order depends on the partition value. Kubernetes will roll out the new image to the pods with the ordinal whereby the order is greater than or equal to the partition value.
Figure 1: Before and After Patching the Oracle Home Image Using the Kubernetes Statefulset RollingUpdate Strategy
In case you need to rollback the patch, you use the same steps as when applying a new image; that is, by changing the image value to the one for the original image. You should retain at least two or three versions of the images in the registry.
There are several ways you can monitor the rolling update progress of your WebLogic domain:
- Use the kubectl command to check the pod status, For example, the following output is produced when doing a rolling update of two Managed Server pods:
kubectl get pod -o wide
kubectl rollout status statefulset
2. You can use the REST API to query the Administration Server to monitor the status of the Managed Servers during the rolling update. For information about how to use the REST API to monitor WebLogic Server, see Oracle WebLogic RESTful Management Services: From Command Line to JavaFX.
- The following example command queries the status of the Administration Server:
- The proceeding command generates output similar to the following:
3. You can use the WebLogic Server Administration Console to monitor the status of the update.
- The server instance wls-domain-ms-1 is stopped:
- The update is done on wls-domain-ms-1, then switched to wls-domain-ms-0 :
4. A fourth way is to use the Kubernetes Dashboard. From your browser enter the URL https:<hostname>:<nodePort>
The process for applying a One-Off Patch or Patch Set Updates to WebLogic Server on Kubernetes is the same as when running in a bare metal environment. When you use a Kubernetes policy of Statefulset, we recommend that you create a new patched image by extending the image with a previous version and then using the update strategy (OnDelete, RollingUpdate, or "RollingUpdate + partition") that is best suited for your environment.
In a future blog, we will explore the patch options available with Kubernetes Operator. You might be able to integrate some of the manual steps shared in above with Operator to further simplify the overall WebLogic Server patching process when running on Kubernetes.