X

Proactive insights, news and tips from Oracle WebLogic Server Support. Learn Oracle from Oracle.

End to end example of monitoring WebLogic Server with Grafana dashboards on the OCI Container Engine for Kubernetes

Marina Kogan
Principal Member of Technical Staff

In previous blogswe described how to run WLS on Kubernetes with the Operator using the Oracle Cloud Infrastructure (OCI) Container Engine for Kubernetes and how to set up the Monitoring Exporter with Prometheus and Grafana. 

In this blog, we will demonstrate the steps to set up and monitor WebLogic Server runtime metrics and corresponding Grafana dashboards for servers, web applications, data sources and JMS services on the OCI Container Engine for Kubernetes (OKE).

Prerequisites

Step-by-Step Guide

This end-to-end sample shows you the steps to set up monitoring for WebLogic domains using Prometheus and Grafana. When completed, you'll have Prometheus, Grafana, and WebLogic domains installed, configured, and running. This sample includes Grafana dashboards to visualize the WebLogic Server runtime metrics. 

Before getting started, clone the WebLogic Monitoring Exporter project from GitHub. It contains many of the samples and helpers we will need.

git clone https://github.com/oracle/weblogic-monitoring-exporter.git

Here is an overview of the process:

  • Configure the persistent volume.
  • Prepare and run a WebLogic domain and operator.
    • Set up MySQL Server.
    • Install the WebLogic Kubernetes Operator.
    • Run a WebLogic domain.
  • Set up Prometheus.
  • Set up Grafana.

Configuring the Persistent Volume

In this sample, we need to create four persistent volumes (PV) and persistent volume claims (PVC) to store data for MySQL, Prometheus server, Prometheus Alert Manager, and Grafana. 

High-Level Steps:

  1. Create a target mount.
  2. Create three file systems, one per persistent volume: MySQL, Prometheus, Prometheus Alert Manager.
  3. Create a storage class.
  4. Create a persistent volume (PV).
  5. Create a persistent volume claim (PVC).

Let’s review all these steps in detail.

Log in to the OCI console. Verify that you are using the correct region, for example, “us-phoenix-1” and department, for example“Quality Assurance”. 

Select‘Menu’->’Developer Services’->’Container Clusters (OKE)’ and verify that your OKE cluster is up and running. 

Select ‘Menu’-> ‘Networking’ -> ‘VNC’. Select the VCN assigned to your cluster and note the CIDR range, for example ’10.7.0.0/16’

From the OCI Menu, select ‘File Storage’ -> ‘Mount Targets’

Click on ‘Create Mount Target’ and select the same Virtual Cloud Network (VNC) as your cluster, for example: mkcluster_vcn

Click on the created Mount Target link, create three file systems by using the ‘Create Export’ button under the ‘Exports’ section. Create three File Systems, each per desired PV.

 

From the OCI Menu, select ‘Networking’ -> ‘Virtual Cloud Networks’

Choose the VCN assigned to your cluster, for example mkcluster_vcn.

On the left of Security Lists, select -> ‘Create Security List’, and create new Security List, for example mkcluster-default-security-list.

Create four Ingresses (two TCP [2048-2050, 111] and two UDP [2048, 111) destination port) and three Egresses (two TCP [2048-2050, 111] and one UDP [111] source ports). The IP range should match the CIDR range in the corresponding VCN, in our example it is ‘10.7.0.0/16’.

Here is an example of the created Ingress Rules:

 

Here is an example of the Egress Rules:

Create the Storage Class with provisioner: oracle.com/oci-fss and the OCID from the Mount Target (for example: ocid1.mounttarget.oc1.phx.abcdby27ve3wk6bobuhqllqojxwiotqnb4c2ylefuzabcd)

Here is the example of storageclass.yaml:

Execute this command to create the Storage Class:

kubectl apply -f storageclass.yaml

Setting up MySQL Server

Modify the file weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/persistence.yaml to use the NFS type for Persistent Volume (PV) . Use your Mount Target IP address (for example: 10.7.10.4) and the File System name (for example: /FileSystem-20190806-2144) for the root of the persistent volume. Use the created Storage Class for PV and PVC (for example: oci-fss). 

Here is the example modified weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/persistence.yaml file from the end-to-end sample:

Push mysql image to the OCIR registry by executing these steps:

 docker pull mysql

 docker tag mysql:latest phx.ocir.io/weblogick8s/db/mysql:5.7

 docker login phx.ocir.io

 Username: weblogick8s/[OCIUsername]

 Password: [Auth Key of OCI]

 docker push phx.ocir.io/weblogick8s/db /mysql:5.7

Create the OCIR secret:

kubectl create secret docker-registry ocirsecret --docker-server=phx.ocir.io --docker-username=weblogick8s/[OCIRUsername] --docker-password=[OCIRPassword] --docker-email=[OCIRDockerEmail] -n default

Modify the file weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/mysql.yaml to add a securityContext section to add permission for the user mysql, add OCIR image info, and pull secret:

Follow the instructions from the end-to-end sample to deploy PV and PVC and create MYSQL database, using the modified mysql.yaml and persistence.yaml files.

Installing the WebLogic Kubernetes Operator and WebLogic Server Domain

Follow the instructions from the end-to-end  sample to create WebLogic Kubernetes Operator.

In our example, we will use the ‘default’ namespace for the WebLogic domain and ‘monitoring’ namespace for running Prometheus and Grafana. Use helm to install the Traefik load balancer. Use the values.yaml in the sample but set kubernetes.namespaces specifically.

Go to the OCI dashboard -> ‘Networking’ -> ‘LoadBalancers’, select your compartment, and verify that the instance was created and running.

Note the public IP address created for the load balancer, for example ‘129.146.192.196’. Ask your DNS provider to create three host names and match them with this IP address. For our example, we have created:

e2e-wls.weblogick8s.org to access WebLogic Domain

e2e-prom.weblogick8s.org to access Prometheus dashboard

e2e-grafana.weblogick8s.org to access Grafana dashboard

In OKE, we will be using the Traefik load balancer. To run WebLogic Monitoring Exporter with the load balancer, we need to add an extra property into the configuration file. Modify the weblogic-kubernetes-monitoring-exporter/samples/kubernetes/end2end/dashboard/exporter-config.yaml configuration file to add the restPort info. Insert this line above ‘queries’:

restPort8001                                                          .

Follow the instructions from the end-to-end  sample to create Domain Image.

Push the created Domain Image to OCIR registry:

docker tag domain1-image:1.0 phx.ocir.io/weblogick8s/endtoend/domain1-image:1.0

docker push phx.ocir.io/weblogick8s/endtoend/domain1-image:1.0

For the domain deployment, replace the image name in the weblogic-monitoring-exporter/samples/kubernetes/end2end/demo-domains/domain1.yaml file to reflect the created domain image name in the OCIR, for example: 

image: phx.ocir.io/weblogick8s/endtoend/domain1-image:1.0

Deploy the rules for Traefik path routing:

 

Verify that you can access the WebLogic domain console page via a browser, using this URL:

http://e2e-wls.weblogick8s.org/console

Verify that the wls-exporter application was deployed and accessible, using this URL:

http://e2e-wls.weblogick8s.org/wls-exporter

Make sure that you can see the generated metrics by clicking on the ‘metrics’ link from the exporter console or by using this URL:

http://e2e-wls.weblogick8s.org/wls-exporter/mertics

Setting up Prometheus

In the end-to-end sample, we have provided the instructions on how to start Prometheus using a hostPath Persistent Volume (PV) . For an OKE environment, let’s modify the persistent volume and persistent volume claims configuration to use the NFS type:

Like the MySQL, PV, and PVC configurations, edit the provided end-to-end sample YAML file,

weblogic-monitoring-exporter/samples/kubernetes/end2end/prometheus/persistence.yamlto replace hostPath with nfs. Use your Mount Target IP address (for example: 10.7.10.4) and the /FileSystem name (for example: /FileSystem-20190806-2142) for the root of the Persistent Volume

Modify the weblogic-monitoring-exporter/samples/kubernetes/end2end/prometheus/alert-persistence.yaml file in the same way, for Prometheus Alert Manager PV; we use  “/FileSystem-20190806-2140”:

Follow the instructions from the end-to-end  sample for Setting Up Prometheus to install all deployments. Check that all Prometheus pods in the monitoring namespace are running.

In order to be able to access Prometheus dashboard in OKE, redeploy the Prometheus service to switch to the ClusterIP type:

kubectl delete svc prometheus-server -n monitoring

Use this override.yaml file to redeploy it:

kubectl apply -f override.yaml

Finally, create the Ingress for Prometheus by using this command:

Now you can access the Prometheus Dashboard by using this URL:

http://e2e-prom.weblogick8s.org

Go to ‘Status’->’Targets’ to verify that the WebLogic Domain targets are up and running.

Setting up Grafana

Install Grafana using a Helm chart with the customized values.yaml file. Modify the provided end-to-end sample values.yaml file to use the internally created Persistent Volume and Persistent Volume Claim. Also switch to the ClusterIP service. Here is the sample of the modified values.yaml:

Create the Grafana administrative credentials:

kubectl --namespace monitoring create secret generic grafana-secret --from-literal=username=admin --from-literal=password=12345678

Install the Grafana chart:

helm install --wait --name grafana --namespace monitoring --values grafana/values.yaml stable/Grafana

Create Grafana ingress by executing this command:

 

Now you should be able to access the Grafana dashboard by using this URL:

http://e2e-grafana.weblogick8s.org

To complete the Grafana configuration, you need to create the data source and WebLogic dashboard. Here is the example how to do it using the Grafana REST API.
Create the Prometheus data source with the predefined JSON file datasource.json by executing this command:

 

Create the WebLogic Server Dashboard with the predefined JSON dashboard.json file.

Log in to the Grafana dashboard, select WebLogic Server Dashboard and check all the available metrics

Click on 'Servers', 'Web Applications', 'Data Source', 'JMS Services' to view the provided panels:

Setting up a Webhook And Firing Alerts

Follow the instructions from the end-to-end sample to set up a Webhook. The only difference for an OKE environment is to push the created Webhook image to the OCI Registry and use that image in the deployment configuration:

docker tag webhook-log:1.0 phx.ocir.io/weblogick8s/endtoend/webhook-log:1.0

docker push phx.ocir.io/weblogick8s/endtoend/webhook-log:1.0

Modify the image name in  weblogic-monitoring-exporter/samples/kubernetes/end2end/webhook/server.yaml to 

phx.ocir.io/weblogick8s/endtoend/webhook-log:1.0

After verifying that your Wehhook deployment is up and running, follow the instructions from the end-to-end sample to fire Alerts.

Summary

In this blog, we have demonstrated all the required steps to set up and run an end-to-end sample that sets up monitoring for WebLogic domains using Prometheus and Grafana in the OCI Container Engine for Kubernetes Cluster (OKE). We have seen how we can access the administration console, how to set up load balancing and expose applications outside the OCI Container Engine for Kubernetes Cluster, and how to set up , install, and run Prometheus, Grafana, and WebLogic domains, and use Grafana dashboards to visualize the WebLogic Server runtime metrics. You can add to the Grafana dashboards provided in this sample by creating your own dashboards to display different metrics you want to monitor.

 

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.