This end-to-end sample shows you the steps to set up monitoring for WebLogic domains using Prometheus and Grafana. When completed, you'll have Prometheus, Grafana, and WebLogic domains installed, configured, and running. This sample includes Grafana dashboards to visualize the WebLogic Server runtime metrics.
Before getting started, clone the WebLogic Monitoring Exporter project from GitHub. It contains many of the samples and helpers we will need.
git clone https://github.com/oracle/weblogic-monitoring-exporter.git
Here is an overview of the process:
In this sample, we need to create four persistent volumes (PV) and persistent volume claims (PVC) to store data for MySQL, Prometheus server, Prometheus Alert Manager, and Grafana.
Let’s review all these steps in detail.
Log in to the OCI console. Verify that you are using the correct region, for example, “us-phoenix-1” and department, for example“Quality Assurance”.
Select‘Menu’->’Developer Services’->’Container Clusters (OKE)’ and verify that your OKE cluster is up and running.
Select ‘Menu’-> ‘Networking’ -> ‘VNC’. Select the VCN assigned to your cluster and note the CIDR range, for example ’10.7.0.0/16’
From the OCI Menu, select ‘File Storage’ -> ‘Mount Targets’
Click on ‘Create Mount Target’ and select the same Virtual Cloud Network (VNC) as your cluster, for example: mkcluster_vcn.
Click on the created Mount Target link, create three file systems by using the ‘Create Export’ button under the ‘Exports’ section. Create three File Systems, each per desired PV.
From the OCI Menu, select ‘Networking’ -> ‘Virtual Cloud Networks’.
Choose the VCN assigned to your cluster, for example mkcluster_vcn.
On the left of Security Lists, select -> ‘Create Security List’, and create new Security List, for example mkcluster-default-security-list.
Create four Ingresses (two TCP [2048-2050, 111] and two UDP [2048, 111) destination port) and three Egresses (two TCP [2048-2050, 111] and one UDP  source ports). The IP range should match the CIDR range in the corresponding VCN, in our example it is ‘10.7.0.0/16’.
Here is an example of the created Ingress Rules:
Here is an example of the Egress Rules:
Create the Storage Class with provisioner: oracle.com/oci-fss and the OCID from the Mount Target (for example: ocid1.mounttarget.oc1.phx.abcdby27ve3wk6bobuhqllqojxwiotqnb4c2ylefuzabcd)
Here is the example of storageclass.yaml:
Execute this command to create the Storage Class:
kubectl apply -f storageclass.yaml
Modify the file weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/persistence.yaml to use the NFS type for Persistent Volume (PV) . Use your Mount Target IP address (for example: 10.7.10.4) and the File System name (for example: /FileSystem-20190806-2144) for the root of the persistent volume. Use the created Storage Class for PV and PVC (for example: oci-fss).
Here is the example modified weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/persistence.yaml file from the end-to-end sample:
Push mysql image to the OCIR registry by executing these steps:
docker pull mysql
docker tag mysql:latest phx.ocir.io/weblogick8s/db/mysql:5.7
docker login phx.ocir.io
Password: [Auth Key of OCI]
docker push phx.ocir.io/weblogick8s/db /mysql:5.7
Create the OCIR secret:
kubectl create secret docker-registry ocirsecret --docker-server=phx.ocir.io --docker-username=weblogick8s/[OCIRUsername] --docker-password=[OCIRPassword] --docker-email=[OCIRDockerEmail] -n default
Modify the file weblogic-monitoring-exporter/samples/kubernetes/end2end/mysql/mysql.yaml to add a securityContext section to add permission for the user mysql, add OCIR image info, and pull secret:
Follow the instructions from the end-to-end sample to deploy PV and PVC and create MYSQL database, using the modified mysql.yaml and persistence.yaml files.
Follow the instructions from the end-to-end sample to create WebLogic Kubernetes Operator.
In our example, we will use the ‘default’ namespace for the WebLogic domain and ‘monitoring’ namespace for running Prometheus and Grafana. Use helm to install the Traefik load balancer. Use the values.yaml in the sample but set kubernetes.namespaces specifically.
Go to the OCI dashboard -> ‘Networking’ -> ‘LoadBalancers’, select your compartment, and verify that the instance was created and running.
Note the public IP address created for the load balancer, for example ‘188.8.131.52’. Ask your DNS provider to create three host names and match them with this IP address. For our example, we have created:
e2e-wls.weblogick8s.org to access WebLogic Domain
e2e-prom.weblogick8s.org to access Prometheus dashboard
e2e-grafana.weblogick8s.org to access Grafana dashboard
In OKE, we will be using the Traefik load balancer. To run WebLogic Monitoring Exporter with the load balancer, we need to add an extra property into the configuration file. Modify the weblogic-kubernetes-monitoring-exporter/samples/kubernetes/end2end/dashboard/exporter-config.yaml configuration file to add the restPort info. Insert this line above ‘queries’:
restPort: 8001 .
Follow the instructions from the end-to-end sample to create Domain Image.
Push the created Domain Image to OCIR registry:
docker tag domain1-image:1.0 phx.ocir.io/weblogick8s/endtoend/domain1-image:1.0
docker push phx.ocir.io/weblogick8s/endtoend/domain1-image:1.0
For the domain deployment, replace the image name in the weblogic-monitoring-exporter/samples/kubernetes/end2end/demo-domains/domain1.yaml file to reflect the created domain image name in the OCIR, for example:
Deploy the rules for Traefik path routing:
Verify that you can access the WebLogic domain console page via a browser, using this URL:
Verify that the wls-exporter application was deployed and accessible, using this URL:
Make sure that you can see the generated metrics by clicking on the ‘metrics’ link from the exporter console or by using this URL:
In the end-to-end sample, we have provided the instructions on how to start Prometheus using a hostPath Persistent Volume (PV) . For an OKE environment, let’s modify the persistent volume and persistent volume claims configuration to use the NFS type:
Like the MySQL, PV, and PVC configurations, edit the provided end-to-end sample YAML file,
weblogic-monitoring-exporter/samples/kubernetes/end2end/prometheus/persistence.yamlto replace hostPath with nfs. Use your Mount Target IP address (for example: 10.7.10.4) and the /FileSystem name (for example: /FileSystem-20190806-2142) for the root of the Persistent Volume
Modify the weblogic-monitoring-exporter/samples/kubernetes/end2end/prometheus/alert-persistence.yaml file in the same way, for Prometheus Alert Manager PV; we use “/FileSystem-20190806-2140”:
Follow the instructions from the end-to-end sample for Setting Up Prometheus to install all deployments. Check that all Prometheus pods in the monitoring namespace are running.
In order to be able to access Prometheus dashboard in OKE, redeploy the Prometheus service to switch to the ClusterIP type:
kubectl delete svc prometheus-server -n monitoring
Use this override.yaml file to redeploy it:
kubectl apply -f override.yaml
Finally, create the Ingress for Prometheus by using this command:
Now you can access the Prometheus Dashboard by using this URL:
Go to ‘Status’->’Targets’ to verify that the WebLogic Domain targets are up and running.
Install Grafana using a Helm chart with the customized values.yaml file. Modify the provided end-to-end sample values.yaml file to use the internally created Persistent Volume and Persistent Volume Claim. Also switch to the ClusterIP service. Here is the sample of the modified values.yaml:
Create the Grafana administrative credentials:
kubectl --namespace monitoring create secret generic grafana-secret --from-literal=username=admin --from-literal=password=12345678
Install the Grafana chart:
helm install --wait --name grafana --namespace monitoring --values grafana/values.yaml stable/Grafana
Create Grafana ingress by executing this command:
Now you should be able to access the Grafana dashboard by using this URL:
To complete the Grafana configuration, you need to create the data source and WebLogic dashboard. Here is the example how to do it using the Grafana REST API.
Create the Prometheus data source with the predefined JSON file datasource.json by executing this command:
Create the WebLogic Server Dashboard with the predefined JSON dashboard.json file.
Log in to the Grafana dashboard, select WebLogic Server Dashboard and check all the available metrics
Click on 'Servers', 'Web Applications', 'Data Source', 'JMS Services' to view the provided panels:
Follow the instructions from the end-to-end sample to set up a Webhook. The only difference for an OKE environment is to push the created Webhook image to the OCI Registry and use that image in the deployment configuration:
docker tag webhook-log:1.0 phx.ocir.io/weblogick8s/endtoend/webhook-log:1.0
docker push phx.ocir.io/weblogick8s/endtoend/webhook-log:1.0
Modify the image name in weblogic-monitoring-exporter/samples/kubernetes/end2end/webhook/server.yaml to
After verifying that your Wehhook deployment is up and running, follow the instructions from the end-to-end sample to fire Alerts.
In this blog, we have demonstrated all the required steps to set up and run an end-to-end sample that sets up monitoring for WebLogic domains using Prometheus and Grafana in the OCI Container Engine for Kubernetes Cluster (OKE). We have seen how we can access the administration console, how to set up load balancing and expose applications outside the OCI Container Engine for Kubernetes Cluster, and how to set up , install, and run Prometheus, Grafana, and WebLogic domains, and use Grafana dashboards to visualize the WebLogic Server runtime metrics. You can add to the Grafana dashboards provided in this sample by creating your own dashboards to display different metrics you want to monitor.