Here's Part 3 of our series on this year's Hands on Lab on setting up a simple DevOps toolchain on Oracle Solaris. In the previous posts Part 1 how to install Oracle WebLogic Server on Oracle Solaris and Part 2 how connect Maven to the Weblogic instance with a plugin. In this post we'll show how to install and configure Jenkins as a build manager to allow for a completely automatic build and deploy of your Java application to the WebLogic instance. This will help complete the build server part of the toolchain and get closer to making our install look like this:
One of the other smaller things we'll need to add to the build server is the Git repository origin of our Java application to ensure that we can signal Jenkins when any updates or changes to the Java application have been committed and pushed. We want to use this repository because we can then later also easily create a clone somewhere else and use that as a source of updates.
In this blog we essentially have 3 subsections. The first is on how to install Jenkins and use Oracle Solaris features like the Service Management Facility (SMF) to keep it always on and secure. The second is getting the application into a Git repository and then push it to a central master repository called origin. And the third section will show how to create a project in Jenkins that will then take the code from this repository, build it, and then deploy it to the WebLogic instance using Maven. This last section will heavily leverage the work we've done in the previous two posts to get that backend ready.
Jenkins is a very popular open source build automation server. Because it has a huge following many plugins have been written to it and pretty much anything can be built, tested, and deployed with it. What's also really nice about it is that it's written in Java so therefore it and it's plugins will work on many different platforms. This includes the many different flavors of Linux as well as Oracle Solaris. It's actually one of the tools we use internally to automate building new versions of Solaris and will work nicely in our DevOps example.
The way it's normally installed is by starting the jenkins.war file with Java and then going through the procedures where it essentially unpacks itself in your directory. But in our case we want to leverage Solaris SMF to create a Jenkins service that we can turn on and off and that will keep Jenkins running over reboots and failures. To do this we'll have to create a SMF manifest (an XML file that describes the service) and then load it into SMF. The nice thing about this is that you can for example define as which user the service will run and what it's allowed to do. This greatly helps you to run services securely as you can create a user that nobody would ever log in as that has very limited privileges and even start the service with even more restrictive privileges. In our case we're going to use some of these features.
So first, as root in the j-m_zone download the latest version of Jenkins from their download site and copy it into the /opt directory:
root@j-m_zone:~# wget http://mirrors.jenkins.io/war-stable/latest/jenkins.war --2017-09-12 06:35:22-- http://mirrors.jenkins.io/war-stable/latest/jenkins.war ... jenkins.war 100%[==========================================================>] 67.35M 2.18MB/s in 31s 2017-09-12 06:35:53 (2.18 MB/s) - ‘jenkins.war’ saved [70620203/70620203] root@j-m_zone:~# cp jenkins.war /opt/.
We copy it to /opt because that's a standard place to put optional software but in reality because Jenkins will download and unpack itself inside JENKINS_HOME this is step is optional. As long as you know where it it so that you can point the SMF manifest in the right location (and user jenkins can read the file).
Now we'll create the SMF manifest. To make this easy there's a command called svcbundle that will create the manifest from a template after which we only have to make a few tweaks to get the manifest in the form we want it. So first run svcbundle:
root@j-m_zone:~# svcbundle -o jenkins.xml -s service-name=site/jenkins -s model=child -s instance-name=default -s start-method="java -jar /opt/jenkins.war"
This will create a file called jenkins.xml in your current working directory. And we can finish the configuration by editing this XML file and replacing this line:
<instance enabled="true" name="default"/>
With these lines:
<instance enabled="true" name="default"> <method_context> <method_credential group='nobody' privileges='basic,net_privaddr' user='jenkins' /> <method_environment> <envvar name='JENKINS_HOME' value='/export/home/jenkins' /> </method_environment> </method_context> </instance>
These extra lines tell SMF to start Jenkins as user jenkins, group nobody, with basic privileges plus the privilege to bind to a privileged port (below 1024), and that it should set JENKINS_HOME to /export/home/jenkins. This way even though it gets started by root the Jenkins process is limited in this way.
Finally we can validate the manifest, load it into SMF, and check it's loaded and running:
root@j-m_zone:~# svccfg validate jenkins.xml root@j-m_zone:~# cp /root/jenkins.xml /lib/svc/manifest/site/. root@j-m_zone:~# svcadm restart manifest-import root@j-m_zone:~# svcs -p jenkins STATE STIME FMRI online 12:06:21 svc:/site/jenkins:default 12:06:21 16466 java
This shows that the service is now up and running with a Java process with PID 16466.
Now we can go to the Jenkins BUI to finalize the install. It will be on port 8080 of the j-m_zone. First you'll see a screen asking for a code to verify yourself as the administrator:
Next we'll configure Jenkins not to install any additional plugins, this will save a lot of time and we'll install the plugins we need later. Select te option "Select plugins to install", then deselect all plugins and continue:
It will then ask for an admin user, please choose a username and password, we use "jenkins" and "solaris11", but you can use what you want of course . Then click "Save and Finish" and now it's ready, click "Start using Jenkins":
Jenkins will restart, it will ask you to log in, and show it's main screen:
At this point Jenkins is up and running and will remain so until you choose to stop it with svcadm disable jenkins which tells SMF to stop the service. svcadm enable jenkins will tell SMF to start it. We'll not change it's state as we want to keep on using it.
Now we'll add the plugins that we need. Click on "Manage Jenkins" and then on "Manage Plugins":
Then click on the "Available" tab and in the "Filter" search on the right fill in "Maven". This should give you a whole list of option, scroll down and select the "Maven integration plugin":
Then go back to the filter and fill in "Git". This also will give quite a long list and select the "Git plugin" from the list:
To start downloading the plugins click on the button "Download now and install after restart" at the bottom of the screen. This should kick off the install the selected plugins and their the plugins they depend upon:
At the very bottom of the list there's an option to restart Jenkins once all the plugins are downloaded, check the box to do this. Now wait, either by once in a while refreshing the screen or by enabling the auto refresh button in the top right hand corner. Once the main Jenkins screen is back you're ready to do some final configuration steps.
Now we go back to the Jenkins BUI and we'll configure Jenkins to use this copy of Maven as well as the local JDK. Go back to the "Manage Jenkins" screen, and this time click on the "Global Tool Configuration" link:
Now go to the JDK section of this page and click on "Add JDK", fill in a name like "System Java", uncheck "Install automatically" and for "JAVA_HOME" fill in "/usr/jdk/latest":
Then scroll down to the Maven section click on "Add Maven" and add the local Maven installation by unchecking "Install automatically" again and filling in "System Maven" and "/opt/apache-maven-3.2.5/" in the fields for "Name" and "MAVEN_HOME":
Now click on the "Save" button at the bottom to save these configurations.
Now we have Jenkins up and running we need to first put our application (in section 6 of Part 2) into a Git repository and then push it to a central origin repository Jenkins is going to pull from. We could also have Jenkins pull from our local repository but when we want to have a remote development environment pull the code and then push changes it's better to already use a central repository.
First initialize the repo and add the Java app to it:
jenkins@j-m_zone:~/mywebapp$ git init Initialized empty Git repository in /export/home/jenkins/mywebapp/.git/ jenkins@j-m_zone:~/mywebapp$ git status On branch master Initial commit Untracked files: (use "git add <file>..." to include in what will be committed) pom.xml src/ target/ nothing added to commit but untracked files present (use "git add" to track) jenkins@j-m_zone:~/mywebapp$ git add * jenkins@j-m_zone:~/mywebapp$ git status On branch master Initial commit Changes to be committed: (use "git rm --cached <file>..." to unstage) new file: pom.xml ... new file: target/maven-archiver/pom.properties jenkins@j-m_zone:~/mywebapp$ git config --global user.email "email@example.com" jenkins@j-m_zone:~/mywebapp$ git config --global user.name "Your Name" jenkins@j-m_zone:~/mywebapp$ git commit -a -m "My first commit." [master (root-commit) 92595fd] My first commit. 16 files changed, 10414 insertions(+) create mode 100644 pom.xml ... create mode 100644 target/maven-archiver/pom.properties
At this point the Java app has been added to the local repository, but as said above, in order for Jenkins to easily use it we're going to create the repository origin, in this case on the same system. We're choosing to put it on the same system but in most cases it will be remote, or possibly even on a site like GitHub.
To create origin we create a bare repository in the home directory of a new git-repo user for which we can set a new password so we can easily access it from another system. First exit from being user jenkins and then create the new user, set the password, and create the bare repository:
jenkins@j-m_zone:~$ exit logout root@j-m_zone:~# useradd -m git-user 80 blocks root@j-m_zone:~# passwd git-user New Password: Re-enter new Password: passwd: password successfully changed for git-user root@j-m_zone:~# su - git-user Oracle Corporation SunOS 5.11 11.3 July 2017 git-user@j-m_zone:~$ git init --bare mywebapp.git Initialized empty Git repository in /export/home/git-user/mywebapp.git/
Now we can go back to user jenkins and set this as the origin of out Java app repository:
git-user@j-m_zone:~$ exit logout root@j-m_zone:~# su - jenkins Oracle Corporation SunOS 5.11 11.3 July 2017 jenkins@j-m_zone:~$ cd mywebapp jenkins@j-m_zone:~/mywebapp$ git remote add origin ssh://git-user@localhost:/export/home/git-user/mywebapp.git jenkins@j-m_zone:~/mywebapp$ git push origin master Password: Counting objects: 35, done. Compressing objects: 100% (25/25), done. Writing objects: 100% (35/35), 43.04 KiB | 0 bytes/s, done. Total 35 (delta 4), reused 0 (delta 0) To ssh://git-user@localhost:/export/home/git-user/mywebapp.git * [new branch] master -> master
At this point the Java app has been pushed to origin and we can start configuring a Jenkins project.
Now Jenkins is up and running and we also have Maven and WebLogic configured and working we can move towards creating the first Jenkins project that will build our Java app and deploy it to our WebLogic Server instance automatically every time we commit a new change to the Java app.
To do this we're going to create a new project by going to the main screen in Jenkins and click on "create new jobs" to start a new project:
Then as the item name fill something in like "MyWLSWebApp" and click on "Maven project" and click "OK":
Under "Source Code Management" select the "Git" and fill in "/export/home/git-user/mywebapp.git":
Note: We're using the direct file path because it's easy and we can. Normally you'd use something like
sshand set up credentials. For the direct file path to work of course the repository needs to at least be readable by user jenkins, which by default it should be.
And under "Build Triggers" make sure "Poll SCM" is checked:
Then under "Build" fill in "pre-integration-test" in the "Goals" field:
Finally click "Save" and we've configured the first project.
At this point Jenkins is ready but we need to make sure that it gets notified when there's a new update to the Git repository. We do this with an "SCM hook" which is part of Git. The hook we're going to use is called the post-receive hook. The way this works is we create a new file in the hooks directory of the origin Git repository, this file contains a script that is invoked after Git receives new code.
The script contains a curl command that in turn pokes Jenkins that there's new code:
So, as user git-user go to the hooks directory and create the post-receive file, not forgetting to make sure the new script is executable:
git-user@j-m_zone:~$ cd mywebapp.git/hooks/ git-user@j-m_zone:~/mywebapp.git/hooks$ vi post-receive git-user@j-m_zone:~/mywebapp.git/hooks$ cat post-receive #!/usr/bin/bash curl http://localhost:8080/git/notifyCommit?url=file:///export/home/git-user/mywebapp.git git-user@j-m_zone:~/mywebapp.git/hooks$ chmod +x post-receive
Go back to the directory that holds the Java apps code and make a small edit to the source code of the src/main/webapp/index.xhtml file. For example change the heading from:
<h2>My Basic Webapp</h2>
and then commit the change to Git and push the new code to origin:
git-user@j-m_zone:~/mywebapp.git/hooks$ exit logout root@j-m_zone:~# su - jenkins Oracle Corporation SunOS 5.11 11.3 July 2017 jenkins@j-m_zone:~$ cd mywebapp/ jenkins@j-m_zone:~/mywebapp$ vi src/main/webapp/index.xhtml jenkins@j-m_zone:~/mywebapp$ git commit -a -m "My first change." [master 8bbe4cc] My first change. 1 file changed, 1 insertion(+), 1 deletion(-) jenkins@j-m_zone:~/mywebapp$ git push origin master Password: Counting objects: 6, done. Compressing objects: 100% (5/5), done. Writing objects: 100% (6/6), 512 bytes | 0 bytes/s, done. Total 6 (delta 2), reused 0 (delta 0) remote: % Total % Received % Xferd Average Speed Time Time Time Current remote: Dload Upload Total Spent Left Speed remote: 100 117 100 117 0 0 1449 0 --:--:-- --:--:-- --:--:-- 1500 remote: Scheduled polling of MyWLSWebApp remote: No Git consumers using SCM API plugin for: file:///export/home/git-user/mywebapp.git To ssh://git-user@localhost:/export/home/git-user/mywebapp.git 4efe4bd..0660e8b master -> master
And if you quickly go back to the Jenkins screen you'll see that it's picked up the new build:
And once it's done you can go back to the Java App page and refresh it, once you've done that it should now show the new heading:
This verifies the build chain is working. At this point the build part of your toolchain is up and running and any change you make to your code that you push to the origin repository will then automatically be built and deployed to the WebLogic instance.
This ends the blog on installing and configuring Jenkins on Oracle Solaris. Check in later for the last part of this series where we install NetBeans as the IDE as the development front-end to this toolchain thereby completing the picture.