In the previous article Automated Testing Of Visual Builder Application we learnt how to create and run test scripts to validate the user interface of the application that you build and deploy using Oracle Visual Builder. The next logical step is to plug in these test scripts into your build system so any changes in the application can be verified and validated before going production.

In this article we will learn how to make use of a docker container with an Selenium based Chrome Image to deploy and run the test scripts on the web application in headless mode using the Oracle Visual Builder Studio architecture and also configure the tests running for any changes happening in the source code of the web application.

Notes

The tools/technologies used are:

Setup

Selenium Chrome Image

We will use the image selenium/standalone-chrome:4.5.0 for running the tests. This docker image contains selenium server 4.5.0 and Chrome 106 and allows to execute functional tests in headless mode.

Create Build Job To Run The Tests

To run the tests on docker, you will need to create a build job.

  • In the Builds section, create a new job called run-tests-on-docker with any executor template which is capable of running docker.
  • In the Git section, add a Git operation and select functional-tests-example.git. Select master as the branch.
  • In the Steps section add a Unix Shell task with following code. 
    
export HOSTNAME=localhost
cd tests
npm install
 
// this is the App URL that we built and deployed using Oracle Visual Builder earlier.
grunt docker --url="https://preview-vbdemo.builder.ocp.oraclecloud.com/ic/builder/rt/functional_tests_example_master/1.0/webApps/example/"
cd ..

Modify Gruntfile And Test Script Files

The test script files need to be updated to make use of the docker for running the test. Open the Gruntfile available under /VBTestApp folder and update it to the following.


module.exports = function (grunt) {
  
  var url = grunt.option('url');
  console.log("will open: " + url);
  
  const testsToRun = grunt.option('tests') !== undefined ? grunt.option('tests') : 'Test*.spec.ts';
  
  require('load-grunt-tasks')(grunt);
  
  var executeMocha = function () {
    // the currently working directory here is the VBAppTest folder
    return `./node_modules/.bin/mocha -r ts-node/register --reporter mocha-multi-reporters --reporter-options configFile=config.json ./tests/mocha-test-setup.ts ./tests/${testsToRun}  --timeout=60000`;
  };
  
  grunt.initConfig({
    vars: {
      selenium: 'selenium',
      output_tests: './'
    },
    shell: {
      run_local_mocha: {
        command: executeMocha,
        options: {
          async: false,
          stdout: true,
          stderr: true,
          failOnError: false,
          execOptions: {
            cwd: '<%= vars.output_tests %>',
            env: {
              APP_URL: url
            }
          }
        }
      },
    remote_mocha_tests:{
            command: executeMocha,
            options:{
                async:false,
                stdout:true,
                stderr:true,
                failOnError:false,
                execOptions:{
                    cwd:'<%= vars.output_tests %>',
                    env: {
                      APP_URL: url,
                      USE_REMOTE_DRIVER: process.env.HOSTNAME
                    }
                }
            }
        },
        remote_start_docker:{
            command:function(){
                return `docker run -d -p 4444:4444 --name "test-server" --shm-size=2g  selenium/standalone-chrome:4.5.0`;
            }
        }
    }
  });
  grunt.registerTask('default', ['shell:run_local_mocha']);
  grunt.registerTask('docker', ['shell:remote_start_docker', 'shell:remote_mocha_tests']);
};

Also note that the file mocha-test-setup.ts under /VBAppTest/tests folder has a condition set for using the remote docker server to run the tests based on the flag USE_REMOTE_DRIVER that is set in the Gruntfile under remote_mocha_tests task.


.....
 
var builder: Builder = new Builder()
    .withCapabilities(capabilities);
if (process.env.USE_REMOTE_DRIVER) {
    console.log("using sever "+"http://"+process.env.USE_REMOTE_DRIVER+":4444/wd/hub");
    builder = builder.usingServer("http://"+process.env.USE_REMOTE_DRIVER+":4444/wd/hub");
}
dm.setBuilder(builder);
.....

Run Tests On Docker

Run the job run-tests-on-docker now using the Build Now button. This will run the tests in docker using the remote_start_docker, remote_mocha_tests tasks defined in Gruntfile and the results will be shown in the Builds page.

To make the tests run automatically after any code changes in functional-tests-example.git, enable the checkbox Automatically perform build on SCM commit in the Git section of the job run-tests-on-docker.

Debugging Tests In Case Of Failures

If headless functional tests are failing and it is not obvious why from the failed test outcome (for example error says that test timed out after 60000ms) it is possible to switch to debug version of docker image which contains VNC server. When you start this debug docker image you will need to have VNC client installed and while the test is running you can connect to it and watch test running in chrome browser in the docker. You can interact via VNC with the chrome browser, open dev tools, debug. The debug docker image can be started by using the docker run command as given in remote_start_docker task in Gruntfile with an additional -p 5900:5900 parameter. All you need to do is connect to VNC server using the IP address and the port 5900 as in this example. Port is configured in the grunt starting the debug docker version and IP address can be found in the Build Executors under VBS Organsation section. You will need to click on Detail of the executor to see the Public IP.

Configure Post Build Steps

Ideally one would like to make the build job fail in case of failed tests. To achieve this visit the After Build section of the job run-tests-on-docker. Add Artifact Achiver task and specify "tests/mochawesome-report/**, audit.json, audit.txt" in Files to archive field. Also add JUnit Publisher task and specify "tests/junit-report/test-results.xml" in the Include JUnit XMLs field and enable the checkbox "Fail the build on failed tests". Save the changes.

Thats it. Now your application is configured to run the tests and check for any breakages as soon as any code changes happen.