Did you know that you can now create selenium based test scripts to run automated tests on Oracle Visual Builder built applications? With new oraclejet-webdriver public libraries you can now create test scripts for your application so any breakage or errors occuring due to version upgrade and changes in code can be caught earlier before releasing the application for production.
In this article you will learn how to create and run tests to validate the user interface of the application that you build and deploy using Oracle Visual Builder and create test reports as well. The scope of this demo is to install, configure, write test scripts, run and report the results. The next article would talk about how to integrate such tests into continual integration provided within Visual Builder Studio.
Notes
The tools/technologies used are:
- Node & npm
- Chai – Assertion framework
- Mocha & Mochawesome – Reporting
- OracleJet Webdriver – UI automation library built on top of selenium-webdriver to work with Oracle JET based application
- Type Script – test script writing
- Chrome Browser and respective ChromeDriver
The source files described in this article are available in the attached zip file VBAppTest.zip.
Setup
The setup requires the following components to be installed and configured.
Install Node / npm
Download and install NodeJS and npm on your computer.
Setup Proxy
In case you are behind a firewall and use a proxy for accessing internet, you must set the proxy configuration for npm to work correctly. Use the following commands to change the proxy settings in npm configuration.
npm config set proxy http://username:password@proxy-server-url:port
npm config set https-proxy http://username:password@proxy-server-url:port
npm config set noproxy localhost,127.0.0.1,.myorg.com
Create Test Folder And package.json
Create a local folder where you will be creating the test scripts. Let's call it VBAppTest. In this folder, create a package.json file and copy the following contents in it. These are the dependencies to write and run basic test, a starting point from which you can diverge and change dependencies and their versions.
{
"name": "VBAppTest", // give a name to your test app
"version": "22.0.4", // version of Visual Builder that VBApp is running
"description": "Execute functional tests on VBApp",
"devDependencies": {
"@oracle/oraclejet-webdriver": "11.1.4", // exact oraclejet-webdriver version corresponding to JET version your Visual Builder application is using
"@types/chai": "^4.1.7",
"@types/fs-extra": "^8.0.1",
"@types/mocha": "^5.2.7",
"@types/node": "latest",
"@types/selenium-webdriver": "^4.0.1",
"chai": "^4.2.0",
"chai-json-pattern": "^1.1.0",
"fs-extra": "^8.1.0",
"grunt": "^1.0.0",
"grunt-contrib-clean": "^2.0.0",
"grunt-contrib-compress": "1.3.0",
"grunt-contrib-concat": "^1.0.0",
"grunt-contrib-copy": "^1.0.0",
"grunt-downloadfile": "2.1.1",
"grunt-shell": "^3.0.1",
"load-grunt-tasks": "^3.5.0",
"mocha": "^7.1.2",
"mocha-junit-reporter": "^1.23.1",
"mocha-multi-reporters": "^1.1.7",
"mochawesome": "^4.1.0",
"selenium-webdriver": "^3.6.0",
"serial-mocha": "0.0.4",
"ts-node": "^8.3.0",
"typescript": "~3.8.3"
},
"main": "Gruntfile.js",
"scripts": {
"test": "mocha"
},
"author": "",
"license": "ISC"
}
Note: Please make sure to have the exact version of @oracle/oraclejet-webdriver in package.json matching the version of oraclejet your Visual Builder Application is using.
Create config.json
In the same folder VBTestApp create a new file config.json and add the following contents in it. This defines mochaswesome as the report creating tool to run along with mocha.
{
"reporterEnabled": "mocha-junit-reporter,mochawesome",
"mochaJunitReporterReporterOptions":
{
"mochaFile": "./junit-report/test-results.xml"
}
}
Create Gruntfile.js
In the same VBTestApp folder create a Gruntfile.js file and add the following contents in it. This is used to build and execute the test tasks. Though the tests can be executed from command line but using the Gruntfile you can customize the test execution as needed.
module.exports = function (grunt) {
var url = grunt.option('url');
console.log("will open: " + url);
const testsToRun = grunt.option('tests') !== undefined ? grunt.option('tests') : 'Test*.spec.ts';
require('load-grunt-tasks')(grunt);
var executeMocha = function () {
// the currently working directory here is the VBAppTest folder
return `./node_modules/.bin/mocha -r ts-node/register --reporter mocha-multi-reporters --reporter-options configFile=config.json ./tests/mocha-test-setup.ts ./tests/${testsToRun} --timeout=60000`;
};
grunt.initConfig({
vars: {
selenium: 'selenium',
output_tests: './'
},
shell: {
run_local_mocha: {
command: executeMocha,
options: {
async: false,
stdout: true,
stderr: true,
failOnError: false,
execOptions: {
cwd: '<%= vars.output_tests %>',
env: {
APP_URL: url
}
}
}
}
}
});
grunt.registerTask('default', ['shell:run_local_mocha']);
};
Install Grunt
Open a command terminal. Change the folder to VBTestApp folder. Execute the following commands to install grunt.
npm install -g grunt-cli
Setup ChromeDriver
Install chromedriver matching with the installed chrome version. Use the following command to install chromedriver.
npm install chromedriver --chromedriver-cdnurl=http://chromedriver.storage.googleapis.com --chromedriver_version=LATEST
Note: You can manually download the driver from https://chromedriver.chromium.org/downloads.
Setup /tests folder
In VBTestApp folder create a new folder called tests. This is the folder where all test files will be created. Create a new file mocha-test-setup.ts and copy the following contents in it. This is a setup for all tests, configures timeouts, remote driver, and captures screenshots. This is all optional and tests could be written and run without this but sooner or later you may need these features.
/**
* This test should be run as the first test so that DriverManager can be
* configured from Mocha before other tests are run. Two things are configured here:
* - Chrome se default web driver for either local or remote execution
* - browser screenshot gets capture when test fails
*/
import * as fs from "fs";
import * as fsExtra from "fs-extra";
import * as path from "path";
import { Builder, Capabilities, WebDriver } from "selenium-webdriver";
import {
DriverManager as dm,
ScreenshotManager as sm,
} from "@oracle/oraclejet-webdriver";
// initialize Chrome default WebDriver:
var capabilities = Capabilities.chrome();
dm.registerConfig({
capabilities:capabilities,
timeouts: {
pageLoad: 60000
}
});
var builder: Builder = new Builder()
.withCapabilities(capabilities);
if (process.env.USE_REMOTE_DRIVER) {
console.log("using sever "+"http://"+process.env.USE_REMOTE_DRIVER+":4444/wd/hub");
builder = builder.usingServer("http://"+process.env.USE_REMOTE_DRIVER+":4444/wd/hub");
}
dm.setBuilder(builder);
// setup screenshot capturing:
const addContext = require("mochawesome/addContext");
let driver: WebDriver;
const screenshotRoot = path.resolve(__dirname, "..");
const screenshotDir = `${screenshotRoot}${path.sep}mochawesome-report${path.sep}screenshots`;
if (String("@captureScreenshot@") === "true") {
sm.set(sm.create(screenshotRoot + path.sep + "mochawesome-report"));
}
try {
fsExtra.mkdirpSync(screenshotDir);
} catch (err) {
console.error(err);
console.error(err.stack);
}
/**
* Global hook handling before each test.
*/
beforeEach(async function () {
const suiteFolder =
screenshotDir
path.sep +
this.currentTest.parent.title.replace(/[^a-z0-9]/gi, "_");
try {
if (!fs.existsSync(suiteFolder)) fs.mkdirSync(suiteFolder);
} catch (err) {
console.error(err);
console.error(err.stack);
}
const TestFolder =
screenshotDir
path.sep +
this.currentTest.parent.title.replace(/[^a-z0-9]/gi, "_") +
path.sep +
this.currentTest.title.replace(/[^a-z0-9]/gi, "_");
try {
if (!fs.existsSync(TestFolder)) fs.mkdirSync(TestFolder);
} catch (err) {
console.error(err);
console.error(err.stack);
}
const screenshotManager = sm.get();
screenshotManager.beforeTest(
this.currentTest.parent.title,
this.currentTest.title
);
});
/**
* Global hook handling each test, generating screenshot when that test fails as
* /screenshotFolder//testname_AT_timestamp.jpeg
*/
afterEach("take screenshot on failure", async function () {
driver = await dm.getDriver();
//Purge screeshots for passed tests
const screenshotManager = sm.get();
screenshotManager.afterTest(this.currentTest.state === "passed");
if (this.currentTest.state !== "passed") {
const suiteFolder = this.currentTest.parent.title.replace(/[^a-z0-9]/gi, "_");
const screenshotF = screenshotDir + path.sep + suiteFolder;
try {
fsExtra.mkdirpSync(screenshotF);
} catch (err) {
console.error(err);
console.error(err.stack);
}
const imageFileName =
this.currentTest.title.replace(/[^a-z0-9]/gi, "_") + ".jpeg";
const screenshot = await driver.takeScreenshot();
try {
fs.writeFileSync(
screenshotF + path.sep + imageFileName,
screenshot,
"base64"
);
} catch (err) {
console.error(err);
console.error(err.stack);
}
const imageWithRelativePath =
"screenshots" + path.sep + suiteFolder + path.sep + imageFileName;
addContext(this, imageWithRelativePath);
}
//Purge the empty test folder
const cleanTestFolder =
screenshotDir
path.sep +
this.currentTest.parent.title.replace(/[^a-z0-9]/gi, "_") +
path.sep +
this.currentTest.title.replace(/[^a-z0-9]/gi, "_");
try {
if (fs.existsSync(cleanTestFolder)) {
const files = fs.readdirSync(cleanTestFolder);
if (!files.length) fs.rmdirSync(cleanTestFolder);
}
} catch (err) {
console.error(err);
console.error(err.stack);
}
//Purge the empty suite folder
const cleanSuiteFolder =
screenshotDir
path.sep +
this.currentTest.parent.title.replace(/[^a-z0-9]/gi, "_");
try {
if (fs.existsSync(cleanSuiteFolder)) {
const files = fs.readdirSync(cleanSuiteFolder);
if (!files.length) fs.rmdirSync(cleanSuiteFolder);
}
} catch (err) {
console.error(err);
console.error(err.stack);
}
});
Create another file utils.ts in same folder and copy the following contents in it. This is a placeholder for utility functions shared by all tests.
import {WebDriver} from 'selenium-webdriver';
class Utils
{
someCommonUtilFunction = async function(driver: WebDriver, foo: string)
{
};
}
export const TestUtils = new Utils();
Before tests can be run, all dependencies specified in package.json needs to be installed. To do that run:
npm install
This command needs to be run only once after dependencies were changed in package.json.
Now execute the following command to run the tests:
grunt
At this point the test would run and report an error that there are no test script files to execute.
Quick Note On App To Test
The Visual Builder app url that will be tested using the tests is as follows. It contains an empty table that gets a new row on click of the Insert Friends button.
https://preview-vbdemo.builder.ocp.oraclecloud.com/ic/builder/rt/functional_tests_example_master/1.0/webApps/example/
Creating First Test Script
In the tests folder create a new file TestScript.spec.ts and copy the following contents in it. Note that the script file name shall start with Test**. You can refer to OracleJet WebDriver for more details on the classes and options available in test script writing.
// imports
import { expect, assert } from 'chai';
import { Builder, By, until, Key, WebElement, WebDriver } from 'selenium-webdriver';
import ojwd, { DriverManager as dm } from '@oracle/oraclejet-webdriver';
import { ojButton, OjButton, ojTable, OjTable, ojTextArea, ojSelectOne, OjSelectOne, OjWebElement } from '@oracle/oraclejet-webdriver/elements';
import { TestUtils } from './Utils';
describe('Test Sample App', function () {
// define variables for table and webdriver
let table: OjTable,
driver: WebDriver;
// "before" function will run always before each test
// and is good place to pre-initialize everything for individual tests:
before(async function () {
// fetch web driver insstance:
driver = await dm.getDriver();
console.log("Before loading index.html");
// load test page:
await ojwd.get(driver, process.env.APP_URL);
console.log("After loading index.html");
// resize browser for your testing needs
await driver.manage().window().setSize(1680, 1050);
// locate page root element:
const pageElement = await driver.findElement(By.css('html')) as OjWebElement;
console.log("Before pageElement.whenReady()");
// and wait for it to be ready:
await driver.wait(pageElement.whenReady());
});
// after web page of app loads, check if the table exists on the web page.
it('Verify table exists', async function () {
table = await ojTable(driver, By.id("addresses")); // addresses is the id of the table component
console.log("After addresses table is loaded");
expect(table).not.null; // assert with expect if table is not found
});
// click on insert button and see if new row gets added to table
it('Verify Insert Button Adds A Row In Table', async function () {
const insertButton = await ojButton(driver, By.id("insertButton")); // insertButton is the id of the Insert button
await insertButton.doAction(); // click on the insert button
await table.whenReady(); // wait for things to load
// check that table has exactly one row
let rowCount = await driver.findElements(By.className("oj-table-body-row"))
.then((elements: OjWebElement[]) => elements.length);
expect(rowCount).to.equal(1);
console.log("row count in table is 1 and expected");
});
after(async function () {
dm.releaseDriver(driver); // release the driver after tests are done executing
});
});
Running Your Test
In the command terminal, change location to VBAppTest folder. Run the following command. The url parameter is the location of the web app being tested.
grunt -url=https://preview-vbdemo.builder.ocp.oraclecloud.com/ic/builder/rt/functional_tests_example_master/1.0/webApps/example/
Note: The application being tested here has anonymous user access. In case your application requires a sign-in, you would create a util function with hard coded username and password which waits for sign-in page to load, enters the username and password, clicks sign-in button and waits for main Visual Builder application page to load. This function should be called from before() function in each test.
The tests specified in TestScript.spec.ts are executed. You should see two tests passing.
will open: https://preview-vbdemo.builder.ocp.oraclecloud.com/ic/builder/rt/functional_tests_example_master/1.0/webApps/example/
Running "shell:run_local_mocha" (shell) task
Test Sample App
Before loading index.html
After loading index.html
Before pageElement.whenReady()
After table is loaded
¿ Verify table exists (126ms)
row count in table is 1 and expected
¿ Verify Insert Button Adds A Row In Table (1190ms)
2 passing (11s)
[mochawesome] Report JSON saved to /Users/vinagarw/Desktop/VBAppTest/mochawesome-report/mochawesome.json
[mochawesome] Report HTML saved to /Users/vinagarw/Desktop/VBAppTest/mochawesome-report/mochawesome.html
Generated Test Report
The test execution report is created in mochawesome.html. In case of any error in the test execution a screenshot is also taken and added into the report.
Test Failure Scenario
To see what results are generated when a test fails, lets make a change in TestScript.spec.ts as follows.
expect(rowCount).to.equal(1); // originally insert button click expects the table to have one row
change this expectation to:
expect(rowCount).to.equal(0); // intentionally making the test to fail
Run the test again as specific above. You should see two tests getting executed but one failing.
will open: https://preview-vbdemo.builder.ocp.oraclecloud.com/ic/builder/rt/functional_tests_example_master/1.0/webApps/example/
Running "shell:run_local_mocha" (shell) task
Test Sample App
Before loading index.html
After loading index.html
Before pageElement.whenReady()
After addresses table is loaded
¿ Verify table exists (125ms)
1) Verify Insert Button Adds A Row In Table
1 passing (11s)
1 failing
1) Test Sample App
Verify Insert Button Adds A Row In Table:
AssertionError: expected 1 to equal +0
+ expected - actual
-1
+0
at Context. (tests/TestScript.spec.ts:40:29)
at step (tests/TestScript.spec.ts:33:23)
at Object.next (tests/TestScript.spec.ts:14:53)
at fulfilled (tests/TestScript.spec.ts:5:58)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
[mochawesome] Report JSON saved to /Users/vinagarw/Desktop/VBAppTest/mochawesome-report/mochawesome.json
[mochawesome] Report HTML saved to /Users/vinagarw/Desktop/VBAppTest/mochawesome-report/mochawesome.html
The generated test report mochawesome.html reports the error along with a screen shot.
Next
The next thing you would do is to plug in these test scripts into your build system so any changes in the application can be verified and validated before going production. This is explained in another blog post Integrating Test Scripts Into Build System In VBS.