Mittwoch Mai 27, 2009

Best Practices For Software Test Code

Best Practices For Software Test Code

Author: Joerg Sievers – Reviewers: Helge Delfs, Venkatesh Devarajan, Arun Kumar S, Srinivas Kg, Mario Schroeder, Elizabeth Matthis – Revision 10 – Last change: 1. July 2009

Test code is no production code

A well skilled Java developer isn't also a well skilled Java test code developer. The focus of a tester is different to a programmer and also automated test cases are different from methods of production code. They can be written in the same language and they maybe look alike but their goals are different. UI-tests should be more like an acceptance test because it is the same view a customer also has.

Explanation: In desktop environments the production code goal could be not to crash under the most worst circumstances. In test code you should stop working (=controlled, well reported stop of the test case execution) if an error occurs because the subsequent steps would report undefined results. It is easier to get an overview what happened if you only see one, the first error and not hundreds more which depended on the first one.

In web environments the production code goal could be to cloud the executed code in the browser. In test cases the feedback should be best traceable to reproduce simple steps if an error occurs.

Set preconditions

Be sure that a evadable default in the application is really the default in all environments. Otherwise you have to set the settings you need in front of the test case (preconditions). [OOO1]

An automated test case should not depend on other test cases

If you need preconditions to execute a test scenario you have to put in the precondition phase which always runs in front of any test case.

A test case needs to be re-runnable

No test pieces which being created while the test is running should hinder a re-run of the test case. If so, you have to remove such pieces in the precondition phase (=”running it always on the greenfield”). You should write automated software test not user scenario tests with undefined states – that would be another type of tests. Every testcase should have 3 sections: Entry, Action, CleanUp. Entry section will be useful to do any specific pre-settings, Action section is where actual testcase runs, CleanUp section is to undo all the actions performed by that particular testcase.

Use global tool settings and routines instead of own created

If an output path, a timeout, an environment variable, or also a helper method is defined globally it should be used instead of creating them in the test case on your own. This is the same for production and for test code to optimize the maintainability.

Explanation: Especially timeouts often defined in many test cases instead of using divisions or multiplications of the global timeout which is set in the most test tools.

Write test cases readable from top-level like use-cases

In production code the object-oriented approach is best practice. In test code, the focus is to simplify the work-flow of steps executed multiple times. The helper classes and abstraction layers should be made based on this focus but always it should be possible to read the important steps to get a test case done on the top-level and not in sub-routines.[MM01]

Useful feedback

A test case is a step-by-step instruction of a work-flow. In production code, feedback is not given if an action was successful. In test code you should do that more often to be able for others to reproduce these steps manually – and not only those which failed. [MF01]

Use defined prefixes for variables

Modern Integrated Development Environments (IDE) are able to show the type of a variable but for a better reading it makes sense to use defined prefixes. [OOO1] If there is no definition made the “ Hungarian Notation” [ Link English] [ Link German 1] can help.


prefix

type

i or n

integer

i

long

d

single

d

double

s

string

b

boolean

The following prefixes must be combined with an additional prefix:

a

array

g

global (from global.inc)

l

list (for variables declared for use of functions from t_list.inc)

Other naming schemes:

t

testcase

s

sub

f

function


Use clear indicators to show when an action is complete.

When an action has been started and you are waiting for the results, you have to have clear indicators for automated software to show when the action has been completed successfully. If you do not have such an indicator, then you have to use "timeouts", but they are always only a last resort.

For example: You want to load a document. To show that the document has been loaded, the status bar of the application changes or certain buttons become active.

Bibliography

OOO1: Thorsten Bosbach, Helge Delfs, Joerg Sievers, Joerg Skottke, OpenOffice.org Cookbook, 2004

MM01: Marc Michaelis, Boon and Bane of GUI Test Automation, 2008

MF01: Mark Fewster, Dorothy Graham, Software Test Automation, 1999





1The German Wikipedia version has some more tables and descriptions


Samstag Nov 11, 2006

Rightsize your Testing Process

Yesterday I was at the „QS-Tag 2006“ organized by imbus AG, Germany to attend a workshop in rightsizing the testing process.

It was a very interesting workshop about process modeling („ Deming-circle“ ), planning- and implementation methods ( QFD - Quality function deployment), TPI ® - Test Process Improvement) and controling mechanisms ( GQM - „Goal, Question, Metric“ ).

It was funny that the instructor talked about „Excel“ sheets but he has also installed NeoOffice on his Apple Notebook and used it spontaneous in the workshop and it looked nice, except an issue in Calc I will submit on monday :-) OpenOffice.org was known by all of the attendees of the workshop and has been used as example very often in that workshop :-)

Some of the testers/test managers were also interested in our new specification template. It is a fact that also other, smaller project have a need to get testable and measurable specifications not only OpenOffice.org and not only Sun.









Freitag Jul 21, 2006

Test case specification template proposal

After the software specification process and the software specification template have been published on OpenOffice.org (OOo) I have now suggested a test case specification template and what to do with it. Both "processes" should help to get a higher quality into the open source office suite.

The project team of the software specification process will held a presentation about their work at the OpenOffice.org Conference (OOoCon) in Lyon, France from 11.-13. SEP 2006. Hopefully see you there!

Donnerstag Jun 29, 2006

OpenOffice.org Specification Process published

After a project team of the OpenOffice.org community has launched a specification template (OpenDocument File Format) now the process documentation has been published in OpenOffice.org Wiki. It's the first time that OOo community members will be able to find fastly and clearly written what they need to introduce a new feature with a high quality level into OpenOffice.org.

Freitag Jan 06, 2006

A specification template preassigned on improving quality

On OpenOffice.org the quality of created specifications (...) revealed between 5 and 60 defects per page (1 page = 300 words) [1]. Too much variation and I am a member of a team which has analyzed the root causes. The first ouput is a suggestion for a new specification template which makes it much easier to write and to test specifications.

As a tester (and also as the writer of a specification for OpenOffice.org) you find all relevant documentation you need in it. Also the UI elements can be tested now easily because the specification writer needs two clicks to add an UI element and all necessary properties will be added to the specification. The specification writer and the tester won't forget a property in future.

With the ruleset to be complete, clear and simple it should be now much better to get a consistent quality of specifications to optimze the engineering work on features for OpenOffice.org.

[1] http://specs.openoffice.org/servlets/ReadMsg?list=dev&msgNo=17

Dienstag Nov 08, 2005

A Testing Glossary

As a software tester often you do not find the correct words to argue your decisions. I use a German and English glossary in the public net which is based on that what international qualification scheme ISTQB® Certified Tester (International Software Testing Qualification Board) also uses.

About

jogi

Search

Archives
« April 2014
MoDiMiDoFrSaSo
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Heute
Bookmarks