Mittwoch Mai 27, 2009

Best Practices For Software Test Code

Best Practices For Software Test Code

Author: Joerg Sievers – Reviewers: Helge Delfs, Venkatesh Devarajan, Arun Kumar S, Srinivas Kg, Mario Schroeder, Elizabeth Matthis – Revision 10 – Last change: 1. July 2009

Test code is no production code

A well skilled Java developer isn't also a well skilled Java test code developer. The focus of a tester is different to a programmer and also automated test cases are different from methods of production code. They can be written in the same language and they maybe look alike but their goals are different. UI-tests should be more like an acceptance test because it is the same view a customer also has.

Explanation: In desktop environments the production code goal could be not to crash under the most worst circumstances. In test code you should stop working (=controlled, well reported stop of the test case execution) if an error occurs because the subsequent steps would report undefined results. It is easier to get an overview what happened if you only see one, the first error and not hundreds more which depended on the first one.

In web environments the production code goal could be to cloud the executed code in the browser. In test cases the feedback should be best traceable to reproduce simple steps if an error occurs.

Set preconditions

Be sure that a evadable default in the application is really the default in all environments. Otherwise you have to set the settings you need in front of the test case (preconditions). [OOO1]

An automated test case should not depend on other test cases

If you need preconditions to execute a test scenario you have to put in the precondition phase which always runs in front of any test case.

A test case needs to be re-runnable

No test pieces which being created while the test is running should hinder a re-run of the test case. If so, you have to remove such pieces in the precondition phase (=”running it always on the greenfield”). You should write automated software test not user scenario tests with undefined states – that would be another type of tests. Every testcase should have 3 sections: Entry, Action, CleanUp. Entry section will be useful to do any specific pre-settings, Action section is where actual testcase runs, CleanUp section is to undo all the actions performed by that particular testcase.

Use global tool settings and routines instead of own created

If an output path, a timeout, an environment variable, or also a helper method is defined globally it should be used instead of creating them in the test case on your own. This is the same for production and for test code to optimize the maintainability.

Explanation: Especially timeouts often defined in many test cases instead of using divisions or multiplications of the global timeout which is set in the most test tools.

Write test cases readable from top-level like use-cases

In production code the object-oriented approach is best practice. In test code, the focus is to simplify the work-flow of steps executed multiple times. The helper classes and abstraction layers should be made based on this focus but always it should be possible to read the important steps to get a test case done on the top-level and not in sub-routines.[MM01]

Useful feedback

A test case is a step-by-step instruction of a work-flow. In production code, feedback is not given if an action was successful. In test code you should do that more often to be able for others to reproduce these steps manually – and not only those which failed. [MF01]

Use defined prefixes for variables

Modern Integrated Development Environments (IDE) are able to show the type of a variable but for a better reading it makes sense to use defined prefixes. [OOO1] If there is no definition made the “ Hungarian Notation” [ Link English] [ Link German 1] can help.


prefix

type

i or n

integer

i

long

d

single

d

double

s

string

b

boolean

The following prefixes must be combined with an additional prefix:

a

array

g

global (from global.inc)

l

list (for variables declared for use of functions from t_list.inc)

Other naming schemes:

t

testcase

s

sub

f

function


Use clear indicators to show when an action is complete.

When an action has been started and you are waiting for the results, you have to have clear indicators for automated software to show when the action has been completed successfully. If you do not have such an indicator, then you have to use "timeouts", but they are always only a last resort.

For example: You want to load a document. To show that the document has been loaded, the status bar of the application changes or certain buttons become active.

Bibliography

OOO1: Thorsten Bosbach, Helge Delfs, Joerg Sievers, Joerg Skottke, OpenOffice.org Cookbook, 2004

MM01: Marc Michaelis, Boon and Bane of GUI Test Automation, 2008

MF01: Mark Fewster, Dorothy Graham, Software Test Automation, 1999





1The German Wikipedia version has some more tables and descriptions


Freitag Nov 14, 2008

Hi Admin, Gnome is gone! We haven't done anything!

Storyboard

I worked at home on my broadband connected Sun Ray and the phone rang. A colleague was on the other side and said that their test zone has no GNOME anymore. He promised that they haven't done anything. It is just gone. The test zone is being used for installing and testing StarOffice (SO), OpenOffice.org (OOo) and StarOffice PDF Conversion Server (Converter). After promises that no other software than SO, OOo or the Converter have been installed in the last days, I tried to analyze the problem....

Analyze phase

root# svcs -x
svc:/application/font/fc-cache:default (FontConfig Cache Builder)
 State: maintenance since Wed Nov 12 22:39:44 2008
Reason: Start method failed repeatedly, last died on Killed (9).
   See: http://sun.com/msg/SMF-8000-KS
   See: fc-cache(1M)
   See: /var/svc/log/application-font-fc-cache:default.log
Impact: \*This service is not running.\*

Me: Upps, have you manipulate the system?
He: Ehm... I remember that I have installed and removed a child workspace and also I removed with prodreg a printer device driver I have needed to evaluate a customer problem.
root# more /var/svc/log/application-font-fc-cache:default.log

(...)
[ Mar 25 09:46:27 Executing start method ("/usr/bin/fc-cache") ]
[ Mar 25 09:46:41 Method "start" exited with status 0 ]
[ Nov 12 14:53:47 Stopping because service disabled. ]
[ Nov 12 14:53:47 Method property group 'stop' is not present. ]
[ Nov 12 14:54:47 Enabled. ]
[ Nov 12 14:54:50 Executing start method ("/usr/bin/fc-cache") ]
ld.so.1: fc-cache: fatal: libexpat.so.0: open failed: No such file or directory
[ Nov 12 14:54:50 Method "start" failed due to signal KILL ]

Me: The system has been manipulated by one of the software under test (SUT) you have removed. It has taken the libexpat-library with it. I will come back to you. [click]
A comparison with sister-system and I was sure that it is a specific problem to this test-system.

Resolving the problem

I copied the library and created the symbolic links to get the following result:
root# ls -lia libexpat\*
   1062318 lrwxrwxrwx   1 root     root          17 Nov 13 10:40 libexpat.so -> libexpat.so.0.5.0
   1062317 lrwxrwxrwx   1 root     root          17 Nov 13 10:39 libexpat.so.0 -> libexpat.so.0.5.0
   1062316 -r-xr-xr-x   1 root     bin       307540 Nov 13 10:37 libexpat.so.0.5.0

Now some commands
root# svcadm clear svc:/application/font/fc-cache:default
root# svcadm enable svc:/application/font/fc-cache:default
root# svcs -x
root# more /var/svc/log/application-font-fc-cache:default.log

to see that the service is back again:
[ Nov 13 10:45:07 Leaving maintenance because clear requested. ]
[ Nov 13 10:45:07 Enabled. ]
[ Nov 13 10:45:07 Executing start method ("/usr/bin/fc-cache") ]

Ringing up the colleague...
Me: Try to log-in again and tell me that GNOME isn't gone....
He: Yes, Gnome is back. Thank you!
Me: Please validate which nice piece of software removed your library that not customer will have the same problem. Thank you.

Samstag Jul 14, 2007

Building OpenOffice.org for the Mac is easy

Yesterday I started the second time building OpenOffice.org (OOo) under the Mac (INTeL) platform. Eric B. has given me the link to the documentation to build the Aqua version under Mac OS X and I started the game again....

Months (years?) before I did the same on the PowerPC version of my iMac to build the X11 version of OOo and it was a hell. You need to get packages downoaded from elsewhere and had to put it under the build tree, the manual and the process was very tricky. It did not work and I have given up. I am not a porter or a developer who is able to understand the problem statements of the build environment, I am a tester and need a build to test as fast a possible...

Today it was totally different and very easy. Only one typo in the Wiki page and the rest was described very easy. I was able to start - after re-installing the Xcode package from Apple where Ant was missing?!? - around 3.40 p.m. and as I looked onto the machine again (on the way to bed) around 10.20 p.m. everything was done and the build was packaged.... Wow!

Dienstag Jun 05, 2007

Flash Player 9 Beta


If you want to watch sometimes videos or going to a newly designed web page where Adobe's Flash Player is being used you could run into trouble because the installed version is outdated. To get things better done you maybe want to give the Flash 9 Beta version for Solaris SPARC and x86 a try.

Do you use OpenOffice.org? / Benutzen Sie OpenOffice.org?


Thomas Krumbein from the German OpenOffice.org community project has created a questionaire to all who wants to help OpenOffice.org to get numbers. Numbers of installed workstations on which operating systems etc. The questionaire is in German.
Thomas Krumbein hat einen Fragebogen erstellt, der OOo helfen soll an Zahlen zu kommen, da die Verkaufszahlen ja nicht gemessen werden. Es wäre toll, wenn Sie sich 2 Minuten Zeit nehmen und mithelfen eine Übersicht der installierten Basis zu erstellen:
http://www.amt-wiesbaden.de/ooo/Fragebogen.html
Fragen: Wieviel Arbeitsplätze (ca.) sind mit OOo ausgestattet?
  • Welche Version von OOo wird aktuell genutzt?
  • Seit wann wird OOo genutzt? (Jahr oder Versionsnummer)
  • Welche Module werden hauptsächlich genutzt?
  • Welches Betriebssystem wird überwiegend eingesetzt?

Donnerstag Mai 03, 2007

We're back on Mac

Sun is supporting the Mac OS X "Aqua" port of OpenOffice.org and the Sun mac users at Hamburg office made a small welcome-party.[Read More]

Freitag Mrz 02, 2007

1st OpenSolaris Developer Conference

Yesterday I traveled to Berlin and visited the 1st OpenSolaris Developer Conference and the German Unix User Group Frühjahrsfachgespräch 2007.

  1. Keynote on 1. March 2007 – Simon Phipps

    Simon started the day with the keynote and talked about

    • the participation age

    • Communities and how the OSS1 economic system works
    • that Open Source not always means free software
    • OSS Business Model 3.0 where the software need to be paid first time when the participant (better word than customer) is using it.
    • Sun as FOSS2 Startup & what Sun has already open sourced
  2. How to make your program privilege aware - Wolfgang Ley

    Wolfgang (also from the Hamburg Sun office where I am located) held a session about a very, very interesting part in Solaris 10+: The privileges. Sometimes you need a good lecturer to understand things. It isn't complicated, it's great! To make it short: UID 0 isn't needed to do jobs which need more privileges, just add that privilege to the user – or add them to the application if it is called by the specific user :-)

  3. Looking into the black-box – How the kernel may impact your application – Thomas Nau

    From the practice Thomas Nau reported what he and his team done with dtrace at the University of Ulm and a T1-processor based T2000 box. First he introduced dtrace and said that this tool “is one of the best, maybe the best innovations in the last ten+ years” (in the IT) – after his session everybody will agree because it was for me the best session of the day where I learned that the move from the old tools (truss(1), apptrace(1), mdb(1), intrstat(1m), busstat(1m), cputrack(1m), ... ) are the old way and dtrace can give us more specific information what really the application dynamically is doing. The “D-Language” which dtrace uses is easy and we have just to start. He also mentioned to spice up our applications and how.

    Really great was the case study about the “slow” Sun T2000 and how they resolved the problem by changing the cache-bin algorithm (coloring) in /etc/system – which seems to be fixed in the meanwhile by newer OpenSolaris build. One small change and the T2000 boost-up as expected.

  4. [GUUG] Innovation in Solaris – today and tomorrow – Franz Haberhauer

    The GUUG sessions were held in German but some of them were also a day or some hours later on the Solaris DevCon track – maybe more from the point of an OS developer view but I am a tester with system administration tasks.

    This one from Franz I found only on the GUUG track and was interesting what the projects “Nemo”, “Crossbow” and “Clearview” will bring in the world of networking and zones and what the project “FireEngine” did already in Solaris 10. dladm will be the tool for monitoring and managing network interfaces and will make things easier.

    Also the desktop is back to the focus of Solaris (the Sun distribution of OpenSolaris) development with updating the existing modules GNOME to 2.16, Firefox to 2.0, Thunderbird to 1.5 and StarOffice to version 8 and adding Egika (video conferencing) and Vino (remote desktop solution on a vnc base).

    My favorite in the additions is “tamarck” which implemented a privileged HAL3 who finds new devices (USB plug-in your device and play... yeah!).

  5. [GUUG] Solaris Container Best Practices – Detlef Drewanz

    Detlef also held AFAIK only once this session in the GUUG track and his hints are very helpful if you think about using zones. Which use cases are possible? How to deploy Solaris Containers and which questions you need answers for? The different zone-types was nothing new to me but the impact on application architectures was very interesting. Deploying applications outside a zone makes things like moving (!) or cloning a zone much easier. The summary is: Plan you infrastructure to get really the full advantage of Solaris Containers.

  6. Back in 2008

    I am looking forward to the next OpenSolaris Developer Conference because the level of sessions was high the small-talks were very interesting.

1Open Source Software
2Free Open Source Software
3Hardware Abstract Layer

Dienstag Jan 16, 2007

Enhancing automated software test environment on OOo

The automated testing team inside Sun Microsystems maintains at the moment 100% of the GUI testing code of OOo but there is hope that also others will start writing test code in future which can be also shared and used by others. That would be a great test case development.
To get others into the boat we want to get a good set of test scripts [1] and will do some enhancements in these areas:

  1. Documentation
  2. Reuse
  3. Structured
  4. Mainenance

We know for 1.) that we need to document how to add a new L10N (=localization) language to the framework and also we need to make it easier (4.).

If you have suggestions for the other areas we should think about, subscribe to dev@qa.openoffice.org mailing list and let us know what your barrier is we need to break down.

We want to collect in the next three weeks which enhancements we need to to do to get the barriers down and you into the boat. I will summarize it and maybe you are alos interested to do some work with us...



[1]
Software Test Automation, Mark Fewster & Dorothy Graham, Addison Wesley, 1999, ISBN 0-201-33140-3, page 69; table 3.1 "Comparison of good and poor sets of test scripts that perform the same set of test cases"

Samstag Nov 11, 2006

Rightsize your Testing Process

Yesterday I was at the „QS-Tag 2006“ organized by imbus AG, Germany to attend a workshop in rightsizing the testing process.

It was a very interesting workshop about process modeling („ Deming-circle“ ), planning- and implementation methods ( QFD - Quality function deployment), TPI ® - Test Process Improvement) and controling mechanisms ( GQM - „Goal, Question, Metric“ ).

It was funny that the instructor talked about „Excel“ sheets but he has also installed NeoOffice on his Apple Notebook and used it spontaneous in the workshop and it looked nice, except an issue in Calc I will submit on monday :-) OpenOffice.org was known by all of the attendees of the workshop and has been used as example very often in that workshop :-)

Some of the testers/test managers were also interested in our new specification template. It is a fact that also other, smaller project have a need to get testable and measurable specifications not only OpenOffice.org and not only Sun.









Mittwoch Okt 11, 2006

Sun Weblog Publisher available for SO and OOo

I am using now the new Sun Weblog Publisher to write my entries. Wow! Just using StarOffice ™ with the full advantages of a word processor. Great! The Sun Weblog Publisher is also usable for OpenOffice.org not only for StarOffice.



Dienstag Okt 03, 2006

Sun's flexible office and Sun Ray at home on TV


The German TV station 3sat has visited Sun's German headquarter in Munich and also my personal workplace at home with a Sun Ray. The brodcast nano extra will be shown at 18.30h (MESZ) on 3. October 2006 on 3sat which is also reachable through satellite.

If you understand German you can read an article about the upcoming brodcast at 3sat. It was funny to show the TV team that it saves my time to start my work at home and not in the office and that it is great not to run into the daily rush hour of Hamburg. Also I have much more time with my daughter which I do not want to miss. It is really great to have an quiet area where nobody can enter the room where you are trying to debug some code or reading an interesting technical article.

I haven't seen a preview of the broadcast and hope that my big advantages to work at home for Sun will be shown there.

Jogi at home

Freitag Jul 21, 2006

Test case specification template proposal

After the software specification process and the software specification template have been published on OpenOffice.org (OOo) I have now suggested a test case specification template and what to do with it. Both "processes" should help to get a higher quality into the open source office suite.

The project team of the software specification process will held a presentation about their work at the OpenOffice.org Conference (OOoCon) in Lyon, France from 11.-13. SEP 2006. Hopefully see you there!

Donnerstag Jun 29, 2006

OpenOffice.org Specification Process published

After a project team of the OpenOffice.org community has launched a specification template (OpenDocument File Format) now the process documentation has been published in OpenOffice.org Wiki. It's the first time that OOo community members will be able to find fastly and clearly written what they need to introduce a new feature with a high quality level into OpenOffice.org.

Samstag Mrz 04, 2006

They needed 24 years to win! Congratulation HSV Hamburg!

They did it! After nearly 24 years the soccer club Hamburger Sportverein (HSV) won against the FC Bayern Muenchen in Munich (1:2) - since a long time the one and only number one in the Bundesliga (premier national soccer league). They also have won the match in Hamburg. They're back in the game!

Freitag Feb 17, 2006

My biggest personal project nearly finished


We, my wife and I, have finished building our house in Ahrensburg, Germany. It is definitely the biggest personal project my wife and I have succesfull finished. If you understand the German language read in my report how happy we were with our building promoter!

About

jogi

Search

Archives
« April 2014
MoDiMiDoFrSaSo
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Heute
Bookmarks