OpenDS--Open Source Improves Quality in Software Development

Gary Williams, a staff engineer and the QA lead of OpenDS, published a great article with Marina Sum on the topic of how working on an open source software project has improved quality in product development.  The process is without challenges which he outlines in the article as well.  However, he also gives great detail about the test harness that is used, the amount of automation and community involvement to address the challenges and get high quality product in community hands more frequently.  The full article is available on the Sun Developer Network here

These are the types of processes that quality open source projects do as a part of the project development process.  Indira Thangasamy, produced a similar article on how they approach QA within the OpenSSO project.  As companies evaluate other open source projects, especially in these challenging economic times where cost reduction provide stronger rational's to consider starting projects using open source software.  The quality approach of communities becomes an important differntiator as companies use open source in production and customer facing systems.

Here is a quick overview of the test harness used on OpenDS:

We use open-source, Java platform-based test tools, such as the following, not only to demonstrate our support for open source but also to ensure that they are accessible to everyone:

Here are a couple of other highlights:

  • Unit Testing and Automation:  "Testing starts in the programming phase with unit tests, which verify that the code works as intended and which must exist for all features. Today, we run 30,000 automated unit tests daily on different Java virtual machines. No code can be integrated without satisfying the precommit requirements."
  • Code coverage — With open-source EMMA, we find out the number of code lines, blocks, methods, and classes that are exposed by the unit and functional tests. Part of that information pinpoints the amount of the code tested as a percentage of the total, defining if we've met the quality criteria. We also define which areas of the code are not tested, called coverage holes, and create new tests to fill them.
  • Feature coverage — OpenDS delivers features that customers want, that is, customer requirements. Each feature is recorded as an issue in the Issue Tracker, a tool that monitors defects. This data tells us the state the features are in and their status: Ready for Test or Tested.
  • Documentation coverage — To ensure that the documentation is reviewed according to the test plan, we adopt a two-phase documentation review process: a technical review of the content followed by a formal QA review. Like the product features, the documentation is divided into categories—books, chapters, and sections—that are recorded in the Issue tracker. Through this coverage, we measure the percentage of the documentation reviewed over time and identify the reviewers and review status.
  • Defect rates — This is a traditional measure. The goal is to have no high-priority bugs open at release time. Our Bug Council constantly studies the defects and assesses the risks to customers. We also plot simple graph trends to gauge how well the project is converging.

Thanks to Gary and Marina for publishing this article and allowing the community to learn from your experience. 

Comments:

Post a Comment:
Comments are closed for this entry.
About

This blog provides information regarding the Oracle Directory Server Enterprise Edition and Oracle Unified Directory products. Use this blog to get the latest breaking information regarding releases and updates plus other technical and non-technical information.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today