Key Points

• Why improving the developer's tests
• How to improve the developer's tests a practical approach

Presentation Abstract

This paper discusses the reasons for improving the developers' tests. Next, a practical approach is given for improving the quality of the developers' tests (i.e. program test and integration test). The emphasis of the approach is on:

• clear testing responsibilities;
• the idea that improvements come from within the development team;
• insight into the quality of the test object by using exit- and entry-criteria.

The paper concludes with two case-stories.

Introduction

Testing theory states the importance of early testing. The developers perform one of the earliest forms of testing. There is sufficient literature on how this kind of testing should be performed, including a lot of techniques and tools. However, there is a large gap between theory and practice. In practice, developers often test based on their intuition and in an unstructured way, in stead of using techniques, etc. Based on our own experiences, this paper gives reasons why better developer testing is important and how improvements can be implemented. The paper concludes with two case-stories. Our experiences originate from the world of administrative automation (financial institutions, government, industry), and "new media" projects (such as internet, knowledge management, data ware housing).

Characteristics of developers' tests

Typical developers' tests are the program test and the integration test. Some characteristics of these tests (and differences with other types of tests) are:

• The finder of a defect is also (often) the solver of the defect. This means communication overhead can be kept to a minimum;
• (In principle) all defects found should be repaired before the product is handed over. Therefore, the amount of reporting is limited;
• A developer differs in attitude from a professional tester; the former wants to demonstrate that the product works, the latter wants to demonstrate the inverse. Time-consuming and thorough testing conflicts with the developer's attitude;
• The tests are an integral part of the development process;
• At the time these tests start, most defects are in the product: this requires cheap and fast repair of defects.

Should developers' tests be improved?

When asking the developers whether their ways of testing need improvement, frequent answers are:

• Not enough time, too expensive;
  Only if given more time and money, better testing is possible. However, development is always short of time and money. But when the project leader is asked for more time and money, that person responds something like: "If I give the developers more time and money, they'll certainly spend it. The big question is, on what?")
• Faith in current way of working, in current product quality;
  It is common knowledge that 100% defect free software is impossible to achieve. Apart from that, developers are proud of the products they develop.
• A good (better) test follows.
  Test professionals perform later, better tests. These people are trained in testing and they typically find a lot of defects. And what's more important, they like testing. Because for most developers à
• Testing is boring!

There are of course a number of reasons for improving developer testing:

• Defects one finds oneself are easier to analyse than defects found by other parties;
• Early rework is cheaper, since knowledge is still fresh and all relevant parties are still there;
• Earlier feedback prevents similar errors;
• White-box techniques used in developers' tests find different defects from black-box techniques used in later tests.
• Because higher quality products are delivered, fewer defects are found in later tests (and in production). As a consequence, developers can spend less time reworking products and more time creating products;
• One of the most uncertain planning factors is the amount of time and resources necessary for rework activities. More certainty about product quality therefore results in better project planning;
• For the same reason, the project lead-time is shortened.

These arguments sound good enough, but in practice seldom win from the arguments given against improvement. However, there is a new development adding arguments in favour of improving. This important development is the increasing strategic use of software, i.e. supporting the primary business of an organisation, directly dealing with the customers and readily adaptable to new challenges. This means a different kind of customer than the traditional IT department, demanding quality software delivered in time. Improving only system and acceptance testing will result in failing to meet these "in time" demands. Improving developers' tests is an important step towards meeting the demands.

How to improve? An approach

Below, a practical approach on how to improve developers' tests is described. The approach has the following key aspects:

• organisation
• use of test design techniques
• use of a test life-cycle model

Organisation

Perhaps the most important aspect of the approach is that an Application Integrator (AI) will be made responsible for the progress of integration and for the quality of the outgoing product. The AI negotiates with the project manager or the development team leader what level of quality is required: under what conditions can the system be released to the next phase (exit-criteria). The AI also demands insight in the quality of incoming modules or programs (entry-criteria). A module or program is only accepted into the integration process if it meets the entry-criteria. In order to prevent mixing interests, the AI should not be development team leader. This creates a deliberate tension between the AI, who is responsible for quality, and the development team leader, who tends to focus on functionality and the amount of time and resources spent. Since the AI is a member of the development team, this generates far less resistance from the developers against improving their testing than other test approaches would do and considerably raises the awareness of quality within the development team. For test specific expertise, test professionals (outside of the project) support the AI. The AI communicates with the customer on quality issues. Because communication is essential, it is very important for the AI to have good social skills.

Use of test design techniques

The approach does not prescribe 100% use of formal test design techniques, as this will (in our experience) generate too much resistance. Instead, the AI, program testers and project manager or development team leader negotiate: important parts of the system will be tested using formal test design techniques, less important parts will be tested using informal techniques or even in the old-fashioned, undocumented way of testing without use of any techniques. A good balance has to be found, for which several aspects play a role:

• risks for the organisation / importance of the system;
• desired quality of the product;
• progress of the project;
• test maturity of the development team;
• test coverage;
• test evidence;
• resource consumption (of using the test design technique).

Popular techniques often applied in our projects are checklists and marking test situations in the functional specifications. Only test situations too complex to be tested directly from these markings are detailed further into specific test cases. Although definitely not watertight, the marked up documents supplied with the tester's initials serve as test evidence.

Use of a test life-cycle model

Exit- and entry-criteria and other agreements are laid down in a test plan (phase 1), test cases are prepared (phase 2) and executed (phase 3). Again, practical use prevails: writing documentation such as a test plan is not a target in itself, but serves as a means of communication between project leader, AI and customer, and should be kept as minimal as possible.

About the Speaker

Since 1989, Ruud Teunissen is employed in the testing world. He has been involved in a large number of ICT projects and has performed several functions within the testing organization: tester, test specialist, test advisor, test manager, etc.

Based on his experience, Ruud participated in the development of the structured testing methodology TMap and is co-author of several books on structured testing. The last years Ruud is involved in implementing structured testing within organizations on the Belgian and Dutch market.

At this moment Ruud is working in Belgium for Gitek n.v. as Manager Testen. Ruud is frequently speaking in Benelux and Great Britain.