QWE2000 Session 4T

Lisa Crispin [USA]

The Need for Speed: Automating Functional Testing in an eXtreme Programming Environment

Full Technical Paper in QWE2000 CD-ROM

Read the Best Presentation Winning Paper, and Best Paper Runner Up (Adobe PDF format; requires Adobe Acrobat Reader)

Key Points

Presentation Abstract

In my two and a half years working in a web environment, where quality and time to market are both essential to success, I've been frustrated by the difficulty in combining these traits within traditional software process. After reading Kent Beck's book, eXtreme Programming Explained, I couldn't wait to try this methodology to enable small teams to deal with short timeframes and changing requirements while still producing high quality software. I recently joined iFactor-e, where we use XP to combine the highest levels of quality and shortest time to market.

Testing in a Web environment can feel like leaping out of a plane. Testing in an XP environment feels like competing in a sky-surfing competition. You have to be better than everyone else, but you don't have much time. You can only hope for a soft landing. While the eXtreme Programming literature (including Ron Jeffries' book, eXtreme Programming Installed), centers around unit and integration testing as part of the XP core process, I felt that functional/acceptance testing from the customer perspective was incompletely defined. The role of the tester in XP is clearly defined - to help the customer choose and write functional tests and to make sure those tests run successfully. The question is, how to do this when the ratio of developers to testers is quite high (8 - 1 is recommended, and we are in a more extreme ratio than that) and the development iterations are so short.

Like an extreme-sports competitor, the XP tester needs courage, speed, stamina and creativity. Working with the developers and with input from an automated test tool vendor, I have developed an approach to designing modularized, self-verifying tests that can be quickly developed and easily maintained. I'll present my basic design and give some examples. I used the test tool WebART, but this methodology should be applicable to any au tomated tool that includes a scripting language.

About the Speaker

I have eighteen years experience in the industry with the last nine in Testing and Quality Assurance. I started out as a programmer and later worked in customer support and QA for large software vendors. In March of 1998, I discovered the world of Web startups, joining TRIP.com as the first test engineer. The challenge of building quality into Web applications while meeting tight development cycles was eye-opening. At TRIP.com, I built a QA department of seven test engineers testing state-of-the-art, first-of-their-kind applications such as flightTracker and intelliTRIP. I felt, however, that we never found a really good process that worked to produce high-quality software in a short amount of time. Missed deadlines were common. Still, we were proud of our accomplishments, as TRIP.com grew to one of the highest-traffic travel Web sites, rated 4.5 out of 5 starts by BizRate and ranked near the top of the Keynote Top 40 websites for performance.

Several developers from TRIP.com left to join a new startup, iFactor-e, devoted to using eXtreme Programming to combine high quality and short time to market to wow the customer. One of these developers loaned me Kent Beck's book. After I read that, I was eager to try XP myself and was fortunate enough to be hired as the first test engineer at iFactor-e in July of 2000. Since then, I have been racing to establish a functional testing methodology that successfully applies the values of XP.

I have given successful presentations at both local and international user and QA conferences to audiences of up to 60 people. I have many years experience training both technical and end users.