sss ssss rrrrrrrrrrr
ssss ss rrrr rrrr
sssss s rrrr rrrr
ssssss rrrr rrrr
ssssssss rrrr rrrr
ssssss rrrrrrrrr
s ssssss rrrr rrrr
ss sssss rrrr rrrr
sss sssss rrrr rrrr
s sssssss rrrrr rrrrr
+===================================================+
+======= Testing Techniques Newsletter (TTN) =======+
+======= ON-LINE EDITION =======+
+======= February 1999 =======+
+===================================================+
TESTING TECHNIQUES NEWSLETTER (TTN), Online Edition, is E-mailed monthly
to support the Software Research, Inc. (SR)/TestWorks user community and
to provide information of general use to the worldwide software quality
and testing community.
Permission to copy and/or re-distribute is granted, and secondary
circulation is encouraged by recipients of TTN-Online provided that the
entire document/file is kept intact and this complete copyright notice
appears with it in all copies. (c) Copyright 2003 by Software Research,
Inc.
========================================================================
INSIDE THE FEBRUARY 1999 ISSUE:
o Quality Week '99 (24-28 May 1999) Technical Program Announced
o Open Letter to Friends of the National Software Council, by John
Marciniak
o Y2K Solution Achieves New Spell Of Sophistication, by Dave
Stringer-Calvert
o CAPBAK/Web Early Release on Windows 95/NT
o TOM -- The Test Organisation Maturity Model, by Paul Gerrard,
Systeme Evolutif Limited, London, UK
o Three for Dinner: Served with Y2K Flavorings, by Ann Schadt
o Software Glitch Plagues E-Trade
o The Y1K Crisis, Canterbury, England, A.D. 999, by John Favaro
o Automated Software Engineering (ASE'99): Call for Papers
o Special Issue on Information Systems Support for E-Commerce
o TTN SUBMITTAL, SUBSCRIPTION INFORMATION
========================================================================
Quality Week '99 (QW'99) (24-28 May 1999) Technical Program Announced
We are pleased to announce the best-ever technical program for QW'99.
There are two days of full-day and half-day tutorials, world-renowned
keynote talks, a three-day multi-track collection of the best speakers,
a tools and services exposition, and special events.
The complete technical program is given below. Full information about
the QW'99 event can be found at the Conference Website:
<http://www.soft.com/QualWeek/QW99>
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
T U T O R I A L S
Monday, 24 May 1999, 8:30 - 12:00 -- AM Tutorials
Mr. Edward Kit [Assisted by Mr. Hans Buwalda] (Software Development
Technologies) "Integrated, Effective Test Design and Automation
(A1)(A2)"
Mr. John McGregor (Clemson University) "Testing Distributed Object
Systems (B1)"
Ms. Karen Bishop-Stone, CSTE, CSQA (Testware Associates, Inc.)
"Practical Software Test Case Design (C1)"
Dr. Norman Schneidewind (Naval Postgraduate School) "Development and
Maintenance Process Assessment Using Reliability, Risk, and Test Metrics
(D1)"
Dr. Linda H. Rosenberg (Software Assurance Technology Center, NASA)
"Writing High Quality Requirement Specifications (E1) (E2)"
Monday, 24 May 1999, 1:30 - 5:00 -- PM Tutorials
Mr. Edward Kit [Assisted by Mr. Hans Buwalda] (Software Development
Technologies) "Integrated, Effective Test Design and Automation
(A1)(A2)"
Mr. Thomas A. Drake (Coastal Research & Technology Consultant)
"Measuring Object-Oriented Software Quality for C++ and Java (B2)"
Mr. William Bently (Mu_Research) "How to Test an Object: The Information
Flow Approach (C2)"
Dr. John D. Musa (Independent Consultant) "Software Reliability
Engineering: More Reliable Software, Faster (D2)"
Dr. Linda H. Rosenberg (Software Assurance Technology Center, NASA)
"Writing High Quality Requirement Specifications (E1) (E2)"
Tuesday, 25 May 1999, 8:30 - 12:00 -- AM Tutorials
Dr. Boris Beizer (Independent Consultant) "An Overview of Testing: Unit,
Integration, System (F1) (F2)"
Dr. Magdy Hanna (Software Dimensions Consulting and Training, Inc.)
"Establishing a Software Inspection Process (G1)"
Mr. Robert Binder (RBSC Corporation) "Modal Testing Strategies for
Object-Oriented Software (H1) (H2)"
Mr. Leonard Verhoef (Human Efficiency) "Improving Software Quality for
Users (J1)"
Mr. Michael Deck (Cleanroom Software Engineering, Inc.) "Requirements
Analysis Using Formal Methods (K1) (K2)"
Monday, 25 May 1999, 1:30 - 5:00 -- PM Tutorials
Dr. Boris Beizer (Independent Consultant) "An Overview of Testing: Unit,
Integration, System (F1) (F2)"
Mr. Tom Gilb (Result Planning Limited) "Advanced Inspection (G2)"
Mr. Robert Binder (RBSC Corporation) "Modal Testing Strategies for
Object-Oriented Software (H1) (H2)"
Ms. Sally Drew (Tescom UK SST) "E-Commerce Testing -- The Clash of the
Titans (J2)"
Mr. Michael Deck (Cleanroom Software Engineering, Inc.) "Requirements
Analysis Using Formal Methods (K1) (K2)"
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
T E C H N I C A L P R O G R A M
Wednesday, 26 May 1999, 8:30 - 10:00 -- KEYNOTE SESSION #1
Dr. Cem Kaner (Independent Consultant) "Facing the Future: The Law
(1P1)"
(Speaker To Be Announced) "(1P2)"
Wednesday, 26 May 1999, 10:30 - 5:00 -- Parallel Technical Tracks
BOFS...
Ms. Elisabeth Hendrickson (Aveo, Inc.) "Mass Market Software Testing
(2B)"
Ms. Peggy Fouts (Compuware Corporation) "Medical and Safety Critical
Application Testing (3B1)"
Mr. Mark S. Wiley (nCUBE) "OS and Embedded System Testing Techniques
(3B2)"
Mr. Larry Apfelbaum (Teradyne) "Testing Telecommunications Software
(4B1)"
Mr. Jon Hagar (Lockheed Martin Astronautics Company) "Testing for
Military and Government Software (4B2)"
TECHNOLOGY TRACK...
Dr. Selim Aissi and Ms. Wendi Hummel (Applied Dynamics International)
"Automating Syntax Testing: The Case of a Real-Time Simulation Tool
(2T1)"
Mr. Kenneth Nagin & Mr. Alan Hartman (IBM Research Laboratory in Haifa)
"TCBeans Software Test Tool Kit (2T2)"
Mr. Brian Miller (Teradyne) "Automated Test Generation for Computer
Telephony Systems (3T1)"
Dr. Bettina Buth, Prof. Dr. Jan Peleska & Dr. Hui Shi (FB3 Informatik)
"Combining Methods for the Analysis of a Fault-Tolerant System (3T2)"
Mr. Ira D. Baxter, Mr. Andrew Yahin, Mr. Srinivas Nedunuri, and Mr.
Leonardo Moura (Semantic Designs) "Lowering Maintenance Costs by Code
Clone Removal (4T1)"
Mr. Christopher Agruss (Autodesk, Inc.) "Automating Software
Installation Testing (4T2)"
APPLICATIONS TRACK...
Mr. Graham Thompson (InCert Software) "Minimizing Testing While
Maximizing Failure Detection (2A1)"
Dr. Frank Ackerman (Institute for Zero Defect Software) "Measuring Fault
Density in the Real World (2A2)"
Mr. Ron Silacci (Lucent Technologies, Inc.) "A Testers' Top 10 List
(3A1)"
Ms. Lisa Boden & Mr. Jon Hagar (Lockheed Martin Astronautics Company)
"How to Build a 20 Year Successful Independent Verification and
Validation (IV&V) Program for the Next Millennium (3A2)"
Dr. Steven Rabin (Interworld Corporation) "eCommerce Performance
Benchmarking -- Methodology and Criteria (4A1)"
Mr. Tsuneo Yamaura (Hitachi Software Engineering) "How To Design
Practical Test Cases (4A2)"
TOOLS & SOLUTIONS TRACK...
Mr. Douglas Hoffman (Software Quality Methods, LLC) "Test Automation
Architectures; Planning for Test Automation (2S1)"
Mr. Alan Ark and Sarah Ackroyd (Thomson Financial Services) "Euro: An
Automated Solution to Currency Conversion (2S2)"
Mr. Rob Oshana (Raytheon Systems Company) "An Automated Testing
Environment to support Operational Profiles of Software Intensive
Systems (3S1)"
Mr. David Carman (Bellcore) "Event-Based Test Generation for Distributed
Systems (3S2)"
Dr. Heesun Park (SAS Institute Inc.) "Optimum Level of Test Automation
for Client/Server Software (4S1)"
Dr. Huey-Der Joseph Chu (National Defense Management College)
"Automating Client/Server Testing in the Real World (4S2)"
MANAGEMENT TRACK...
Mr. Roger M. Records (Boeing Commercial Airplanes) "Deploying SQA in
Very Small Projects (2M1)"
Mr. Mike Ross (Quantitative Software Management, Inc.) "Size Does
Matter: Continuous Size Estimating and Tracking (2M2)"
Mr. John N. Romanak (Bellcore) "Life as a CMM Level 5 Test Organization
(3M1)"
Dr. Ilene Burnstein, Ms. Ariya Homyen, Dr. Taratip Suwannasart, Mr.
Robert Grom & Mr. Gary Saxena (Illinois Institute of Technology) "Using
the Testing Maturity Model (TMM) to Assess and Improve Your Software
Testing Process (3M2)"
Mr. Herb Krasner (Krasner Consulting) "Using the Cost of Quality
Approach for Software (4M1)"
Mr. Michael Deck (Cleanroom Software Engineering, Inc.) "Process
Diversity: How I Stopped Worrying and Learned to Love Chaos (4M2)"
QUICKSTART...
Mr. Rainer Pirker and Mr. Andreas Rudolf (IBM) "Millennium is getting
closer -- The Quickstart to Y2K Testing (2Q)"
Mr. Tom Gilb (Result Planning Limited) "Evolutionary Project Management
(`Evo') (3Q)"
Mr. Doug Hoffman and Mr. Cem Kaner (Independent Consultant) "Thoughts on
Oracles and Software Test Automation (4Q)"
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Thursday, 27 May 1999, 8:30 - 10:00 -- KEYNOTE SESSION #1
Mr. Martin Pol (IQUIP Informatica B.V.) "Facing the Future Means Facing
Test Maturity (5P1)"
Mr. Roger Sherman (Independent Consultant) "Facing the Future:
Commercial Product Testing (5P2)"
Thursday, 27 May 1999, 10:30 - 5:00 -- Parallel Technical Tracks
BOFS...
Dr. Cem Kaner (Independent Consultant) "Status Report on US Software
Quality Laws (6B1)"
Mr. Richard Denney (Landmark Graphics) "Blue Collar Formal Methods
(6B2)"
Ms. Johanna Rothman (Rothman Consulting Group, Inc.) "Life as a New
Test Manager (7B)"
Mr. Rodney Wilson (KLA-Tencor) "Care and Feeding of a Testing Career
(8B1)"
Mr. Mark Johnson (OrCAD) "Productivity in Small Integrated Teams (8B2)"
TECHNOLOGY TRACK...
Ms. Fan Yang, Mr. Trung Nguyen & Mr. Anant Adiga (Sequent Computer
Systems, Inc.) "A Web-based System Testing Repository Model (6T1)"
Ms. Frances I. Medina (AT&T Operational Technology Center) "Test
Automation of a GUI WEB Based Application: An Experience Developing
Reusable Automated Testing (6T2)"
Mr. Patrick Copeland (Microsoft) "Approaches to Testing Componentization
in the Windows CE Operating System (7T1)"
Mr. Sergio Cherskov (Microsoft) "Testing Windows CE 3.0 Real-Time Kernel
(7T2)"
Mr. Steven Toeppe & Mr. Scott Ranville (Ford Motor Company) "An
Automated Inspection Tool For A Graphical Specification and Programming
Language (8T1)"
Mr. John Kent (CISS Ltd.) "Advanced Automated Testing Architectures
(8T2)"
APPLICATIONS TRACK...
Ms. Mei-Hwa Chen and Mr. Ming-Hung H. Kao (SUNY Albany) "Investigating
Test Effectiveness on Object-Oriented Software -- A Case Study (6A1)"
Mr. Glen Xia (Deloitte Consulting) "An Industrial Case Study of
Quantitative Management for Object Oriented Software Testing (6A2)"
Mr. Leon Slota (Neoglyphics Media Corporation) "Developing Load and
Performance Requirements for Web Sites (7A1)"
Mr. Sam Guckenheimer (Rational Software Corporation) "Effective Testing
for Java-Based Web Software (7A2)"
Mr. Lorenzo Lattanzi & Mr. Mario Musmeci (Alenia Aerospazio) "Safety
Critical S/W Development for a Satellite Based Navigation System (8A1)"
Matias Vierimaa, Ms. Minna Makarainen & Mr. Atte Kinnula (VTT
Electronics) "Improving DSP Software Engineering Processes from the
Testing Viewpoint (8A2)"
TOOLS & SOLUTIONS TRACK...
Mr. Hanania T. Salzer (RTS Software Ltd.) "ATRs (Atomic Requirements)
Used Throughout Development Lifecycle (6S1)"
Mr. Larry Apfelbaum and Mr. Steve Meyer (Teradyne, AT&T) "Use Cases are
Not Requirements (6S2)"
Mr. Jim Williams (CableData, Inc.) "Testing for Y2K Compliance: A Case
Study (7S1)"
Mr. Gunther Chrobok-Diening, Dr. Andreas Ulrich & Mr. Peter Zimmerer
(Siemens AG) "Test Architectures for Testing Distributed Systems (7S2)"
Mr. Mark Charles (Vector Research, Inc.) "Testing a System With Dynamic
Requirements (8S1)"
Mr. Yuri Chernak (Valley Forge Consulting, Inc.) "In-Process Validation
and Improvement of Test-Case Effectiveness (8S2)"
MANAGEMENT TRACK...
Mr. Stale Amland (Avenir (UK) Ltd.) "Risk Based Testing and Metrics
(6M1)"
Mr. Tom Gilb (Result Planning Limited) "Risk Management Technology: A
Toolkit for Identifying, Documenting, Analyzing and Coping with Project
Risks (6M2)"
Mr. Philip Lones (Lucent Technologies) "Revolutionary? A Development
Method That Works (7M1)"
Mr. Nick Borelli (Microsoft Corporation) "Seizing Control of the
Development Lifecycle (7M2)"
Mr. James Bindas (Intel Corporation) "Tactical Improvement Projects:
Real-Life Lessons in Leading Change (8M1)"
Mr. Scott Young (Perot System Corporation) "Them and Us: Communication
between Development and Test (8M2)"
QUICKSTART...
Mr. Bill Deibler (Software Systems Quality Consulting) "Making the CMM
Work: Streamling the CMM for Small Projects and Organizations (6Q)"
Mr. Ted Hammer (NASA GSFC SATC) "Continuous Risk Management at NASA
(7Q)"
Dr. Cem Kaner (Independent Consultant) "Interviewing Software Testing
Job Candidates (8Q)"
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Friday, 28 May 1999, 8:30 - 10:00 -- Parallel Technical Tracks
BOFS...
Mr. Mark D. Anderson (Discerning Software Corporation) "Client/Server
Load Testing (9B1)"
Ms. Carla Oexmann (Chromatic Research) "Running a Nightly Test (9B2)"
TECHNOLOGY TRACK...
Mr. Bor-Yuan Tsai, Dr. Simon Stobart, Mr. Norman Parrington & Dr. Ian
Mitchell (University of Sunderland) "A State-Based Testing Approach
Providing Data Flow Coverage in Object-Oriented Class Testing (9T1)"
Ms. Martina Marre, Ms. Monica Bobrowski & Dr. Daniel Yankelevich
(Universidad de Buenos Aires) "A Software Engineering View of Data
Quality (QWE'98 Best Paper) (9T2)"
APPLICATIONS TRACK...
Mr. Jon Hagar (Lockheed Martin Astronautics Company) "Industrial
Experiences in Establishing Laboratories and Software Models to
Effectively Execute Software Test (9A1)"
Ms. Johanna Rothman (Rothman Consulting Group, Inc.) "Using Quality to
Drive Product Development Processes (9A2)"
TOOLS & SOLUTIONS TRACK...
Alain Kerbrat (Verilog) "Automated Test Generation from SDL/UML
Specifications (9S1)"
Mr. E.L. (Ed) Safford III (Lockheed Martin Tactical Aircraft System)
"Development of an Integrated Real-Time Avionics Software Testbed (9S2)"
MANAGEMENT TRACK...
Mr. Tom Wissink (Lockheed Martin Mission Systems) "Test Engineering -- A
"Value Add" Career Path (9M1)"
Mr. Keith Stobie (BEA Systems, Inc.) "Creating a Testing Culture (9M2)"
QUICKSTART...
Ms. Elfriede Dustin & Jean Paul (CSC) "Moving From Conventional Testing
to Object Oriented Testing (9Q)"
Friday, 28 May 1999, 10:30 - 12:00 -- KEYNOTE SESSION #3
Dr. Jakob Nielsen (Nielson Norman Group) "Facing the Future: Usability
Aspects of Quality (10P1)"
Mr. Brian Marick (Reliable Software Technologies) "Facing the Future:
Trapped by Models (10P2)"
Dr. Boris Beizer (Independent Consultant) "The Mavin (10P3)"
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
QW'99 ADVISORY BOARD MEMBERS
Frank Ackerman (Consultant) * Tim Anderson (Segue)
Larry Apfelbaum (Teradyne) * Walter Baziuk (Nortel, Canada)
Boris Beizer (Analysis) * Bill Bently (Mu_Research)
Larry Bernstein (Consultant) * Antonia Bertolino (IEI/CNR, Pisa, Italy)
Robert Binder (RBSC, Inc.) * Robert Birss (PricewaterhouseCoopers)
Jack Bishop (Silicon Valley Networks) * Rita Bral (SR/Institute)
Lori Clarke (UMass) * Tom Drake (CRTI)
Walt Ellis (SW Metrics) * William Everett (SPRE)
Danny Faught (HP) * Dick Hamlet (Portland State)
Bill Howden (UC/San Diego) * Neil Hunt (Rational)
Andre Kok (CMG, Netherlands) * Brian Marick (RST)
Edward Miller (SR, Inc.) * John Musa (Consultant)
Emilia Peciola (Ericsson, Sweden) * Martin Pol (IQUIP, Netherlands)
Rob Schultz (Motorola) * Antonio Serra (MetriqS, Italy )
Keith Stobie (BEA Systems) * Otto Vinter (Bruel & Kjaer, Denmark)
Tony Wasserman (Software Methods+Tools) * Lee White (CWRU)
Hakan Wickberg (Volvo, Sweden)
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Q W ' 9 9 R E G I S T R A T I O N I N F O R M A T I O N
Complete registration with full information about the conference is
available on the WWW at
<http://www.soft.com/QualWeek>
where you can register on-line.
We will be pleased to send you a QW'99 registration package by E-mail,
postal mail or FAX on request. Send your E-mail requests to:
qw@soft.com
or FAX or phone your request to SR/Institute at the numbers below.
QW'99: 24-29 May 1999, San Francisco Bay Area, California USA
+-----------------------------------+----------------------------------+
| Quality Week '99 Registration | Phone: [+1] (415) 861-2800 |
| SR/Institute, Inc. | TollFree (USA): 1-800-942-SOFT |
| 1663 Mission Street, Suite 400 | FAX: [+1] (415) 861-9801 |
| San Francisco, CA 94103 USA | E-Mail: qw@soft.com |
| | WWW: http://www.soft.com |
+-----------------------------------+----------------------------------+
========================================================================
Open Letter to Friends of the National Software Council
by
John Marciniak
As some of you may know, the NSC as we know it has evolved into the
Center for National Software Studies. The Board of Directors believes
that a center operation enables the forging of strategic partnerships
needed to carry out the program envisioned and to accomplish the goals
and objectives that the NSC had set forth.
The CNSS will address software issues through a three-part program
composed of forums, communications, and studies. CNSS forums will bring
together professional, academic, and government leaders to address
national software issues and develop findings and recommendations. These
forums will increase communication and understanding, and bring focus to
issues that require in- depth study and analysis. The CNSS will
establish an extensive program of communications through published
reports, internet sites, and conference participation to promulgate its
findings and solicit feedback and participation in its work. Studies
will be initiated by the CNSS or commissioned by supporters, and will
develop findings and recommendations on software issues. Studies will
be carried out by CNSS staff and Fellows, monitored by the Board of
Trustees, and reviewed by the National Software Council.
At the current time, the CNSS Board of Directors is building a base of
support for the CNSS as a prelude to seeking monetary support. The CNSS
concept is set forth in a preliminary prospectus, which can be viewed at
our Web site: <http://www.Cnsoftware.org>
We ask you to review the prospectus, and if you believe as we do in this
concept, provide a letter of support. Topics to consider are:
Issue(s) that you are concerned about/with National software issues
Areas that the CNSS should be involved with
We also appreciate comments.
If you decide to send us a letter please let us know if we have your
permission to include this letter in the prospectus package.
With the prospectus, and the letters of support, we will be able to
approach potential benefactors.
Please send your letter to:
Alan Salisbury, President
Center for National Software Studies
P. O. Box 8981
Reston, VA 20195-2981
Many thanks in advance for your support!
========================================================================
Y2K SOLUTION ACHIEVES NEW SPELL OF SOPHISTICATION
submitted by
Dave Stringer-Calvert
To: All Computer Users
Date: Thu, 11 Feb 1999 11:46:49 -0800
Subject: Memo on Y2K (source unidentified)
Dear Boss:
I hope that I haven't misunderstood your instructions. Because to be
honest, none of this Y to K problem makes any sense to me. At any rate
I have finished the conversion of all of the months on all the company
calendars for next year (year 2000). The calendars have returned from
the printer and are ready to be distributed with the following new
months:
Januark
Februark
Mak
Julk
I've also changed the following days:
Mondak
Tuesdak
Wednesdak
Thursdak
Fridak
Saturdak
Sundak
In general, all references to "Day" were changed to "Dak" (e.g.
"President's Dak"). And all references to "Birthday" were changed to
"Birthdak" (e.g. "Washington's Birthdak").
I had a hard time deciding about "New Year's Day", "Martin Luther King,
Jr. Day", "Yom Kippur", and "Hanukkah", but I finally changed them to
"New Kear's Dak", "Martin Luther Ying, Jr. Dak", "Kom Yippur", and
"Hanuyyah".
========================================================================
CAPBAK/Web Early Release on Windows 95/NT
We will be making our initial version of CAPBAK/Web [IE] 1.4 available
for download later this month.
This product is described in an extensive "Frequently Asked Questions"
style format at the following URL on our WebSite:
<http://www.soft.com/Products/Web/CAPBAK/faq.html>
In addition, CAPBAK/Web generally meets all of the requirements that we
set out in this Technology Note about WebSite verification and
validation:
<http://www.soft.com/Products/Web/Technology/challenge.html>
CAPBAK/Web is distinguished by these characteristics:
o Full object mode operation for capture and playback of user
activity on a WebSite.
o "C" language script interface for easy to edit, easy to modify
playback scripts.
o Complete WebBrowser capability based in the current release of IE.
Versions based on Netscape (Mozilla) and on Amaya coming soon.
o Built-in 1 msec. resolution fully resettable timer to accurately
measure WebSite performance.
o Interfaced to SMARTS for complete test suite control.
If you are interested in trying out this completely new approach to
testing and analyzing your WebSite analysis and testing please send
email to and request to be put on the early evaluation
list.
For a hint of some of the things we're doing with this new product
please take a look at:
<http://www.soft.com/Products/Web/CAPBAK/pppp.html>
========================================================================
TOM -- The Test Organisation Maturity model
by
Paul Gerrard, Systeme Evolutif Limited, London, UK
The Need for a Test Process Maturity Model
Evolutif have been conducting test process improvement projects since
1991. To help us to improve our clients' test practices by focusing on
what is most important, we continue to refine our approach to test
process improvement. All process improvement methods require an initial
assessment of current practices and this is used to measure the current
capability, identify shortcomings and guide the improvement process. For
several years we have been seeking a process model that helped us to
assess an organization's testing maturity objectively, and which could
be used to identify a set of appropriate improvements.
The Capability Maturity Model (CMM) is the best known Software
Engineering process model. It has as its foundation an incremental set
of maturity levels. Each level consists of a set of Software Engineering
practices that an organization must use for them to reach that maturity
level. To reach the next level of maturity, the organization must
implement the practices identified as part of the next level and so on.
The CMM attempts to provide a sequenced set of process improvements to
reach the ultimate process capability.
The CMM (and related models) have been found lacking when it comes to
testing practices. The detail presented in these models is sparse, to
say the least. Attempts have been made to refine the CMM as well as to
come up with alternative testing-specific models. We have found that
even these models did not match the way we conducted testing improvement
projects. These models are based on assessments that identify whether
certain good practices are used or not, and present a staged sequence of
process improvements. The recommended process improvements consist of
the good practices not currently being adopted. The assumption is that
these practices will increase testing effectiveness and improve software
quality.
Remedy-Based Models are Inadequate
Several problems with such approaches (particularly the CMM) have been
documented but we would emphasize one in particular. We believe that
these models are all solution or 'remedy-based' and miss the point.
Consider what might happen, if a doctor adopted a remedy-based diagnosis
process. If you had a headache, and the doctor asked you a series of
questions relating to possible remedies, this would probably perplex
you: 'are you taking aspirin?', 'are you taking penicillin?'... These
questions are not related to the problem and would be very
unsatisfactory, unless of course, you were a hypochondriac and wanted to
take a lot of pills.
Process assessments that are remedy-based are also unsatisfactory. Most
organizations wishing to improve their test practices have one or more
specific problems they wish to solve. E.g. 'testing costs too much',
Answering NO to questions such as, 'do you conduct inspections?', 'do
you use a tool?', 'are incidents logged?' should not mean that
inspections, tools and incident logging are automatically the best
things to do next. The remedies recommended may be based on the
sequencing of practices in the model, not because it will help the
organization solve its software development problems.
We fear that many organizations use remedy-oriented approaches blindly.
Assuming that an organization's problems can be solved by adopting the
adopting new practices may outweigh the marginal benefit of using them.
For example, an organization might use 80% of CMM level 2 practices and
60% of level 3 practices, but would not be assessed at a level higher
than level 1. If the organization adopted the last 20% of level 2
practices would they automatically benefit? There might be some benefit,
but it is more likely that those practices are not adopted because the
benefits are marginal or negative at this time.
We believe that process improvement methods that use remedy-based
approaches are inadequate because they do not take existing problems,
objectives and constraints into consideration.
Process Models and Process Improvements
In our experience, the major barriers to improved practices are
organizational, not technical. Most of the difficulties in the
implementation of improved practices are associated with changing
management perceptions, overcoming people's natural resistance to change
and implementing workable processes and management controls.
For example, management may say 'testing takes too long' and believe
that an automated tool can help. Buying a tool without further analysis
of the problems would probably waste more time than it saves: time is
spent getting the tool to work, the tool doesn't deliver the benefits
promised, so the situation is made worse and the tool would end up as
shelfware.
The underlying issue to be addressed is most likely due to a combination
of problems. Management doesn't understand the objectives of testing;
the cost of testing is high but difficult to pin down; developers,
testers, users may never have been trained in testing; the quality of
the product delivered into testing is poor, so takes forever to get
right. To address the management problem, a mix of improvements is most
likely to be required: management awareness training; testing training;
improved definition of the test stages and their objectives; measurement
of the quality of the product at each stage etc. etc.
We believe that the assessment model must take account of the fact that
not all improvements are a good idea straight away. Some improvements
are expensive; some save time, but the changes to the way people work
may be dramatic; some improve the quality of the testing, but take
longer to perform. Very few improvements save time, improve quality,
cause minimal change and pay back after two weeks. Recommended
improvements must take account of other objectives, constraints and
priorities.
The Test Organisation Maturity Model (TOM(tm))
Evolutif have developed a Test Organisation Model, TOM(tm) to address
the primary concern that the outcome of the assessment should address
the problems being experienced. The assessment process is based on a
relatively simple questionnaire that can be completed and a TOM(tm)
score derived without the assistance of a consultant.
The questionnaire appears on the following pages and is governed by the
following:
* The questions focus on organizational rather than technical issues and
the answers, in most cases, can be provided by management or
practitioners (try both and compare).
* The number of questions asked is small (twenty).
* The objectives of the organization assessed should be taken into
consideration and prioritized. (Do we want to get better, or do we want
to save money?)
* Questions relate directly to the symptoms, not remedies. (What's going
wrong, now?)
* Symptoms are prioritized. (Release decisions are made on 'gut feel'
and that's bad, but we are more concerned that our sub-system testing is
poor).
* The scoring system is simple. All scores and priorities are rated from
one to five.
The Improvement Model
A potential process improvement may help to solve several problems. The
improvement model is a simple scoring/weighting calculation that
prioritizes potential improvements, based on the assessment scores and
priorities. The model has a library of 83 potential testing
improvements. For each symptom, a selection of improvements has been
deemed most appropriate, and weighted against the objectives and
constraints.
When the questionnaire is completed, the scores are entered, and a
prioritized action list of potential process improvements is generated.
How the TOM(tm) Questionnaire is used
When completed, the questionnaire can be used to calculate a TOM(tm)
level. Since you must answer twenty questions with scores of 1-5, you
can score a minimum of 20 and a maximum of 100. If you repeat the
questionnaire after a period, you can track progress (or regress) in
each area. If you send the completed questionnaire to us, we will enter
the data into our TOM(tm) database. We are using assessment data to
survey testing practices across industry. The database also has a
built-in improvement model. Based on the assessment data entered, the
model generates up to seventy prioritized improvement suggestions. You
can use these to identify the improvements that are likely to give the
most benefits to your organization.
Completing the Questionnaire
The questionnaire has four parts. Parts one and two are for
administrative and analysis purposes. Parts three and four are used to
calculate a TOM(tm) level and, and then analyzed in the testing
improvement model to generate prioritized improvement suggestions.
If you mail completed questionnaires to Evolutif, we will return a
printed version of the assessment and a prioritized list of up to
seventy potential test process improvements FREE. Information provided
on these questionnaires is treated in the strictest confidence.
If you wish to download a copy of the TOM assessment questionnaire with
full instructions on its use, please visit:
<http://www.ftech.net/~evolutif/>
Paul Gerrard
Business Development Manager
paulg@evolutif.co.uk
SYSTEME EVOLUTIF LIMITED SOFTWARE TESTING SERVICES
Assessment, Education, Improvement, Outsourcing, Unification
Gloucester House, 57/59 Gloucester Place, London, W1H 3PE, UK
Tel: +44 (0)171 487 4705 Fax: +44 (0)171 487 2960
Web: http://www.ftech.net/~evolutif
========================================================================
Three for Dinner: Served with Y2K Flavorings
By Ann Schadt
Boris Yeltsin, Bill Clinton, and Bill Gates were invited to have dinner
with God. During dinner, God told them, "I invited you to dinner,
because I needed three important people to send my message out to all
people Tomorrow, I will destroy the Earth!"
Yeltsin immediately called together his cabinet and told them, "I have
two really bad announcements to make. First, God really does exist, and
second, tomorrow He will destroy the Earth."
Clinton called an emergency session of Congress and told them, "I have
good news and bad news. The good news is that God does exist, and the
bad news is that He will destroy the Earth tomorrow."
Gates went back to Microsoft headquarters and told his people, "I have
two FANTASTIC announcements! First, I am one of the three most
important people on Earth, and second, the Year 2000 Problem has been
solved!"
========================================================================
Software Glitch Plagues E-Trade
Customers of online brokerage E-Trade could not place orders on the
company's Web site for more than an hour yesterday due to problems with
a software application that was changed the night before.
Most areas of the company's Web site continued to operate normally, but
customers who attempted to conduct a trade found that the system would
not accept their instructions. "They couldn't execute their orders,"
says an spokeswoman. Once the company's IT staff identified the problem,
they reversed the change made to the software, she says. E-Trade says it
has about 500,000 active accounts, and a study by Credit Suisse First
Boston ranks the brokerage's share of the burgeoning online stock market
as third after Charles Schwab Corp. and Toronto-Dominion Bank's
Waterhouse Investor Services.
The E-Trade incident is the latest in a series of problems experienced
this year by online brokerages, which have been struggling to shore up
their information systems to cope with an unexpected surge in trading
volume. Trade volumes at Schwab were up more than 600% last year, and
E-Trade experienced 20% compounded increases in daily volume over a
period of four days last month. While frantically trying to keep up,
Schwab, Waterhouse, and other brokerages have also seen their Web sites
go down or have experienced sluggish performance. --Gregory Dalton
Grateful acknowledgement to <http://www.informationweek.com>.
========================================================================
The Y1K Crisis, Canterbury, England, A.D. 999
(Sent in by John Favaro, favaro@pisa.intecs.it)
An atmosphere close to panic prevails today throughout Europe as the
millennial year 1000 approaches, bringing with it the so-called "Y1K
Bug," a menace which, until recently, hardly anyone had ever heard of.
Prophets of doom are warning that the entire fabric of Western
Civilization, based as it now is upon monastic computations, could
collapse, and that there is simply not enough time left to fix the
problem.
Just how did this disaster-in-the-making ever arise? Why did no one
anticipate that a change from a three-digit to a four-digit year would
throw into total disarray all liturgical chants and all metrical verse
in which any date is mentioned? Every formulaic hymn, prayer, ceremony
and incantation dealing with dated events will have to be re-written to
accommodate three extra syllables. All tabular chronologies with three-
space year columns, maintained for generations by scribes using
carefully hand-ruled lines on vellum sheets, will now have to be
converted to four-space columns, at enormous cost. In the meantime, the
validity of every official event, from baptisms to burials, from
confirmations to coronations, may be called into question.
"We should have seen it coming ," says Brother Cedric of St. Michael
Abbey, here in Canterbury. "What worries me most is that THOUSAND
contains the word THOU, which occurs in nearly all our prayers, and of
course always refers to God. Using it now in the name of the year will
seem almost blasphemous, and is bound to cause terrible confusion. Of
course, we could always use Latin, but that might be even worse -- The
Latin word for Thousand is Mille which is the same as the Latin for
mile. We won't know whether we are talking about time or distance!"
Stonemasons are already reported threatening to demand a proportional
pay increase for having to carve an extra numeral in all dates on
tombstones, cornerstones and monuments. Together with its inevitable
ripple effects, this alone could plunge the hitherto-stable medieval
economy into chaos.
A conference of clerics has been called at Winchester to discuss the
entire issue, but doomsayers are convinced that the matter is now one of
personal survival. Many families, in expectation of the worst, are
stocking up on holy water and indulgences.
========================================================================
Automated Software Engineering
ASE'99
14th IEEE International Conference
October 12 - 15, 1999
Cocoa Beach Hilton
Cocoa Beach, Florida, USA
http://sigart.acm.org/Conferences/ase/
Electronic Abstracts Due: May 1, 1999
Paper Submission Deadline: May 10, 1999
CALL FOR PAPERS
The IEEE International Conference on Automated Software Engineering
brings together researchers and practitioners to share ideas on the
foundations, techniques, tools and applications of automated software
engineering technology. Both automatic systems and systems that support
and cooperate with people are within the scope of the conference, as are
models of software and software engineering activities. ASE-99
encourages contributions describing basic research, novel applications,
and experience relevant to automating software engineering activities.
Solicited topics include, but are not limited to:
- Automated software specification and analysis
- Automating software design and synthesis
- Category-theoretic approaches
- Computer-supported cooperative work, groupware
- Domain modeling
- Knowledge acquisition
- Maintenance and evolution
- Process and workflow management
- Program understanding
- Re-engineering
- Requirements engineering
- Reuse
- Software Architecture
- Testing
- Tutoring, Help, and Documentation Systems
- User interfaces and human-computer interaction
- Verification and validation
The IEEE International Conference on Automated Software Engineering,
formerly called the Knowledge-Based Software Engineering Conference,
provides a forum for researchers and practitioners to discuss the
application of automated reasoning and knowledge representation to
software engineering problems. In conjunction with the name change two
years ago, the scope of the conference expanded to encourage
international participation and to reach other scientific communities
concerned with formal methods, partial evaluation, process support,
human-computer interface support, requirements engineering, reverse
engineering, testing, or verification & validation.
All accepted papers will be published in the proceedings. In addition,
several of the highest quality papers will be selected for a special
issue of The Journal of Automated Software Engineering (Kluwer). ASE-99
will also include invited talks, tutorials, panel discussions, and
project demonstrations for which separate calls for participation will
be issued. Following on the success of previous years, ASE-99 will
again feature a doctoral consortium for selected thesis students whose
work has not yet reached a publishable stage (see conference web page
for details).
Complete details from:
General Chair:
Dorothy Setliff
Department of Electrical Engineering
University of Pittsburgh
setliff@ee.pitt.edu
Program Chairs:
Robert J. Hall, ASE'99
AT&T Labs Research
180 Park Ave, Bldg 103
Florham Park, NJ 07932
Enn Tyugu
Kungliga Tekniska Hoegskolan, Sweden
tyugu@it.kth.se
========================================================================
SPECIAL ISSUE ON INFORMATION SYSTEMS SUPPORT FOR ELECTRONIC COMMERCE
Guest Editors: Michael P. Papazoglou and Aphrodite Tsalgatidou
INFORMATION SYSTEMS
http://www.elsevier.com/locate/infosys
Fall 1999
Electronic Commerce is the ability to conduct business via electronic
networks such as the Internet and the World Wide Web. Although
Electronic Commerce is based on the principles of Electronic Data
Interchange (EDI) it goes far beyond EDI in that it aims at supporting
the complete external business process, including the information stage
(electronic marketing, networking), the negotiation stage (electronic
markets), the fulfillment (order process, electronic Payment) and the
satisfaction stage (after sales support).
Emphasis these days is on business-to-business E-Commerce applications:
taking orders, scheduling shipments, providing customer service and so
on. However, present E-Commerce implementations automate only a small
portion of the electronic transaction process. Moreover, E-Commerce is
hampered by closed (self-contained) markets that cannot use each other's
services; incompatible frameworks that cannot interoperate or build upon
each other; and a bewildering collection of security and payment
protocols. In general, E-Commerce applications do not yet provide the
robust transaction, messaging and data access services typical of
contemporary client/server applications. While there is considerable
interest in developing robust Internet applications, protection of
significant investments in client/server technology and interoperation
with mainframe transaction servers and legacy systems is a serious
requirement.
The purpose of this special issue is to cover enabling technologies,
critical technical approaches and business-centered design methodologies
that address shortcomings of contemporary E-Commerce applications and
that can have a major impact on the evolution of business-to-business
E-Commerce. Emphasis is given to information systems technologies and in
particular how these meet the requirements of Internet-enabled business
(vertical) applications that span locational as well as organizational
boundaries. Topics addressed by this special issue include:
Architectures for E-Commerce Marketplaces
Middleware and Interoperable Platforms
E-Commerce Security Protocols and Architectures
Transaction Technologies for E-Commerce
Workflow Systems and E-Commerce
E-Commerce Brokering and Matchmaking
Negotiation Protocols and Services
Intelligent Searching Techniques
Contracting and Billing Services
Business Languages for E-Commerce
Multi-Agent Systems and E-Commerce
Multi-Media Shopping Malls and Kiosk Systems
Integrated/Virtual Enterprises
Inter-Corporate Business Engineering Methodologies
Important dates:
Submission of papers: 1 March 1999.
Author notification: 17 April, 1999.
Final papers due: 18 August, 1999.
Scheduled appearance: Fall 1999.
Instructions for authors: Five copies of original high-quality
submissions, following the general author instructions of Information
Systems ftp://kubin.informatik.rwth-aachen.de/pub/InfSys/styles/latex,
should be sent to one of the guest editors:
Michael P. Papazoglou
Tilburg University
INFOLAB
P.O. Box 90153
5000 LE Tilburg
The Netherlands
E-mail: mikep@kub.nl
http://infolabwww.kub.nl:2080/infolab/people/mikep
Aphrodite Tsalgatidou
University of Athens
Department of Informatics
Panepistimiopolis, TYPA Buildings
Ilisia, Athens 157 71,
Greece
E-Mail: afrodite@di.uoa.gr
http://www.di.uoa.gr/~afrodite/
========================================================================
------------>>> TTN SUBMITTAL POLICY <<<------------
========================================================================
The TTN Online Edition is E-mailed around the 15th of each month to
subscribers worldwide. To have your event listed in an upcoming issue
E-mail a complete description and full details of your Call for Papers
or Call for Participation to "ttn@soft.com".
TTN On-Line's submittal policy is as follows:
o Submission deadlines indicated in "Calls for Papers" should provide at
least a 1-month lead time from the TTN On-Line issue date. For
example, submission deadlines for "Calls for Papers" in the January
issue of TTN On-Line would be for February and beyond.
o Length of submitted non-calendar items should not exceed 350 lines
(about four pages). Longer articles are OK and may be serialized.
o Length of submitted calendar items should not exceed 60 lines (one
page).
o Publication of submitted items is determined by Software Research,
Inc. and may be edited for style and content as necessary.
DISCLAIMER: Articles and items are the opinions of their authors or
submitters; TTN-Online disclaims any responsibility for their content.
TRADEMARKS: STW, TestWorks, CAPBAK, SMARTS, EXDIFF, Xdemo, Xvirtual,
Xflight, STW/Regression, STW/Coverage, STW/Advisor, TCAT, TCAT-PATH, T-
SCOPE and the SR logo are trademarks or registered trademarks of
Software Research, Inc. All other systems are either trademarks or
registered trademarks of their respective companies.
========================================================================
----------------->>> TTN SUBSCRIPTION INFORMATION <<<-----------------
========================================================================
To SUBSCRIBE to TTN-Online, to CANCEL a current subscription, to CHANGE
an address (a CANCEL and a SUBSCRIBE combined) or to submit or propose
an article, use the convenient Subscribe/Unsubscribe facility at
<http://www.soft.com/News/TTN-Online>. Or, send E-mail to
"ttn@soft.com" as follows:
TO SUBSCRIBE: Include in the body the phrase "subscribe {your-E-
mail-address}".
TO UNSUBSCRIBE: Include in the body the phrase "unsubscribe {your-E-
mail-address}".
QUALITY TECHNIQUES NEWSLETTER
Software Research, Inc.
1663 Mission Street, Suite 400
San Francisco, CA 94103 USA
Phone: +1 (415) 861-2800
Toll Free: +1 (800) 942-SOFT (USA Only)
Fax: +1 (415) 861-9801
Email: qtn@soft.com
Web: <http://www.soft.com/News/QTN-Online>
## End ##