xxxxxxx x
xxxxxxxxxxxx xx
xxxxxxxxxxxx xxxx
xxxxxxxxx xxxxx
xxxxxxxx xxxxx
xxxxxxxx xxxxx xxx x
xxxxxx xxxxx xxxxxxx xx
xx xxx xxxxxxxxx xxx
x xxxxxxxxxx xxxx
xxxxxx xxxxx
xx xxxxxx xxxxxx
xx xxx xxxxxx xxxxxxx
xxx xxxxx xxxxxx xxxxxxx
xxxx xxxxxx xxxxxx xxxxxxx
xxxx xxxxxxx xxxxxxx
xxx xxxxxxxx xxxxxxx
xxx xx xxxxxxxx xxxxxxx
xxx xxxxxxxx xxxxxxx
xx xxxxxxxx xxxxxxxxx
xx xxxxxxxxx xxxxxxx
xxxxxxxxx xx xxxxx
xxxxxxx xxxxxxxxx xxx xxxxx
xxxxxxxxxxxxxxxxxxxxxxxx xxxx xxxx
xxxxxxxxxxxxxxxxxxxxxxxx xxxxx xxxx
xxxxxxxxxxxxxxxxxxxx xxxxxx xxxx
xx xxxxxxxx xxxxxx xxxx
xxxxxx xxxxx
xxxxxx xxxxx
xxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxx xxxxxxxxxxxxxxxx
+===================================================+
+===================================================+
+======= Testing Techniques Newsletter (TTN) =======+
+======= ON-LINE EDITION =======+
+======= June 1994 =======+
+===================================================+
+===================================================+
TESTING TECHNIQUES NEWSLETTER (TTN) is E-mailed on a monthly basis to
support the Software Research, Inc. (SR) community and to provide infor-
mation of general use to the testing community.
(c) Copyright Software Research, Inc. 1994. Permission to copy and/or
re-distribute is hereby granted to any recipient of the TTN On-Line so
long as the document/file is kept intact and this copyright statement
appears with it.
TRADEMARKS: Software TestWorks, STW, STW/Regression, STW/Coverage,
STW/Advisor, X11 Virtual Display System, X11virtual and the SR logo are
trademarks of Software Research, Inc. All other systems are either
trademarks or registered trademarks of their respective companies.
========================================================================
INSIDE THIS ISSUE:
o James Bach: "Why do Inexperienced Testers Often Think They
Understand Testing?"
o ISSTA '94 Preliminary Information
o 7th International Software Quality Week
o Object World Germany '94 To Be Bigger Than Ever in Frankfurt
o Call: 17th ICSE, Seattle, April 1995
o Calendar of Events
o TTN Submittal Policy
o TTN Subscription Information
========================================================================
WHY DO INEXPERIENCED TESTERS OFTEN THINK THEY UNDERSTAND TESTING?
James Bach
"The Data Guy"
Languages Quality Assurance
Borland International
jbach@netcom.com, (408)431-1476
(EDITORS NOTE: James Bach, a QW94 speaker and a regular contributor to
the "Computer Software Testing" InterNet Newsgroup, submitted this piece
to TTN-Online a few weeks ago and it seems to be of rather general
interest.)
Software testing is a funny kind of profession. With almost no training
or experience a reasonably well-organized child could do it. Seriously.
The result would be mediocre, but better than nothing. On the other
hand, a new college graduate with a degree in Computer Science-- let's
say an advanced degree-- but no further training or experience, would do
only a slightly better job. Computer Science graduates can work longer
hours, and have better organizational skills, otherwise they are not
qualitatively better software testers than children.
Okay, let's say this graduate really applies himself, reads some QA
books, and gains real-world project experience. In a year or eighteen
months he will probably be a pretty good software tester. Maybe, in
another ten years, he will be very good.
(Of course, long before that, it's a sure bet that he will happily aban-
don software testing and go into programming, which is better respected
and better paid.)
The learning curve in software testing is deceiving and frustrating.
Most people who find themselves testing software stay with it for a cou-
ple of years, never receive any sort of training, and go on to something
else with the idea that testing is simple monkey work. This is under-
standable, because the basic idea of testing is so simple. This
apparent simplicity fools them into thinking that they've moved far up
the learning curve.
The simple idea of testing is to compare the software to what it's sup-
posed to be. At first this is an obvious task. Only when the result of
testing is not good enough, when it costs too much money or takes too
much time, do we begin to probe deeper into the process. That's when
the waves of complexity break over us. And the more questions we ask,
the more black boxes we pry open, the less we know than when we started.
Here's what I mean. Consider the following statement:
1. Software testing is the process of comparing software to its
specification so as to reveal bugs.
Fair enough. But let's say we have to determine how many testers will
be needed to carry out this misson on a particular software package. We
decide we need to decompose the process further in order to understand
how much work there is to do.
This is the beginning of the yellow-brick road. The day is sunny and
bright. Munchkins dance. Let's begin by making sure we understand
statement #1:
1.1. What does it mean to compare software with its specification?
1.2. How are bugs revealed?
Hmm... keep going...
1.1.1. What is software?
1.1.2. What is a specification?
1.2.1. What is a bug?
Almost there...
1.1.1.1. Software is a program of instructions.
1.1.1.2. Software is a box containing user manuals, install
disks, utility programs, application programs, data files, and online
help files.
1.1.1.3. Software is the phenomenon we experience when the pro-
gram executes in a certain operating environment, on a certain hardware
platform.
1.1.1.4. Software is the total interaction between the human user
and the program over time.
1.1.1.5. Software is a solution provided to a customer.
Uh oh...
1.1.2.1. A specification is an understanding, on the part of the
developers, testers, documenters, technical supporters, marketers,
management and users of how the software is supposed to behave.
1.1.2.2. Part of a specification may be a document delivered at
the beginning of the project.
1.1.2.3. Part of a specification may be a prototype or previous
version of the software.
1.1.2.4. Part of the specification may be an ongoing dialog
between the developers, testers, documenters, technical supporters,
marketers, management and users of the software.
1.1.2.5. Part of the specification may be aspects of other soft-
ware packages that solve similar problems.
1.1.2.6. Part of the specification may be implicit in the
behavior of other software packages that this software must interoperate
with.
1.1.2.7. Part of the specification may be implicit in the operat-
ing environment of the software.
1.1.2.8. Part of the specification may be implicit in the condi-
tions of the development of the software, i.e. time constraints,
developer skill, equipment availability.
1.2.1.1. A bug is a difference between the software as it behaves
and as it is specified.
1.2.1.2. A bug is any failure of the software to meet the needs
or expectations of the developer, tester, documenter, technical sup-
porter, marketer, management or user.
1.2.1.3. A bug may be incorrect output.
1.2.1.4. A bug may be incorrect behavior.
1.2.1.5. A bug may be confusing output, or behavior.
1.2.1.6. A bug may be incorrect or confusing output or behavior
with respect to a particular operating environment, kind of user or mode
of use.
We would have to expand this outline quite a bit in order to reach the
level where the tasks and entities are directly comprehensible to an
inexperienced engineer.
This illustrates how an apparently simple process like testing is not
simple at all.
Food for thought!
-JB
========================================================================
ISSTA `94 Preliminary Information
International Symposium on Software Testing and Analysis
Seattle, Washington
August 17-19, 1994
The International Symposium on Software Testing and Analysis is the
latest in a series of leading-edge research conferences that began in
1978 (under the name Testing, Analysis, and Verification (TAV) in the
late 1980s). ISSTA provides a forum for the latest results on program
testing and analysis from academic and industrial research organiza-
tions, and it assures a meeting place for practitioners and researchers
working in this area.
The Symposium will be held at the Edgewater Inn on the waterfront in
Seattle, Washington, USA. The Inn is built out over the Puget Sound
harbor, and located within walking distance of waterfront restaurants
and shops, including the famous Pike Street Public Market. Its water-
side rooms, some with balconies, have an impressive view of the Sound
and the Olympic Mountains beyond. The meeting room also has a spectacu-
lar water-side view.
The program includes 16 regular papers; a special "workshop session" for
new ideas comprising 8 additional papers; an invited address on communi-
cations protocol testing; and two panels, on software testability and on
experimental design in software testing. There will be an "open mike"
session where any participant can make a brief presentation of new
research.
Registration is $300-$350 depending on ACM memberships and date (early
registration closes July 15). Attendance is limited to about 125.
For additional information, a copy of the program, and registration
forms, contact Dick Hamlet, (503) 725-3216, InterNet: hamlet@cs.pdx.edu,
FAX (503) 725-3211.
========================================================================
7th INTERNATIONAL SOFTWARE QUALITY WEEK
Synopsis of May 17-20, 1994 Conference
Copies of Proceedings Availabled
Software Research Institute's (a not-for-profit subsidiary of SR) 7th
International Software Quality Week (QW'94) held in San Francisco, May
17-20, was the most successful Quality Week conference so far with more
than 630 conference and exposition attendees. Represented were 300+
different companies from throughout the U.S. and from 12+ foreign coun-
tries.
QW'94 provided presentations addressing a wide range of issues with
respect to advances in software analysis methodologies and automated
test methods that can effectively help an organization implement a
manageable, well-defined repeatable process. Seven plenary sessions,
ten tutorial/seminars and 38 regular papers were presented by industry
experts, academics and practitioners in QA/QC/Testing, including the
seven keynote talks:
* Dr. Boris Beizer (ANALYSIS): "On Being Competitive"
* Mr. Robert Binder (Independent Consultant): "Testing Object-Oriented
Systems: State of the Art, State of the Practice"
* Mr. Walter Ellis (Software Process and Metrics, Washington, DC):
"National Software Council: Staus of the Initiative"
* Dr. Herb Hecht (SoHaR): "Software Dependability in Distributed Sys-
tems"
* Dr. Monika Muellerburg (GMD): "Why Systematic Testing is Difficult:
The Problem of the Sample"
* Dr. John Musa (Bell Laboratories): "Software Reliability Engineering
Practice"
* Mr. Shel Siegel (Objective Quality): "Where Are the Software Quality
Leaders and What Are They Doing?"
A special 90-minute Mini-Tutorial "What's on in Europe? European Soft-
ware Quality Initiatives?" was presented by Mr. Hans-Juergen Kugler
(K&M Technologies, Ltd.).
A limited number of the bound copies of the QW'94 Conference Proceedings
are available for $250 per copy from SR and the QW'94 Tutorial Notes are
available for $100 per copy.
For further information send your request to "qw@soft.com" or call
SR/Institute at: [1] (415) 550-3020 or FAX your request to: [1](415)
550-3030
========================================================================
OBJECT WORLD GERMANY `94 TO BE BIGGER THAN EVER IN FRANKFURT
(EDITORIAL NOTE: This conference announcement arrived over the InterNet
and the subject, being a "hot" one, may be of interest to TTN readers.)
Object World Germany returns to Frankfurt at the Intercontinental Hotel
on September 27-29, 1994, with a much-expanded program in its third year
as a leading industry forum for all system vendors, IT managers, and
software developers.
More comprehensive than ever, this year's program features 12 full-day
and half-day tutorials, 40 conference sessions, three panels, a number
of free vendor seminars, and a free exhibition.
The exhibition portion of OW Germany is expected to grow from last
year's total of more than 30 exhibitors to around 50 this time out.
Conference sessions, also expanded, will this year be divided into four
separate tracks: Management, Databases, Tools & Languages, and Applica-
tions. Each track will explore key industry themes from a different per-
spective.
Four Top Keynotes As a highlight, this year's OW Germany will feature
not two but four separate keynote addresses by industry leaders.
Philippe Kahn, President of Borland International, will open with a dis-
cussion of the strategical and practical impact of using OT for applica-
tion development.
Hans Breidler, Director of Systems Strategy at Siemens Nixdorf, will
deliver the second address, focusing on Open System Architectures.
Bertrand Meyer, President of Interactive Software Engineering, will then
present "The New Software Culture: Object-Orientation and Its Impact on
the Corporation."
Finally, attendees will learn about the World Cup USA `94 Soccer
Championship's Advanced Information Systems Project, as presented by
Juan Jose Vidal, Vice President of Technology for World Cup USA.
First-Ever CORBA Showcase A special CORBA Showcase will be on offer for
the first time in Germany at the OW show. The showcase will include a
full-day tutorial on OMG CORBA Interface Definition Language (IDL); two
presentations by OMG's Technical Director Richard Soley focusing on
"Interoperability Using Objects: OMG Status and Goals" and "CORBA and
COSS: Building Object-Based Distributed Applications"; and presentations
from representatives of Digital and IBM, who will share their experi-
ences in implementing CORBA.
In addition, the CORBA Showcase will include a special series of presen-
tations entitled "Trending Towards Objects", open to all attendees, and
hosted by John Slitz, Vice President Marketing of OMG.
New-User Case Studies In accordance with this year's heightened emphasis
on end-users, a new conference stream will be dedicated exclusively to
user case studies. Among the speakers are representatives from Deutsche
Aerospace, Dresdner Bank, Lufthansa and Telenorma.
Computerwoche Object Application Awards
This year, Object World Germany makes its recognition of real-world user
applications from OT leaders official with the launch of the first Com-
puterwoche Object Application Awards. Computerwoche, sister publication
to IDG Communications' American COMPUTERWORLD, will grant awards in five
catagories to honor the best end-user applications utilizing OT.
User Group Meetings Also new this year is OW Germany's hosting of six OT
user group meetings, ranging from C++ to Smalltalk to Telecommunication
and Multimedia.
Object World Germany is a joint project between IDG WEC, OMG and LogOn
Technology Transfer GmbH.
For more information about Object World Germany, please contact Roberto
Zicari at LogOn Technology Transfer GmbH, Tel: +49-6173-2852, or Fax:
+49-6173-940420, or Email: roberto_zicari@omg.org.
========================================================================
CALL: 17th International Conference on Software Engineering
Seattle, Washington USA
April 24-28, 1995
Sponsored by ACM SIGSOFT and IEEE Computer Society TCSE
GENERAL CHAIR: Dewayne Perry, AT&T Bell Laboratories (USA)
PROGRAM CO-CHAIRS: Ross Jeffery, University of New South Wales (AUSTRALIA)
David Notkin, University of Washington (USA)
The purpose of the ICSE-17 is to provide a forum within which to promote
the development of software engineering as both an industrial practice
and an academic discipline, extending the frontier of both the state of
the art and the state of the practice.
We solicit both research papers and experience reports on all topics in
software engineering, as well as tutorial and tools fair submissions.
In addition, five pre-Conference workshops will explore research issues
at the intersection between Software Engineering and other disciplines.
The workshop topics are Architectures for Software Systems, Program
Transformation for Software Evolution, Formal Methods Application in
Software Engineering Practice, Research Issues in the Intersection of
Software Engineering and Programming Languages, and Software Configura-
tion Management (SCM5).
IMPORTANT DATES FOR THE CONFERENCE
* Deadline for paper and tutorial submissions: September 1, 1994 * Dead-
line for tools fair submissions: January 31, 1995 * Notification of
acceptance: December 1994 * Final versions of papers due: February 1995
IMPORTANT DATES FOR THE WORKSHOPS
* Deadline for submissions: November 15, 1994 * Notification of accep-
tance: January 15, 1995 * Final versions of position papers due: March
1995
FOR MORE INFORMATION
* Up-to-the-minute information about the conference is available on the
World Wide Web (WWW) at http://www.research.att.com/#calls *
PostScript and ASCII versions of the full calls are available by
anonymous
FTP at ftp.cs.washington.edu, in directory pub/se/icse17 * Questions
can be sent by email to icse17@cs.washington.edu
========================================================================
------------------->>> CALENDAR OF EVENTS <<<-------------------
========================================================================
The following is a partial list of upcoming events of interest. ("o"
indicates Software Research will participate in these events.)
+ July 26-28, 1994: Software Quality Management
Edinburgh, SCOTLAND
Contact: Sue Owen
Conference Secretariat,
Wessex Institute of Technology
Phone: 44 (0) 703 293223
========================================================================
--------->>> TTN SUBMITTAL POLICY <<<---------
========================================================================
The TTN On-Line edition is forwarded on the 15th of each month to sub-
scribers via InterNet. To have your event listed in an upcoming issue,
please E-mail a description of your event or Call for Papers or Partici-
pation to "ttn@soft.com".
The TTN On-Line submittal policy is as follows:
o Submission deadlines indicated in "Calls for Papers" should pro-
vide at least a 1-month lead time from the TTN On-Line issue date.
For example, submission deadlines for "Calls for Papers" in the
January issue of TTN On-Line would be for February and beyond.
o Length of submitted items should not exceed 68 lines (one page).
o Publication of submitted items is determined by Software Research,
Inc., and may be edited as necessary.
========================================================================
-------------->>> TTN SUBSCRIPTION INFORMATION <<<--------------
========================================================================
To request a FREE subscription or submit articles, please E-mail
"ttn@soft.com". For subscriptions, please use the keywords "Request-
TTN" or "subscribe" in the Subject line of your E-mail header. To have
your name added to the subscription list for the quarterly hard-copy
version of the TTN -- which contains additional information beyond the
monthly electronic version -- include your name, company, and postal
address.
To cancel your subscription, include the phrase "unsubscribe" or
"UNrequest-TTN" in the Subject line.
Note: To order back copies of the TTN On-Line (August 1993 onward),
please specify the month and year when E-mailing requests to
"ttn@soft.com".
TESTING TECHNIQUES NEWSLETTER
Software Research, Inc.
901 Minnesota Street
San Francisco, CA 94107 USA
Phone: (415) 550-3020
Toll Free: (800) 942-SOFT
FAX: (415) 550-3030
E-mail: ttn@soft.com
## End ##