sss ssss rrrrrrrrrrr
ssss ss rrrr rrrr
sssss s rrrr rrrr
ssssss rrrr rrrr
ssssssss rrrr rrrr
ssssss rrrrrrrrr
s ssssss rrrr rrrr
ss sssss rrrr rrrr
sss sssss rrrr rrrr
s sssssss rrrrr rrrrr
+===================================================+
+======= Testing Techniques Newsletter (TTN) =======+
+======= ON-LINE EDITION =======+
+======= December 1998 =======+
+===================================================+
TESTING TECHNIQUES NEWSLETTER (TTN), Online Edition, is E-mailed monthly
to support the Software Research, Inc. (SR)/TestWorks user community and
to provide information of general use to the worldwide software quality
and testing community.
Permission to copy and/or re-distribute is granted, and secondary
circulation is encouraged by recipients of TTN-Online provided that the
entire document/file is kept intact and this complete copyright notice
appears with it in all copies. (c) Copyright 1998 by Software Research,
Inc.
========================================================================
INSIDE THIS ISSUE:
o QW'99 Advisory Board Announced
o Special Issue on Information Systems Support for Electronic
Commerce: Call for Papers
o Software Fault Injection: Innoculating Programs Against Errors, J.
Voas an G. McGraw (Wiley Computer Publishing)
o TestWorks Product Updates
o Summary of Dot Graham's Plenary Session Survey from QWE'98
o A Reader's Suggestion
o Millennium Event Planning (Humor)
o The Basis Path Test Path Controversy (Revisited)
o The Hidden Code of Y2K
o Minimize Testing by Removing Redundant Code
o TTN SUBMITTAL, SUBSCRIPTION INFORMATION
========================================================================
S e a s o n s ' s G r e e t i n g s
We here at at SR and SR/Institute
would like to take this opportunity
to wish you and yours our
Very Best Wishes
for the upcoming
Holiday Season!
And we extend to everyone our hope for a
Peaceful and Happy New Year!
========================================================================
QW'99 ADVISORY BOARD ANNOUNCED
We are pleased to announce the Advisory Board for QW'99. The Advisory
Board reviews all submitted papers and abstracts and generally is
responsible for assuring quality in the conference.
This year in response to many suggestions we have added a number of
members from Vendor Companies. Composition of the QW'99 board stands at
around 18% non-USA and around 21% vendors.
Frank Ackerman * Consultant * FAckerman@aol.com
Tim Anderson * Segue * taa@segue.com
Larry Apfelbaum * Teradyne * larry@sst.teradyne.com
Walter Baziuk * Nortel, Canada * baziuk@nortelnetworks.com
Boris Beizer * Analysis * bbeizer@sprintmail.com
Bill Bently * Mu_Research * wbently@fourway.net
Larry Bernstein * Consultant * lbernstein@worldnet.att.net
Antonia Bertolino * IEI/CNR, Pisa * bertolino@iei.pi.cnr.it
Robert Binder * RBSC, Inc. * rbinder@rbsc.com
Robert Birss * Price-Waterhouse * robert.birss@us.pwcglogal.com
Jack Bishop * Slicon Valley Networks * jack@svnetworks.com
Rita Bral * SR/Institute * bral@soft.com * Executive Director
Lori Clarke * UMass * clarke@cs.umass.edu
Tom Drake * CRTI * tom.drake@coastalresearch.com
Walt Ellis * SW Metrics * waltelli@erols.com
William Everett * SPRE * w.w.everett@computer.org
Danny Faught * HP-Convex * faught@rsn.hp.com * BOFS Chair
Dick Hamlet * Portland State * hamlet@cs.pdx.edu
Bill Howden * UC/San Diego * howden@cs.ucsd.edu
Neil Hunt * Rational * nhunt@rational.com
Andre Kok * CMG, Netherlands * andre.kok@cmg.nl
Brian Marick * RST * marick@rstcorp.com
Edward Miller * SR/Institute * miller@soft.com * General Chair
John Musa * Consultant * j.musa@ieee.org
Emilia Peciola * Ericsson, Sweden * E.Peciola@rd.tei.ericsson.se
Rob Schultz * Motorola * schultz@cig.mot.com
Keith Stobie * BEA Systems * Keith.Stobie@BEAsys.com
Tony Wasserman * Software Methods & Tools * tonyw@methods-tools.com
Hakan Wickberg * Volvo, Sweden * it1.wickberg@memo.volvo.se
Lee White * CWRU *leew@alpha.ces.cwru.edu
Complete information about QW'99 is available at the Conference Website:
<http://www.soft.com/QualWeek/QW99> or by Email from qw@soft.com.
Remember: Title & abstract and presentation proposals for QW'99 are due
18 December 1998.
Please mark your calendars NOW for 1999:
Quality Week '99
QW'99
San Jose, California
24-28 May 1999
Quality Week Europe '99
QWE'99
Brussels, Belgium
1-5 November 1999
========================================================================
INFORMATION SYSTEMS
http://www.elsevier.com/locate/infosys
CALL FOR PAPERS
SPECIAL ISSUE ON
INFORMATION SYSTEMS SUPPORT FOR ELECTRONIC COMMERCE
Guest Editors: Michael P. Papazoglou and Aphrodite Tsalgatidou
Fall 1999
Electronic Commerce is the ability to conduct business via electronic
networks such as the Internet and the World Wide Web. Although
Electronic Commerce is based on the principles of Electronic Data
Interchange (EDI) it goes far beyond EDI in that it aims at supporting
the complete external business process, including the information stage
(electronic marketing, networking), the negotiation stage (electronic
markets), the fulfillment (order process, electronic Payment) and the
satisfaction stage (after sales support).
Emphasis these days is on business-to-business E-Commerce applications:
taking orders, scheduling shipments, providing customer service and so
on. However, present E-Commerce implementations automate only a small
portion of the electronic transaction process. Moreover, E-Commerce is
hampered by closed (self-contained) markets that cannot use each other's
services; incompatible frameworks that cannot interoperate or build upon
each other; and a bewildering collection of security and payment
protocols. In general, E-Commerce applications do not yet provide the
robust transaction, messaging and data access services typical of
contemporary client/server applications. While there is considerable
interest in developing robust Internet applications, protection of
significant investments in client/server technology and interoperation
with mainframe transaction servers and legacy systems is a serious
requirement.
The purpose of this special issue is to cover enabling technologies,
critical technical approaches and business-centered design methodologies
that address shortcomings of contemporary E-Commerce applications and
that can have a major impact on the evolution of business-to-business
E-Commerce. Emphasis is given to information systems technologies and in
particular how these meet the requirements of Internet-enabled business
(vertical) applications that span locational as well as organizational
boundaries. Topics addressed by this special issue include:
Architectures for E-Commerce Marketplaces Middleware and Interoperable
Platforms E-Commerce Security Protocols and Architectures Transaction
Technologies for E-Commerce Workflow Systems and E-Commerce E-Commerce
Brokering and Matchmaking Negotiation Protocols and Services Intelligent
Searching Techniques Contracting and Billing Services Business Languages
for E-Commerce Multi-Agent Systems and E-Commerce Multi-Media Shopping
Malls and Kiosk Systems Integrated/Virtual Enterprises Inter-Corporate
Business Engineering Methodologies
Important dates:
Submission of papers: 15 February, 1999. Author notification: 17 April,
1999. Final papers due: 18 August, 1999. Scheduled appearance: Fall
1999.
Instructions for authors:
Five copies of original high-quality submissions, following the general
author instructions of Information Systems ftp://kubin.informatik.rwth-
aachen.de/pub/InfSys/styles/latex, should be sent to one of the guest
editors:
Michael P. Papazoglou
Tilburg University
INFOLAB
P.O. Box 90153
5000 LE Tilburg
The Netherlands
E-mail: mikep@kub.nl
http://infolabwww.kub.nl:2080/infolab/people/mikep
Aphrodite Tsalgatidou
University of Athens
Department of Informatics
Panepistimiopolis, TYPA Buildings
Ilisia, Athens 157 71,
Greece
E-Mail: afrodite@di.uoa.gr
http://www.di.uoa.gr/~afrodite/
========================================================================
Book Published:
Software Fault Injection -- Innoculating Programs Against Errors,
by J. Voas and Gary McGraw
Wiley Computer Publishing
ISBN 0-471-18381-4
Software development is a peculiar process, half science, half art. Now
that software is doing things like controlling airplanes and bank
accounts, the artsy part had better be backed by solid engineering
practice. Consumers are no longer satisfied by code that mostly works.
Fault injection is a useful tool in developing high-quality, reliable
code. Its ability to reveal how software systems behave under
experimentally controlled anomalous circumstances makes it an ideal
crystal ball for predicting how badly good software can behave.
This complete, how-to guide to a revolutionary new approach to software
analysis gets developers, programmers, and managers up to speed on
cutting-edge fault injection techniques. Fault-injection pioneers
Jeffrey Voas and Gary McGraw use real-world case studies and code to
demonstrate the unique benefits and challenges associated with these
techniques. Fault injection is useful in multiple domains including:
* Testing--predicting where faults are most likely to hide
* Safety--simulating failures in real software environments and
estimating worst-case scenarios
* Law--predicting the level of liability incurred by a piece of
code
* Security--uncovering potential security vulnerabilities during
the development cycle
* Reuse--obtaining a more accurate read on crucial maintenance and
reuse issues
* Engineering--seamlessly introducing fault-injection methods into
your software process
========================================================================
TestWorks Product Updates
Major product releases and upgrades are always noted in the "What's New"
section of our Website <http://www.soft.com/News/whatsnew.html>. Here
is a summary of recent changes made to the TestWorks Product suite:
o TCAT/C-C++ for Windows has been upgraded to provide support for
MicroSoft Visual C++ Ver. 6. TCAT/C-C++ now supports both Ver. 5
and Ver. 6 (there are two separate downloads depending on which
compiler you use because the installations are slightly different
depending on the compiler).
o TCAT for Java has been upgraded to be better integrated with
Microsoft's Visual Java VJ++ Ver. 6.0 as well VJ++ 1.n.
In addition, this latest revision of TCAT for Java provides full
support for Ver.1.2 of Java, full support for instrumentation of
all features in the most-recent JFCs, and full support for all JDC
Ver.1.2 features.
o The license software for all of the Windows Products (CAPBAK,
SMARTS, TCAT/C-C++ and TCAT for Java) has been reorganized to make
management of the keys simpler. All of the keys for any
combination of TestWorks products can now reside in a single
directory.
o We have available two new PowerPoint presentations that describe
two important new thrusts for TestWorks development:
o Remote Testing Technology (RTT). This powerful new
collection of tools and associated techniques offers
developers the means to mechanize collection of user
behavior information prior to product release. The RTT
PowerPoint presentation shows the structure of the RTT
offering and outlines many of the details of how RTT works.
o WebSite Validation. This PowerPoint presentation gives an
overview of the new CAPBAK/Web facility that will form the
basis for TestWorks/Web based validation of websites, with
particular emphasis on E-commerce applications.
To obtain a copy please send Email to sales@soft.com and ask for
either the RTT PTT document or the WebSite PPT document.
o Regular public TestWorks product training weeks are now set for:
15-19 February 1999
12-16 April 1999
For complete details including day-by-day course descriptions
please check the website pages at:
<http://www.soft.com/Training/index.html>
========================================================================
Summary of Dot Graham's Plenary Session Survey
At QWE'98 one of the Conference Keynoters took an informal survey of the
attendees regarding their relative adoption of software tools. Here are
the results of that survey (which, to be strictly correct, was scored
very approximately).
How many have tool support of the following kinds:
Test execution including Capture/Playback? 85%
Test Management including traceability? 50%
Performance or load testing? 10%
Static/dynamic analysis (metrics, memory leaks)? 10%
Coverage meaurement? 20%
Test input generation? 2%
Defect tracking? 80%
Configuration management? 80%
CM for testware? 60%
How many want more tools? 40%
How many have achieved some benefit? 90%
How many are satisfied with test
automation (i.e. achieved significant benefits)? 5%
How many experienced significant problems? 50%
How many have shelfware? 20%
========================================================================
A READER'S SUGGESTION
> If you have ... a gripe about what quality assurance
> quality control is all about ...
> ... I urge you to compose your item and forward
> to me for possible inclusion in a future issue of TTN-Online.
A suggestion would be to raise the issue of the broader definition of QA
such that the industry -- especially IT/DP shops and the recruiters and
HR folks who advertise for them -- stops limiting QA to testing only. I
find this to be a disservice to efforts to improve understanding about
quality. And I think, truthfully, it trivializes testing's role/value
as more value-added than just bug finding.
Scott P. Duncan
SoftQual Consulting
http://www.mindspring.com/~softqual/
========================================================================
Millennium Event Planning
THE RITZ-CARLTON MILLENNIUM EXPERIENCE
If you want to do it right, Ritz-Carlton offers "The Ultimate Y2K
Experience" at $100,000. Here's what you get:
- 3 night stay
- The Ritz-Carlton Suite, the presidential suite
- Men's and ladies' 18-karat gold Bvlgari watches
- A Jaguar automobile for the weekend
- 24-hour private butler service
- Magnum of Taittinger champagne
- Two Baccarat flutes in custom-designed gift box
- Daily massage for two in room,
- Commemorative photo album & engraved stationery,
- Bvlgari's luxurious Eau Parfumee amenities
- Limo transportation to/from airport
- Plus signature services reflective of each location.
========================================================================
THE BASIS PATH TEST PATH CONTROVERSY (REVISITED)
> Dear TTN:
>
> Heard someone mention an example of a problem with the McCabe's basis path
> model having to do with loops...
>
> Could you please provide an example?
>
> Thanks
> Bob
Dear Bob:
Don't know exactly what you heard, but there is one possible limitation
in the basis-path testing idea.
The problem arises in programs which have structures that don't map 1-1 into
pure-structured programs (i.e. those with only WHILES and IF's and no GOTOs).
Actually, any digraph with a multiple-exit loop will do it.
The easiest way to show this is with this program fragment:
procedure ()
startup
while (loop)
body1
if (condition)
response
return
endif
body2
endwhile
closedown
return
end
This program has 1 while and 1 if. As a digraph it has 4 nodes and 5
edges. The cyclomatic number (the number of basis paths) can be
computed with one of two formulas:
#if's + #while's + 1 = 3
#edges - #nodes + 2 = 3
If you do the basis path thing you miss at least one of the actual paths
in the program. We see the four paths paths as:
1 startup closedown
2 startup body1 response
3 startup (body1 body2)**k closedown
4 startup body1 (body2 body1)**k response
But the basis path calculation suggests #if's + #while's + 1 = 3 which
clearly leaves one path out.
Conundrum: Why does the basis-path idea miss the mark here? What is
left out?
-TTN
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Bob's Response:
Dear TTN:
> Dear Bob:
> See the above. Actually, if you can find any flaw in this I'd be glad to
> hear about it...
>
> But this has been hanging around for a long time unresolved
> to my mind...
>
> Basically, do you believe there really are four paths in
> that fragment?
>
> And, are these four paths significant?
Yes, here's why:
Let:
s = startup
w = loop header
g = body1 + predicate
h = body2
x = response
c = close
r = return
Your four cases are:
1 swcr
2 swgxr
3 sw(gh)*cr
4 sw(gh)*gxr (reformulating a little)
Make a segment table
s w g h x c r
-----------------------------------
1 x x x x
2 x x x x x
3 x x x x x x
4 x x x x x x
The unique segment combinations are determined by g, h, x, c. There are
only four cases, but each is a unique combination (with any n segments,
there are 2**n combinations, but most are excluded by structure, and
others could be infeasible.)
If we take any three of these four, we must omit a unique combination of
segments. So, yes, I agree that there are four paths, which are not
just an artifact of iteration.
The "basis path" model is satisfied with any three and therefore will
miss at least one unique combination of segments. But this is because
it is a bogus model.
Here's my example
void bar(){
if (a==b) {
obj.foo(a);
} else {
obj.foo(b);
}
if (c==d) {
obj.foo(c);
} else {
obj.foo(d);
}
return;
}
The cyclomatic number for this method is 3. However, there are 4
distinct entry-exit paths (denoted by the argument sent to foo) a-c, a-
d, b-c, b-d. You can achieve cyclomatic coverage and not exercise one
of these paths. The problem is worse if there are deeper nestings of
predicates in series. In fact, it is easy to show that there are
relatively simple methods for which cyclomatic coverage does not even
require statement coverage!
This is a result of the fact the cyclomatic number is not defined for
directed graphs -- computing this number for a directed graph is like
trying to count the corners in a circle.
Best Bob
========================================================================
THE HIDDEN CODE OF Y2K
Robert Roskind
Y2K Solutions Group, Inc.
roskind@y2kvideos.com
Only two things threaten our existence: (1) a breakdown of society as
we know it; (2) the continuation of society as we know it.
Every lesson that mankind must learn to continue life on this planet is
inherent within one seemingly random computer event, Y2K. These lessons
are:
(1) Our competitive, power-driven instincts must quickly evolve
into co-operative ones;
(2) We must deal with our nuclear weapons, plants and waste as
they pose a continuing threat to the entire species;
(3) We must stop polluting at our present level. Most pollution
comes from the first world (one American pollutes more than
500 Nepalese).
(4) We must transform our near-gluttonous materialism and our
intoxication with physical pleasuring as it is the source of
much of the pollution;
(5) We must return our value system to one more centered in
people and relationship, as opposed to the present one
centered on owning and things;
(6) We must further develop our decentralized, non-polluting
sources of energy (wind, solar, wave, etc.);
(7) We must take our other technological threats seriously
(global warming, biological, nuclear and chemical weapons,
El Nino, ozone level, pollution, etc.);
(8) We must become more self-reliant, depending more on
ourselves and each other as opposed to our leaders and
government, who often do not have our best interest at
heart;
(9) The large corporations must be dismantled or adjust their
value system to include not only profit but concern for the
environment and people as well. Presently, their greed and
short term thinking are eating through the earth.
(10) A renewal of the use of non-hybrid seeds. At the present
rate that seed companies are promoting and selling hybrid
seeds ( seeds that are only good for one season), in 20
years there will be almost no non-hybrid seeds available. We
would have genetically engineered all food seeds on the
planet to be non-renewable.
(11) Just as American indian leaders planned seven generations
ahead, so must we consider and plan for the generations that
will follow us.
Every lesson we must learn is above. None are left out. Not learning any
one of them continues to put our species at risk. All are contained
within Y2K. The chances of this being coincidence is infinitesimally
small. And just to be sure our interest was caught, it arrives on the
nanosecond of the new millennium.
The above lessons will create a major paradigm shift on the planet. One
that is absolutely necessary if we are to continue here much longer.
These lessons will not be learned easily. They will be suffering and
pain. But they will be learned. This event is an intelligently created
and guided unfolding imbued with almost unimaginable love for our
species.
========================================================================
ICSE99 WORKSHOP, LA, MAY '99
CALL FOR PAPERS
Empirical Studies of Software Development and Evolution (ESSDE)
Tuesday May 18th, 1999
*OVERVIEW The 1st Process Modelling and Empirical Studies of Software
Evolution Workshop was held in Boston, USA, preceding ICSE97. The
success of that workshop has motivated us to hold the workshop again at
ICSE99 in LA.
*SCOPE Empirical studies applied to process models. Empirical studies of
evolving systems. The role of empirical studies in process improvement.
Empirical studies of the object-oriented paradigm. Approaches to
raising awareness of empirical research and its application in industry.
Exploration and comparison of different methods for empirical research.
*REQUIREMENTS FOR ATTENDANCE We encourage both researchers and
practitioners to participate in this workshop. Potential attendees
should submit a 3-5 page position paper outlining a contribution of
understanding, experience, or some other proposal relevant to the
workshop. Position papers must arrive no later than February 8th 1999.
Submission via email (uuencoded compressed postscript, straight
postscript or Word97) is encouraged. Notification of acceptance will
be sent by March 31st.
*WORKSHOP CHAIR Rachel Harrison (University of Southampton, UK)
*PROGRAM COMMITTEE Lionel Briand (Fraunhofer Institut (IESE), Germany);
Rachel Harrison (University of Southampton, UK); Marc Kellner (Software
Engineering Institute, CMU, USA); Manny Lehman (Imperial College, UK);
Steve MacDonell (University of Otago, New Zealand); Linda Ott (Michigan
Technological University, USA); David Raffo (Portland State University,
USA); Carolyn Seaman (University of Maryland, USA); Martin Shepperd
(Bournemouth University, UK); Lawrence Votta (Bell Labs, Lucent
Technologies Inc., USA).
*STEERING COMMITTEE Lionel Briand (Fraunhofer Institut (IESE), Germany);
Marc Kellner (Software Engineering Institute, CMU, USA); Martin Shepperd
(Bournemouth University).
*DETAILS: http://www.staff.ecs.soton.ac.uk/~rh/ICSE99/cfp.html
========================================================================
Minimize Testing by Removing Redundant Code
We recently finished a product ("CloneDR") for finding and removing
redundant code clones from programs. It finds 10% of the code of many
software systems is redundant and can be removed by abstraction. Our
motivation was to reduce the cost of maintenance by minimizing the
amount of code to be maintained.
Reading TTN, it suddenly struck me that a very interesting special case
is minimization of testing brought about by the same means. Since
testing cost is one of the largest in the software lifecycle, reduction
of code volume can have significant impact on testing, especially if it
path-based testing. I invite readers to look at the technology, (a
refereed journal paper is available, as well as slides describing how it
works) and to download a free demo copy of the clone remover (C/C++ for
now) at our website, www.semdesigns.com.
Ira Baxter, Ph.D. email: idbaxter@semdesigns.com
Semantic Designs, Inc. voice: 512-250-1018 x140
12636 Research Blvd. #C-214 FAX: 512-250-1191
Austin, Texas 78759-2200 http://www.semdesigns.com
========================================================================
TTN-Online Articles Wanted
TTN-Online, sent to over 5000 subscribers worldwide, is seeking articles
about testing, quality technology, and test automation.
If you have a pet story about testing, a gripe about what quality
assurance quality control is all about, or a sage observation you think
ought to be shared with the community, I urge you to compose your item
and forward for possible inclusion in a future issue of TTN-Online.
Articles should be in pure-ASCII format and should include an
authorization to print/re-print.
Best wishes,
Edward Miller
TTN-Online Publisher
Send articles to info@soft.com or to miller@soft.com
========================================================================
------------>>> TTN SUBMITTAL POLICY <<<------------
========================================================================
The TTN Online Edition is E-mailed around the 15th of each month to
subscribers worldwide. To have your event listed in an upcoming issue
E-mail a complete description and full details of your Call for Papers
or Call for Participation to "ttn@soft.com".
TTN On-Line's submittal policy is as follows:
o Submission deadlines indicated in "Calls for Papers" should provide at
least a 1-month lead time from the TTN On-Line issue date. For
example, submission deadlines for "Calls for Papers" in the January
issue of TTN On-Line would be for February and beyond.
o Length of submitted non-calendar items should not exceed 350 lines
(about four pages). Longer articles are OK and may be serialized.
o Length of submitted calendar items should not exceed 60 lines (one
page).
o Publication of submitted items is determined by Software Research,
Inc. and may be edited for style and content as necessary.
DISCLAIMER: Articles and items are the opinions of their authors or
submitters; TTN-Online disclaims any responsibility for their content.
TRADEMARKS: STW, TestWorks, CAPBAK, SMARTS, EXDIFF, Xdemo, Xvirtual,
Xflight, STW/Regression, STW/Coverage, STW/Advisor, TCAT, TCAT-PATH, T-
SCOPE and the SR logo are trademarks or registered trademarks of
Software Research, Inc. All other systems are either trademarks or
registered trademarks of their respective companies.
========================================================================
----------------->>> TTN SUBSCRIPTION INFORMATION <<<-----------------
========================================================================
To SUBSCRIBE to TTN-Online, to CANCEL a current subscription, to CHANGE
an address (a CANCEL and a SUBSCRIBE combined) or to submit or propose
an article, use the convenient Subscribe/Unsubscribe facility at
<http://www.soft.com/News/TTN-Online>. Or, send E-mail to
"ttn@soft.com" as follows:
TO SUBSCRIBE: Include in the body the phrase "subscribe {your-E-
mail-address}".
TO UNSUBSCRIBE: Include in the body the phrase "unsubscribe {your-E-
mail-address}".
TESTING TECHNIQUES NEWSLETTER
Software Research, Inc.
901 Minnesota Street
San Francisco, CA 94107 USA
Phone: +1 (415) 550-3020
Toll Free: +1 (800) 942-SOFT (USA Only)
FAX: +1 (415) 550-3030
E-mail: ttn@soft.com
WWW: <http://www.soft.com/News/TTN-Online>
## End ##