|
12th INTERNATIONAL SOFTWARE QUALITY WEEK (QW'99) 24-28 May 1999, San Jose / Silicon Valley, California USA PAPER AND PRESENTATION ABSTRACTS |
|
|
12th INTERNATIONAL SOFTWARE QUALITY WEEK (QW'99) 24-28 May 1999, San Jose / Silicon Valley, California USA PAPER AND PRESENTATION ABSTRACTS |
The 12th International Software Quality Week
Technical Program is organized into separate tracks depending on the content of the presentations.
Below are abstracts for all QW'99 half-day Tutorials, Keynotes, QuickStart Mini-Tutorials,
and Regular Technical Papers.
Version updated 2 May 1999.
HomePage
| Mission
| Board
| Download: { Flyer,
Brochure }
| Send Brochure
Preview
| Tour
| REGISTER
| PROGRAM
| Abstracts
| Biographies
Sponsors
| Exhibitors
| Hotel
| Car Rental
| Bay Area
| Kudos
[ AB
| CD
| EF
| GH
| IJ
| KL
| MN
| OP
| QR
| ST
| UV
| WX
| YZ ]
X-Mailer: AOL 4.0 for Windows 95 sub 4 Status: R Although failures per usage unit (reliability) is the ultimate customer- oriented measure of software quality, a more meaningful measure for software development management is fault density. The two measures are of course directly related, that is, in general higher fault density will result in higher failure rates. One would think, then, that fault density would be routinely measured and tracked throughout the software industry. Not so. One reason for this is that the measure that is wanted is faults pre new and changed lines of code. In large commercial projects making this count can be non-trivial. This presentation will describe how such a count was made and how the lessons learned there might be applied to industry in general.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
The user's first encounter with a new software package is typically with its installer program, making it especially important that the installer be of the highest possible quality. Testing software installation is a relatively constrained and repetitive task that lends itself nicely to automated methods. One of the prime advantages in doing this is that the installer tests can be run in parallel against an array of different machine configurations, minimizing the repetitive tedium of manual testing. A well-crafted automated system can verify many aspects of an installation, including: the accuracy of the purported disk space requirements, confirming that the expected files are written to disk, and checking that the program can be launched successfully afterwards. With a modular design, many of these same routines can be used to verify uninstallation of the software as well. After describing our experience with creating such a system, we'll also discuss some readily available freeware, shareware, and commercial tools that we found helpful for automating installation testing.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Syntax testing allows for formal user-input definitions for real-time simulation tools. A commercial Record/Replay tool was used as a vehicle to automatically generate various sets of test scripts that cover "clean" and "dirty" syntax. The proposed method is faster and more predictable than traditional Record/Replay testing. A Test Efficiency of 1.7 defect-categories per hour of test execution was achieved.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Capturing requirements accurately is essential to developing correct software. An often-utilized mechanism for requirements capture has been use cases.Unfortunately for most systems it is not feasible for engineers to create large numbers of use cases. Furthermore, the actual behavioral requirements of the system are only implicitly defined by these scenarios.
This paper will present a methodology where a behavioral model describing the systems actions can replace multiple use cases as a compact means of describing them. Benefits of this approach include easy analysis of behavior, rapid response to changes in specifications of requirements, tests generation correlated to requirements as well as automated generation of requirements based tests.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Customers have grown to expect that the dial town will be there, that their calls will be completed quickly, and their telephone bills will be accurate. Those famous words, "the software will be fixed in the next release", do not satisfy telecommunications customers. And if their current vendor or service provider can't offer them the quality level they expect, there are several other suppliers queuing up for their business who will.The communications industry is experiencing unprecedented growth and change with the explosion of the Internet, Client/Server computing, and the Telecommunications Reform Act of 1996. Boundaries are being reset as communications service providers and other companies are entering new markets with revolutionary technology. This boom is fueled by businesses who are becoming increasingly dependent on communications technology as part of their "mission-critical" systems and networks, where failure just isn't an option. At the heart of these systems are sophisticated electronics which handle high-speed control switching, enhanced services, billing, database access - just to name a few. Both hardware and software must be thoroughly tested to ensure that these systems will operate error free and that they withstand the rigors of peak loading conditions prior to actual deployment.
This session will explore the issues, trends and some of the techniques employed to be successful in the real practice of telecommunications testing.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
This presentation examines how First Call handled the Euro conversion and its verification. Converting the financial data from the 11 countries to the Euro is a staggering challenge due to over 40 million data points involved. Using Perl and SQL, an automated solution was created for verification purposes. We completed the design and implementation in less than 140 hours and have equipped our organization with a robust, tested, portable, and reusable currency conversion process and infrastructure.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Most software quality initiatives focus on process to achieve product improvement. We accept the importance of this, but also believe that the human factors in the processes have not been properly examined. In this paper we examine the impact of software practitioners have on product quality. We explore softer aspects of quality improvement, and emphasise the continuum between the human and technical aspects of quality during the development process.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Software sources contain typically 5-20% duplicated ("cloned") code. This paper describes a method for automatically removing such clones, decreasing maintenance and testing costs by a corresponding amount. The method uses a language grammar to achieve language independent detection of clones in spite of formatting, comment and content changes. Examples for C++ and COBOL will be shown.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
This is an overview of the testing field. Its purpose is to provide you with the technical and conceptual vocabulary of testing. Testing has emerged as a field within software engineering and has acquired a big vocabulary. It has progressed, in the past 20 years, from intuition to science -- from personal heuristics to well-understood practices rooted in theory and confirmed by use and experiments.
(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
The industry trend toward software components. such as JavaBeansTM, shifts the focus of object-oriented testing to the unit level. The central problem at this level is how to select method invocation sequences for an object. Program analysis has the potential to identify these sequences in an automated fashion. Conventional forms, such as data flow analysis, do not provide a complete answer. This tutorial introduces a new theory of object testing based on dynamic information flow analysis. The theory yields fresh insights and a more comprehensive solution. The presentation wil be informal, using simple JavaTM examples to explain the general approach.OUTLINE:
unique aspects of testing an object
foundations of structural testing
- flowgraphs
- paths
review of classical structural testing theorybeyond classical theory: information flow analysis
- control flow analysis
- data flow analysis
path structure of an object
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. James R. Bindas (Intel Corporation) [USA]
Paper Title: Tactical Improvement Projects: Real-Life Lessons in Leading Change (8M1)Leading a Tactical Improvement Project (TIP) team is a challenging task. This presentation is based on my experiences as a TIP leader in cross-organizational improvements. Specifically the presentation will detail an understanding of the typical lifecycle of a TIP, how to lead a TIP effort, and the common experiences encountered during this type of effort.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Robert V. Binder (RBSC Corporation) [USA]
Tutorial Title: Modal Testing Strategies for Object-Oriented Software (Tutorial H, Part I & II)Classes (objects) have distinctly different behavior patterns (modes). A mode must be identified to select an effective test strategy. This tutorial presents new approaches for domain/state modeling to characterize class modality and shows how to produce effective test suites from these models.Participants in this seminar will learn how to:
- Identify the mode of the class under test.
- Develop a domain model of a class.
- Develop a state model of class behavior.
- Develop a domain-based test plan using the vertex probe strategy.
- Develop a behavior-based test plan using the FREE state model.
- Develop a test suite that achieved either vertex or N+ state coverage.
(Click here for Bob Binders's Home Page)
(rbinder@rbsc.com)
(TOP OF PAGE)
RETURN TO QW'99 PROGRAM
Ms. Karen Bishop-Stone (TestWare Associates, Inc.) [USA]
Tutorial Title: Practical Software Test Case Design (Tutorial C1)This tutorial is designed to introduce the audience to practical cost-effective test case design techniques for every level of software testing from unit to user acceptance. A discussion of how to determine quality testing rather than execute quantities of tests will open the tutorial followed by explanations and exercises of various techniques.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Rex Black (Rex Black Consulting Services) [USA]
Stand-by Paper Title: Charting the Progress of System Development Using Defect Data (SBA1)Aggregate defect data, presented graphically, allow testers to give project and executive management `dashboard' information they need to drive development projects to successful conclusions. I discuss four charts that distill meaning from test findings in terms of product stability, quality, bug repair, root cause, and affected subsystems.
(TOP OF PAGE)
RETURN TO QW'99 PROGRAM
Ms. Lisa Boden & Mr. Jon Hagar (Lockheed Martin Astronautics Company) [USA]
Paper Title: How to Build a 20 Year Successful Independent Verification and Validation (IV&V) Program for the Next Millennium (3A2)IV&V is common on many critical software government programs. IV&V can save missions and improve the product quality when done right. There must be a balance of people with the right skills, management, as well as processes aimed at verification of each life cycle step, validation with simulation/analysis tools, and a hardware based test facility. Automation of testing/IV&V helps to reduce cost, but having the processes with good people is a requisite. Over 20 years of error trending data is examined and confirms problem rates are initially high but reduce as an organization matures. While IV&V is not for every software program because of redundancies, but high risk projects where there are critical cost or safety factors can benefit from some level of IV&V.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Bruce Boes (Software Emancipation Technology, Inc.) [USA]
Paper Title: Improve Your Software Quality through Quality Process-Control and Management (VT15)Software content is rapidly increasing in all products. Due to the complexity of software and the increasing rate of information present in an organization, there is a growing need for software development information management. Most organizations have software development processes that are optimized for an ideal world, but are often sacrificed by the pressures of deadlines and budgets. This session explores building quality software while facing management challenges of acquiring and managing new people while retaining flexibility, and moving from project to project. The session concludes with best practices for improving software quality, reducing maintenance costs and management of software development information.You'll obtain practical and technical information on how to:
You should attend this technical session if you need to:
- Improve software quality and reduce cost of maintenance
- Reduce the number of software defects in the product
- Adhere to programming standards
- Improve the quality of submissions
- Understand the impact of changes on the entire system
- Capture company standards for software development
- Measure and assess initial internal quality
- Monitor software quality and assure that quality remains consistently high
- Evolve your software development process from Craftsmanship to Engineering
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Nick Borelli (Microsoft Corporation) [USA]
Paper Title: Seizing Control of the Development Lifecycle (7M2)In this talk, I'll discuss scenarios where the Testing/Quality organization builds, drives and owns part or all of the Development Process and how that can help you achieve the types of Process Improvement and Quality gains that you are looking for.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Eric Bowden (KeyLabs) [USA]
Paper Title: Outsourced or Out-Of-Luck (VT14)With so much work to be done and not enough hours in a day, hardware and software companies are deciding that it is more cost effective for them to focus their resources on the things that they do best. This sometimes means finding someone else to make sure that the product meets design requirements. Outsourcing your QA testing can be advantageous, but simply having good intentions will not guarantee that your outsourced relationship will be a successful one. This paper will help you decide when outsourcing your testing needs is appropriate and will also help you understand the common pitfalls when dealing with an outsourced partner.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Michael Bowden (CYRANO) [USA]
Paper Title: Push the Limits of Your Applications and Deploy with Confidence (VT4)A description of how CYRANO solutions helped a worldwide travel agency to properly size their ERP implementation and reduce their costs. By optimally tuning their system and testing the entire application, this CYRANO customer increased their ROI, productivity and quality of service. After 16 months of performance issues, CYRANO solutions solved the problem within 4 weeks, allowing the customer to have a smooth deployment.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Ilene Burnstein, Ms. Ariya Homyen, Dr. Taratip Suwannasart, Mr. Robert Grom & Mr. Gary Saxena (Illinois Institute of Technology) [USA]
Paper Title: Using the Testing Maturity Model (TMM) to Assess and Improve Your Software Testing Process (3M2)Our research focuses on the development of a Testing Maturity Model that is designed to guide and assist software organizations with evaluating, and improving their testing processes. We report on the TMM maturity level framework, the TMM assessment model, and results of a trial evaluation of the TMM in industry. Maturity model integration issues are also discussed.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Bettina Buth, Prof. Jan Peleska & Dr. Hui Shi (Bremen University) [Germany]
Paper Title: Combining Methods for the Analysis of a Fault-Tolerant System (3T2)The acceptance of Formal Methods in industries essentially depends on their applicability in large scale realistic industrial projects. In this paper we report experiences in using a combination of methods and related tools for the analysis of the fault-tolerant data management system for the International Space Station (ISS). Our approach allows us to break down the verification task into manageable sub-tasks.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. David Carman (Bellcore) [USA]
Paper Title: Event-Based Test Generation for Distributed Systems (3S2)A fundamental test design approach that simplifies both test development and scripting will be presented. This approach can support the rapid design and automation of scalable and portable test cases. A case study demonstrating how this approach was used to test a distributed call processing system will be provided.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Eugenio Cervetto (Performance Research) [Italy]
Paper Title: Model-based performance analysis of an EDP/ERP-oriented wide area network (VT5)Many people think that the system-level performance modeling of a wide-area network is such a complex and time-consuming issue to discourage its application when a short time optimization is forced by the user dissatisfaction. Those people usually manage the efficiency issue at component level and, eventually, manage unavoidable system level problems as a matter of experience and chance. Experience shows that, as the system complexity increases, this way of thinking fails in providing predictable and satisfactory solutions and, sometimes, even drives the "performance risk" to become unmanageable. PREDICTA by Performance Research is a performance modeling tool that has been specifically designed to face the system-level complexity of a large network and provide an analytic view of its performance behavior.In this paper, we report about a recent industrial application of PREDICTA by FIAT Auto in Italy. PREDICTA was used to investigate a severe efficiency problem on its ERP/EDP WAN serving all the production, commercial and financial departments in Argentina, and, therefore, deeply impacting the performance of the organization. The modeling activity was performed at system level using model libraries describing the hardware layers (mainly IBM RS6000 servers, CISCO routers, physical links, controllers, protocols, etc.) and the IBM AIX operating system. The software layer models represent the most relevant exposure of this activity:
Finally, special emphasis is given to the performance oriented "what-if" analysis and the obtained results.
- BAAN Triton functions including Product Data Management, Work-item tracking, Enterprise Resource Planning and Computer Aided Process Planning.
- Data Base Management functions using Oracle.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Mei-Hwa Chen, Mr. Ming-Hung H. Kao & Ms. Mei-Huei Tang (SUNY Albany) [USA]
Paper Title: Investigating Test Effectiveness on Object-Oriented Software -- A Case Study (6A1)We present a case study that investigates the effectiveness of traditional testing techniques on detecting faults in object-oriented programs. In this study, we applied a black-box approach: functional testing and two white-box approaches on which statement and decision coverage criteria were used in test case selection and as a guideline for determining test adequacy. These testing techniques were applied to three industrial systems with sizes ranging from 5.6k to 21.3k LOC. The investigation began with classifying faults found in these three systems for the past three years. Based on their relevance to the object-oriented features, the faults are classified into three types where {\em type I} is strongly related to the object-oriented features such as inheritance and polymorphism; {\em type II} is related to object management, and {\em type III} is the type of faults that can be found in the non-object-oriented software as well. After applying the three testing techniques to these faulty systems, we observed that the majority of {\em type I } and {\em type II} faults still remained in the systems. This result implies the likelihood that traditional testing techniques are not adequate for detecting object-oriented faults. Furthermore, we investigated the correlation between software metrics and the distribution of each type of fault, so that, based on the structure and the complexity of the system, one can allocate test resources more effectively.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Yuri Chernak (Valley Forge Consulting, Inc.) [USA]
Paper Title: In-Process Validation and Improvement of Test-Case Effectiveness (8S2)This article focuses on in-process validation of functional test cases and the improvement of their effectiveness in finding defects. In-process validation has the benefit of providing the project team with feedback on the test process during the test cycle, i.e. before a system is released into production. The paper overviews related sources that discuss these issues. We propose a new metric -- Test Case Effectiveness -- that can be used for the purpose of in-process test suite validation. We also describe a framework for improving test-case effectiveness based on defect causal analysis. A case study is used to illustrate how the new metric and improvement framework can be applied in software projects.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Sergio Cherskov (Microsoft) [USA]
Paper Title: Testing Windows CE 3.0 Real-Time Kernel (7T2)Presenting the testing procedures and goals that were used to test and verify real-time kernel during development of Microsoft Windows CE 3.0 operating system. Presentation details the methods used to test the multi-tasking kernel (scheduling, priorities, inter-process communications), as well as the real-time performance measurements such as ISR (Interrupt Service Routine) and IST (Interrupt Service Thread) latencies.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Huey-Der Chu (National Defense Management College) [Taiwan]
Paper Title: Automating Client/Server Testing in the Real World (4S2)This paper applies mobile agents to automated test execution across multiple platforms. A test driver can be launched by a mobile agent to remote client sites to run the tests. During the testing, the mobile agent will use the Network Class Server to dynamically load the classes relevant to this test. The test result on each client site will be sent back by the mobile agent. The mobile agent roams across different platforms and finally it arrives at the application server site to bring back the trace file for inspecting the interaction behaviour among clients. This work can really let the tests run on multiple client sites across different platforms.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Patrick Copeland (Microsoft) [USA]
Paper Title: Approaches to Testing Componentization in the Windows CE Operating System (7T1)The introduction of the Microsoft Windows CE required the QA team to rethink the way it tests software. This new embedded OS supports componentization. Using the Microsoft Platform Builder, componentization allows users to create a spectrum of configurations from a simple kernel to a rich Win32 API subset. This talk will focus on the innovative testing approaches that allowed the delivery of a robust system capable of supporting numerous applications from hotdog cookers to terminals to sub-PCs to next-generation high-end 3-D game machines.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Manu Das (Soffront Software, Inc.) [USA]
Paper Title: Defect Tracking and Analysis, Software Quality, and the Internet (VT7)Now that the door has been opened on the support organization, users are bound to start asking some obvious questions. "If my computer is already connected to support, why do I need to call them on the phone when I find a bug?" "Why can't reporting a defect or requesting an enhancement be as easy as using help?" "Why can't the system simply capture configuration and program status information when a defect is encountered, send it to support with a problem report, and automatically download a fix if one is available?" "Why can't users be automatically notified about maintenance updates?"This technical seminar will explore these issues in more depth and attempt to identify facilitating technologies and obstacles to implementation of systems to meet these emerging requirements. The target audience will be development, quality assurance and support engineers trying to understand the future direction of their discipline and anyone interested in helping to stimulate an examination of this subject. Topics to be discussed will include the following:
1. The Evolution of Defect and Request Tracking Systems
2. Current State of the Art in Tracking Systems
3. Impact of the Internet and Other Technologies
4. Future of the Tracking Systems
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Michael Deck (Cleanroom Software Engineering, Inc.) [USA]
Tutorial Title: Requirements Analysis Using Formal Methods (Tutorial K, Part I & II)Inadequate, incomplete, and ambiguous requirements are a significant problem for almost every software project. This course teaches how to use formal methods to analyze, improve, document, and manage software requirements. It provides “informal ways to think formally” about requirements to reveal domain gaps, incompleteness, and other problem spots early in the product cycle. The course looks at three dimensions of the requirements and specification process: categories such as usability and performance, practices such as reviewing and formalizing, and phases such as elicitation and analysis. The focus will be on requirements analysis through formalization though we will also discuss elicitation, management, and documentation.Participants will be introduced to notations and structures for reducing the ambiguity inherent in natural language requirements specifications. Developers will use this to understand potential problems and tradeoffs before development and can use these specifications to guide formal inspections of designs and code. Testers will use these specifications to construct test cases and decide whether a test case has “failed.” Analysts and other “user representatives” will use these specifications to identify and close gaps in requirements understanding.
The expected audience includes developers, testers, managers, analysts, and QA professionals. No specific programming-language is assumed although programming experience is a plus.
Paper Title: How I Stopped Worrying and Learned to Love Choas (4M2)Process improvement efforts will fail if they assume processes are uniform across large organizations. The basic set of practices should be uniform, but we certain projects or subprojects will have needs that require specialized processes. This experience report will share some techniques of process diversity documentation and management, and will describe lessons learned.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Bill Deibler (Software Systems Quality Consulting) [USA]
Quickstart Title: Making the CMM Work: Streamlining the CMM for Small Projects and Organizations (4Q)This presentation allows participants to identify and leverage the strength of the CMM to improve software development practices in their company. The presentation prepares the attendee to build durable, maintainable software development practices that exploit the CMM framework. The presentation ensures that the participant will be able to:
- Implement a realistic and useful strategy for deploying software development practices in small organizations and projects
- Simplify the CMM to support appropriate, effective, flexible software development processes for any small organization or project
- Resolve apparent discrepancies between the guidance in the CMM and the needs of small, commercial and internal software development projects and organizations
- Identify and prioritize elements of advanced levels that should be considered by every organization.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Richard Denney (Landmark Graphics) [USA]
BOFS Topic: Blue Collar Formal Methods (7B2)There are many areas in commercial QA where FM could make useful contributions. But QA groups often operate in a "MASH" like mode to support a much larger development group: there is no time for tidy, thorough operations on a single patient; rather surgeons are forced to triage and hastily perform rudimentary operations on a large number of incoming. For this type environment there needs to be increased focus on "Blue Collar FM", i.e. FM for the working folks. It's about trying to identify that 20% of FM that gives developers & QA engineers 80% of the bang for the buck. Example areas for applying blue collar FM are technical peer reviews, test specification, function point like metrics based on semi-formal specs, operational profiles and reliability engineering, and formal specification styles as a basis for natural language spec templates. This BOF session will collect experiences from those using FM in commercial QA settings, and identify fruitful areas for future cross-pollenation of the two fields.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Thomas A. Drake (Coastal Research & Technology, Inc.) [USA]
Tutorial Title: Measuring Object-Oriented Software Quality for C++ and Java (Tutorial B2)Many software development organizations are increasingly using the object oriented (OO) approach as the basis for their information technology development strategies and delivered solutions. But how does one measure the quality of object oriented development as a process and the associated software as the product? This tutorial presentation will provide practical and useful knowledge centered on measuring object oriented software quality using emerging OO code level analysis and process techniques and automated tool technology support as well as defining OO quality and what it means. Examples will be drawn from C++ and Java.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Sally Drew (Tescom UK SST) [UK]
Tutorial Title: E-Commerce Testing -- The Clash of the Titans (Tutorial J2)The Internet presents many unique challenges to testers. This Tutorial, based on numerous real-life E-Commerce and New Media testing projects, will lead the audience through these challenges. It will describe practical techniques which work, experiences using automation solutions and how traditional testing techniques mapped onto this environment (or not!). Additionally, it will include a number of pitfalls encountered and lessons learned, as well as addressing some of the cultural issues which needed to be addressed. Its aim is to provide the audience with the resources and confidence to successfully deliver E-Commerce testing solutions.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Elfriede Dustin & Mr. John Paul (CSC / Freddie Mac) [USA]
Quickstart Title: Moving From Conventional Testing to Object Oriented Testing (9Q)This presentation outlines the process required for a test team to transition from traditional test methods, associated with a structured development environment, to an object oriented test methodology required to implement object oriented test automation.Test managers faced with the evolution toward object oriented test automation within the software industry need to evaluate how to best make this transition. The test manager will need to address several questions. How soon will such a transition be required? What can be done to re-equip/train the present test team in order to perform within an object oriented paradigm? What tools can be used to make the transition smooth.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Peggy Fouts (Compuware Corporation) [USA]
BOFS Topic: Medical and Safety Critical Application Testing (3B1)Medical and other safety critical applications present unique challenges for QA and testing personnel. Meeting tight schedules while performing adequate testing and fulfilling imposed documentation requirements can be tricky.Depending on the desires of the participants in this BOF session, we will cover such topics as:
1. Setting up processes which meet regulatory requirements, including tools which facilitate this
2. Methods of testing for specific categories of applications
3. Attitude required
4. What works and what doesn't?
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Tom Gilb (Result Planning Limited) [NORWAY]
Tutorial Title: Advanced Inspection (Tutorial G2)Software Inspections started at IBM in about 1973. This tutorial will focus on the improvements and generalizations of the Inspection process which Tom Gilb (and clients and professional friends) have made since then. The improvements have tried to make Inspection more useful for more purposes, more interesting, and less costly. Some examples of improvements are using sampling, focusing on Major defects, more detailed well-defined process, defect-checking specialist-roles, real-time Inspection process control, application of the process to any type of specification, including management documents.
RETURN TO QW'99 PROGRAM
Paper Title: Evolutionary Project Management (3Q)(Abstract to be supplied.)
RETURN TO QW'99 PROGRAM
Paper Title: Risk Management Technology: A Toolkit for Identifying, Documenting, Analyzing and Coping with Project Risks (6M2)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Steve Goodhall (Compuware) [USA]
Year 2000: Lessons Learned (VT6)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Sam Guckenheimer (Rational Software Corporation) [USA]
Effective Testing for Java-Based Web Software (7A2)Java technology is relatively new to the programming scene, and has already caught on faster than any other programming language in history. Because of this rapid evolution, people responsible for rolling out Java Web applications have significant quality-assurance challenges. Of course, Java applications are rolled out on notorious "Internet-time" schedules. What's more, the user load can vary greatly, the runtime browser environment is unpredictable, and the technology base changes very fast. This presentation offers the QE practitioner practical approaches to understanding, then testing Java applications, to ensure their reliability, functionality, and performance. We discuss testing Web time, the planning of tests, and whether and how to automate testing.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Jon Hagar (Lockheed Martin Astronautics Company) [USA]
Paper Title: Industrial Experiences in Establishing Laboratories and Software Models for Testing Software (9A1)In recent years, there has been more interest in software testing supported by models as well as creating test environments to realistically test the software. There has been an increase in the number and types of support programs or tools available from commercial suppliers in these areas. Organizations within Lockheed Martin (LM) have experience creating test facilities using models to help plan, define, run, and analyze tests. We examine what models are and how they are used in testing to accomplish: simulation in a test facility environment, calculation of inputs, and determination of output correctness. There are different types of modeling tools as well as different levels that tools can operate at. Planning for use of models can be part of a mature or new test organization that is looking to improve automation and maximize the use of skilled staff. Modeling tools have helped find many of the common types of errors that occur within software.
BOFS Topic: Testing for Military and Government Software (4B2)
In many ways the government and military have been progressive when it comes to software testing. We have had standards that required high levels of testing in the defense, aviation, medical, and nuclear areas for years. Many of the concepts that are now common practice in software testing grew out of government efforts. But we still have issues, e.g., Y2K, systems that have errors, high cost, etc. Much government software is done by commercial companies, so while it is different, it is still the same. So what are the ideas from commercial practice government efforts should consider? What ideas from the government world should the commercial people consider, e.g. many tools and things like the CMM grew out of government areas? And, what are things that will never be the same, e.g., areas like medical devices and airplanes because the concern to the "general public"? These questions and others will be considered during this session.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Ted Hammer & Dr. Linda Rosenberg (NASA GSFC SATC) [USA]
Quickstart Title: Continuous Risk Management at NASA (7Q)NPG 7120.5A, "NASA Program and Project Management Processes and Requirements" enacted in April, 1998, requires that "The program or project manager shall apply risk management principles..." The Software Assurance Technology Center (SATC) at NASA GSFC has been tasked with the responsibility for developing and teaching a systems level course for risk management that provides information on how to comply with this edict. This risk management structure of functions has been taught to projects at all NASA Centers and is being successfully implemented on many projects. The course was developed in conjunction with the Software Engineering Institute at Carnegie Mellon University, then tailored to the NASA systems community. This presentation will briefly discuss the six functions for risk management: (1) Identify the risks in a specific format; (2) Analyze the risk probability, impact/severity, and timeframe; (3) Plan the approach; (4) Track the risk through data compilation and analysis; (5) Control and monitor the risk; (6) Communicate and document the process and decisions.Finally, the presentation will give project managers the information needed to implement Continuous Risk Management successfully at a cost they can afford.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Magdy Hanna (International Institute for Software Testing) [USA]
Tutorial Title: Establishing a Software Inspection Process (Tutorial G1)Software inspections have proved to be very effective in capturing more defects early enough to avoid the cost of rework. Yet, many organizations are struggling to make the inspection process effective and to achieve the real benefits of inspections. This course will examine the different elements that make an effective inspection process.
Paper Title: Disciplined Software Testing Practices and Certification (VT20)
The demand for software testing professionals is growing at such a high rate that software organizations are unable to perform adequate testing to software systems that control different aspects of our lives. Many software organizations are attempting to close the gap as quickly as possible by growing their own test professionals. In spite of the increasing demand, there is no one educational institution that teaches the software testing as a discipline. It is important that software testing professionals acquire adequate training in software testing and attain certification of their proficiency in the field. In this talk, Dr. Magdy Hanna will discuss the mission of the International Institute for Software Testing to achieve this goal through the Certified Software Testing Professionals program.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Avi Harel (ErgoLight Ltd.) [Israel]
Paper Title: Measuring the Quality of GUI Usability (VT13)ErgoLight Ltd. develops and markets solutions for enhancing the usability of Windows applications. The term "usability" is typically interpreted in various ways, to mean friendly appearance, ease of use, ease of learning, user productivity and preventing accidents caused by user's errors. ErgoLight solutions for usability labs and QA departments allow local or remote testing, over the Internet. The test results are usability profiles, composed of objective measures of the user operation and detailed analysis of the user errors.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Elisabeth Hendrickson (Aveo, Inc.) []
BOFS Title: Mass Market Software Testing (2B)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Douglas Hoffman (Software Quality Methods, LLC) [USA]
Paper Title: Test Automation Architectures: Planning for Test Automation (2S1)Designing a practical test automation architecture provides a solid foundation for a successful automation effort. This talk describes elements of automated testing that need to be considered, models for testing that can be used for designing a test automation architecture, and considerations for successfully combining the elements to form an automated test environment.Quickstart Title: Thoughts on Oracles and Software Test Automation (With Cem Kaner) (6Q)
Automated regression testing is the most popular approach to software test automation but it is often neither the most effective nor the most cost-effective approach. It is one strategy among many. We look at several testing strategies, including (for example) manual exploratory testing, traditional regression testing, and simulation based testing using strong and weak oracles. These are all good methods. They all have their time and their place and their value. The question that we have is, when should we choose one over the others.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Mark Johnson (OrCAD) [USA]
BOFS Topic: Productivity in Small Integrated Teams (8B2)The stories we hear about amazing new products and incredible feats of team productivity are generally from very small startup organizations. In addition to a lot of hours, what are the reasons a small focused team can achieve so much, while the larger organizations struggle? Part of the answer is breaking down barriers that block creativity and communications. One way to do this is to integrate small teams of development and QA people with shared responsibility for all the aspects of developing a product.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Cem Kaner (Attorney at Law) [USA]
Keynote Title: Facing the Future: The Law (K5P1)(Abstract to be supplied.)
BOFS Topic: Status Report on US Software Quality Laws (7B1)
(Abstract to be supplied.)
Quickstart Title: Interviewing Software Testing Job Candidates (8Q)
1999 through mid-2000 should be the peak session for Y2K-related software testing. There is far too much testing work to do, so your test group will probably try to expand this year and will probably discover that it is hard to find really good people. Hiring badly is one of the worst mistakes that a manager can make. So how do you hire well? Especially, how to you hire well in a tight market?The paper accompanying this session surveys the broad problem, considering such issues as:
- A behavioral approach to gathering information
- Legal issues
- Consensus-driven hiring
- A strategy for gathering information
- Characteristics of the person you're seeking
- Defining the position and characteristics needed for it
- Who approves the hiring
- How to find candidates
- Sifting through resumes
- Evaluating the candidate's public materials
- The phone screen
- The application form
- Preparing for the interview
- Asking questions during the interview
- Reviewing work samples
- Reviewing sample test documentation (of yours)
- Writing bug reports
- Using tests and puzzles
- Debates and controversial questions
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. John Kent (CISS Ltd.) [UK]
Paper Title: Advanced Automated Testing Architectures (8T2)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Alain Kerbrat & Mr. Iulian Ober (Verilog) [France]
Paper Title: Automated Test Generation from SDL/UML Specifications (9S1) (VT1)Automated test generation from formal specifications presents a lot of promises, either in cost control or test suite correctness. The telecommunications domain is a field where some interesting tools begin to emerge, either prototypes or industrial strength tools. We present TestWriter, a new industrial test generation tool. TestWriter allows the generation of conformance tests, from SDL/UML specifications. It comes with TestPlayer, which allows to execute test cases written with the MSC language. Both tools are part of ObjectGEODE, an industrial toolset for real time systems development.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Ed Kit & Mr. Hans Buwalda (Software Development Technologies / CMG Finance) [USA/Netherlands]
Tutorial Title: Integrated Test Design and Automation (Tutorial A, Part I & II)This tutorial provides a practical guide for addressing three essential testing challenges. First, how to design and document a highly inspectable test suite. Second, how to cost-effectively architect an automated regression test library using the minimum number of highly technical, programming-proficient testers. Third, how to also bridge test design and automation to provide an integrated solution. Examples and demonstrations from today's true testing leaders who have achieved all three of these challenges will be given.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Ara Kouchakjdian (Q-Labs) [USA]
Paper Title: Experience using Statistical Usage Testing at the FAA (VT3)This presentation will discuss the application of Statistical Usage Testing (SUT) to a project at the FAA Technical Center in Atlantic City, New Jersey. It will discuss how SUT was applied, lessons learned, and results.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Herb Krasner (Krasner Consulting) [UK]
Paper Title: Using the Cost of Quality Approach for Software (6M1)Cost of software quality (CoSQ) is a technique that enables our understanding of the economic tradeoffs involved in delivering good quality software, and eliminating the wide-spread and debilitating effects of poor software quality. This paper discusses the context for using CoSQ, then defines how it is used to differentiate the costs involved with: handling nonconformances, achieving acceptable quality, and preventing poor quality.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Lorenzo Lattanzi & Mr. Mario Musmeci (Alenia Aerospazio) [Italy]
Paper Title: Safety Critical S/W Development for a Satellite Based Navigation System (8A1)This paper will analyze the problems related to the software development for a safety critical system. The analysis is performed in the frame of a navigation application field and in particular using as a reference the RTCA standard "Software Considerations in Airborne Systems and Equipment Certification" (DO-178B) and the direct experience made on the European Geostationary Navigation Overlay Service (EGNOS) project.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Philip Lones (Lucent Technologies) [USA]
Paper Title: Revolutionary? A Development Method That Works (7M1)The software development process has improved over the past several years but there are still too many projects that fail to deliver the right product or any product at all. This paper describes a process based on the EVO model that has allowed us to develop a product on time, within budget and is the right product for our customers.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Brian Marick (Reliable Software Technologies) [USA]
Keynote Title: Facing the Future: New Models for Test Development (10P2)If you use phrases like "unit testing", "integration testing", or "black box testing", your thinking has been trapped by inappropriate models. We need test development models built around the special activities that make up testing, not ones that are adornments to code development models. In this talk, I provide one.
marick@testing.com
http://www.stlabs.com/marick/root.htm
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Martina Marre, Ms. Monica Bobrowski & Dr. Daniel Yankelevich (Universidad de Buenos Aires) [Argentina]
Paper Title: A Software Engineering View of Data Quality (9T2) QWE'98 BEST PAPER!Thirty years ago software was not considered a concrete value. Everyone agreed on its importance, but it was not considered as a good or possession. Nowadays, software is part of the balance of an organization. Data is slowly following the same process. The information owned by an organization is an important part of its assets. Information can be used to competitive advantage. However, data has long been underestimated by the software communityUsually, methods and techniques apply to software (including data schemata), but the data itself has often been considered as an external problem. Validation and verification techniques usually assume that data is provided by an external agent and concentrate only on software.
In this work we present different issues related to data quality from a software engineering point of view. We propose three main streams that should be analyzed: data quality metrics, data testing, and data quality requirements in the software development process. We point out the main problem and opportunities in each of them.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. John McGregor (Clemson University) []
Paper Title: Testing Distributed Object Systems (Tutorial B1)This tutorial presents techniques for testing components and systems composed of components. The tutorial is built around a simple, but comprehensive, example that illustrates the basic concepts of the various component models and interconnection technologies. The approach described uses the standard patterns of development for each of the component models to serve as the basis for building test software and selecting test cases. JavaBeans and CORBA are used as the examples.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Frances I. Medina & Mr. Andrew Van Kraanen (AT&T Operational Technology Center) [USA]
Paper Title: Test Automation of a GUI WEB Based Application: An Experience Developing Reusable Automated Testing (6T2)The GUI Interface for our application is making use of a Web Browser (Netscape). The user of this application is exposed to several WEB windows which some of them dynamically display information to the end user; others are used for provisioning or to send request or action to a server or to equipment. Several scenarios are tested for this application; submitting information in forms, dynamic changing applets triggered by an external event, clicking buttons, html links etc. To be able to test all those scenarios for each release would be very time consuming. A commercially available tool was purchased to do the GUI automation. Then the question was what and how to automate, such that the test automation could be used for different software releases. This presentation shares our experience with web automation tools, methodology used in out testing and improvements made by using test automation tools.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Brian Miller (Teradyne) [USA]
Paper Title: Automated Test Generation for Computer Telephony Systems (3T1)Great progress has been made in automating the front-end application design process and back-end test execution process for today's Computer Telephony applications. Sophisticated App Gen tools coupled with configurable IVR platforms allow end users, integrators, VARS or dealers to build sophisticated IVR applications. Robust automated test execution systems can automatically emulate protocols and voice to greatly increase the thoroughness and repeatability of functional test, load testing, regression testing and in-service monitoring. What's missing is an automated test generation process that integrates the upfront call flow descriptions of the App Builders with back-end automated test execution environments. This paper will examine the issues with the current process and discuss the requirements for an automated approach.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. John D. Musa (Independent Consultant) [USA]
Tutorial Title: Software Reliability Engineering: More Reliable Software, Faster (Tutorial D2)This tutorial will quickly, efficiently teach you the basics of how to apply Software Reliability Engineering (SRE) to your project to make software more reliable and to develop and test it faster and cheaper. You will learn about the major activities of SRE in considerable detail: defining "necessary" reliability, developing operational profiles, preparing for testing, executing tests, and applying failure data to guide decisions. It requires no previous knowledge other than general experience with software development. It is therefore an excellent introduction to SRE for software testers, software developers, systems engineers, quality assurance engineers, reliability engineers, project managers, and other interested parties.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Kenneth Nagin & Mr. Alan Hartman (IBM Research Laboratory in Haifa) [Israel]
Paper Title: TCBeans Software Test Tool Kit (2T2)TCBeans is a framework designed to assist testers in developing, executing and organizing function tests directed against Application Program Interfaces (APIs) and software protocols written in Java, C or C++. GOTCHA is a tool for generating an abstract test suite for a finite state machine driven by a coverage model. TCBeans enables automatic testing by simulating the behavior expressed in the GOTCHA model on the application under test.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Jakob Nielsen (Nielsen Norman Group) [USA]
Keynote Title: Facing the Future: Usability Aspects of Quality (10P1)Usability engineering is the systematic approach to improving user interface quality by applying a set of established methods throughout the system development lifecycle. Typically, applying these methods can cut training costs in half and increase the users' productivity by at least 25%. My work on "discount usability engineering" aims at making usability methods cheaper, easier, and faster to use. Not only do these revised methods save money, they may be the only chance for getting any usability work done in companies with tight development schedules (a particularly common characteristic of Internet projects).
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Gerry Ocampo (Data Dimensions) [USA]
Paper Title: Testing Considerations for Web-Enabled Applications (VT18)With the popularity of web applications, QA departments are facing a real challenge due to the wide variety of browsers, technologies and environments. This session will explore some of the testing considerations and also discuss some of the techniques that can be applied to testing web-enabled applications.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Carla Oexmann (ATI Research) [USA]
BOFS Topic: Running a Nightly Test (9B2)Nightly testing refers to a process in which a software product is built at regular intervals and put through a series of tests to verify that the product is still sane. Errors are caught within 24 hours of their introduction, which makes it far easier to find and fix them.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. John Okanishi (Computer Associates) [USA]
Paper Title: Automating Cleanroom Management (VT16)This presentation from Computer Associates stresses the importance of "Cleanroom Management" in managing application source code and components. This process ensures and enforces both the quality and reliability of an organization's application systems, a process that is becoming even more critical as Year 2000 remediated code is being implemented into production.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Rob Oshana (Raytheon Systems Company) [USA]
Paper Title: An Automated Testing Environment to support Operational Profiles of Software Intensive Systems (3S1)An automated testing environment was developed that uses statistically generated test scripts based on user and usage profiles as well as other operational profiles to drive the project testing effort. The testing approach is based on the concepts of software testing based on statistical principles. This paper will focus on the successes and issues associated with developing a statistical testing environment for an industrial software project. The paper will also describe how both statistical testing based on software models and traditional testing based on unit and other functional tests can be combined into an effective approach to testing large software intensive systems.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Ingrid B. Ottevanger (IQUIP Informatica BV) [Netherlands]
Stand-by Paper Title: Test Factory (TSite): A Next Step in Structured Testing (9S2)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Federico Pacquing, Jr. (TechWave Inc.) [USA]
Paper Title: Usability Testing in E-Commerce Applications (4A2)This presentation will focus on the conducting of Usability Studies and analysis as a key and fundamental tool in the evaluation of e-commerce sites. Usability problems are often overlooked or placed as a lower priority than technical defects. However, the impact of these issues on schedule, user-experience, technical support and ultimately the financial "bottom-line" can be just as devastating as high severity technical problems. The use of these test techniques in the e-commerce space is critical - where user experience can make or break a web site.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Heesun Park (SAS Institute Inc.) [USA]
Paper Title: Optimum Level of Test Automation for Client/Server Software (4S1)This paper focuses on how to determine and achieve optimum level of test automation that produces highest test throughput in client/server environment. It describes the advantages and the tradeoffs of different levels of test automation and the strategies that we have developed at SAS to overcome the pitfalls of test automation.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Rainer Pirker & Mr. Andreas Rudolf (IBM) [Austria]
Quickstart Title: Millennium is getting closer -- The Quickstart to Y2K Testing (2Q)This presentation will show what you can realistically expect if you start your Y2K tests now. It attacks the test from the business point of view, so you can concentrate on the most critical transactions where your main business depends on. The approach is based on working experience from various long running Y2K test projects starting with the first project in 1996. You get a number of measurements which help you to estimate your project and identify your risks. It gives you guidelines on how to perform time-warp tests and on which dates you should put your main test focus. We give you composed from various projects the critical success factors and tell what worked well and what did not. Furthermore, we explain how to set up a contingency management for those areas where compliant solutions can not be implemented and tested in time because of the Y2K time restrictions.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Martin Pol (IQUIP Informatica B.V.) [Netherlands]
Keynote Title: Facing the Future Means Facing Test Maturity (1P1)Apart from continuous evolutions in technical infrastructure (e.g. Client/Server, GUI's and the Internet) and in the development infrastructure (e.g. OO, RAD) the challenges for testing lie in the integration of business processes and related innovations such as electronic commerce and computer telephony integration. We will connect customers, suppliers, employees, public services, etc., 24 hours per day, 7 days a week, world-wide. Complexity of the IT-solutions and testing will grow dramaticly. The industry will focus on reuse of components, components of tailor-made software, packages, odd data files and incompatible technical infrastructure have to play together. Reusability and adaptability will be major quality requirements. The future requires new test solutions and more mature test processes. Many organisations still struggle with the implementation and consolidation of structured testing. Frequently, day-to-day's pressure causes reinvention of the testing wheel over and over again. Structured, continuous test improvement programmes are required to establish a stable foundation for the challenges to come. A part of this presentation is the introduction of the TPI D2-model, which is based on current state-of-the-art test process improvement practices. The model gives practical guidelines for assessing the maturity level of testing in an organisation and for step by step improvement of the process. The purpose of such improvement could be reaching CMM level 3.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Dave Priest (Sunpower Computing) [USA]
Paper Title: (To be supplied.) (VT11)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Shel Prince (Software Development Technologies) [USA]
Paper Title: Automating Software Inspections (VT12)The power of the Web has now been brought to Technical Review and Inspection -- the most effective software defect detection and removal method. Online data and metrics collection allow easy collaboration for distributed development teams and timely executive reporting. Join companies like Sun, US West, and the Navy in taking advantage of this innovative solution.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Steven Prothero (Intrinsa Corporation) [USA]
Paper Title: Concurrent Software Development and Quality Assurance: New Tools and Techniques (VT9)Ever increasing software product complexity and shorter time to market demands are wreaking havoc on software quality. As the software development cycle evolves, new techniques are being used to address software quality earlier in the development process. This presentation reviews how utilizing state-of-the-art Quality Assurance tools during the coding stage can significantly reduce code defects, improve software quality, and reduce product development cycles.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Steven Rabin (Interworld Corporation) [USA]
Paper Title: eCommerce Performance Benchmarking -- Methodology and Criteria (4A1)One of the key aspects of a successful eCommerce initiative is performance. While this is often overlooked or taken for granted very few customers will return to a site that exhibits sluggish performance. This session discusses the key elements in designing an eCom based performance testing benchmark and the methodology that goes along with it. This includes workload characteristics, transactional definitions and site assumptions. A framework for eCom performance benchmarking and the criteria that need be measured is also introduced.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Roger M. Records, Mr. Jay G. Ahlbeck, Ms. Linda J. Lin, Mr. Jon S. Scharer & Ms. Margaret A. Stocking (Boeing Commercial Airplanes) [USA]
Paper Title: Deploying SQA in Very Small Projects (2M1)To deploy SQA in one or two person projects requires the definition of an SQA function which can be implemented with minimum resource investment, yet adding value to the small project. An infrastructure of instruction from Subject Matter Experts using model plans, processes and reviews that can be adopted by the small project and tailored to their needs has proven to be a feasible implementation.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. John N. Romanak (Bellcore) [USA]
Paper Title: Life as a CMM Level 5 Test Organization (3M1)Quality, timeliness, cost-effectiveness - for years software professionals have believed you cannot have all three. Bellcore has proven otherwise. Bellcore's approach to Software Quality has made us a leading provider of World-Class software. Bellcore is ISO9000 and SEI CMM Level Three certified. Positioning Bellcore for SEI CMM Level Five in 1999.Bellcore's software quality journey began in the 1992/1990 timeframe. Testing was viewed as providing low value and there was a very poor relationship between testing and development.
Bellcore's SQA Independent Testing Organization is now industry-recognized as best in class.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Linda Rosenberg (SATC, NASA) [USA]
Tutorial Title: Writing High Quality Requirement Specifications (Tutorial E, Part I & II)The requirements specification establishes the basis for all of the project's engineering management and assurance functions. If the quality of the requirements specification is poor it can give rise to risks in all areas of the project. This workshop will educate project managers and software developers in effective development of quality requirement specifications. It will also provide them with ideas and methods they can incorporate immediately into their project plan and find a productive return in documentation evaluation and comprehension.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Mike Ross (Quantitative Software Management, Inc.) [USA]
Paper Title: Size Does Matter: Continuous Size Estimating and Tracking (2M2)Estimating the size of a software system is a critical development process activity. Not only does size impact the technical; it also impacts the project management solution. It is therefore insufficient to estimate size only once, at the beginning of the project when the least is known about the system being developed. This paper describes a quantitative process for managing the size of software development projects through continuous estimation. The process includes a probabilistic approach to estimation coupled with tracking and assessment of trends to determine whether or not stability exists. This process has direct applicability to the SEI CMM Level 2 KPAs for Software Project Planning and Software Project Tracking and Oversight. It also serves as a Level 4 Quantitative Process Management tool for measuring the effectiveness of an organization's size estimating and requirements management process.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Johanna Rothman (Rothman Consulting Group, Inc.) [USA]
Paper Title: Using Quality to Drive Product Development Processes (9A2)Companies create products, for many reasons. Since a projects' quality goals are intimately related to the reasons for creating the release, you can't use a cookie cutter approach to project management in software development. This paper discusses how to choose quality goals and a product lifecycle to meet the company's product goals.
BOFS Topic: Life as a New Test Manager (6B)
We all have our own ways of becoming successful test managers. Please attend this BOF to share what's worked for you or your manager. Some of the topics we'll discuss: drawing the line between project management and test management, how to get the rest of the organization to listen to you, how to know that you're managing the right things.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Hanania T. Salzer (RTS Software Ltd.) [Israel]
Paper Title: ATRs (Atomic Requirements) Used Throughout Development Lifecycle (6S1)Atomic requirements (ATRs) provide the means to test exactly what the programmer must implement. By making requirements indivisible, ATRs implement concepts such as the segregation of the functional coverage from a test's "depth" and the inheritance of standards defined as lists of ATRs. Individuals can enjoy the benefits that ATRs bring, even without management cooperation. The author suggests that CASE and test automation tools use ATRs to link design with testing.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Fred Scheck (Interim Technology) [USA]
Paper Title: Beyond Y2K & The New Software Quality Professional (VT10)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Norman Schneidewind (Naval Postgraduate School) [USA]
Paper Title: Development and Maintenance Process Assessment Using Reliability, Risk, and Test Metrics (Tutorial D1)Practitioner and research software engineers and managers will benefit from this tutorial by learning how to make product reliability measurements and predictions to assess the risk of deploying software and how these measurements and predictions can also be used to assess the stability of the process that develops and maintains the product.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. J. P. Schroeder (Teradyne) [USA]
Stand-by Paper Title: Model-based Testscript Generation for a C++ Class (SBS2)This presentation illustrates how to generate tests for a C++ class using a state-machine-based modeling approach.C++ classes enable software engineers to design objects that have attributes (typically data) and behavior (typically implemented as functions). The member functions of a class are commonly called methods. Testing a class involves exercising its methods to ensure that they are internally and externally consistent. Internal consistency means the methods work correctly with all other methods of the class. External consistency involves correct operation with objects in the larger software application. This presentation only includes testing for internal consistency.
TestMastertm, an automated test design and generation tool, is used to demonstrate creating "black box, state machine models of a system's behavior". In this case, the System-Under-Test is a C++ class. The behavior of a class is defined by its public functions. Black box testing meshes neatly with the information hiding inherent in class definitions. The implementation details of the methods are purposely ignored. The focus of the testing is (again) the class's public functions. State machine notation is used to describe and partition the class's external behavior. The invocation of a method typically produces very different results depending on the class object's state. The state machine notation allows you to set-up and invoke the methods under these differing conditions.
Finally, the model is used to create tests, each in the form of a small, compatible C++ Test Driver.
All the examples shown use the TestMaster automated test design tool. The presentation also provides a step-by-step method for analyzing the components of a class and modeling these components. Given these examples, Testers can create their own state machine models manually or using the example tool.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Jeff Schuster (Rational Software Corporation) [USA]
Keynote Title: Facing the Future: E-Commerce Quality and YOU! (Keynote 1P2)Web-based electronic commerce applications signify the stability of an organization and expose the organization to unpredictable numbers of external users. Applications that fail can cost companies revenue. For this reason, organizations have the responsibility to ensure the reliability, robustness, and integrity of its applications. Performance Testing solves this problem. Load (or stress) testing tools enable organizations to simulate application performance (in terms of application integrity and response time) with virtual users prior to deployment, and help companies predict how Web apps. will function once thousands of customers access a Web site at the same time. By understanding the breaking points of a Web application, organizations can gain visibility to what their customers will experience (in terms of application integrity and response time) once the Web app. is deployed and identify and correct problems before their customers find them. This presentation discusses the importance of performance testing electronic commerce applications and offers the QA practitioner practical approaches to performance testing.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Roger Sherman (Consultant) [USA]
Keynote Title: Facing the Future: Commercial Product Testing (5P2)In the second half of the 20th century we have seen the beginning of a new technical profession: software tester. What challenges will testers of commercial software face in the next century? What kinds of tools will be needed? What research do we need to commission? And will management ever learn how to "do software?"
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Ron Silacci (Lucent Technologies, Inc.) [USA]
Paper Title: A Testers' Top 10 List (3A1)System testing is a vital component of the service/product development cycle at any company. The current competitive environment requires our project teams to deliver higher quality products faster while meeting or exceeding customer expectations. In an effort to meet this challenge, more and more pressure is being put upon the testing effort. From practical experiences in testing many Telecommunications systems, a testers' top 10 list has been compiled. This list contains the top 10 factors that will make testing a product successful and a testers work life easier. The paper discusses these top 10 key factors of success.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Leon Slota (Neoglyphics Media Corporation) [USA]
Paper Title: Developing Load and Performance Requirements for Web Sites (7A1)In the web world where the competition is only a click away performance and load testing have become a necessity. However, load and performance requirements are often note clearly defined. This presentation discusses ways to develop requirements which answer the questions "How fast should it be?" and "How many users should it support?".
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Michael L. Smith (McCabe & Associates) [USA]
Paper Title: Software Quality (VT2)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Keith Stobie (BEA Systems, Inc.) [USA]
Paper Title: Creating a Testing Culture (9M2)This talk covers several areas related to sharing best practices for software testing:
- Body of knowledge defined and mapped to Job level and skills
- Recommended training curriculum
- Engineering wide and Test specific process
- Test Patterns, Test Catalogs, Test matrices
- Interviewing skills and questions
- Software metrics: raw data (Defects, Code Size, Dates) to information (e.g. Projected defect discovery and closure rates)
- Test Reporting & concept of "test points"
- Test Automation: Cross platform API and GUI testing
BOFS Topic: Building a Common Knowledge Base of Testing (SBB1)
How can we leverage Brian Marick's catalogs (The Craft of Software Testing), Cem Kaner's matrices (Testing Computer Software) and Robert Binder's Test Design Patterns (http://www.rbsc.com/pats/TestPatternTemplate.html) to build a common knowledge base of testing? Can we Catalog/Matrix/Pattern testing of integers? file opening or writing? URL processing? XML Syntax errors?Start of position paper:
In Brian Marick's The Craft of Software Testing he describes "a catalog is a collection of test requirements, indexed by variables and operations". Example catalogs are appendices in the book. They cover for example
In Cem Kaner's course Testing Computer Software (based on book by the same name) he describes "matrices for whenever you can specify multiple tests to be done on once class of object". An example given is
counts:
- -1, 0, 1, >1, max value, max value +1
Test Matrix for a Numeric Input Field
Finally, Robert Binder, building the on software engineering Design Patterns, has started cataloging Test Design Patterns (http://www.rbsc.com/pats/TestPatternTemplate.html) "Test patterns simply frame and focus the issues that must be resolved to accomplish any kind of testing." An example mentioned is
- nothing, valid value, lower bound (LB), upper bound (UB), LB-1, UB+1, ...
Category-Partition:
- Design test suite based on input/output analysis.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Peter Szirmak (Information Balance) [USA]
Paper Title: Y2K: The tasks remaining and the challenge of leveraging knowledge gained for the future (VT8)Most North American companies are well on their way to becoming Year 2000 compliant. During the remainder of 1999, some key areas remain to be addressed including: Testing of both applications and desktop systems, Quality Assurance (or independent code verification) and Independent Program Reviews. Considered the "Due Diligence" of the Year 2000 world, these activities govern the quality and integrity of Year 2000 Programs by providing an independent assessment of an organization's processes and results, ensuring that no issues fall between the cracks.An equally great challenge lies ahead for companies once the deadline has passed : to assess what they have gained and learned from their Year 2000 program experience, and to leverage this insight into improved development, testing and quality assurance processes. This presentation will discuss our experiences in dealing with various aspects of Y2K projects including quality assurance activities, and how we are working to transfer our knowledge and expertise into viable ideas for future process improvements.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Graham Thompson (InCert Software) [USA]
Paper Title: Minimizing Testing While Maximizing Failure Detection (2A1)When is less better than more? In testing, it’s when the testing is focused on failure detection. This presentation describes in detail how to streamline test suites while still effectively exposing failure points within an application. Experiences will be shared from multiple clients, who through the pressures of Year 2000, were forced to minimize their test suites in order to complete a practical and reasonable level of testing by an unmovable date. This did not mean that testing was to be any less effective, but that test data to drive their applications had to be minimized while still exercising all possible paths. These and other lessons learned from Year 2000 testing will be communicated with the productivity gains that have been achieved.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Steven Toeppe & Mr. Scott Ranville (Ford Motor Company) [USA]
An Automated Inspection Tool for a Graphical Specification and Programming Language (8T1)Ford Motor Company has adopted a graphical language for powertrain control development specifications and design. An internal style standard has been developed. An automated consistency checker tool has been developed that is used to automatically verify that the graphical models conform to over 250 specific rules. This paper addresses the style issues, some of the features of the tool and some of the general analysis methods applicable to the tool.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Bor-Yuan Tsai, Dr. Simon Stobart, Mr. Norman Parrington & Dr. Ian Mitchell (University of Sunderland) [UK]
Paper Title: A State-based Testing Approach Providing Data Flow Coverage in Object-Oriented Class Testing (9T1)State-based testing transforms a class behaviour model into test cases, and data flow testing generates test case based on patterns of definitions and uses of the variables in a class. A novel object-oriented class testing approach, called MACT, combines both techniques to ensure the quality of classes under test.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Andreas Ulrich, Mr. Peter Zimmerer & Mr. Gunther Chrobok-Diening (Siemens AG) [Germany]
Paper Title: Test Architectures for Testing Distributed Systems (7S2)This paper deals with new advances in designing and implementing test architectures for testing distributed, concurrent software systems. It defines requirements to the test architecture that need to be matched to support execution of tests and their correct assessment afterwards. The paper finally presents the "Test and Monitoring Tool" (TMT) for testing distributed systems.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. George O. van der Veen (Testmasters Inc.) [USA]
Paper Title: Y2K Independent Verification & Validation (VT17)The Y2K IV&V (Independent Verification & Validation) process is used to determine if the Y2K planning, remediation, testing and preparedness reporting efforts are efficient & effective. The primary objectives of the IV&V are to make sure that the systems are Y2K compliant and help to limit the risk of litigation.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Leonard Verhoef (Human Efficiency) [Netherlands]
Tutorial Title: Improving Software Quality for Users (Tutorial J1)This seminar shows how to design Windows software enabling users to work efficiently. The basis for this seminar is not a vague concept, like unfriendliness, but psychological knowledge used for improvement of human performance.During the seminar, the consequences for software design will be demonstrated by an extensive amount of examples illustrating the difference between stimulus-response interfacing and cognitive interfacing. In addition, empirical data and experiments will be demonstrated. Obtained knowledge will enable participants to evaluate rationally the ergonomic costs and benefits of the design options that are available. In this way, the difference between the 20th and the 21st century interface design becomes visible and concrete.
In sum, this seminar offers:
- insight in movement, perception, memory and reasoning;
- how these processes interact and operate with software;
- a translation of this theoretical basis into practical guidelines for design;
- application of these guidelines.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Matias Vierimaa, Ms. Minna Mäkäräinen & Mr. Atte Kinnula (VTT Electronics / Nokia) [Finland]
Paper Title: Improving DSP Software Engineering Processes from the Testing Viewpoint (8A2)This paper presents the efforts to improve the DSP (Digital Signal Processing) software testing at Nokia Mobile Phones. Ltd (henceforth NMP). We describe the improvement approach used, together with the intermediate results and give a brief overview of the improvement actions initiated. The focus of this article is on the intermediate results which identify the key issues that affect the testing of DSP software and indicate where and how in the DSP software engineering process the testing issues should be taken into account.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Siegfried Voessner, Mr. Michael O'Sullivan & Mr. Joachim Wegener (Stanford /Daimler-Chrysler) [USA / Germany]
Stand-by Paper Title: Genetic Algorithm with Cluster Analysis for Software Testing (8S1)Testing is one of the most complex and time-consuming activities within the development of embedded systems. Usually, embedded systems have to fulfill real-time requirements, and correct system functionality depends on their logical correctness as well as on their temporal correctness. An investigation of existing testing approaches shows a lack of support for testing temporal behavior. For this reason, we developed a new approach, called "Evolutionary Testing", which uses genetic algorithms and a technique based on cluster analysis for detecting temporal errors. It is an iterative testing procedure, which approximates the worst-case execution times of the system under test within several generations. Mathematically speaking, the test is stated as a discontinuous, nonlinear, optimization problem, with the input domain as search space, sets of test data as decision variables, and execution times as objective values.=20We will show how to get more information about the structure of the search space - something entirely unknown if the source code of the test object is too complicated to be analyzed or not available. Furthermore we can gather information about local optima that is useful to optimize the program code and detect several performance leaks within one test run. Finally our proposed test algorithm provides a reliable termination criterion for the test process. To illustrate the proposed algorithm we will present several experiments. The results show a significant improvement in performance over previous results.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Dr. Tony Wasserman (Software Methods & Tools) [USA]
BOFS Topic: Testing Web Applications (SBB2)(Abstract to be supplied.)
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Reinhard M. Weiss (Further Inspections, Inc.) [USA]
Paper Title: Automated Load Testing of DB2 Client/Server Systems (SBA2)This presentation will help make the audience aware of the solutions that are available in automated performance tests. This presentation will also give advice in making design decisions based on testability. The presentation will cover my many experiences in using LoadRunner to performance test mainframe based Client/Server systems. I will discuss the special challenges involved in testing these systems and provide suggestions on design choices that can be made by those who are considering future DB2-PC Client Development.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Mark S. Wiley (nCUBE) [USA]
BOFS Topic: OS and Embedded System Testing Techniques (3B2)Come exchange ideas with other professionals on how to automatically test the software of operating systems and embedded systems. Attendees will be encouraged to discuss the pros and cons of the tools and techniques they use.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Jim Williams (CableData, Inc.) [USA]
Paper Title: Testing for Y2K Compliance: A Case Study (7S1)Automated testing methods are used to confirm Y2K compliance for a complex, multi-module, multi-platform application under test (AUT). The 2-year effort resulted in some 40,000 test scripts and slightly over 400,000 saved comparison images. Excellent pay-back from test mechanization and highly enhanced confidence in the AUT are the results.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Rodney Wilson (KLA-Tencor) [USA]
BOFS Topic: Care and Feeding of a Testing Career (8B1)Questions and discussion for this Birds Of A Feather will center around the following three topic areas:1. What are core competencies for software test and quality engineering professionals?
2. What are potential career destinations for software test and quality engineers?
3. How can you keep from falling into a career rut?
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Tom Wissink (Lockheed Martin Mission Systems) [USA]
Paper Title: Test Engineering -- A "Value Add" Career Path! (9M1)In the not so distant past the title of "Test Engineering" was not used or even considered. This paper takes the position that Test Engineering is not only valid but can provide significant value to the person and his or her company. Can Test Engineering be a career path, what steps are in that path, what skills are needed? These questions and others will be addressed.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Glen Xia (Deloitte Consulting) [Australia]
Paper Title: An Industrial Case Study of Quantitative Management for Object Oriented Software Testing (6A2)In this paper, we will report our experience in managing the testing aspect of a large object oriented software development project through quantitative analysis. The novel approach, we developed in estimating development effort and the number of defects will be presented. We will demonstrate the analysis and visualization techniques used in tracking, monitoring and controlling the testing process. Finally, the process improvement initiatives, based on the Orthogonal Defects Analysis (ODC), will also be reported.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Ms. Fan Yang, Mr. Trung Nguyen & Mr. Anant Adiga (Sequent Computer Systems, Inc.) [USA]
Paper Title: A Web-based System Testing Repository Model (6T1)This paper attempts to stand on a global viewpoint to define the automation process for system testing instead of the traditional test automation mechanism that is focused on automating test scenarios under test harnesses. To overcome a constant challenge in reducing the testing cycles and to meet the Time-To-Market's (TTM) pressure, we specifically propose a web-based system testing process model.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM
Mr. Scott Young (Perot System Corporation) [USA]
Paper Title: Them and Us: Communication between Development and Test (8M2)It is essential for the Test team to recognize their dependence upon the Development team as a source of technical and functional information on a development project. This paper addresses the problems if the relationship is ignored and suggests technical and procedural techniques to foster strong relationships between the groups.
JUMP TO TOP OF PAGE
RETURN TO QW'99 PROGRAM