3rd INTERNATIONAL SOFTWARE QUALITY WEEK EUROPE (QWE'99) 1-5 November 1999, Brussels, Belgium PAPER AND PRESENTATION ABSTRACTS

Mr. T. Ashok (VeriFone India Ltd) [India]
Tutorial Title: A Model for a High Performance Test Organization(Tutorial H1)

The tutorial focuses on: How test engineering, test management, processes, metrics and people com together to build a high performance test organization Issues and solutions to tackle people aspects in test groups Testing as a formal engineering activity, implementation of a continuously improving test process Lessons to be learned from this presentation: What it takes to setup a great test group How can a test group be value-adding to a company People side to testing Implementation of testing as a formal engineering activity How blind automation may not be very useful How to inculcate a method of criticizing objectively JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. T. Ashok, Hema Gollamudi, & Piyali Biswas (VeriFone India Ltd.) [India]
Paper Title: A Model for Analysis of Software Testing Metrics for Process Improvements (Paper 11A)

This paper describes a framework for analysis of product data quality gathered during system test to discover process incapability and suggest methods to aid in improvement of processes used in SDLC. The analysis has been carried on data collected in the Software Test and Analysis Group (STAG) an independent software testing group at VeriFone India Ltd. Bangalore, India, a division of Hewlett-Packard over the last two years. The data encompasses over twenty five projects of various sizes done in the last two years. This paper outlines how the defect-oriented data collected during system test cycles can be analyzed to give a better insight of process implementation issues, and aid in continual process improvement. Also this model builds a empirical defect prediction and prevention model. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Rob Baarda & Mr. Tim Koomen (IQUIP Informatica BV) [Netherlands]
Paper Title: Risk Based Test Strategy (Paper 7M)

In this presentation, the steps to define a testing strategy will start with business risks. This gives all parties involved better insight. The stepwise defining of the test strategy can be used for any test level and also for an overall strategy, including and coordinating all test levels and even inspections. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Kishor Bapat (EWO Software) [USA]
Paper Title: Clean Management Begins with Clean Applications (Paper 2M)

In this paper we will discuss some new Configuration Management methodology and how an IS organization can adopt a successful and complete Clean Management process. We will discuss two basic requirements of Clean Management a). ensuring that any application being considered for Clean Management is "clean" to begin with and, b). periodic auditing/verification is necessary for the application to maintain its "clean" status. We will not be discussing the day-to-day maintenance activities involved in Clean Management. We feel that this particular topic has been extensively analyzed by a number of other authors. Clean Management is one of the latest topics to hit the IT field. However, unlike a number of other fads or buzzwords, Clean Management is rooted in some very solid principles. A lot of the principles are simply common sense. In fact, the surprise is only that it took so many Fortune 500 Information Systems Departments so long to recognize the importance of it. In this discussion we will discuss only the aspects of Clean Management as they apply to software applications. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Torsten Baumann, Mr. Rex Black, Mr. Serban Teodorescu, & Mr. Gordon Page (Interactive Media) [Canada]
Paper Title: Integrated Test Automation of IVR-Telephony Applications and Client-Server Call Center Applications (Paper 7A)

While test tools do exist exist for IVR Telephony, Client-Server and GI Testing independently, there is not one that can test across the various systems. We have developed an approach that can be used in an integrated test environment. The system under test (SUT) is described as follows. There are four main components: The Customer service application The IVR Telephony application The Content monitoring application The CTI server and PBX. The testing performed by our test group is called Black Box Testing. This paper will discuss the following areas of our test effort in more detail: Client-server GUI testing CTI client-server testing IVR applications testing using simulated calls Product-wide integration using multi-tools (CallSim, Fake-Pipe, QA Partner) Approach Management considerations JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Walter Baziuk (Nortel Networks Inc.) [Canada]
Paper Title: Developing Highly Reliable Products in "Internet Time" (Paper 5A)

Today, competition is fierce and customers are first. We all have competitors who are responsive to customers and quick to ship their products. Our experience tells us that those who are first to market tend to win a greater market share. Market differentiation is critical! Streamlining development to respond quickly to market needs means: making effective fact-based decisions, emphasizing project management for consistency and accountability and focusing employees on a manageable number of features. We all work hard. To compete in today's market, we must increase the reliability of our products while significantly reducing product development time. These two product aspects are ones that customers acknowledge and will pay a premium for. Developing highly reliable products in "Internet time" is not a fad, it's a requirement to stay in business. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Gualtiero Bazzana & Mr. Enrico Fagnoni(ONION S.p.A.) [Italy]
Tutorial Title: Testing of Web-Based Applications(Tutorial G2)

The tutorial focuses on testing methods and tools which can be successfully applied to the testing of Web-based applications, notably: Internet WWW servers Intranet dynamic applications Extrenet e-commerce application For each application, the following aspects are dealt with: testing peculiarities testing methods testing techniques testing tools case studies The tutorial is constituted of over 150 slides which are constantly kept up-to-date w.r.t the unprecedented growth rate and dynamism of the Internet technologies and e-business. The distinguishing feature of the tutorial is that it combines methodological aspects with in-depth knowledge and experience of technical aspects, together with an overview of over 20 testing tools which can be applied depending on various phases. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Gualtiero Bazzana, Mr. C. Baresi & Mr. G. Rumi (ONION S.p.A.) [Italy]
Paper Title: RADIUM - Applying RAD to innovative ERP/E-commerce projects(Paper 11B - EI)

ERP systems are experiencing a tremendous growth owing to new requirements imposed by the evolution of the market and its new paradigms. Introduction of such systems is a big effort, both from an organisational and technical point of view. On the other side Internet and its new technologies are growing faster and faster and give to companies the opportunity to reach new markets. The integration of ERP systems and Internet technologies is the only way to satisfy new market requirements. This paper presents the experiences matured in the introduction of SAP R/3 following an "on spec, on time, on budget" strategy. The methodological approach combines the adoption of ASAP (Accelerated SAP, a methodology specifically intended for reducing delivery time) with innovative approaches like management on Intranet of the project key information. The approach has been successfully applied (within the scope of the RADIUM project) both to the introduction of classic ERP features as well as to the integration of ERP features with WWW environment. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Bill Bently (mu-Research) [USA]
Tutorial Title: How to Test an Object: The Information Flow Approach (Tutorial G1)

The industry trend toward software components, such as JavaBeanstm, shifts the focus of object-oriented testing to the unit level. The central problem at this level is how to select method invocation sequences for an object. Program analysis has the potential to identify these sequences in an automated fashion. Conventional forms, such as data flow analysis, do not provide a complete answer. This tutorial introduces a new theory of object testing based on dynamic information flow analysis. The theory yields fresh insights and a more comprehensive solution. The presentation will be informal, using simple JavaTM examples to explain the general approach. OUTLINE unique aspects of testing an object foundations of structural testing flowgraphs paths review of classical structural testing theory control flow analysis data flow analysis beyond classical theory: information flow analysis path structure of an object JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Robert V. Binder (RBSC Corporation) [USA]
Tutorial Title: Modal Testing Strategies for Object-Oriented Systems (Tutorial E, Parts I & II)

Classes (objects) have distinctly different behavior patterns (modes). A mode must be identified to select an effective test strategy. This tutorial presents new approaches for domain/state modeling to characterize class modality and shows how to produce effective test suites from these models. Participants in this seminar will learn how to: Identify the mode of the class under test. Develop a domain model of a class. Develop a state model of class behavior. Develop a domain-based test plan using the vertex probe strategy. Develop a behavior-based test plan using the FREE state model. Develop a test suite that achieved either vertex or N+ state coverage. (TOP OF PAGE) RETURN TO QWE'99 PROGRAM

Mr. Robert V. Binder (RBSC Corporation) [USA]
Keynote Title: Lessons Learned: The State of the Art in OO Testing (Keynote 2)

(Abstract to be supplied.) (TOP OF PAGE) RETURN TO QWE'99 PROGRAM

Mr. Nick Borelli (Microsoft) [USA]
Keynote Title: Seizing Control of the Development Lifecycle - (QW'99 Best Paper) (Keynote 6)

In this talk, I'll discuss scenarios where the Testing/Quality organization builds, drives and owns part or all of the Development Process and how that can help you achieve the types of Process Improvement and Quality gains that you are looking for. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Bart Broekman & Mr. Christiaan Hoos (IQUIP Informatica BV) [Netherlands]
Tutorial Title: How to Automate Your Tests with Lasting Benefits (Tutorial A, Parts I & II)

The tutorial includes: Growing need for test automation to achieve more efficient use of precious testing time, more re-use of developed testware and higher test quality. An approach to structure this automation process. A model is provided, which includes a life cycle, activities, technical and organizational aspects. Effective use of state-of-the-art test tools: Proven techniques (including "data driven" and "framework" architectures) to develop flexible and maintainable automated test suites. Life demonstrations of such techniques. A significant part of the tutorial is a case on the PC which will show the attendees step-by-step how to build a maintainable automated test suite. Cost / benefit examples from the real world. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Luigi Buglione & Ms. Elixabete Ostolaza (European Software Institute) [Spain]
Paper Title: Achieving Business Excellence in SPI: Applying the EFQM/SPICE Integrated Model in Industry (Paper 2B)

This work analyses the increasing importance for a Software Intensive Organisation (SIO) to implement and take advantage from a TQM model tailored for the software field in order to achieve business excellence through continuous improvement. ESI has developed the EFQM/SPICE Integrated Model, that combines the strengths of two well-known and accepted models: EFQM - for the business side, taking advantage from its capability to link results with specific business goals through a causal chain. SPICE (ISO/IEC 15504) - for the software improvement side, taking advantage from its effective process-based approach. The experience gained from 1998 trials is presented, as well as the consequent suggestions, useful starting points for the improvements of the upcoming model evolution. The new 1999 Trials design is also presented. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Marcelo Dalceggio & Mr. Alvaro Ruiz de Mendarozqueta (Banco Rio de la Plata SA) [Argentina]
Paper Title: Automated Software Inspection Process (4M)

Due to the huge effort that inspection process demands, we developed an automated inspection process based on our experience with Y2K code control in order to increase the efficiency of the inspection process. Our systems have more than 10 million LOC and we produce/modify 900 programs per month and these programs have to be compliant with more than 150 rules established with different standards. It's almost impossible to inspect manually that volume of code and to pay attention to the rules during inspection. Our inspection process has four basic steps: Selection of the inspection candidates, Identification of the attributes of each component, Inspection execution and reporting, Defect removal tracking and verification. We built a compliance policy and rules database based on different sources: code standards, failure tracking lessons learned, technical books suggestions, prohibited commands, software reuse and we developed a system that reviews source code and matches rules with statements producing an inspection report. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Roberto Delmiglio, F. Basili, E. Bendinelli, A. Manini, A. Rappelli, & G. Rumi (Italtel SpA) [Italy]
Paper Title: Test Automation in Telecommunications Software: A Case Study on GPRS (Paper 2T)

The paper describes the test automation experiences gained at Italtel Reti Mobili - Research and Development (RM-RD) department focusing on specific methods and tools for GPRS service. The following aspects are dealt with: a short company profile and a global description of the testing challenges for mobile telephony; the reference development process, the testing practices, the Software Process Improvement (SPI) activities within which test automation activities have been experienced; an overview about the GPRS service in terms of expected technical and business benefits; the test automation activities run in the last years, including a technical description of the environment and a quantitative effectiveness analysis of the introduced practices and tools; a detailed technical presentation of the methods and tools internally designed and developed by the RM-RD department to drive the testing of the GPRS feature; a general appraisal of lessons learned and future goals. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. K. Dinesh, Dr. Pankaj Jalote, Mr. M.R. Bhashyam, & Mr. S. Raghavan (Infosys Technologies Ltd.) [India]
Paper Title: Managing the Transition from ISO to High Maturity Levels of the CMM (Paper 8M)

In an effort to improve its processes, Infosys, a large ISO9000-certified software house, adopted the CMM framework. Infosys was able to successfully transition from ISO to level 4 of the CMM. A key success factor in this achievement was that the transitioning was treated and managed like an aggressive project. This article describe some of the important aspects of managing this transition. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Misha Dorman (IPL Information Processing Ltd.) [United Kingdom]
Paper Title: Testing C++: Why It's Hard and How To Do It Well (Paper 6T)

Object-oriented technologies, and the C++ language in particular, have enjoyed great popularity in recent years. Proponents have promised many benefits, including reduced lifecycle costs, increased reliability and improved maintainability. Unfortunately, not all of these benefits have been fully realised. One area where OO methods are relatively immature is testability and testing. The na=EFve application of traditional testing techniques to object-oriented C++ software has not been completely successful. Isolation testing =96 a technique successfully used to =93divide and conquer=94 traditional testing problems becomes unmanageable when applied to OO software systems. This paper explores the problems from a practical point of view, and discusses an improved integration testing approach based on an extension to the widely used source code instrumentation technique. This approach provides many of the advantages of isolation testing without forcing the tester to laboriously write simulated code (=93stubs=94) for the rest of the system. Measuring test effectiveness of OO software using traditional coverage and complexity metrics can be unreliable and misleading. This paper discusses some metrics which provide more insight into the static and dynamic complexity of the software under test. In particular, they attempt to measure the complexity of the interactions between classes and the extent to which the polymorphic features of the software have been exercised. In addition, modifications are proposed to the traditional coverage metrics, to make them more appropriate to C++ software. By combining the enhanced integration testing techniques and new metrics described in this paper, together with appropriate tool support, C++ testing will be more efficient and more effective. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Thomas A. Drake (Coastal Research & Technology, Inc.) [USA]
Tutorial Title: Testing Software Based Systems - People, Process & Technology (Tutorial D, Parts I & II)

This tutorial will examine the challenges associated with testing our increasingly complex, network centered, software based information technology systems. The tutorial will reveal and discuss the limitations of current testing practices, the use of testing as part of a larger program risk mitigation strategy, the need for quality testing, the impact of bugs, key activities necessary for both defect detection and detect prevention and discovering what we can learn about testing from industry through some dramatic examples and real world case studies. The tutorial will also provide a streamlined "just-in-time" practical approach for testing software developed and delivered under real time constraints and process limitations with an eye toward what is emerging in the testing discipline as "best practice" for the future. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Thomas A. Drake (Coastal Research & Technology, Inc.) [USA]
Paper Title: Extreme Quality: What Can we Learn from the Computer and Video Games Industry (Paper 10T)

The next generation of programs will be more complex than ever before. What can we learn from the computer and video games industry that is relevant for quality and points to the growing necessity for sustained high quality software development? What does the fascinating and ever exciting world of computer and video console games hold out for the future of quality in our increasingly software-centric information technology systems and services? An industry survey and analysis of this interesting and highly dynamic segment of the business provides some perspective on where both testing and quality are heading in a business where "game play" is everything for both the development house and the customer who "plays" the game. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. David Eade (McCabe & Associates) [United Kingdom]
Presentation Title: Integrated Quality for the Real World (VT 7)

The practice of software development has evolved rapidly and dramatically in recent years. From its inception as a collection of numerical calculations to its widespread use in military and scientific applications to its ubiquity in consumer devices to its dominance of the infrastructure of virtually all corporations, software has taken an increasingly prominent role in our society. Most recently, the growth of the Internet has lent an unprecedented urgency to software development as time-to-market can be measured in minutes rather than the traditional quarters. There has been a tremendous proliferation of software design methodologies, programming languages, and development environments, which have enabled larger, more complex software systems to be delivered on shorter schedules than ever before. However, for those systems to be assets rather than liabilities they must work properly when delivered, and remain stable when modified. This seminar will highlight sound techniques and proven technology to manage the quality and testing of those systems at the same high levels of sophistication that we apply to their design and coding. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. John Elliott & Mr. Peter Raynor-Smith (Defence Evaluation and Research Agency UK) [United Kingdom]
Paper Title: Customer Satisfaction Through Improved Requirements Understanding (Paper 3M)

Customer satisfaction is one of the key drivers in systems development, for IT and non-IT applications. However, widespread customer satisfaction is not attained largely due to problems of inadequate 'requirements understanding'. This paper describes the key concepts, a process improvement theme and case study that has been directed towards better customer satisfaction through improved through-life requirements engineering and management. The themes include business need modeling, customer developer interfaction and requirements management. The case study will refer to integrating these themes using some of the concepts within the Dynamic Systems Development Method as applied to general evolutionary systems development. This case study arises from the EU supported project, REJOICE, a Process Improvement Experiment within the ESSI initiative. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Peter Fröhlich & Mr. Johannes Link (ABB Corporate Research) [Germany]
Paper Title: Modeling Dynamic Behaviour Based on Use Cases (Paper 9A)

Use Cases are a popular formalism for capturing functional requirements. While the advantages of use cases for requirements engineering are widely accepted, it is less clear how the use case-based requirements shall be further formalized in the design phase. The UML meta model offers three different notations for designing dynamic system/subsystem/class behavior based on use cases, as discussed in: Activity diagrams interpret use cases as branching processes. Interaction diagrams can be used to formalize single scenarios contained in use cases. State diagrams specify the behavior of a system/subsystem/class in reaction to event from actors. After an evaluation of previous work on the alternative formalizations, the current paper introduces a novel transformation from a structured use case template into a nested UML state machine. We discuss the benefits of this new method, which include increased expressiveness (e.g. the ability to express pre- and post- conditions and the systematic formalization of dependencies among the use cases), less need for scenario integration compared to the other approaches and new perspectives for automated testing. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Ms. Nathalie Fuchs & Mr. Steven Van Schil (Gitek n.v.) [Belgium]
Paper Title: Structured Testing, a Must in a Validation Critical Environment (Paper 1A)

This presentation covers the following topics: 1. The growing need for a structured test process as part of a business improvement process at JRF 2. The JRF Validation Policy 3. The V-model, which includes a life cycle, activities and organisational aspects 4. Refining the V-model at project level by introducing TMap® 5. Lessons learned: facts and figures, measures to be taken 6. JRF objectives for the future JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Ian Gilchrist (IPL) [Belgium]
Presentation Title: C++ Software Testing and Cantata++ (VT 8)

C++ testing presents a number of difficulties. This short talk illustrates how these may be overcome with tool support from Cantata++. In particular it will cover aspects of black and white box testing, coverage analysis, simulating external software, and test reuse in the context of inherited code and templates. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Martin Gonzalez-Rodriguez (University of Oviedo) [Spain]
Paper Title: Mr. Martin Gonzalez-Rodriguez (University of Oviedo) "Automatic Usability Testing for Hypermedia Navigational Graphs (Paper 4T)

Multiple navigational graphs can be obtained as the result of the design stage of a hypermedia-based artifact. The only way to know which one adapts better to the user navigational metaphor is by mean of usability testing. This technique is expensive in terms of the number of human resources needed to perform it, and it isn't able to record spontaneous user behavior. It also introduces external noise because users could feel nervousness or confusion while they are under observation. The use of automatic testing tools is an interesting and cheap alternative that avoids the problems commented. We designed and developed our own automatic navigability testing system (ANTS), which can be used with any kind of hypermedia device including web sites. During the presentation we will dedicate a deep look at the design of ANTS, so audience will get the basis of developing similar systems, in order to monitor user behaviour in their own hypermedia prototypes. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Mats Grindal (Enea Data AB) [Sweden]
Paper Title: A post-mortem Analysis of a Semi-Successful Client Server System Test Project (Paper 10A)

This presentation will answer the questions: What worked well? What did not work? What can be done to improve the next project? For each phase of the project (planning, preparation, test execution and reporting) some key factors contributing to the end result are examined and commented on. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Pedro Gutierrez (European Software Institute) [Spain]
Paper Title: Roadmaps for SPI (Paper 1B)

Until recently, the only possibilities for an organisation when planning an improvement were either to rely on accepted and well established SPI models such as the CMM® or to start from zero. The current number of SPI experiences in Europe however, provides sufficient experience so as to help these organisations. It is now possible to design roadmaps for SPI based on real experiences. These roadmaps provide answers to questions such as: "What are the alternatives for increasing the quality of software products?" "Which are the steps for each of the alternatives?", or "Do organisations with different characteristics follow the same steps?" The present paper depicts the current scenario of SPI experiences in Europe (within the ESSI initiative) and presents roadmaps in order to help organisations to plan an improvement initiative. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Prof. Wolfgang A. Halang & Dr. Matjaz Colnaric (FernUniversitaet, Faculty of Electrical Engineering) [Germany]
Paper Title: Engineering Safety Related Control Software in Developing Countries (4A)

The development and rigorous verification of high integrity software for safety related industrial applications is an area of software engineering, which does not require geographically close contact between software engineers and end users. In the paper, a technology is presented which particularly facilitates the outsourcing of development in this domain. The idea presented in the paper is that it is reasonable to take pattern from the hardware design: HW architects take off the shelf chips and wire them together into complex circuits. To verify their behaviour they do not need to verify the operation of the chips: it has already been done and is taken for granted. Only the interconnections must be verified, which are far less complex. Similarly, it is reasonable to identify pieces of code that most often appear in such applications. Studies have been done showing that the number of such short programs is reasonably small (less than 100). They can be coded and verified very thoroughly because of their small scale (up to two pages). Once verified (and, possibly, licensed), put into ROMs in order not to allow for their changing, they can be treated as "SW chips" that only need to be "wired" together into more complex programs whose verification becomes feasible with reasonable effort. To facilitate such design, the IEC1131-3 standard for programming industrial controllers has been used: it provides practical means in the form of function block diagrams. Each block represents such a routine with inputs and outputs which are interconnected into programs. Once the blocks are proven correct, the verification of the latter is reduced to the verification of interconnections of the blocks. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Hans-Ludwig Hausen (GMD) [Germany]
Tutorial Title: Software Metrology Basis (Tutorial B1)

Overall aim of the course is to make the attendees familiar with the methods and principles of software metrics for procedural, object-oriented and agent-based systems. Attendee will exercise proven techniques for goal directed measurement, scaling and assessment as part of an industry proven, standardized procedure for concurrent software quality assurance and final evaluation of certification. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Hans-Ludwig Hausen (GMD) [Germany]
Paper Title: Seven Views to Website Quality Modeling and Assessment (Paper 11B)

A website is to be considered as a composed artifact comprising a number of connected, version-controlled artifacts (i.e. web pages). Each of the web pages of a website is defined via content and metacontent using the respective data models or standards. Both websites as well as web pages are defined in either a programming language (C, Java, etc.) or a markup language (SGML, XML, HTML, etc.) or a mixture of the two. Metacontent is defined in terms of a metacontent language (e.g. MCF, RDF). Content is given in a mixture of text-language, picture-language, or video-language. The approach to quality modelling uses seven views to define aspects of quality of an IT system. In particular we have: goals and objectives for the system, artifacts comprising the system, service provided by the system, methods to measure and assess the system, tools to instrument the measurement and assessment of the system, characteristics required and observed on the system, and finally the mission of the quality assessment for the system. The latter is defined in terms of the other six problem domains. Using the language-based approach to quality modelling we have the possibility to specify and assess web objects together with programs that handle them. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Dermot Hore (Optimal Systems Ltd) [Ireland]
Paper Title: The Impact on Staff of the Implementation of PVCS Technology (12B - EI)

Optimal Systems provides a comprehensive solution for interactive modelling, design and analysis of transmission and distribution lines. Through regular surveillance audits, information flow has been identified as the underlying cause of a number of problems being experienced. These include project overruns, continual requirements change, and errors in transition from change requests/bugs received via email too, requirements specification and onwards to system test. The experiment will assess the performance of PVCS Version Control and Tracker modules to improve the information flow between the agents and development staff and also among the development staff. The experiment, funded by the European Commission, will test the expectation that for small to medium size software development companies, improved information flow, through the use of software tools, will improve the quality of the end product, gain better control over project effort and improve effort estimation and scheduling. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Michael Jacobsen-Rey (Reasoning Software GmbH) [Germany]
Presentation Title: Reasoning Inspector Plus - Protection against Software Failures through Automated Inspection (VT 2)

In the world of E-business and high reliability requirements - testing of software is not enough! In order to shorten quality assurance and testing cycles, to reduce time and cost of finding and fixing software defects as well as making applications more stable and reliable, Inspector Plus provides the tool and methodology to recognize defects and measures program and module software fragility. The underlying consultancy process assures improved application quality through staged code inspections, technical review meetings and analysis of defect tracking information. Reasoning is the inventor and leader of "Y2K Automated Code Inspection" and has inspected more than 500 million lines of code for Y2K defects. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Narayana Jayaram (University of Westminster) [United Kingdom]
Paper Title: Towards a Test Process Framework for Distributed Component-based Software (Paper 8T)

The imperative of minimizing costs and time factors in the development of software for distributed systems applications, and the push for software re-use, have generated increasing interest in building distributed software architectures from Componentware. Further, the necessity to integrate business processes with real-time processes as found in, for example, Communications and Security protocols within the fast expanding domain of Enterprise systems - evidenced by growing E-Commerce and Supply-Chain network applications - provides a compelling case for the increasing use of Componentware. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Narayana Jayaram & Mr. James G. Henderson (University of Westminster) [United Kingdom]
Paper Title: Addressing Quality of Service Issues in E-Commerce Environment (Extra Paper)

The growing importance of electronic commerce is due to its recognition as a key driver of global markets where small and large enterprises compete as equal players. In the electronic commerce environment, the architecture and business processes from the business-centric system also referred to as the digital nervous system, where such critical corporate functions as business operations, reflexes and logistics as well as customer interactions are supported. The success of the business-centric system depends largely on the extent to which the quality of service (QOS) issues are addressed. Hence this paper will identify and examine these issues, define the maturity model that we are developing to formalise the quality assessment framework, and present our analysis on standards that impact the quality process. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Elie Kanaan (Mercury Interactive) [France]
Paper Title: E-Commerce: Where Application Testing means Staying in Business (Paper 12B)

Can E-Business mean big business? Yes. But is the WebSite fast enough? Does it scale? Is it secure? Is it reliable? There is serious money to be made in E-Business. But it's not enough to just put your applications on the Net. You have to ensure a positive user experience - with short response time, secure interactions and intuitive interface. You need also to deploy, on time, a scalable system that performs flawlessly all day, every day. But how can you be sure of all that before you go on-line? And during the lifetime of your Website? The answer is by comprehensive automated testing before deployment and performance management during the lifecycle of your website. Test in order to deploy with confidence and ensure a memorable user experience: Test for performance & scalability Test for functionality and usability And manage you testing environment Performance as seen by the users in order to keep them coming and coming back: Measure & feel the performance of the lack of it as users would Take preventive measures to keep performance at the top Avoid corrective measures as in E-Business they usually come too late! During this presentation I will share with the audience the experience of some of the Fortune 1000 companies such as IBM, SAP, AOL, J.P. Morgan, Fedex, Cisco Systems, Lucent Technologies, American Airlines, Citibank, Microsoft, Ford Motors and Deutsche Telekom, in testing their E-Business applications and managing their continuous performance, and the benefits they derived from it. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Tim Koomen (IQUIP Informatica BV) [Netherlands]
Paper Title: Test Process Improvement Experiences: Everything You Always Wanted To Know... (Paper 9M)

The presentation is split in two parts. The first part deals with the TPI*-model, which is based on current state-of-the-art test process improvement practices. The model gives practical guidelines for assessing the maturity level of testing in an organisation and for step by step improvement of the process. In the second part of the presentation, we will discuss three cases from our own experiences in using the model, both successful and not so successful. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Tim Koomen & Mr. Rob Kuijt (IQUIP Informatica BV) [Netherlands]
Paper Title: Improving Developer's Tests (Paper 2A)

The presentation discusses the reasons for improving the developers' tests (i.e. program test and integration test). Next, a practical approach is given for improving the quality of the developers' tests . The emphasis of the approach is on: clear testing responsibilities; insight in the quality of the test object by using exit- and entry-criteria. The presentation concludes with two short case-stories. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Tim Koomen & Mr. Ruud Teunissen (IQUIP Informatica BV / Gitek n.v.) [Netherlands / Belgium]
Tutorial Title: Stepwise Improvement of the Test Process using TPI (Tutorial F1)

This tutorial deals with the TPI*-model, which is based on current state-of-the-art test process improvement practices. The model gives practical guidelines for assessing the maturity level of testing in an organisation and for step by step improvement of the process. The purpose of such improvement could be reaching CMM level 3. The tutorial includes a general description of the application of model, which deals with how to implement and how to consolidate the improvements. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Ara Kouchakdjian & Mr. Larry Apfelbaum (Q-Labs / Teradyne SST) [USA]
Tutorial Title: State Modeling for Usage-Based Testing (Tutorial C2)

In software testing, it's not only important to know what you're doing, but to know what you're not. There are a huge number of possible usage scenarios. During a test suite review, a designer might be asked to explain what the set of included usage scenarios covers and what it does not. Many designers cannot answer these questions, particularly the second. Using state models to describe usage provides a framework for understanding what a suite of usage scenarios is and is not doing. This same framework enables the systematic generation (manually or automatically) of a well-defined set of usage scenarios. As with any modeling technique, building the model provides a constructive focus to learning about the software and interacting with users and developers. This tutorial will survey and describe: usage-based test approaches including function-based testing, use cases, and beta testing basic state models, their components, and various types of usage-oriented supplements to these models such as probability information the constructive dynamics of model generation a variety of associated test generation strategies and coverage criteria using statistically generated test suites for reliability certification JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Ara Kouchakdjian (Q-Labs) [USA]
Presentation Title: Experience using Statistical Usage Testing at the FAA (VT 10)

This presentation will discuss the application of Statistical Usage Testing (SUT) to a project at the FAA Technical Center in Atlantic City, New Jersey. It will discuss how SUT was applied, lessons learned, and results. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Axel Lankenau & Mr. Oliver Meyer (Bremen Institute of Safe Systems) [Germany]
Paper Title: Formal Methods in Robotics: Fault Tree Based Verification (Paper 1T)

The intention of the paper is to emphasize the importance of employing formal methods for the design of robotic systems. After a brief survey of current research in this area, a set of requirements is discussed that formal development process should fulfill. As an extended example, a general verification approach for reactive systems is described in detail. It is based on a CSP specification of a fault tree that observes the behaviour of the target system. A template for the modelling of fault tree leaves and nodes is given, and it is instantiated by a "real-world" application taken from the field of mobile robotics. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Lorenzo Lattanzi & Mr. Mario Musmeci (ALENIA AEROSPAZIO) [Italy]
Paper Title: Safety Critical S/W Development for a Satellite Based Navigation System (Extra Paper)

This paper will analyze the problems related to the software development for a safety critical system. The analysis is performed in the frame of a navigation application field and in particular using as a reference the RTCA standard "Software Considerations in Airborne Systems and Equipment Certification" (DO-178B) and the direct experience made on the European Geostationary Navigation Overlay Service (EGNOS) project. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Jin-Cherng Lin (Tatung Institute of Technology) [China]
Paper Title: An Efficient Measurement for Assertion Placement (Paper 6A)

Software testing is one of the most powerful methods to improve the software quality directly. Usually, testing costs on the large scales of 50% during software system development phase. How to reduce the testing cost thus becomes an important issue. Software Testability, the degree for software to reveal its faults during testing, is an important issue for quality assurance and verification. When testability is low, testers often want advice on how to increase it. We propose using data state errors measure for assertion placement. Software assertions are one relatively simple trick for improving testability. When knowing where data state errors appear likely to hide from a test, we have insight into where assertions are beneficial. We explore using data state errors that measure testability as a method for where and to inject software assertions in their placement. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Minsheng Liu & Mr. Arthur Gorges (Hüngsberg AG) [Germany]
Paper Title: Improving Quality in the Software Development Process on the Basis of UML (Paper 11T)

This paper represents the results of the execution of the project funded by the European Systems and Software Initiative (OFTPIVE.PIE). In the project the object-oriented method with UML and a new paradigm approach have been introduced to the application environment of the software development and to improving the process of the software development. With the aid of the UML, the weaknesses in the process of our software development have been recognised and the process has carefully been redefined in detail and improved. The quality of each step in the software development process has been controlled. It is shown that the introduction of the UML, CASE tool and a new paradigm is beneficial to the software development process. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Dave Locke & Ms. Catherine Connor (Rational Software Corporation) [USA]
Paper Title: How to Implement a Effective Requirements Management Process (Paper 8A)

This paper presents the elements of an effective requirements management process, including how to: elicit, document, organize and track changing requirements; manage iterative and unanticipated changes throughout the software development lifecycle; share information and unify cross-functional project teams across a development organization; and ensure software system quality and reliability. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Ms. Hanna Luden (Getronics Software) [Netherlands]
Paper Title: Software Project Evaluation as a Vehicle for Software Process Improvement (Paper 5M)

Once an organization begins with the software process improvement adventure, a long-term path is entered. Already during the assessment software professionals and their managers are encouraged to discuss and evaluate their current practices, with their pros and cons in order to improve them. The long lasting silence following the assessment activities, due to the need to carefully analyze and plan the improvements, forms a risk of losing the gained momentum. This paper discusses project evaluation as a simple - almost obvious - tool that can effectively help preserve the gained momentum. It is also a means to spread awareness for the need for change and to exchange information about best practices. We discuss our experience with qualitative project evaluations (QPE). QPE must be carefully planned, carried out following a script, and documented in the broadly accessible project evaluation database. Thus, the evaluation serves as an important vehicle for continuous improvement, providing the organization with direct information about its actual practices. It offers early evaluation of relevant aspects of the software process, helps set priorities for improvement actions, allows monitoring the effects of already introduced improvements and of culture, and bridges the gap between software professionals and the software process improvement program. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Benedikt Lutz (Siemens AG Osterreich) [Austria]
Paper Title: Writing a Software Engineering Handbook for the Intranet (Paper 10B)

If extensive technical documentation (like manuals, process descriptions, handbooks) is put as a hypertext into the internet/intranet, then you have to struggle quite soon with problems of user acceptance and practical usefulness of such tests. Therefore it is strongly advisable to consider aspects of usability and usefulness in the design of highly structured large websites. For the design of such online information the findings and principles of Test Linguistics, Communication Studies and Usability Engineering can be very useful for us. This paper I will presents a concrete example (with live presentation) what it means to apply such principles to the design and text structure of a large online handbook: In the last years we revised the Software Engineering Method of Siemens AG Austria, Program and Systems Engineering, which is now available as an Intranet Application consisting of about 1000 html pages. Some keywords/themes of the paper: Hypertext design and definition of node and link types Structure, granularity and length of a page "Informational surplus" through Hypertext Interconnection of text and graphic elements Multiple access structures for the users and more ... JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Regis Mauger (Mercury Interactive) [France]
Presentation Title: Testing E-Business Applications (VT 1)

In this presentation, we will focus on Testing E-Business Applications to deploy with confidence in order to be sure the Business is working and it will keep working under load overtime. We will be able to cover: How to capture Business Processes? How to use these Business Processes to be sure the application will work properly with a single user and will deliver an acceptable response time under load with 10, 1000, 10000...Users? How to monitor the Service Level delivered in production? JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Edward Miller (Software Research, Inc.) [USA]
Papar Title: E-Commerce Reliability and Web Testing (Paper 7B)

(Abstract to be supplied.) JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Larry Moffett & Mr. Jouko Luukas (e-Strategy / CCC Professionals) [Belgium / Finland]
Presentation Title: RiskDRIVER: Risk Management for Projects and Process Improvement (VT 6)

We will present RiskDRIVER, a European web-based initiative to promote best practices in project risk management. After introducing the concepts of project risk, risk management, and risk-driven project management, we cover the five steps of the risk management process: 1. Identification 2. Assessment 3. Action plan 4. Follow-up 5. Knowledge capitalisation The RiskDRIVER website (www.riskdriver.com) is a one-step resource for risk management practitioners, expert and novice. We will demonstrate some of its interactive features: RiskProbe self-assessment tool RISKMAN online risk catalogue Discussion forum Best practices documentation Our presentation will conclude with a demo of RISKMAN, the only software package to provide thorough support of a risk management methodology and full integration of risk management in all phases of project management. Interested participants are welcome to visit our booth in the Exhibit Hall to see an in-depth demonstration, pick up product literature, or receive a complimentary copy of the RiskDRIVER CD-ROM. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Pietro Moro & Dr. Antonio Cicu (Optec / QualityLab Consortium/MetriQs) [Italy]
Paper Title: Improving Software Documentation: Results of the ESSI PIE DOCPROVE Project (Paper 3B)

The DOCPROVE project is an ESSI PIE (Process Improvement Experiment) funded by the European Commission and undertaken by Optec Srl, an Italian small company operating in the development and commercialisation of optical and opto-electronic custom systems. The paper will cover the following main points of the improvement project: the essentials of the approach (driven by business goals), including the ami and GQM approach for metrics; the details of the application of SPICE process models to software documentation (in the framework of a SPICE trial), with emphasis to the adopted systemic approach; the effectiveness and ease of use of the adopted templates and tools; the results of the measurements and of the SPICE final assessment; the lessons learned regarding what has been really improved versus sustained costs, also from the point of view of motivation, training and reaction of people; and finally, an evaluation of experience, replicability inside and outside Optec. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Feng Nan (IBM China Research Lab) [China]
Paper Title: A New Tool for Measuring Server's QoS with Multi-Workload (Paper 7T)

This paper addresses the following: What is an appropriate notion of quality of service for the Internet server? The QoS presented in the paper is aimed at measuring a site how it should respond to external requests, how efficiently it responds to critical requests and how many requests it can respond to. What are drawbacks of the old performance testing tools? The old testing tools regard the Internet server only as a file server (or web server), so only the response time and throughput are tested. And these tests are often performed without other kinds of workloads. What is the workflow of e-tester? The workflow of e-tester contains four steps shown below: 1. Workload case generation. 2. Workload case customization. 3. Perform testing. 4. Report. Different workload kinds may have different workload case generation step and workload customization step. How can such a QoS test tool be implemented? In this section, we describe the detail of the test tool. The e-tester consists of three parts: 1. Workload controller. Workload controller runs on a monitor workstation to control and monitor the results from all agents. 2. Agent. Agents started and controlled by the workload controller gather the test results from engine and send them to controller. 3. Workload engine. One workload engine started by one agent run test script on the destination server and returns the test results to agent. What is the open interface for 3rd parties developing another workload? In order to make sure the reusable of this framework, we define two Java interfaces for 3rd parties to develop and other workloads. One is iConfig which is used to generate a new workload case, the other is iWorkload which is an engine execute the new workload case. Any workloads developed by 3rd party implemented these interfaces can be integrated into this framework. What is the QoS of current popular systems? We performed QoS test with our tool on several popular platforms such as NT, Linux, and S/390, compared the results and discussed the reasons of such results. The remainder of this paper provides the background material on the related work and describes the test environment in our research work. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Hugh Newsam (Pi Technology) [United Kingdom]
Paper Title: Automated Testing of COM Software in a RAD Environment (Paper 5T)

Pi Technology develops software for both embedded and Windows environments. The embedded teams have been using advanced automated test techniques for a long while now but the windows development area certainly needs to catch up. A general overview of the objectives presented in this paper would be: To investigate improving test coverage at each release of the product. The experiment aims to do this by using automated testing, therefore reducing the time required to run a full set of tests on the product at each release. By using automated testing much more regression testing can be performed when a new release is made. The rules of COM development aid this as they state that an interface, once published must not change. One should also make sure that the semantics of the object interfaced should remain constant although the underlying mechanism may undergo change. The experiment also aims to produce recommendations based on difficulties faced during the implementation of the automated testing. The paper discusses the issues involved in choosing appropriate testing tools for the job of developing tests for this particular product. The choice of tool depends on the application under test. The types of things we have to perform during an automated test depends on the application under test. The types of things we have to perform during an automated test are described and then reasons for the choice of tool based on those requirements are presented. The choice of coverage tool is also discussed in the paper. All the coverage tools we looked at provided the same level of functionality and so the choice was more dependent on the tool's integration with the automated testing tool and our current build process. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Henrik Oertel, Mr. Klaus Alvermann, Mr. Stephan Graeber & Mr. Lothar Thiel (Institut fur Flugmechanik) [Germany]
Paper Title: Developing Embedded Software for a Helicopter Testbed (Paper 3A)

Based on the current development of an experimental on-board equipment for a new helicopter testbed, the design tool ObjecTime is used. The paper gives an overview over the baseline project, describes the software development procedure, presents an evaluation of case tools and concludes with lessons learned. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Mark Paap (IQUIP Informatica BV) [Netherlands]
Paper Title: Test Tool Selection: Best Practices (Extra Paper)

Why do organizations buy test tools? The most common answer for organizations to buy test tools are the wish to: optimize the testing process decrease lead time of test execution decrease the need for human resources improve quality of testware and test execution standardize the products of the testing process What is the problem with test tool selection? Selecting test tools is not something that is part of the daily activities of organizations. The people that are involved in selecting the test tools are not the same people that are involved in the selection of other software packages like financial packages, case tools, etcetera. This means that experience in the selection process is missing. There is a risk that the test tool vendors will "take over" the selection process and the buyers become following instead of leading. To deal with this problem, it is necessary to have a clear picture of the test tool selection process. During this process there are some aspects of test tools that need special attention. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Jens Pas (ps_testware) [Belgium]
Paper Title: Managing Your Test Costs (Paper 12M)

In his presentation, ps_testware's Operations Manager Jens Pas, concentrates on the different aspects of managing test costs. The speaker will discuss in depth how to forecast a testing budget, how to optimise test costs and how to manage test costs while testing. The presentation is intended for an audience of advanced test professionals, test and business managers. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Geert Pinxten (ps_testware) []
Presentation Title: Test Automation: Levering your Test Results (VT 5)

This presentation is a case study that illustrates the evolution of the test automation approach at ps_testware. The presentation will sketch the evolution of the experiences gathered during the many attempts to automate test process in various projects, which finally resulted in a specific approach. The audience will be warned of the pitfalls of automation and will get the solution towards a successful implementation strategy. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Martin Pol (IQUIP Informatica BV) [Netherlands]
Keynote Title: Lessons Learned: Test Process Improvements (Keynote 3)

The emphasis of the presentation is focus on: Testing challenges require solutions Growing test Maturity Continuous improvement of the test process Since time-to-market, competition, globalisation and the quality of services, including the quality of software systems, have become a serious condition to survive for many organisations, the need for an adequate test process is becoming increasingly important. Besides, the complexity of the IT-solutions is growing dramatically, e.g. with Component Based Development and the integration of business processes and related innovations like internet and electronic commerce. These challenges for testing require improvement of the test process and new solutions from the testing community. New methods and techniques are necessary and automation is a prerequisite. The growing need for testing and the ever changing challenges has made testing more and more mature. New testing methods, techniques and tools of a satisfactory level are developed, trained and experienced testing staff are more and more available and testing approaches which are generic to some degree enable organisations to deal with the continuous technology push. However, in practice it turns out to be hard to define what steps to take, and in what order, for improving and controlling the test process. A reference model for test process improvement is required to support the growing test maturity with the definition of small and controlled improvement steps. Since Software Process Improvement models such as CMM/SEI generally are less specific for testing, suitable Test Process Improvement models are developed to enable structured test process improvement. The TPIŽ model is an increasingly popular model that supports the selection and prioritisation of improvement steps. It also provides a large number of practical details and instructions on how to implement the improvement steps. Improving the test process is required to establish the stable testing foundation, which is a condition to face future challenges. (Abstract to be supplied.) JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Prof. David Powell (Interim Technology) [United Kingdom]
Paper Title: Test Management -- Solutions for Project Improvement (Paper 10M)

Testing of information systems is a major concern for many organisations. It is often perceived as costly, time consuming, boring and difficult to manage. But testing is necessary to avoid even greater problems in the "live" environment. Automation of the test process with test tools has drawn a lot of attention as a possible solution to the problems of manual testing. These tools are known as CAST: Computer Aided Software Testing. But without a proper approach things can get worse instead of better, and there have been many organisations which have given up on their use, and therefore lose the benefits they were designed to bring. In 1994 work began on a new approach to testing and test automation which takes the automation of testing a step further than the commonly used "record and playback" method. It has been in use in over 300 projects across Europe among them banks, oil companies, telecommunication, government, chemical companies, stock exchanges, insurance firms, flight operators, etc. At its heart is a structured approach for test management/organisation, test development, and test automation/execution. This approach has been given the name of TestFrameTM as it provides a framework for the Test Manager to use to improve automated and manual testing. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Steve Quirke (X-pert Group) [South Africa]
Paper Title: How we Implemented a Change Control Process in a Large Financial Services Organization (Paper 1M)

How do you introduce key Software Quality processes in an environment where all the old-timers say "we tried that 20 years ago and it didn't work", the young techies don't understand the words you are using and everyone in between thinks of a different excuse not to do it? The IT department consisted of about 900 people supporting different arms of business. The systems ranged from a massive suite of 30 year old systems written in Assembler, to World Wide Web sites which had won international awards. A Software Engineering Process Group had been set up to define and implement processes based on international best practice. The situation was not unusual. In the new PC and Server computing environments the old mainframe disciplines were more difficult to maintain and some of the controls had fallen away. There was also suspicion around the tightness of controls in the mainframe environment itself. There was a change control process in the mainframe environment but there seemed to be loopholes and the library management and version control processes were not adequate. In this presentation I will show how we successfully implemented a Change Control process by bringing together three tried and tested disciplines, Software Engineering, Project Management and Organisational Development. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Steven Rabin (Interworld Corporation) [USA]
Paper Title: e-Commerce Functional Testing and Performance Benchmarking - Methodology and Criteria (Paper 7B)

One of the key aspects of a successful eCommerce initiative is performance. While this is often overlooked or taken for granted very few customers will return to a site that exhibits sluggish performance. This session discusses the key elements in designing an eCom based performance testing benchmark and the methodology that goes along with it. This includes workload characteristics, transactional definitions and site assumptions. A framework for eCom performance benchmarking and the criteria that need be measured is also introduced. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Cristian Radu (Integri) [Belgium]
Presentation Title: Test Environment for Terminal Certification (VT 3)

The test environment presented in this talk is suited for certifying any smart card accepting device like Point-of-Sale terminals, ATMs, and GSM handsets. The tool controls the interaction of the terminal under test with a set of test agents and can configure them to answer various testing purposes. A new concept - the State Transition Tree - facilitates the definition and programming of the test scripts. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Ms. Suzanne Robertson (Atlantic) [England]
Tutorial Title: Making Requirements Visible: A Strategy for Quantifying Requirements and Constraints (Paper B2)

This tutorial explores the quantifiable and hence testable characteristics of requirements and constraints. The emphasis is on making requirements visible so that we can apply a series of tests to ensure that they really are requirements. The idea is to arrive at a common understanding of each requirement. Then, throughout the project, we have a much better ability to negotiate, communicate, trace and test solutions to requirements. The tutorial also illustrates how to create testable clusters by linking requirements using business events and use cases. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Andreas Rudolf & Mr. Rainer Pirker (IBM) [Austria]
Paper Title: How to Test the EURO Effectively (Paper 4B)

The presentation will show how to test the changeover to EURO in software projects. It attacks the test from various points of view starting from unit-, function- to system- and integration-tests where the main focus is on testing critical business transactions. It gives you guidelines how to perform time-warp tests for each different EURO-phase to successfully introduce the EURO implementation to a company. It also tells you how the implementation of the EURO differs from implementing other currencies. The approach is based on working experience from various long running test projects that also includes Y2K- and new-application-tests. You get a number of measurements which help you to estimate your project and identify your risks. We give you composed the critical success factors and tell what worked well and what did not. Key Points: How to perform time-warp tests for each different EURO phase. Guidelines for selecting the right testcases with focus on critical business transactions. Measurements and critical success factors for EURO projects. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Johannes Ryser & Dr. Martin Glinz (University of Zurich) [Switzerland]
Paper Title: A Practical Approach to Validating and Testing Software Systems Using Scenarios (Paper 12T)

Scenarios (Use cases) are used to elicit and document requirements. They also form a kind of abstract level test cases. Yet they are seldom used to derive concrete system test cases. This presentation focuses on a three-step-procedure to use scenarios to systematically derive test cases for system test: Scenario creation Formalization of narrative scenarios into statecharts Path traversal in the statecharts to determine concrete test cases JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Robert A. Sabourin (Purkinje Inc.) [Canada]
Tutorial Title: Bug Priority and Severity (Tutorial H2)

In this interactive half day tutorial the concepts of defect priority and severity are explored. The fundamental question in software engineering "How do you know when you are finished?" is examined. The journey begins on a freezing cold Canadian Winter day on an elevator ride during which Robert Sabourin accidentally overheads some strangers discussing problems with the most important software project taking place in the company! The class is invited to help Robert in his job and to effectively define and identify characteristics of the problem, most importantly the priority and severity of the issue! The concept of the four-quadrants of priority and severity are taught and the class is clearly shown how business factors influence the quadrant of a bug! The class includes a review of some actual defect arrival graphs from recent commercial product development efforts and provides an answer to the fundamental question of software engineering. Practical aspects of bug tracking, defect logging and bug review meetings are included. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Mika Salmela, Jarmo Kalaoja & Jukka Korhonen (VTT Electronics) [Finland]
Paper Title: Automated Test Reuse for Product Families (Paper 1M)

Efficient testing methods are needed for software product families based on reusable components. In our paper we propose two methods for developing reusable tests and test suites. The first one uses the structures of the product feature model to organise test material. The other method applies test case componentation to generate test suites. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Hanania T. Salzer (RTS Software Ltd.) [Israel]
Tutorial Title: ATRs (Atomic Requirements) Writing Workshop (Tutorial C1)

A half day hands-on tutorial where participants will become reasonably trained in writing atomic requirements (ATRs.) The tutorial is extensive enough to enable the participants to study and write more than a hundred different ATRs. Participants will receive a lexicon of different forms of ATRs and learn which should be used in which situations. They learn to avoid common mistakes in the phrasing of ARTs. Participants will write ATRs under various circumstances: when designing a small program, a GUI and a database table, when specifying the features of an undocumented application, when documenting standards, and more. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Chris C. Schotanus (CMG) [Netherlands]
Presentation Title: TestFrame, organising the use of record and playback tools with Action Words (VT 4)

CMG has introduced an entirely new method to the market that improves the use of record and playback tools in an end user environment: Automated testing with action words. This method is five years old now and is used by hundreds of people in countless mainly large and complex projects. Using available test tools standard test software has been developed which improves the use of those tools. In separate files (i.e. spread sheets) called clusters, records are defined containing an action word and all the parameters used during this action. The action word is linked to one or more functions in tailor made test scripts. The main advantages of this method are: Early start of test development: since the test logic is defined in separate clusters the definition of test cases can start as soon as the development of the system has started. Test software is easy to maintain: Test script are programmed, not record. This implies a clear structure of the technical test products Separation between test scripts and test logic: Test cases can be accessed by people who are not experts in test tools or even testing in general Clear functional report back of the results of the test in separate test reports. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Lieven Schouwaerts (Gitek n.v.) [Belgium]
Paper Title: How to Implement Structured Testing in Narrow Time Boxed Projects (Paper 6M)

In this paper, I would like to share my experience in introducing structured testing in narrow time boxed projects. The paper is based on the introduction of structured testing at a large Belgian bank, where a lot of relatively small projects, with alot of interfaces, had to be released simultaneously. Extra risk factors were: Alot of shared resources in the different projects Due to a recent merger, people with a different "culture" had to work together Fixed time boxes JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Jeff Schuster (Rational Software Corporation) [USA]
Paper Title: Effective Testing for Java-based Web Software (Paper 9B)

Developing and implementing Java Web and e-business applications pose significant quality assurance challenges. "Internet Time" schedules, n-tiered complexities, high cost of failure, unpredictable user load, and varying runtime configurations make the problem even more acute. This presentation offers the QE practitioner practical approaches to understanding, then testing Java applications, to ensure their reliability, functionality, and performance. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Harry Sneed (SES Software-Engineering Service GmbH) [Germany]
Keynote Title: New Technology: Control Flow Animation as a Means of Class Testing (Keynote 1)

Source code animation is a technique to confirm the correctness of C++ and JAVA classes, as well as their interactions with one another. Since it is very difficult and also time consuming to build up a class test environment, animation may prove to be a viable alternative, especially when used in connection with the UML state transition and sequence diagrams. Developers can check how their code executes as demonstrated by the animator against how it should execute as specified by the UML diagrams. The speaker who is responsible for quality assurance in Vienna will also point out the problems of module testing with such large distributed object-oriented systems. He suggests that the conventional concepts of unit testing have to be revised to fit the testing requirements of modern we-based, distributed software. The old approaches no longer apply. To this end, several recommendations are made to alter existing test standards and to provide new kinds of unit testing tools. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Fawzy Soliman (University of Technology) [Australia]
Paper Title: Managing the E-Business Operations (Extra Paper)

The number of internet users has been growing rapidly with most of this growth coming from newly registered commercial enterprises. Organisations use the Internet for shortening the development cycle of products, responding to customers and receiving their feedback faster and more effectively, communicating with suppliers and experts from around the world, and accessing supercomputers for research and development. Conducting business over the Internet involves many phases. The most significant phase is the Internet E-Commerce Phase. This new method of conducting business has affected the management of many organisations very significantly. Accordingly, a model for the management of the implementation of Internet E-Business has been developed and presented in this paper. The model shows that using the Internet to conduct business in organisations is an evolutionary process and that the implementation of the Internet is a three-phase introduction. The first phase is concerned with creating and improving the Corporate Image and a Web site presence on the Internet. The second phase is the Electronic Commerce (E-Commerce) phase. At this E-Commerce phase companies use the Internet to conduct some of their usual business operations. During the second phase organisations gradually create: on-line catalogues with interactive credit card payment facility; on-line order taking and order placing; communicating with suppliers; conducting banking and making payments to suppliers and employees; and assisting the process of making delivery to customers, suppliers and stakeholders. The third phase commences when the company decides to embark on full-scale business activities using the Internet. Most of these activities are complex business functions. The model also highlights the importance of: Organisational Communication, Human Resources Management and, Organisational culture as factors affecting the implementation of Internet E-Business. The model also shows that during each phase, the Internet E-Business is used for three distinct applications; namely Business-to-Business, Business-to-Customer and Business-to-Employees. It is also found that the level and the intensity of information exchange between the business and trading partners increases as the business moves from one phase into the other. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Scott Yara (Sandpiper Networks, Inc.) [USA]
Paper Title: Performance 2000: Ensuring Consistent Web Site Performance (Paper 6B)

(Abstract to be supplied.) JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Niels-Bruun Svendsen (B-K Medical A/S) [Denmark]
Paper Title: Error Trending, Why & How (Paper 12A)

When developing systems and software an inevitable management question is: "When is the system ready for release?". On the bottom line the answer on when to release a new product for production and sales is a matter of being able to estimate the cost of releasing, as well as the cost of postponing the release. In calculation of the cost of releasing a product the number of remaining unknown errors is a major factor. Therefore error detection trends during the system-testing phase have been introduced as means of estimating the number of remaining unknown errors. This presentation will share the experiences gained and the lessons learned from introducing error trending as an estimation tool and highlight the benefits found as well the problems encountered. Error trending was introduced in order to support the decision on when to release a new product, but remarkable impact was seen during all of the system testing phase, and for all groups involved. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Johan Swinnen (Gitek n.v.) [Belgium]
Paper Title: TESPRA: A Practical and Integrated Method for Test Control, Reporting and Estimation (Paper 9T)

This presentation is about TESPRA, a practical, integrated method for Test Control, Reporting and Estimation. TESPRA was originally developed by Corné de Koning, IQUIP Informatica B.V., The Netherlands. The main goals of TESPRA are: to gain control over the test process; to give a detailed reporting on the test process to the project leader; to have experience based estimations. TESPRA is used during (test) projects at Janssen Research Foundation (JRF), a part of Janssen Pharmaceutica (Johnson & Johnson Company), Beerse, Belgium. For JRF, TESPRA has been extended by Johan Swinnen, Gitek n.v., Antwerp, Belgium. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Rob van der Pouw Kraan (Qualityhouse BV) [Netherlands]
Paper Title: Test Outsourcing: The First Experiences (Paper 12M)

(Abstract to be supplied.) JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Ronnie Van Parijs (Computer Associates) [Belgium]
Presentation Title: Automating Cleanroom Management (VT 9)

This presentation from Computer Associates stresses the importance of "Cleanroom Management" in managing application source code and components. This process ensures and enforces both the quality and reliability of an organization's application systems, a process that is becoming even more critical as Year 2000 remediated code is being implemented into production. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Otto Vinter (Brüel & Kjaer Sound & Vibration Measurement A/S) [Denmark]
Keynote Title: Experience-Based Approaches to Process Improvement (Keynote 6)

Software process improvement (SPI) is usually based on well-known models of software process maturity such as the Software Engineering Institute's Capability Maturity Model (CMM) and the European counterpart Bootstrap. This presentation reports on alternative approaches to SPI based on knowledge and experience that is already available in the organisation. Rather than a formal comprehensive assessment of the all software development processes in the organisation a "problem diagnosis" is performed. The problem diagnosis aims at identifying the most important (process) issues as perceived by the powerful organisational actors. Improvement actions are then planned and implemented in close collaboration with these actors. The presentation focuses on the results of using such problem diagnosis techniques at Brüel & Kjaer as an alternative SPI strategy. The presentation will report on problems and successes, and relate these results to formal assessments performed in parallel by an external body. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Mr. Otto Vinter (Software Engineering Mentor) [Denmark]
Paper Title: How to Apply for Funding from the IST Programme (Paper 5B)

Key Points: Prerequisites for application Formal application procedures and schedule Evaluation criteria and typical pitfalls Benefits of receiving funding JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Ms. Isabella Wieczorek, L. Briand, B. Freimut, M. Ochs, D. Pfahl, & J. Wuest (Fraunhofer IESE) [Germany]
Tutorial Title: Cost and Quality Measurement in Practice (Tutorial F2)

There are many cost and quality related decisions that need to be made throughout the software life cycle. Measurement can provide an objective and reliable basis for making many of these decisions. Much experience has been accumulated during the last 20 years. Many of the technical and practical problems that one faces when using software measurement can be addressed by readily available techniques and strategies. This tutorial provides the basic concepts and guidelines related to practical software measurement when used to address software cost and quality issues. JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM

Dr. Rainer Zimmerman (European Commission) [Belgium]
Keynote Title: (To be supplied.) (Keynote 5)

(Abstract to be supplied.) JUMP TO TOP OF PAGE RETURN TO QWE'99 PROGRAM