COTS-Based Systems Workshop
OverviewUSC/SEI/CeBASE COTS-Based-Systems Workshop Barry Boehm, USC
The February 7-9, 2001 COTS-Based-Systems (CBS) Workshop was sponsored by the University of Southern California (USC), the Carnegie Melon University Software Engineering Institute (SEI), and the Center for Empirically-Based Software Engineering (CeBASE), which includes USC, the University of Maryland, Mississippi State University, and the University of Nebraska at Lincoln.
This CBS Workshop was fortunate in assembling an outstanding group of experts from industry, government, academia, and research institutes with a wide variety of experience in CBS development and life cycle support. This web site assembles the key Workshop materials, including:
* Charts from the Workshop presentations on February 7;
* Summary charts from the three Working Groups on CBS Process and Life Cycle Considerations; CBS Metrics and Models, CBS Architectures;
* A workshop summary prepared by one of the participants, Prof. David Klappholt of Stevens Institute of Technology;
* A CeBASE "CBS Top-10 List" paper. A draft of this paper was prepared for and substantially improved at the workshop. A version of the paper is being published in IEEE Computer in May 2001.
The major conclusion of the Workshop was that a CBS is not just a somewhat different form of software reuse, but a fundamentally different form of software development and life cycle management. The increasing number of organizations whose information systems are largely COTS-based have to deal with several significant new sources of risk. These include having no visibility into COTS internals; no control over COTS evolution; and unpredictable interactions among independently-developed COTS products. For such risks, traditional programming-oriented software engineering guidelines are occasionally useful, frequently irrelevant, and often dangerous. Examples of the latter are traditional approaches to software requirements, object-oriented design and development, and complementary black-box/white-box testing.
Some of the CBS experience presentations provided useful new conclusions. For example, the practice of escrowing COTS source code for use if a COTS vendor goes out of business fails more than it succeeds. And the practice of understanding your COTS vendor's win conditions and helping the vendor to succeed is much more effective than an adversarial approach (Mike Moore, NASA).
The CBS Process and Life Cycle Working Group identified another important factor: that there is no one-size-fits-all CBS process model. Significant differences exist between COTS-supported (largely requirements-driven) products, for which variants of traditional models are relevant; and COTS-driven products, which require new approaches to such elements as requirements and testing (and there is a large gray area in between).
The CBS Metrics and Models Working Group explored estimation models for some of the SEI's CBS sources of effort, particularly COTS evaluation effort. It determined that COTS evaluation is another source of effort for which "how much is enough?" can be answered via risk analysis. This involves balancing the risk exposure of doing too little COTS evaluation (picking the wrong combination of COTS products) with the risk of doing too much COTS evaluation (of incurring too much CBS delay).
The CBS Architectures Working Group similarly concluded that determining an effective CBS architecture is substantially different from determining the architecture of a build-it-yourself "greenfield" system, particularly for heterogeneous CBS. The role of architectural connectors becomes particularly significant for heterogeneous CBS.
Although the Workshop made some useful progress, much more remains to be done. An important next step involves the upcoming International Conference on COTS-Based Software Systems. It will be held February 4-6, 2002, in Orlando, Florida, and is co-sponsored by the SEI, the National Research Council Canada, and USC/UMD/CeBASE. Its web URL is http://wwwsel.iit.nrc.ca/iccbss/.
Note: On the day before the workshop there were the following related presentation as part of CSE's Annual Research Review.
10:30 - 11:50pm-----CeBASE Status and Plans
-------------------------Basili - Results to Date
-------------------------Shull - Experimental Studies
-------------------------Brown/Henninger - SE Knowledge Repository and Portal
-------------------------Boehm - Opportunities to Collaborate2:00 - 3:20pm--------COCOMO™ Suite
-------------------------Abts - COCOTS Update
------------------------------------------------------------
Wednesday, 7 Feb 2001
USC-SEI-CeBASE Workshop on COTS-Based Systems (CBS)7:30 - 8:00am .........Continenetal breakfast
8:00 - 8:20am .........Workshop Overview -Boehm
8:20 - 9:45am .........Executive Perspectives on CBS (Barry Boehm chair)
......... ......... ......... ..(85 mins) Includes 5 minutes for chair comments and then:
......... ......... ......... . Art Pyster (FAA)
......... ......... ......... . Mike Moore (NASA)
......... ......... ......... . Steve Cross (SEI)9:45 - 10:00am ......Break (15 mins)
10:00 - 11:25am ....CBS Project Experience (John Foreman chair)
......... ......... ......... .(85 Mins) Includes 5 minutes for chair comments and then:
......... ......... ......... .Don Andres (TRW)
......... ......... .......... Greg Hinchman (Lockheed Martin)
......... ......... ...........Bill Roetzheim (Marotz)
......... ......... .......... Stephen Fox (Raytheon)11:25 - 12:25pm....break and lunch (60 mins)
12:25 - 2:10pm...... CBS Empirical Studies (Vic Basili chair)
......... ......... .......... (105 mins) Includes 5 minutes for chair comments and then:
......... ......... .......... Chris Abts (USC)
......... ......... .......... Brad Clark (SEI/Software Metrics)
......... ......... .......... Jeff Poulin (LMCO)
......... ......... .......... Nancy Eickelmann (Motorola)
......... ......... .......... Vic Basili (UMaryland)2:10 - 2:25pm....... Break (15 mins)
2:25 - 3:50pm....... CBS Experience Summaries and Guidelines (Don Reifer chair)
......... ......... .......... (85 mins) Includes 5 minutes for chair comments and then:
......... ......... .......... Betsy Clark (Software Metrics)
......... ......... .......... Tony Jordano (SAIC)
......... ......... .......... David Tobey (GroupSystems.com)3:50 - 4:05pm....... break
4:05 - 5:00pm....... Working groups context setting
......... ......... .......... 10 mins per Working Group to overview their area and set
......... ......... .......... initial agenda/discussion topics5:00 - 7:00pm....... Reception
----------------------------------------
Thursday, 8 Feb 2001 --Working Groups
7:30 - 8:00am....... Continental breakfast
8:10 - 8:20am....... Working Group Guidelines (John Foreman)
8:20 - 8:45am....... 1st International Conference on COTS-Based Software Systems
....... ....... ....... ......(John Foreman)
8:45 - 9:00am....... break, adjourn to WGs
9:00 - 11:30am......WGs
11:30 - 12:30pm....break, lunch
12:30 - 5:00pm......WGs
....... ....... Working Groups
....... ....... ....... 1. CBS Process Issues (Carney, Betsy Clark, Oberndorf; Al-Said)
....... ....... ....... 2. CBS Architecture Issues (Seacord, Medvidovic; Roshandel)
....... ....... ....... 3. CBS Metrics and Predictive Models (Abts, Brad Clark; Lee)-----------------------------------------
Friday, 9 Feb 2001
(AM; Finished by 11:30 am)
7:30 - 8:00am....... Continental breakfast
8:00 - 8:10am....... Introduction (Barry Boehm)
8:10 - 9:40am....... Working Group Reports 1
9:40 - 10:00am......Break
10:00 - 11:00am....Working Group Reports 2
11:00 - 11:30am....Working Group Reports 3
