2005

PhD Dissertations

Zhihao Chen's Dissertation (PDF)

Title: Reduced-Parameter Modeling for Cost Estimation Models

Date: May 2006

In this research, parametric software cost estimation models and their related calibration methods have been analyzed, especially for the COCOMO model and the Bayesian calibration approach. This research combines machine learning techniques and statistical techniques. With this approach, the prediction powers of the COCOMO parametric software cost model are shown to be significantly improved while the variability is decreased with respect to the dataset being analyzed. This research studies not only the accuracy but also the variances of the model and the variables. It can improve the confidence of people who use software cost estimation models, show the prediction power of software cost estimation models after calibration, and make it easier and better to perform software data collection and analysis. However, the research also identifies risks in using the approach, such as dropping parameters that will vary on future projects. This research provides methods that can help an organization to reason about the relationship between the characteristics of the organization and its projects' software development costs and schedules. The methods can thus
help the organization to make more cost-effective development decisions and investment decisions. The research also provides new insights on how to combine calibration, stratification, hold-out, and machine learning techniques to produce more accurate parametric models for particular organizations or situations.

Sunita Chulani's Dissertation ( PDF ) Title: Bayesian Analysis of Software Costs and Quality Models

Software cost and quality estimation has become an increasingly important field due to the increasingly pervasive role of software in today's world. In spite of the existence of about a dozen software estimation models, the field continues to remain not-too-well-understood, causing growing concerns in the software-engineering community.

In this dissertation, the existing techniques that are used for building software estimation models are discussed with a focus on the empirical calibration of the models. It is noted that traditional calibration approaches (especially the popular multiple-regression approach) can have serious difficulties when used on software engineering data that is usually scarce, incomplete, and imprecisely collected. To alleviate these problems, a composite technique for building software models based on a mix of data and expert judgement is discussed. This technique is based on the well understood and widely accepted Bayes' theorem that has been successfully applied in other engineering domains including to some extent in the software-reliability engineering domain. But, the Bayesian approach has not been effectively exploited for building more robust software estimation models that use a variance-balanced mix of project data and expert judgement.

The focus of this dissertation is to show the improvement in accuracy of the cost estimation model (COCOMO II) when the Bayesian approach is employed versus the multiple regression approach. When the Bayesian model calibrated using a dataset of 83 datapoints is validated on a dataset of 161 datapoints (all datapoints are actual completed software projects collected from Commercial, Aerospace, Government and non-profit organizations), it yields a prediction accuracy of PRED(.30) = 66% (i.e. 106 or 66% of the 161 datapoints are estimated within 30% of the actuals). Whereas the pure-regression based model calibrated using 83 datapoints when validated on the same 161 project dataset yields a poorer accuracy of PRED(.30) = 44%.

A quality model extension of the COCOMO II model, namely COQUALMO, is also discussed. The development of COQUALMO from its onset enables one to understand how a comprehensive modeling methodology can be used to build effective software estimation models using the Bayesian framework elaborated in this dissertation.

Bradford K. Clark's Dissertation ( PDF ) Title: The Effects of Software Process Maturity on Software Development Effort

This research examines the effects of Software Process Maturity, using the Software Capability Maturity Model, version 1.1, on software development effort. The technical challenge in this research is determining how much change in effort is due solely to changing Process Maturity when this change generally occurs concurrently with changes to other factors that also influence software development effort. Six mathematical models used in this research support the conclusion that for one hundred twelve projects increasing Process Maturity one level results in a 15% to 21% reduction in effort.

Cristina Gacek's Dissertation (PDF: body , appendix ) Title: Detecting Architectural Mismatches During Systems Composition

The USC Architect's Automated Assistant (AAA) tool and method version 0.1 [Abd-Allah 1996] provides a capability for early detection of software architectural style mismatches among four architectural styles: Main-Subroutine, Pipe-and-Filter, Event-Based, and Distributed Processes. For these four styles, mismatch detection is based on a set of seven conceptual features distinguishing each style, and a set of bridging connectors characterizing compositions among the four styles. However, it was a significant open question whether these conceptual features and connectors were sufficient to characterize composition of other architectural styles.

The work presented here formalizes some additional architectural styles--namely Blackboard, Closed-Loop Feedback Control, Logic Programming, Real-Time, Rule-Based, Transactional Database, and Internet Distributed Entities styles--and extends the mismatch analysis capability to cover interactions of the original four styles with the new ones. The analysis results tested various hypotheses, such as the extensibility of the conceptual feature framework for mismatch detection, and the sufficiency of the original seven conceptual features to characterize the broader set of styles and their composition. In our work we found that the underlying conceptual feature framework could work to cover a broader range of styles and systems, with some extensions. However, the conceptual feature set and the underlying Z-language formal model were not sufficient to cover the full range of styles and systems interactions. We have developed extensions to the conceptual feature set and Z formal model to cover the full set of compositional interactions analyzed. Additionally, we provide means for checking each and every mismatch at the model level, including the dynamic ones, as well as a fully operational tool. We also provide an initial discussion of a more formal basis for detecting and classifying architectural conceptual features, thus providing a formal framework for extending the models.

Hoh In's Dissertation ( PDF ) Title: Conflict Identification and Resolution for Software Attribute Requirements

A critical success factor in requirements engineering involves determining and resolving conflicts among candidate system requirements proposed by multiple stakeholders. Many software projects have failed due to requirements conflicts among the stakeholders.

The WinWin system developed at USC provides an approach for resolving requirements conflicts among the stakeholders. The WinWin system provides a framework for negotiation between the stakeholders to identify and resolve these conflicts. However, such systems do not scale well for large software projects containing many requirements.

Based on an analysis of the options for addressing this problem, I have focused on semiautomated tools and techniques for identifying and resolving conflicts among software quality attributes. I have developed two prototype support tools, QARCC and S-COST, which expand the capabilities of the WinWin system. QARCC focuses on software architecture strategies for achieving quality attribute objectives. S-COST focuses on tradeoffs among software cost, functionality, and other quality attributes. I have also developed portions of underlying theories and models which serve as the basis for the prototype tools.

Finally, I evaluated the theories, models, and tools with the results of WinWin negotiations, such as the CS577 15-project samples.

Mingjune Lee's Disseration ( PDF ) Title: Foundations of the WinWin Requirements Negotiation System

Requirements Engineering (RE) constitutes an important part of Software Engineering. The USC WinWin requirements negotiation system addresses critical issues in requirements engineering including (1) multi-stakeholder considerations, (2) change management, and (3) groupware support. The WinWin approach to date has primarily involved exploratory prototyping. The system is now converging on a relatively stable set of artifacts and relationships. This makes it feasible and important to formalize these artifacts and relationships to provide a solid scientific framework for the WinWin system. This is the focused problem addressed by the research presented in this paper.

Ricardo Valerdi's Disseration ( PDF )

Title: THE CONSTRUCTIVE SYSTEMS ENGINEERING COST MODEL (COSYSMO)

Date: August 2005

As organizations develop more complex systems, increased emphasis is being placed on Systems Engineering (SE) to ensure that cost, schedule, and performance targets are met. Correspondingly, the failure to adequately plan and fund the systems engineering effort appears to have contributed to a number of cost overruns and schedule slips, especially in the development of complex aerospace systems. This has resulted in a recent increased emphasis on revitalizing systems engineering in government and commercial organizations.
This dissertation presents a parametric model that can help people reason about their decisions related to systems engineering. COSYSMO, the Constructive Systems Engineering Cost Model, is an “open” model that contains eighteen parameters: four size drivers and fourteen effort multipliers. It is built on a framework similar to its wellknown predecessor, COCOMO II, and integrates accepted systems engineering standards to define its scope. Funded by industry affiliates, the model focuses on large-scale systems for military applications that employ a disciplined approach to systems engineering. Data was collected from six aerospace companies in the form of expert opinion and historical project data to define and calibrate the model. In reduced form, the model yields a PRED(30) of 50% for programs within a defined productivity range. In principle, the model should apply similarly to commercial systems engineering, but there is a lack of data to test this hypothesis.

The ultimate contributions of this dissertation can be found in at least two major areas: (a) in the theoretical and methodological domain of systems modeling in the quest of a more quantitative cost estimation framework, and (b) in advancing the state of practice in the assessment and tracking of systems engineering in the development of large aerospace systems.

Return to Technical Report System

To the Center for Software Engineering Home Page

The written material, text, graphics, and software available on this page and all related pages may be copied, used, and distributed freely as long as the University of Southern California as the source of the material, text, graphics or software is always clearly indicated and such acknowledgement always accompanies any reuse or redistribution of the material, text, graphics or software; also permission to use the material, text, graphics or software on these pages does not include the right to repackage the material, text, graphics or software in any form or manner and then claim exclusive proprietary ownership of it as part of a commercial offering of services or as part of a commercially offered product.