This guide describes the documentation policies and procedures for the CS577b class projects. The following documents shall be developed by each team to form a set of project baselines.
a. Use the supplied electronic templates for writing the documents. The templates are located on the CS577b web site. Adherence to the formats are encouraged, however alternative formats may be used subject to concurrence of the professor.
b. If the information that belongs in a paragraph does not apply to a project, include the paragraph with the phrase "NOT APPLICABLE", unless specific instructions indicate that subparagraphs may be deleted when not applicable. If the information is not available at the time the document is written, include the phrase "TO BE DETERMINED" with a completion date.
c. Reference other applicable documentation rather than duplicating information.
Acceptance Testing. Acceptance testing is conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system.
Application. An application is an aggregation of software that satisfies an end use function and is designated for separate configuration management.
Application qualification testing. Application qualification testing is performed to demonstrate to the customer that application requirements have been met. It covers the system requirements in the Software Requirements specifications (SRSs) and in any associated interface requirements specifications . If an application is developed in multiple builds, qualification testing of the completed application will not occur until the final build. Application qualification testing in each build is interpreted to mean planning and performing tests of the current build of the application to ensure that the system requirements implemented in that build have been met.
Build. A build is a set of software functions or capabilities which satisfy a subset of system requirements.
Configuration Item (CI). A CI is an aggregation of system elements (hardware, software, or both) that satisfies an end use function and is designated for separate configuration management.
Dry-run. To dry-run a system test is to run the test to ensure that the system is ready for witnessed testing.
Environment. Environment is the natural and induced conditions experienced by the system including its people, products, and processes. The natural environment (e.g., weather, climate, terrain); threat environment (e.g., effects of existing or threat system including communication interception); operations environment (e.g., dust, fog, thermal, shock, vibration) transportation and storage environment; and maintenance environmental test environments.
Evaluation. Evaluation refers to the examination and judgment of a system (or an element of a system) in terms of worth, quality of performance, degree of effectiveness, condition, and the like. The purpose is to determine the true characteristics of the system and to ensure that it successfully fulfills its intended mission.
Formal Testing. Formal testing is that testing and reporting conducted in accordance with approved test plans, and generally witnessed by the customer.
Functional Testing. Functional testing verifies that the functions of a system are present as specified. Functional testing is conducted to verify that the software and hardware satisfy the system's functional (operational) requirements.
Hook. A hook is a built-in capability that produces or makes accessible data normally not accessible. If not a system requirement, the hook must be removed prior to system test.
Integration. Integration is a process by which components are aggregated to create larger components.
Integration Testing. Integration testing is testing done to show that even though the components were individually satisfactory, as demonstrated by successful passage of component tests, the combination of components are incorrect or inconsistent.
Item. An item is a non-specific term used to denote any product, including systems, subsystems, assemblies, subassemblies, units, sets, parts, accessories, computer programs, or computer software.
Non-developmental Item (NDI). An NDI is any item that is available in the commercial marketplace or elsewhere, including COTS, GOTS, or any previously developed item that is either in use or not yet in use.
Operational Testing. Operational testing is performed by the end user in an operational environment.
Performance. Performance is a quantitative measure characterizing a physical or functional attribute. Performance attributes include quantity (how many or how much), quality (how well), coverage (how much area, how far), timeliness (how responsive, how frequent), and readiness (availability, MTBF).
Performance-Based Tests. Performance-based tests concentrate on demonstrating system performance capability; that is, explore the system limits.
Performance Requirements. Performance requirements define the extent to which a mission or function must be executed, generally measured in terms of quantity, quality, coverage, timeliness, or readiness.
Probe. A probe is an external means of accessing data within a system component.
Qualification Testing. Qualification testing deals with tests of the hardware, e.g., maintainability demonstrations, reliability demonstrations, environmental tests, human factors, etc.
Regression Testing. Regression testing is selective testing to verify that modifications have not caused unintended adverse side effects or to verify that a modified system still meets requirements. Regression tests need to be identical to the qualification/acceptance tests that were performed and by which the customer agreed that the system met requirements.
Subsystem. A subsystem is a set of interrelated elements satisfying a logical group of functions within a particular system.
System. A system is two or more objects or components interacting cooperatively to achieve some goal or purpose. A system is an interrelated set of elements that act as a whole, as a unit; the interaction of several elements to produce an effect. A man-made system is developed to achieve a preplanned goal or purpose. A system is an integrated composite of people, products, and processes that provide a capability to satisfy a stated need or objective.
System Testing. System testing verifies the system to its initial objectives/requirements. System tests are conducted on the target system with all system components present.
System qualification testing. System qualification testing is performed to demonstrate to the customer that system requirements have been met. It covers the system requirements in the system/subsystem specifications and in any associated interface requirements specifications. If a system is developed in multiple builds, qualification testing of the completed system will not occur until the final build. System qualification testing in each build is interpreted to mean planning and performing tests of the current build of the system to ensure that the system requirements implemented in that build have been met.
Test Category: A test category is the arbitrary grouping of tests for the purpose of discussion. Also referred to as test types or test classes. Test categories include such groupings as stress tests, maximum capacities tests, timing tests,
Test Levels. A test level indicates the phase or time within the system life-cycle where a test will occur.
Test Method. A test method is the method by which a requirement is verified. Test methods are:
a. Inspection/Examination. A method such as visual inspection of physical
characteristics of an item or the documentation associated with the item,
and comparison of the appropriate characteristics with a predetermined
standard to determine conformance to requirements.
b. Analysis. A method such as reduction or translation of data, analysis
of test data, review of analytical data, or performance of detailed analysis.
c. Test. A method such as actual operation wherein the element of verification
is instrumented, measured, or displayed directly.
d. Demonstration. A method such as actual operation whereby the element
of verification is logically obvious, as being a necessary constrain to
some other result, but not itself displayed.
Testing. (1) Testing is the process of establishing confidence that a program or system does what it is supposed to ("measuring" or "evaluating", not trying to find things wrong). (2) Testing is the process of executing a program or system with the intent of finding errors. (3) Testing is the measurement of software quality. (4) Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements or to identify differences between expected and actual results.
Tool-Proof. To tool-proof is to test the tool or procedure to ensure that is performs as required.