This document :
1) describes the test preparations, test cases, and test procedures to
be used to perform qualification testing of a library application and
2) records the results of the qualification testing, enabling an assessment
of the testing and it's results.
Provide full identification of the system and the hardware/software to which this document applies; including, as applicable, title(s), abbreviation(s), version number(s), and release number(s).
State purpose of the system and the hardware/software to which this document applies. Describe the general nature of the system; summarize the history of the system development, operation, and maintenance; identify the project stakeholders; identify current and planned operation sites; and list other relevant documents.
Summarize the purpose and contents of this document and describe any security or privacy considerations associated with its use.
List by number, title, revision, and date all documents referenced in this plan.
Identify a test (one of the tests in the test set comprising the application testing addressed by this test description) by project-unique identifier and provide a brief description of the test. Use a separate paragraph for each test. Reference earlier paragraphs, if test descriptions contain duplicate information.
Describe the procedures needed to prepare the hardware for the test, including support hardware (e.g., test equipment). Reference any operating manuals, if applicable. Provide the following., as applicable: a. The specific hardware to be used, identified by name and, if applicable, number b. Any switch settings and cabling needed to connect the hardware c. One or more diagrams to show hardware, interconnecting control, and data paths d. Step-by-step instructions for placing the hardware in a state of readiness
Describe the procedures needed to prepare the software for the test, including support software (e.g., simulators, data recording/reduction software). Reference any software manuals, if applicable. Provide the following., as applicable: a. The specific software to be used, identified by name and, if applicable, version number b. The storage medium of the software (magnetic tape, diskette) c. Instructions for loading the software, including required sequence d. Instructions for software initialization common to more than one test case
Describe any other pre-test personnel actions, preparations, or procedures needed to perform the test.
Identify a test (one of the tests in the test set comprising the application testing addressed by this test description) by project-unique identifier and provide the information specified below for the test. Note: the “tests” in this paragraph are collections of test cases covering a specific area or function within the application test described by this test description.
Identify a test case for the above identified test, by project-unique identifier, state its purpose, state its purpose, and provide a brief description.
Identify the application requirements addressed by this test case OR provide this in 5.
Identify any prerequisite conditions that must be established prior to performing the test case. Discuss the following a. Hardware and software configuration b. Flags, initial breakpoints, pointers, control parameters, or initial data to be set/reset prior to test commencement c. Preset hardware conditons or electrical states needed to run the test case d. Inital conditions to be used in making timing measurements e. Conditioning of the simulated environment f. Other special conditions peculiar to the test case
Describe the test inputs needed for the test case. Provide the following, as applicable: a. Name, purpose, and description (e.g., range of values, accuracy) of each test input b. Source of the test input and the method to be used for selecting the test input c. Whether the test input is real or simulated d. Time or event sequence of test input e. The manner in which the input data will be controlled to: 1) Test the item(s) with a minimum/reasonable number of data types and values 2) Exercise the item(s) with a range of valid data types and values that test for overload, saturation, and other “worst case” effects 3) Exercise the item(s) with invalid data types and values to test for appropriate handling of irregular inputs 4) Permit retesting, if necessary
Identify all expected test results for the test case, both intermediate and final test results, as applicable.
Identify the criteria to be used for evaluating the intermediate and final results of the test case. Provide the following information for each test result, as applicable. a. The range or accuracy over which an ouput can vary and still be acceptable b. Minimum number of combinations or alternatives of input and output conditions that constitute an acceptable test result c. Maximum/minimum allowable test duration, in terms of time or number of events d. Maximum number of interrupts, halts, or other system breaks that may occur e. Allowable severity of processing errors f. Conditions under which the result is inconclusive and retesting is to be performed g. Conditions under which the outputs are to be interpreted as indicating irregularities in input test data, in the test database/data files, or in test procedures h. Allowable indications of the control, status, and results of the test and the readiness for the next test case (may be output of auxiliary test software) i. Additional criteria not mentioned above
Define the test procedure for the test case. The test procedure is a series of individually numbered steps listed sequentially in the order in which the steps are to be performed. The procedures may be included in an Appendix, with a reference here to the numbered appendix. A sample test procedure template is shown below.
| Step No. | Step Description | Expected Result | Observed Result | Pass/Fail |
Provide the following for each test procedure, as applicable:
a. Test operator actions and equipment operation required for each step,
including commands, as applicable to: 1) Initiate the test case and apply
test inputs 2) Inspect test conditions 3) Perform interim evaluations of
test results 4) Record data 5) Halt or interrupt the test case 6) Request
data dumps or other aids, if needed 7) Modify the database/data files 8)
Repeat the test case, if unsuccessful 9) Apply alternate modes as required
by the test case 10) Terminate the test case
b. Expected result and evaluation criteria for each step
c. If the test case addresses multiple requirements, identification of
which test procedure step(s) address which requirements or provide in 5.
d. Actions to follow in the event of a system stop or indicated error,
such as: 1) Recording of critical data from indicators for reference purposes
2) Halting or pausing time-sensitive test-support software and test apparatus
3) Collection of system and operator records of test results
e. Procedures to be used to reduce and analyze test results to accomplish
the following, as applicable: 1) Detect whether an output has been produced
2) Identify media and location of data produced by the test case 3) Evaluate
output as a basis for continuation of test sequence 4) Evaluate test output
against required output
Identify any assumptions made and constraints or limitations imposed in the description of the test case due to system or test conditions, such as limitations on timing, interfaces, equipment, personnel, database/data files. Identify any waivers or exceptions to specified limits and parameters, if they have been approved, and identify the effects and impacts upon the test case.
a. Include traceability from each test case identified in this test description to the application requirements it addresses.
b. Include traceability from each application requirement that has been covered by this test description to the test case(s) that address it. Account for all application requirements in the applicable Software Requirements Specification (SRS).
Cover the following topics: a. Provide an overall assessment of the system as demonstrated by the test results in this report b. Identify any remaining (open) deficiencies, limitations, or constraints that were detected by the testing performed. Problem/change reports may be used to provide deficiency information. c. For each remaining (open) deficiency, limitation, or constraint, describe: 1) Its impact on system performance, including identification of requirements not met 2) The impact on system design to correct it 3) A recommended solution/approach for correcting it
Provide an assessment of the manner in which the test environment may be different from the operational environment and the effect of this difference on the test results.
Provide any recommended improvements in the design, operation, or testing of the system tested. Describe the impact on the system, for each recommendation. If no recommendation is provided state, indicate: None.
Identify a test (the application test addressed by an associated test description) by project-unique identifier and provide the information specified below for the test. Note: the “tests” in this description are collections of test cases covering a specific area or function within the application test described by the test description.
Summarize the results of the system test. Include the completion status of each test case associated with the test (for example, all results as expected, problems encountered, deviations required, etc.). Use a table or chart for the summary.
For each test case in which one or more problems occurred, include the following paragraph OR indicate. None.
Identify the test case by project-unique identifier and provide the following: a. A brief description of the problem(s) that occurred b. Identification of the test procedure step(s) in which they occurred c. Reference(s) to the associated problem/change report(s) and backup data, as applicable d. The number of times the procedure or step was repeated in attempting to correct the problem(s) and the outcome of each attempt e. Back-up points or test steps where tests were resumed for retesting.
For each test case in which one or more deviations from the case/test procedures occurred, include the following paragraph OR indicate. None.
Identify the test case by project-unique identifier and provide the following: a. A description of the deviation(s); for example, substitution of required equipment, procedural steps not followed, schedule deviations. Red-lined test procedures may be used to show the deviations. b. The rationale for the deviation(s) c. An assessment of the deviations’ impact on the validity of the test case.
Present a chronological record of the test events covered by this report. (Use a table or appendix to present this log.) Include in the test log the following: a. The date(s), time(s), and location(s) of the tests performed b. The hardware and software configurations used for each test including, as applicable, part/model/serial number, manufacturer, revision level, and calibration date of all hardware, and version number and name for the software components used. c. The date and time of each test-related activity, the identity of the individual(s) who performed the activity, and the identities of witnesses, as applicable.
Provide any general information and rationale pertinent to the information contained in this test description and results report. Include a list of acronyms and abbreviations, and a list of terms and their definitions used in this test report. This list contains definitions needed to understand this test report. Include additional definitions and delete those not applicable.
Use appendixes to provide information that is published separately or that does not fit conveniently in the body of the document. Each appendix should be referenced in one of the above sections where data would normally have been provided.