| 2.2.7.2-1-1 Ensure test procedure is operationally realistic |
Phase D1 |
|
Ensure the test procedures were not set up strictly to sell off requirements, but rather to include Test Like You Fly concepts and Day In The Life implementations. Ensure testing is consistent with the ConOps.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-2 Ensure the system support environment used for developmental test was sufficient for determining operational readiness |
Phase C |
Phase D1 |
|
Ensure simulation software and hardware are validated. Assess how the simulation tools were validated. Assess the operational realism of the products and processes used to complete DT and assure OT success.
|
AFMAN 63-119 A15.10; AFMAN 63-119 A29.3
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|
| 2.2.7.2-1-3 Ensure stability and stress tests are provided in the developmental test plan and procedures, and that those tests cover off-nominal cases, backup locations and data paths, and worst-case scenarios |
Phase C |
Phase D1 |
|
Ensure test cases exercise all portions of code. Ensure stability and stress test (network loading, operational interfaces, off-nominal conditions) procedures exercise maximum data volume, maximum operator interface, etc.
|
AFMAN 63-119 A15.9
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|
| 2.2.7.2-1-4 Ensure all operational databases, whether real-time or off-line, are complete and sufficient for operational test |
Phase D1 |
|
Ensure the pedigree of the data in the operational data bases. Assess the level of testing that used the actual data to be used in operations.
|
AFMAN 63-119 A15.4
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|
| 2.2.7.2-1-5 Ensure the developmental test plans and procedures cover off-nominal use conditions and threads |
Phase C |
Phase D1 |
|
Ensure developmental testing covers as many off-nominal threads as possible.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-6 Ensure maximum use of real-time equipment in all aspects of developmental test |
Phase C |
Phase D1 |
|
Ensure live data are used for developmental test whenever possible - off-nominal threads excepted. Ensure review of simulated data inputs for sufficiency to verify test results. This task helps assess the operational realism of the entire test program.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-7 Ensure operators have been continuously involved with development, and that feedback from operators has been captured, and adjudicated by both the provider and receiver of the products |
Phase B |
Phase C |
Phase D1 |
|
Ensure the operational stakeholders have input into the design and use of the system to be fielded from the beginning of the program. Examples are conducting focus groups with operators when designing HMI and using operators during the testing.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-8 Ensure operational test objectives are reflected in the developmental test program |
Phase B |
Phase C |
Phase D1 |
|
Ensure relevant operational test cases, for both operations and maintenance, are included in the developmental test program. Ensure the developmental test (DT) program reviews operational test (OT) plans as soon as the OT plans are approved. Ensure all OT test cases are included in the DT program. Ensure any gaps found in the review of DT and OT test program contents is remediated, including closure of open issues and DT/OT disconnects. Ensure integrated DT/OT testing does not compromise DT or OT test objectives. Ensure DT data supplements OT whenever possible. Ensure test items are controlled. This task helps assess developmental test coverage of planned operational test scope.
|
AFMAN 63-119 A22.5.1; AFMAN 63-119 A22.5.2; AFMAN 63-119 A28.1.1
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|
| 2.2.7.2-1-9 Ensure involvement of operational testers in developmental test |
Phase C |
Phase D1 |
|
Ensure the developmental test staffing includes operational test team members. This provides invaluable insight to operations, which can help avoid surprises at entry to operational test. This task helps assess developmental test coverage of planned operational test scope.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-10 Ensure the concepts of operations with which the contractor developed the system has not evolved without commensurate operations involvement, test plan and procedure changes, and requirements changes |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor has developed and maintained complete descriptions of how the delivered system will be operated. Ensure those descriptions are understood by the operational command. Determine if there has been an evolution of the conops during the development cycle. Assess level of involvement of end users, especially "super users." Are there direct connections of Conops to development and to operations? This task helps assess consistency of operational concepts during development.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-11 Ensure the operational units were involved in development of operations concepts, implementation reviews, and developmental test activities |
Phase B |
Phase C |
Phase D1 |
|
Ensure operations is completely engaged with contractor/government test personnel. Ensure operations personnel understand implications of problems and their criticality. Ensure continued and effective communications and involvement of and with operators before and during developmental test. This task helps assess consistency of operational concepts during development.
|
NA
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness
|
| 2.2.7.2-1-12 Ensure open problem reports have closure plans, are categorized appropriately, and have no impact on acceptance |
Phase D1 |
|
Ensure proper communications between the developers, government and operational leadership during the DR adjudication and analysis process. Ensure deficiencies, including those originating from operational assessments, have full community concurrence on disposition, documented and controlled corrective action, and applicability to the system being delivered to operational test.
|
AFMAN 63-119 A18.3; AFMAN 63-119 A18.3.2; AFMAN 63-119 A18.3.3; AFMAN 63-119 A18.4; AFMAN 63-119 A19.4.1; AFMAN 63-119 A19.5; AFMAN 63-119 A19.5.1; AFMAN 63-119 A19.5.2; AFMAN 63-119 A19.6; AFMAN 63-119 A19.6.1; AFMAN 63-119 A19.6.2; AFMAN 63-119 A19.7; AFMAN 63-119 A19.7.1; AFMAN 63-119 A21.2; AFMAN 63-119 A21.2.1; AFMAN 63-119 A21.2.2; AFMAN 63-119 A21.3; AFMAN 63-119 A21.6
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|
| 2.2.7.2-1-13 Ensure oversight includes the early determination of critical metrics, the gathering of these metrics, and assessments of them briefed to both provider and receiver government organizations |
Phase D1 |
|
Ensure gathered metrics reasonably depict specifics of progress and successful operational transition. Ensure limitations in operational test are fully described and addressed for future test increments. This task helps assess maturity of the system to be fielded.
|
AFMAN 63-119 A22.6; AFMAN 63-119 A22.6.1; AFMAN 63-119 A22.6.2
|
NA
|
TOR-2013-00219, Assessing and Preparing for Operational Readiness; AFMAN 63-119 or equivalent
|