| 2.2.6-1-1 Ensure integration and test data results and packages are documented, stored, and evaluated |
Phase D1 |
|
Ensure the requirements verification products contain complete and accurate data, and that they support requirements verification and contractors' conclusions. Ensure contractor data are in acceptable formats. Weak supporting evidence unusually requires supporting details such as additional tests and/or subsystem specialists' analyses.
|
AFMAN 63-119 A13.3.1; AFMAN 63-119 A13.6; AFMAN 63-119 A14.1
|
NA
|
Mission Assurance Guide, TOR-2007(8546)-6018, Rev B; AFMAN 63-119 or equivalent
|
| 2.2.6-1-2 Ensure systems integration is completed and verified |
Phase C |
Phase D1 |
Phase D2 |
|
Ensure all integration is completed for all segments, with supporting evidences, before entry into system testing.
|
SFWC 4.2; AFMAN 63-119 A15.5
|
NA
|
AFMAN 63-119 or equivalent
|
| 2.2.6-1-3 Ensure an integrated system checkout is performed in accordance with approved procedures |
Phase D1 |
Phase D2 |
|
Ensure all integration efforts culminate in system checkouts. This includes the issue of selected system test threads to increase confidence in entry to system testing.
|
SFWC 4.9, SFWC 4.8, SFWC 4.7
|
NA
|
NA
|
| 2.2.6-1-4 Ensure both flight and ground software undergo integrated testing before certification for operational use |
Phase C |
Phase D1 |
|
Ensure integration includes end-to-end software testing. This would be an end-to-end test that may not exercise all S/W functions of each subsystem: These should have been done at a lower subsystem level. This is especially important when flight and ground software are in their target environments for the first time.
|
SFWC 2.4.2.2.4
|
NA
|
NA
|
| 2.2.6-1-5 Ensure assessment of system testing, acceptance testing, pathfinder and first article testing for compliance with contractor procedures, designs and test plans |
Phase C |
Phase D1 |
|
Ensure assessment includes ground control hardware/software, support equipment, and processing facility capability. Testing must demonstrate the system and/or components are meeting the performance parameters at prescribed threshold levels and within defined time frames at each step in development.
|
AFMAN 63-119 A13.3
|
NA
|
Mission Assurance Guide, TOR-2007(8546)-6018, Rev B; AFMAN 63-119 or equivalent
|
| 2.2.6-1-6 Ensure each system test is successfully completed per approved procedures, test products are saved and evaluated, and system requirements are verified per the test plans and/or Verification Cross-Reference Matrix |
Phase D1 |
Phase D2 |
|
Ensure red-lines to test procedures are captured. Ensure completion of all testing related to each system requirement, and closure of all critical deficiencies, before entry into operational test. Sufficient workarounds acceptable to operational test organizations are identified for vulnerabilities and deficiencies. Sufficient testing must be accomplished with other systems to support end-to-end cybersecurity and interoperability certifications. Ensure system performance parameters and Critical Technical Parameters (CTPs) are measured and saved. Ensure all changes to specification threshold (pass/fail) values are documented with rationale. Ensure all discrepancies are resolved and any system requirements satisfied at a lower level of testing are documented in the system requirements sell off package. Ensure all test organizations, including the contractor, use a common database for archival and storage of test data. Ensure parameters, formats and test problem severity are agreed upon between all test teams. Ensure configurations for all developmental and operational tests are rigorously controlled.
|
SFWC 4.7, SFWC 4.11; AFMAN 63-119 A11.3; AFMAN 63-119 A11.3.1; AFMAN 63-119 A11.3.2; AFMAN 63-119 A13.5.3; AFMAN 63-119 A13.2.7; AFMAN 63-119 A13.7; AFMAN 63-119 A14.4.1; AFMAN 63-119 A14.6.1; AFMAN 63-119 A14.6.2; AFMAN 63-119 A14.6.4; AFMAN 63-119 A14.9.1
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.2.6-1-7 Ensure product satisfies system key performance parameters |
Phase C |
Phase D1 |
|
Ensure performance specification meets validated user requirements. Ensure any design or other changes still provide for meeting mission requirements.
|
SFWC 2.4, SFWC 2.4.1, SFWC 2.4.2, SFWC 2.4.2.2; AFMAN 63-119 A14.6.3
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.2.6-1-8 Ensure all anomalies and non-conformance dispositions are completed |
Phase C |
Phase D1 |
|
NA
|
SFWC 2.4.1.1
|
NA
|
NA
|
| 2.2.6-1-9 Ensure contractor records, investigates, and corrects all failures/test anomalies |
Phase C |
Phase D1 |
|
Ensure all anomalous behavior is diagnosed and fix or workaround is validated and that anomalous integration and test results are managed and documented by a Failure Reporting and Corrective Action System (FRACAS).
|
SFWC 2.4.2.1.1, SFWC 2.2.3.2; AFMAN 63-119 A13.5.2
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.2.6-1-10 Ensure all test procedures, including operational test procedures, have undergone dry runs |
Phase D1 |
|
Ensure all test procedures undergo a dry run before the start of operational test. Like You Fly test dry runs may involve multiple stakeholders. Problems discovered during dry runs may need multiple participants to properly determine correction path.
|
AFMAN 63-119 A22.5.6; AFMAN 63-119 A24.3
|
NA
|
Test Like You Fly Process Guide for Space, Launch, and Ground Systems, TOR-2014-02537 Rev A, September 2016; TLYF; AFMAN 63-119 or equivalent
|
| 2.2.6-1-11 Ensure development test requirements, specs and standards are identified and closed |
Phase D1 |
|
Ensure readiness to exit development test, including requirements closure and test program adequacy. Ensure sufficient relevant government testing is performed to verify system requirements before operational testing.
|
AFMAN 63-119 A14.6
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.2.6-1-12 Ensure verification analyses demonstrate that the design meets requirements with adequate contingency and margin |
Phase D1 |
|
Ensure the product design completed, is producible, and meets performance requirements.
|
SFWC 2.1.3.1.7
|
Design Review Data Package
|
NA
|