| 2.3.11-5-1 Ensure integration planning is correct, feasible, and complete |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the integration planning includes an incremental build up of the software units into software items. Ensure there are software unit to software unit test cases that test those interfaces. Ensure the integration planning includes a build up of the software items into the overall software system. Ensure the integration planning includes software item to hardware item integration. Ensure the build objectives in the software master build plan are met. Ensure the integration planning includes status of discrepancy reports.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-2 Ensure the ground system integration and qualification planning defines the test philosophy and planning from a software perspective |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the contractor's system integration and qualification planning (Master Test Plan or equivalent) includes software and is consistent with integrating and verifying multiple software builds. Ensure the system level testing includes testing software across external interfaces. If software in one segment has an interface with software in another segment (e.g. interface between Ground Segment and Space Segment), then it is important that the planning includes testing and verifying this software interface as part of the overall system interface testing. Ensure the contractor's integration planning includes status of discrepancy reports.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-3 Ensure software integration and qualification test planning is consistent with the software master build plan |
Phase 0 |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the capabilities to be tested in each of the qualification tests are consistent with the software master build plan. Ensure the software test plan includes a set of tests for each build that tests the new capabilities added in that software build. Ensure the software test plans include a set of regression tests used to check that the capabilities added in previous builds still work. Ensure the software test plans include the resources needed to test each build. Ensure the software test plan includes status of discrepancy reports. Ensure the planned resources include any drivers or stubs necessary to supplement any partial builds and any platforms or simulators, or GFX required to run the build.
|
AFMAN 63-119 A15.14
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit; AFMAN 63-119 or equivalent
|
| 2.3.11-5-4 Ensure the correctness and completeness of the contractor's higher level integration and verification test processes |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the problem reporting process requires that all anomalies or problems encountered in the integration and verification tests are documented, tracked to resolution, and the appropriate integration or qualification tests are repeated to ensure the anomalies and problems are fixed, and discrepancy reports are dispositioned. Ensure the verification test process includes peer reviews of the test cases and procedures.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-5 Ensure the levels of system, segment, element, and subsystem test planning adequately incorporate software and interface requirements verification |
Phase A |
Phase B |
Phase C |
|
Ensure the test planning at the system, segment, element, and subsystem (as applicable) levels include provisions for verifying software-to-software and software-to-hardware interfaces that cannot be verified at lower levels. Ensure the testing of these interfaces are adequate for completing the verification of these requirements. Ensure the testing are adequate for the closure of discrepancy reports. Ensure the testing of end-to-end requirements across software items, subsystems, elements, segments, and systems (as applicable) is complete and represents valid user operational scenarios.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-6 Ensure planning for independent testing of critical software is correct and complete |
Phase A |
Phase B |
Phase C |
|
Ensure the completeness coverage in the Software Development Standard is used for the worst cases including adequate testing of closed discrepancy reports.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit; Test Like You Fly Process Guide for Space, Launch and Ground Systems, TOR-2014-02537
|
| 2.3.11-5-7 Ensure the software test plan fully supports the system integration and qualification test plan levels in the area of information assurance strategies and policies |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the software test plans follow DoD Net-Centric and IA strategies. Ensure the software test plans support system integration tests that test for vulnerabilities.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-8 Ensure the software qualification tests for information assurance and cyber security requirements are correct and complete |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure interfaces to trusted security components are tested to the same common criteria (CC) level as the trusted security components' level. Trusted security components at higher CC levels should have tests that support those higher CC levels.
|
NA
|
Integration and Test Plans; Test and Evaluation Master Plan (TEMP)
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-9 Ensure the integration and verification test environments are correct, complete, and have sufficient resources to accomplish the necessary testing within program cost and schedule constraints |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure enough resources are present to perform the necessary software testing. Ensure the correctness and completeness of the test beds and test facilities. Ensure the correctness and completeness of the simulators, emulators and other test tools. Interfaces and data should accurately represent the target environment. Ensure using the simulators/emulators or executing in the target environment is transparent to the software under test. Simulators/emulators and other test tools should be qualified/validated to the level appropriate. Ensure the availability of the test beds and test facilities, and any GFX needed, aligns with the start of integration or qualification test. Test facilities with the necessary trained personnel should be up and running prior to the start of integration or qualification test. Test tools should be qualified/validated prior to start of integration or qualification test.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-10 Ensure processes and tools exist for developing the software integration and qualification environments |
Phase A |
Phase B |
Phase C |
|
Ensure the software integration and qualification environment processes cover environment development and qualification/validation, provide an environment that supports the program's software integration and qualification requirements, handle resolution of competing needs for facility support, and address maintenance of the environment(s) throughout the life cycle. Ensure the processes are updated and maintained to reflect changes that impact the integration and qualification environment(s). Ensure sufficient resources are allocated to handle updating and maintaining the integration and qualification environment(s) throughout the life cycle.
|
NA
|
Integration and Test Plans
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|
| 2.3.11-5-11 Ensure the software test plans, cases, and procedures are correct, complete and fully verify the requirements for each software item |
Phase A |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure the verification coverage requirements in the Software Development Standard are met. Ensure all necessary verification methods (I, A, D, T) are used in order to fully verify the software requirements and that the plans, cases, and procedures cover all verification methods. Ensure there are software-to-software test cases that trace to the interfaces defined in the software architecture and design. Ensure there are software-to-hardware test cases that trace to each of the software-to-hardware interfaces. Ensure the test cases have test procedures that satisfy the test case objectives. Ensure the bi-directional traceability of the software requirements to the tests in the test plan, test cases, and test procedure steps. Ensure all test cases have specified the inputs, expected outputs, and evaluation criteria.
|
NA
|
Integration and Test Plans
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment; ATR-2011(8404)-3 An Increase in SW Testing Robustness: Enhancing the SW Development Standard for Space Systems; TR-2006(1472)-1 Recommended Process for Qualification Testing of Payload Software Prior to Use on Orbit
|