| 2.2.5-11-1 Ensure concept testability is evaluated |
Phase A |
Phase B |
|
Assess Candidate system concepts and technologies for testability
|
NA
|
NA
|
Mission Assurance Guide, TOR-2007(8546)-6018, Rev B
|
| 2.2.5-11-2 Ensure assessment of the contractor test capability |
Phase A |
Phase B |
Phase C |
Phase D1 |
|
NA
|
NA
|
NA
|
NA
|
| 2.2.5-11-3 Ensure contractors utilize a consistent test methodology |
Phase C |
Phase D1 |
|
NA
|
SFWC 2.4.2.1
|
NA
|
NA
|
| 2.2.5-11-4 Ensure independent verification and validation of contractor's plans |
Phase B |
Phase C |
Phase D1 |
|
Ensure contractor's plan is correct, complete, consistent, clear, unambiguous, and feasible. Ensure that the IV&V Plan includes verification of requirements, design and code. Ensure that schedule is sufficient to allow problems found to be addressed. Ensure that the IV&V Plan includes validation of software. Ensure the IV&V Plan identifies resources needed for effort (simulators/emulators, flight (or flight-like) hardware, and personnel).
|
AFMAN 63-119 A13.2
|
NA
|
Mission Assurance Tasks for Software TOR, IEEE Std 1012(TM)-2004, IEEE Standard for Software Verification and Validation, TOR-2001(1465)-0934e, Software Independent Verification and Validation; AFMAN 63-119 or equivalent
|
| 2.2.5-11-5 Ensure system verifications / test procedures reviewed/approved for release |
Phase C |
Phase D1 |
|
Ensure test plans and procedures cover all T&E events, and include how they will be accomplished. Ensure duplications and voids in testing are minimized or eliminated. Ensure planning and plans for developmental test and operational test contain adequate margins (schedule, facilities and budget) for re-accomplishment. Ensure organizational test roles, responsibilities, and assignments are declared.
|
SFWC 4.9, SFWC 4.10; AFMAN 63-119 A11.4; AFMAN 63-119 A11.4.1; AFMAN 63-119 A11.4.2; AFMAN 63-119 A14.9; AFMAN 63-119 A22.5.4; AFMAN 63-119 A22.5.5
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.2.5-11-6 Ensure intersegment test perceptivity |
Phase B |
Phase C |
|
Ensure Test-like-you-fly process is implemented and like-you-fly tests are allocated to the unit/component level as appropriate.
|
NA
|
NA
|
Space Vehicle Systems Engineering Handbook, TOR-2006(8506)-4494
|
| 2.2.5-11-7 Ensure the ground control test plans are adequately perceptive |
Phase B |
Phase C |
Phase D1 |
|
Evaluate the contractors allocation of ground control tests and on-orbit testing to verify and/or validate functional, performance, and or interface requirements. Ensures that the ground testing is complete and perceptive.
|
NA
|
NA
|
Mission Assurance Guide, TOR-2007(8546)-6018, Rev B
|
| 2.2.5-11-8 Ensure MCFA results and mission-related work products are used as a basis for designing operationally realistic LYF tests |
Phase C |
Phase D1 |
|
LYF test objectives need to be appropriate to the aspect of the mission being validated and/or for capturing the evidence to surface or exonerate associated flaw paths. LYF tests must be designed to provide the data needed to exonerate mission critical faults. Even those LYF tests meant to prove the ability of the system to perform the mission need to be able to detect mission critical faults, especially any that might have eluded the MCFA. MCFA results consist of mission-ending failure situations with paths and contributors to failure. Mission-related work products consist of mission critical events and critical mission contributors
|
NA
|
NA
|
Test Like You Fly Process Guide for Space, Launch, and Ground Systems, TOR-2014-02537 Rev A, September 2016; TLYF
|
| 2.2.5-11-9 Ensure evaluation criteria used to determine operational effectiveness and suitability are consistent between all individual test plans and Test and Evaluation documents |
Phase C |
Phase D1 |
|
Ensure all test plans and procedures are internally consistent and cross-program consistent with respect to definitions, formulas and evaluation criteria. Ensure the plans and procedures provide for correct and complete evaluation of operational effectiveness and suitability.
|
AFMAN 63-119 A11.2; AFMAN 63-119 A22.3.5
|
NA
|
AFMAN 63-119 or equivalent
|
| 2.2.5-11-10 Ensure the On-Orbit Tests are Adequately Perceptive |
Phase C |
Phase D1 |
|
Evaluate the contractors allocation of on-orbit testing to verify and/or validate functional, performance, and or interface requirements.
|
NA
|
NA
|
Mission Assurance Guide, TOR-2007(8546)-6018, Rev B
|
| 2.2.5-11-11 Ensure assessment of operational test plans and procedures |
Phase D1 |
|
Ensure assessment of the operational test plans and procedures for completeness, feasibility and clarity. Assess the operational test plans and procedures for software-related areas that incorrectly describe the functioning of the software in the system or are beyond the scope of the development contractor's responsibility for delivery. Assess the analysis of software implications, or provide such analysis if one does not exist. Assess the plans and procedures for evaluation of software maturity to ensure software maturity will be evaluated correctly and completely. Assess the plans and procedures for evaluating the measure of operational effectiveness and suitability and the critical technical parameters to ensure these items will be evaluated correctly and completely.
|
AFMAN 63-119 A22.3.8
|
NA
|
TOR-2006(8506)-5749, Mission Assurance Tasks for Software; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Software Mission Assurance Chapter; TR-2006(8550)-1; Reducing Software Acquisition Risk: Best Practices for the Early Acquisition Phases; AFMAN 63-119 or equivalent
|