| 5.2-8-1 Ensure that all test failures have been analyzed for cause and disposition |
Phase D1 |
|
Ensure any failure or unexpected behavior of configuration controlled hardware, software, or both is documented per the failure reporting process defined in the program plans. Ensure that all reported failures are analyzed and root cause identified; If not possible, that a most probable cause has been identified and justified. Ensure hardware and or software associated with each identified cause is addressed by rework, repair, redesign or replacement with fault free equivalent, following procedures and instructions approved per program directives. Ensure all hardware and or software that have undergone any changes are retested, including regression and penalty tests as required by program engineering and quality requirement, and that related documentation is updated to be consistent with changes. Ensure the disposition of all reported failure with identified root cause, includes corrective action intended to prevent future reoccurrences of the same problem.
|
AFMAN 63-119 A15.13
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System; AFMAN 63-119 or equivalent
|
| 5.2-8-2 Ensure that all configuration items or portions thereof being integrated, are sufficiently tested, with approved integration and test procedures, to verify compliance with their respective functional, performance, design and interface requirements, in an environment representative of operation |
Phase D1 |
|
Ensure that, at each level of integration, each configuration control item or portion thereof being integrated, has been verified to comply with all its form, fit, function, performance, quality/reliability/maintainability, and interface requirements, using approved verification procedures. Ensure that all configuration items or portions thereof being integrated, have been verified to meet requirements within the extremes of environmental conditions defined in their respective requirements documents. Ensure that all software items are verified in an environment representative of the operational environment. Ensure that all lower levels of integration have completed their associated integration and test, and verification procedures successfully, as specified in program plans. If the software is developed using an iterative life cycle model, the software being integrated will generally constitute a "build" containing a portion of the completed software. For lower levels of integration prior to the current level, ensure that the components, assemblies, sub segments, products, and software builds are subjected to sufficient testing, including nominal and off nominal, to ensure operational effectiveness, usability, trainability, interface conformance, conformance with specified requirements, producibility, and supportability.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-3 Ensure units, assemblies and subsystems, and software units are fabricated, coded, or assembled in accordance with approved drawings and software design documents, manufacturing planning and procedures, software development procedures, and assembly/integration documentation |
Phase D1 |
|
At each level of integration, review manufacturing or coding, quality and procurement records as needed for each element to be integrated: assembly, unit, sub segment and equivalent software builds. Ensure that the element is manufactured or coded in accordance with approved design baseline. Ensure all components used to manufacture hardware elements have been verified to comply with program Parts, Materials & Process (PM&P) requirements. Ensure software elements have been developed and coded per approved program coding standards, processes and procedures. Ensure all element assembly and integration has been performed in accordance with program approved assembly and integration processes, manufacturing planning, program practices, and procedures. Ensure COTS hardware, software, or both are verified against procurement and manufacturing specifications, prior to integration.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-4 Ensure that all configuration control items or portions thereof being integrated into the current level are integrated per approved processes and design baseline, in an environment representative of operation conducted with approved integration and test procedures |
Phase D1 |
|
Ensure that the configuration control items or portions thereof, are integrated into the current level in accordance with the approved design baseline. Ensure that the current level of integration has been carried out per approved integration processes, manufacturing planning, and approved procedures and program practices. Ensure that the current level of integration has been completed per approved integration and test processes, procedures and program practices. Ensure that software integration and test have been conducted in an environment representative of operation.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-5 Ensure that design reviews and technical audits have been conducted as required and that all action items from the reviews and audits have been resolved |
Phase C |
Phase D1 |
|
Ensure that design reviews and technical audits have been conducted and recorded as required by the contract, the systems engineering master plan, required standards, and program instructions and processes. The objective evidence should include what was reviewed or audited, what criteria was used for the review or audit, who participated in the reviews or audits, and what issues and action items resulted from the review or audit. Ensure all open issues, comments and action items are captured, tracked to closure, and the closure concurred upon by originators of the action, and the program office, before the start of the current level of integration.
|
NA
|
Design Review Data Package
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-6 Ensure test readiness review is held prior to start of integration test for the current level of integration to verify that integration test procedures are complete, approved and released |
Phase D1 |
|
Ensure that a test readiness review is held prior to the start of integration test for the current level of integration. Ensure that all test stakeholders participate in the review. Ensure that the review demonstrates the following: integration test procedures, including data sheets as required, are complete, reviewed, consistent with test plans, approved and configuration controlled; required test facilities are ready and configured to support test; standard test equipment and tools are in place, in calibration, and configuration controlled; special test equipment in place, verified against its requirements and configuration controlled; required consumables of applicable quality level are available and ready for test; test data files and databases, test software, test drivers and data reduction software are complete, verified against their requirements and configuration controlled; for software testing, the test environment (hardware and software) are configured per approved test procedure, verified against its approved requirements, and configuration controlled.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-7 Ensure integration sequence for the current level of integration, as defined in the integration plans and procedures, is complete and adequate rationale for the sequence is provided |
Phase C |
Phase D1 |
|
Ensure that all configuration control items, or portions thereof, to be integrated in the current level, both hardware and software, are included in the integration sequence as defined in the approved integration plans and procedures. Ensure that sound logic and justification are provided for the particular integration sequence selected. Ensure that any deviations from the approved sequence are fully documented and analyzed, and fully justified. Ensure that any risks associated with deviations have thorough risk assessment and that mitigations are carried out.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-8 Ensure that requirements and design for all necessary integration equipment and fixtures, special test equipment, test facilities, test software, test files, and test databases, and other needed test assets, are defined, reviewed, captured and configuration controlled per program requirements |
Phase B |
Phase C |
Phase D1 |
|
Ensure that integration equipment, special test equipment, fixtures, test facilities, test software and test files, databases, test data reduction software, and any other needed test assets, are identified in approved test plans. Ensure that, based on the planned tests, the functional and performance requirements for these special test items, such as equipment, software, test data files and databases, are defined, captured, configuration controlled and maintained. Ensure that detailed design documentation is generated, captured and configuration controlled for each special test item; this documentation should include definition and design of all interfaces between the test items, the test articles, and standard test equipment and facilities specified in the test planning. Ensure that all test asset requirements are developed, reviewed and approved following the process outlined under the requirements development task. Ensure that special test items are reviewed along with segment hardware/software, at regular program reviews. Ensure that all test asset requirements are provided to organizations responsible for the development of the assets.
|
NA
|
System/Segment Specification (SSS); Design Review Data Package
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-9 Ensure integration and test procedures are fully executed and data is captured, reviewed, and approved per program practice |
Phase D1 |
|
Ensure that approved integration and test procedures are executed by knowledgeable experts, and witnessed as required by program practices. Ensure that all as-run integration and test procedures, test data sheets, test data files, test databases, and other information and evidence gathered from executing the procedures, has been captured, verified by witnesses, QA, and other stakeholders as required by program processes and practices. Ensure that all analysis and simulations performed in support of integration and test, are fully documented, and reviewed per program practices. Ensure all software, databases, and test data files are captured in the as-run configuration. Ensure that all integration and test data and evidence is captured and configuration controlled as part of the program history.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-10 Ensure that an integration process, or set of processes, have been developed, captured, controlled, and followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure that the integration processes describe the steps to generate the integration plans and procedures for all aspects of hardware, software and interfaces of the ground segment. Ensure the process describes the approach to developing test assets, including special test equipment and facilities; software test environment, test cases and files, test databases, test drivers and data reduction software; and interface drivers and simulators. Ensure the process describes the approach for determining sequence of integration, the execution and verification of integration steps, and the capture of all results. Ensure the process includes stakeholder reviews of its major steps. Ensure that all evidence of integration process implementation is captured and configuration controlled as part of the program history. Ensure the processes are followed.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-11 Ensure failures are analyzed for cause and disposition |
Phase D1 |
|
Ensure that any failure or unexpected behavior of configuration controlled hardware or software, or both, is documented per the failure reporting process defined in the approved program plans. Ensure that all reported failures are analyzed and that their root cause is identified, or if that is not possible, that a most probable cause is identified and justified. Ensure that the hardware or software, or both, associated with each identified root cause is reworked, repaired, redesigned or replaced with fault free equivalent, following procedures and instructions approved per program directives. In the case of software, rework includes revising the software code and affected related documentation. All hardware/software that is changed in any way should undergo retest for re-verification purposes, and penalty/regression test. Ensure soundness after modification. Ensure that the disposition of all reported failure with identified root cause includes corrective action intended to prevent future reoccurrences of the same problem. Ensure that all hardware or software, or both, that have undergone any of the above changes, have been retested, including regression and penalty tests as required by program engineering and quality requirement.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-12 Ensure that verification by analysis or inspection is planned and executed in accordance with the verification plans per approved program practices, fully reviewed, approved and documented |
Phase D1 |
|
Ensure that verification plans define inspections to be performed to verify requirements. Ensure the plans describe verification analyses including the models, simulations, equations and algorithms to be used, input data requirements including databases if any, type of outputs to be generated and pass/fail criteria. Ensure that all verification inspections are fully documented in the quality history records, including a complete definition of configuration inspected, results, and buy-offs (signatures or QA stamps or both) of inspectors and witnesses. Ensure that verification analyses are fully documented including all inputs used, all models and equations employed, all results, and a pass/fail statement. Copy of all data, including databases, and copies of all software used in the analyses, should be capture and impounded, and linked to the corresponding analysis reports. The analysis records should include a detailed definition of the data processing software and hardware environment used in the formal verification analyses.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-13 Ensure tests are executed as planned, and all verification data is captured, reviewed, and approved |
Phase D1 |
|
Ensure that all approved verification procedures scheduled for completion have been run, all tests have passed, and verifications were run per approved procedures and program practices and processes. Ensure that all as-run verification procedures, test data sheets, test data files, test databases, and other information and evidence gathered from executing the procedures, has been captured, verified by witnesses, QA, and other stakeholders as required by program processes and practices. Ensure that all analysis and simulations performed in support of verification, are fully documented, reviewed and approved per program practices. All software, databases, and test data files used, should be captured in the as-run configuration and linked to their corresponding verification results. Ensure that all integration and test data and evidence is captured and configuration controlled as part of the program history.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-14 Ensure that requirements and design for all necessary verification equipment and fixtures, special test equipment, test facilities, test software, test files, and test databases, and other needed test assets, are defined, reviewed, captured and configuration controlled per program requirements |
Phase B |
Phase C |
Phase D1 |
|
Ensure that verification equipment and fixtures, special test equipment, test facilities, test software and test files, databases, and any other needed test assets, are identified in approved test plans. Ensure that, based on the planned tests, the functional and performance requirements for these special test items, such as equipment, software, test data files and databases, are defined, captured and configuration controlled and maintained. Ensure that detailed design documentation is generated, captured and configuration controlled for each special test item; the documentation should include definition and design of all interfaces between the test items, the test articles, and standard test equipment and facilities specified in the test planning. Ensure that all test asset requirements are developed, reviewed and approved following the process outlined under the requirements development task. Ensure that special test items are reviewed along with segment hardware/software, at regular program reviews. Ensure that all test asset requirements are provided to organizations responsible for the development of the assets.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|
| 5.2-8-15 Ensure that verification process, or set of processes, have been developed, captured controlled and followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure that the verification processes describe the steps to generate the verification plans and procedures for all aspects of hardware, software and interfaces of the ground segment. Ensure the process describes the approach to developing test assets, including special test equipment and facilities; software test environment, test cases and files, test databases, test drivers and data reduction software; and interface drivers and simulators. Ensure the process describes the approach for determining the sequence of verification, and the capture of all results. Ensure the process includes stakeholder reviews of its major steps. Ensure the process includes the use of peer reviews, dry runs and rehearsals, regression and penalty testing, and pre and post verification reviews and analysis to ensure completeness and accuracy of all verification activities. Ensure the process includes discussion of traceability of all plans, procedures and verification results to each other and to applicable requirements. Ensure that all evidence of verification process implementation is captured and configuration controlled as part of the program history. Ensure the processes are followed.
|
NA
|
NA
|
SMC-S-001, SMC Systems Engineering Requirements and Products or equivalent; TOR-2012(8960)-2, Proposed Test Requirements for Ground Systems; IEEE 1220: For Practical Systems Engineering; ANSI/EIA-632, Processes for Engineering a System
|