| 5.4-5-1 Ensure the bidirectional traceability between the software code and software requirements and between the software code units and the software design units is correct and complete |
Phase B |
Phase C |
Phase D1 |
|
Ensure the bidirectional traceability between software implementation units, including newly developed, reuse, open source, modified reuse, and modified open source, COTS, GOTS, and model-based software, and their allocated software requirements, including software interface requirements, is complete and correct. Ensure all software requirements, including software interface requirements, are traced to at least one implementation unit and that the collection of implementation units to which each requirement is traced will actually satisfy the requirement. Ensure all implementation units are traced to one or more software requirements, including software interface requirements, or are derived from documented design decisions and are traced to those decisions. Ensure the bidirectional traceability between the design units and the implementation units is complete and correct. Ensure all design units are traced to at least one implementation unit and that the collection of implementation units to which the design unit is traced will actually implement the design documented by the design unit. Ensure all implementation units are traced to at least one design unit.
|
NA
|
System/Segment Specification (SSS); Segment Design Specification; Configuration Item (Hardware, Software and Firmware) Requirements Specifications; Configuration Item (Hardware, Software and Firmware) Design Specifications
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-2 Ensure contractor code peer reviews are effective and follow the program's peer review processes |
Phase B |
Phase C |
Phase D1 |
|
Ensure each contractor code peer review identified the required and desired personnel, that the required personnel are appropriately qualified, and that the peer review is not held without the presence of all required personnel. Ensure the appropriate type of peer review is held for each code unit. Ensure qualified software personnel participate in contractor peer reviews, on a sampling basis, as long as the contractor's internal processes allow for such participation or such participation is required by the contract. Ensure all attendees, contractor or Program Office software personnel, spend an adequate amount of time preparing for the peer review and that they participate in the peer review by identifying problems or issues. Ensure the peer reviews evaluate the code against criteria established prior to the peer review meeting (e.g., established code peer review checklists). Ensure all action items, whether initiated by contractor or Program Office software personnel, are captured and tracked to closure. Ensure software quality assurance personnel participate in the peer reviews, on a sampling basis if necessary, and that they review the peer reviews for adherence to the peer review process and review the code for adherence to the program's coding standards, practices, procedures, and conventions.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-3 Ensure the code is complete, consistent with the architecture and design, correctly implemented and peer reviewed with all action items closed |
Phase B |
Phase C |
Phase D1 |
|
Ensure all code required by the architecture and design is developed, obtained or modified, including reuse or open source code, modified reuse or open source code, newly developed code, and application program interfaces (APIs) for COTS, GOTS, reuse, open source, and model-based code, and that the code is consistent with the architecture and design. Ensure the code for each design unit is complete. Ensure interface code adheres to the interface design or required Interface Control Document (ICD). Ensure the code is correctly implemented or modified. Ensure an appropriate type of peer review is held for each code unit and that action items from the peer reviews are closed. Ensure the program's implementation processes are followed for all types of code. Ensure all changes to code adhere to these same criteria.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-4 Ensure the contractor's software coding standards, practices, procedures, and conventions are documented, complete, correct, effective |
Phase B |
Phase C |
Phase D1 |
|
Ensure software coding standards, practices, procedures, and conventions exist, that they are documented and updated when necessary, that they are correct and complete, and that they are effective for the software application and are efficient. Ensure developers know and follow the coding standards, practices, procedures, and conventions. Ensure coding standards, practices, procedures, and conventions exist for all languages in use on the program and all types of code (newly developed, unmodified reuse and open source, modified reuse, open source, and model-based). Ensure the coding standards, practices, procedures, and conventions enforce dependability and cyber security requirements via allowed types of constructs that avoid mistakes by programmers and insertions of vulnerabilities. Ensure the coding standards, practices, procedures, and conventions are consistent with industry best practices. Ensure the coding standards, practices, procedures, and conventions are enforced on the program by peer reviews and code analyses. Ensure the coding standards, practices, procedures, and conventions include government, commercial and international standards as appropriate for the program's contract, scope, and constraints.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-5 Ensure all code adheres to the coding standards, practices, procedures, and conventions |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor has enforced adherence to the coding standards, practices, procedures, and conventions via code analyses using tools and peer reviews. Ensure all action items involving non-adherence discovered by code analyses and peer reviews are closed.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-6 Ensure information assurance and cyber security vulnerabilities have been identified, either by the contractor or independently and that all identified vulnerabilities have been mitigated |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor performs code analyses to identify information assurance and cyber security vulnerabilities. Ensure the contractor analyzes all software selected for use on the program, including COTS, GOTS, reuse and open source, to identify vulnerabilities. Ensure all identified vulnerabilities are documented and adequate risk mitigation plans are developed and are being successfully implemented. Ensure these code analyses are performed independently if the contractor has not performed them, or perform them independently in addition to the contractor's analyses in order to ensure as many vulnerabilities as possible are found and mitigated.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-7 Ensure code analyses are performed, either by the contractor or independently, to identify memory leaks, vulnerabilities, type mismatches and other common software defects, and dead code |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor has performed code analyses to identify memory leaks, vulnerabilities, type mis-matches and other common software defects, dead code, and similar problems, using automated structure analysis, static analysis, dynamic analysis, and complexity analysis, as appropriate. Ensure these code analyses are performed independently if the contractor has not performed them, or perform them independently in addition to the contractor's analyses in order to ensure as many problems as possible are found. Ensure all problems and issues found by these analyses are documented as discrepancy reports or action items. Ensure all discrepancy reports and action items generated by these code analyses are tracked to closure.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-8 Ensure the contractor's software unit test cases, procedures, data, and other unit test artifacts are documented, controlled, correct, complete, and adhere to all unit testing standards, practices, procedures, and conventions |
Phase B |
Phase C |
Phase D1 |
|
Ensure unit testing products are prepared and documented for all units for which unit testing is complete. Ensure the unit testing products are created by following the unit testing processes and that the unit testing products adhere to all unit testing standards, practices, procedures, and conventions. Ensure the unit testing products are complete and correct. Ensure contractor software quality assurance personnel have performed compliance checks on the unit testing products and that all non-compliance action items are corrected and tracked to closure. Ensure the contractor performs the required peer reviews of test cases and test results. Participate in these peer reviews on a sampling basis.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-9 Ensure the contractor's software unit testing standards, practices, procedures, and conventions are documented, complete, correct, effective for the software application, efficient, and followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure unit testing standards, practices, procedures, and conventions exist, that they are documented and updated when necessary, that they are correct and complete, and that they are effective for the software application and are efficient. Ensure developers know and follow the unit testing standards, practices, procedures, and conventions. Ensure unit testing standards, practices, procedures, and conventions exist for all types of code (newly developed, modified reuse and open source, and where needed, unmodified reuse and open source). Ensure the unit testing standards, practices, procedures, and conventions are consistent with industry best practices. Ensure the unit testing standards, practices, procedures, and conventions are enforced on the program by peer reviews and use of tools. Ensure the unit testing standards, practices, procedures, and conventions include government, commercial and international standards as appropriate for the program's contract, scope, and constraints.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-10 Ensure the contractor's software implementation processes for the program are documented, complete, correct, effective for the software application, efficient, and followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure program processes for software implementation exist, that they are documented and updated when necessary, that they are correct and complete, and that they are effective for the software application and efficient. Ensure developers know and follow the processes. Ensure the implementation processes address all categories of software (newly developed code, reuse code (unmodified and modified), open source code, COTS, and GOTS). Ensure the processes address model-based development, if applicable to the program's software. Ensure the processes address both initial coding and updating code due to changes or fixing defects. Ensure the processes are consistent with all required specifications, standards, and constraints. Ensure processes for code analyses are included for automated structure analysis, static analysis, dynamic analysis, complexity analysis, and analyses for memory leaks, vulnerabilities, type mismatches, and dead code. Ensure the processes include criteria for determining when a specific code analysis is appropriate. Ensure the processes require code peer reviews for all new and modified reuse and open source code. Ensure the type of peer reviews required are consistent with the type and criticality of the software item and that the peer reviews and code analyses enforce the coding standards and practices. Ensure any tailoring of corporate software implementation processes is appropriate for the program. Ensure the contractor's code development tools are adequate and support the implementation processes.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-11 Ensure the contractor's software unit test processes are documented, complete, correct, effective for the software application, efficient, and followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure program processes for software unit testing exist, that they are documented and updated when necessary, that they are correct and complete, and that they are effective for the software application and efficient. Ensure developers know and follow the processes. Ensure the unit test processes address criteria for what software must be unit tested, and that those criteria include all newly developed software and all modified reuse and open source software as well as all unmodified reuse and open source software designated as critical or with a known track record for defects. Ensure the processes address unit testing of initial coding, re-unit testing of any code updated due to changes or fixing defects, and regression testing of the unit's functionality after changes are made. Ensure the unit testing processes are consistent with all required specifications, standards, and constraints. Ensure the processes for unit testing contain test coverage requirements and that those requirements include: all branches and statements; all error and exception handling; all interfaces internal and external to the unit, including valid and invalid values, values just inside the boundary of acceptable range values, boundary values, values just outside the boundary, and extreme values; all algorithms; nominal and off-nominal test cases; fault detection, isolation, recovery, and data capture and reporting; and where applicable, start-up, termination, and restart. Ensure the unit test processes require documentation of unit test cases, procedures, and results and control of all unit testing artifacts. Ensure the processes require peer reviews for all unit test cases and results. Ensure the type of peer reviews required are consistent with the type and criticality of the software item.. Ensure any tailoring of corporate software unit testing processes is appropriate for the program. Ensure the contractor's unit testing tools are adequate and support the unit testing processes.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-12 Ensure the unit testing fully verifies that the software unit is correctly coded and that the code matches the design |
Phase B |
Phase C |
Phase D1 |
|
Ensure the unit test results for each unit test case are correct. Ensure the unit test cases and results demonstrate that the coded unit fully and correctly implements the unit's design and satisfies the unit's allocated requirements. Ensure the contractor has performed peer reviews of the unit test results. Participate in these peer reviews on a sampling basis.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-13 Ensure the unit testing performed completely covers the software units under test |
Phase B |
Phase C |
Phase D1 |
|
Ensure the unit testing for all units under test included testing the following: all branches and statements; all error and exception handling; all interfaces internal and external to the unit, including valid and invalid values, values just inside the boundary of acceptable range values, boundary values, values just outside the boundary, and extreme values; all algorithms; nominal and off-nominal test cases; fault detection, isolation, recovery, and data capture and reporting; and where applicable, start-up, termination, and restart. Ensure the unit testing has been performed for all newly developed code, all modified reuse and open source code, all unmodified reuse and open source software designated as critical or with a known track record for defects, and all model-based code. For any exceptions to this coverage, the risk has been assessed, documented, and communicated.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-14 Ensure all reuse, COTS, GOTS, and open source software selected during implementation meets the program's established evaluation criteria |
Phase B |
Phase C |
Phase D1 |
|
Ensure the justification for the use of any legacy, reuse, COTS, GOTS, open source, and other non-developmental item (NDI) software selected during implementation is documented, complete, kept up to date, and contains sufficient detail to support the use of said product. Ensure the documented justifications are based on a robust set of evaluation criteria.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|
| 5.4-5-15 Ensure an independent visual inspection of selected code has been performed |
Phase B |
Phase C |
Phase D1 |
|
Ensure an independent visual inspection of a selected portion of the code is performed by experts in the language in which the code is written. Ensure the inspection checks for dangerous constructs that might lead to vulnerabilities or errors that cause software dependability problems are included. Ensure the selected portion of the code included all code considered critical as well as an appropriate samples across different areas of the code and different developers.
|
NA
|
NA
|
TOR-2004(3909)-3537 SW Development Standard for Space Systems; TOR-2011(8591)-20 Space Segment SW Readiness Assessment
|