| 2.3.12-12-1 Ensure the contractor's Security verification test planning is consistent with the contractor's master build plan |
Phase B |
Phase C |
Phase D1 |
|
Ensure the Security test plan includes a set of tests for each build that tests the new capabilities added in that Security build. These tests are intended to probe deeper into the areas that are new for this build. Ensure that the Security test plans include a set of regression tests to check that the capabilities added in previous builds still work. Ensure that the Security test plans include the resources needed to test each build.
|
NA
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development
|
| 2.3.12-12-2 Ensure the contractor's Security test plans, cases, and procedures are correct and complete and that they fully verify the requirements for each Security item |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor's test plans, cases, and procedures fully verify the requirements for each Security item. Ensure that the verification coverage requirements in the Software Development Standard are met. Ensure that all necessary verification methods (Inspection, Analysis, Demonstration, Test) are used in order to fully verify the Security requirements and that the plans, cases, and procedures cover all verification methods. Ensure that the test cases have test procedures that satisfy the test case objectives. Ensure the bi-directional traceability of the Security requirements to the tests in the test plan, test cases, and test procedure steps. Ensure that all test cases have specified the inputs, expected outputs, and evaluation criteria. Ensure cyber resiliency assessments are integrated into developmental and operational testing.
|
AFMAN 63-119 A12.1.2
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development; AFMAN 63-119 or equivalent
|
| 2.3.12-12-3 Ensure the contractor's Security verification planning (Master Test Plan or equivalent) defines the test philosophy and planning from an Security perspective |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor's verification planning includes Security. Ensure that the system level testing includes testing Security across external interfaces, and system owners agree on rules of engagement and reciprocity. Ensure that the system Test and Evaluation and Management Plan (TEMP) includes the Security verification test planning, cyber test infrastructure, and test documentation. Ensure that the contractor's test plan supports the system level test plan. Ensure a Risk Management Framework (RMF) is implemented and the T&E community invited to observe and participate in process activities. Ensure compliance with applicable DoD Information Enterprise Architecture directives. Ensure applicable plans, guides and framework-related documents are complete, consistent with the Test and Evaluation Master Plan, and available to the test and evaluation community as early as possible. Ensure funding is available to complete cooperative vulnerability, penetration assessment, and adversarial assessment test events.
|
AFMAN 63-119 A9.1.3.4; AFMAN 63-119 A9.1.4; AFMAN 63-119 A9.2; AFMAN 63-119 A9.2.1; AFMAN 63-119 A10.7.6.2; AFMAN 63-119 A12.2; AFMAN 63-119 A12.2.1; AFMAN 63-119 A12.2.2; AFMAN 63-119 A12.2.4
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development; AFMAN 63-119 or equivalent
|
| 2.3.12-12-4 Ensure planning for independent testing of critical Security mechanisms and components is correct and complete |
Phase B |
Phase C |
Phase D1 |
|
Ensure independent testing of critical Security mechanisms and components is correctly and completely planned with all the resources needed. Ensure that the independent testing includes negative testing and boundary conditions or the worst possible scenario. Ensure that the completeness coverage in the Software Development Standard is used for the worst cases.
|
AFMAN 63-119 A12.3.5; AFMAN 63-119 A15.12
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development; AFMAN 63-119 or equivalent
|
| 2.3.12-12-5 Ensure the correctness and completeness of the contractor's Security test plan |
Phase B |
Phase C |
Phase D1 |
|
Ensure the contractor's test planning and procedures fully verify the requirements for each Security item. Ensure that the security verification requirements and application security references, e.g., the DISA Application Security & Development STIG, and if available, the contractor Secure Development Standard which both should be referenced in the Software Development Standard, are met. Ensure that all necessary verification methods (Inspection, Analysis, Demonstration, Test) are used to satisfy complete verification of the Security requirements. Ensure that there are Security-to-Security test cases that trace to the interfaces defined in the Security architecture and design. Ensure that the test cases have test procedures that satisfy the test case objectives. Ensure the bi-directional traceability of the Security requirements to the verification test cases. Ensure the bi-directional traceability of the Security architecture and design to the integration test cases. Ensure that all test cases have specified the inputs, expected outputs, and evaluation criteria. Ensure interoperability or net-ready certification activities are complete. Ensure developer/test passwords, password scripts, and accounts in use during system development are deleted prior to operational testing. Ensure cybersecurity vulnerability alerts do not impact operations or system certification, or potentially invalidate test data. Ensure data passed to and from interoperable systems is compatible. Ensure operational test readiness reviews possess interoperability statements. Ensure the test organization has acquired proper authorities prior to entering developmental and operational test. Ensure systems comply with USAF Electromagnetic Compatibility Program and Radio Frequency Spectrum Management guidelines. Ensure systems and subsystems required to interoperate with the test articles (including external systems) are available. Ensure test resource planning includes updated resources and costs associated with cooperative vulnerability, penetration assessment and adversarial test events.
|
AFMAN 63-119 A9.4; AFMAN 63-119 A9.4.1; AFMAN 63-119 A9.4.2; AFMAN 63-119 A9.4.3; AFMAN 63-119 A9.4.4; AFMAN 63-119 A9.4.5; AFMAN 63-119 A9.5; AFMAN 63-119 A9.6; AFMAN 63-119 A12.2.3; AFMAN 63-119 A12.3.6
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development; DISA Application Security & Development STIG; AFMAN 63-119 or equivalent
|
| 2.3.12-12-6 Ensure the correctness and completeness of the verification tests for information assurance and Cybersecurity requirements |
Phase B |
Phase C |
Phase D1 |
Phase D2 |
Phase D3 |
|
Ensure interfaces to trusted security components are tested to the same common criteria (CC) level as the trusted security components' level. Trusted security components at higher CC levels should have integration tests that support those higher CC levels. Ensure applicability of network defender participation in adversarial assessment team OT&E events. Ensure security constraints, permissions and rules of engagement are in place before start of DT&E and OT&E. Ensure limitations of effects on real-world operations are identified and documented. Ensure quantitative cyber resiliency factors and tailored measures are identified, and anti-tamper testing is integrated into DT&E and OT&E as much as possible. Ensure the security test approach is included in the TEMP if applicable. Ensure the Plan of Action and Milestones (POA&M) shows how cybersecurity DRs and vulnerabilities will be resolved.
|
AFMAN 63-119 A12.3.2; AFMAN 63-119 A12.3.3; AFMAN 63-119 A12.3.4; AFMAN 63-119 A12.4; AFMAN 63-119 A12.4.1; AFMAN 63-119 A12.4.2; AFMAN 63-119 A12.6; AFMAN 63-119 A19.9
|
NA
|
Security Considerations in the System Development Life Cycle, NIST SP-800-64; OWASP Software Assurance Maturity Model - A Guide to Building Security into Software Development; AFMAN 63-119 or equivalent
|
| 2.3.12-12-7 Ensure operational test planning addresses required cybersecurity content |
Phase B |
Phase C |
Phase D1 |
|
Ensure the operational test plans contain master plan linkage, architecture, intelligence community-validated cyber threat, operational environment, evaluation structure, time and resources, cooperative vulnerability and penetration assessment, and adversarial assessment. Plans should also address cybersecurity software assurance considerations.
|
AFMAN 63-119 A12.1.2.1
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Integration, Test, and Evaluation Chapter
|
| 2.3.12-12-8 Ensure risk management framework processes are followed |
Phase B |
Phase C |
Phase D1 |
|
Ensure Risk Management Framework (RMF) steps (1. Categorize System, 2. Select Security Controls, 3. Implement Security Controls, 4. Assess Security Controls, 5. Authorize System, 6. Monitor Security Controls) are followed. Ensure an ITT sub-group is established to monitor and control, if necessary.
|
AFMAN 63-119 A12.1.3
|
NA
|
AFMAN 63-119 or equivalent; Mission Assurance Guide, TOR-2007(8546)-6018, Rev B, Program Assurance Chapter
|