DEFENSE TECHNICAL RISK ASSESSMENT METHODOLOGY (DTRAM) CRITERIA VOLUME (VERSION 6.4, May 2023) Office of the Under Secretary for Defense for Research and Engineering (OUSD(R&E)) Abstract The document includes evaluation criteria and a framework for use in technical risk assessments of Department of Defense (DoD) acquisition programs. Assessments may include (1) Independent Technical Risk Assessments (ITRA) as directed by statute and policy, (2) Test and Evaluation Sufficiency Assessments, (3) informational reviews of readiness for milestone decisions, (4) assessments of the adequacy of Systems Engineering Technical Reviews (SETRs) and Test and Evaluation events, and (5) reviews of technical planning documentation (e.g., Systems Engineering Plans, Test and Evaluation Master Plans, and Program Protection Plans). This methodology is applicable to technical assessments for all DoD Component and Office of the Secretary of Defense (including technology and manufacturing) programs, prototypes, and systems, regardless of DoD acquisition pathway. Distribution Statement A. Approved for public release. Distribution is unlimited. DOPSR Case # 23-S-2052 DEFENSE TECHNICAL RISK ASSESSMENT METHODOLOGY (DTRAM) CRITERIA VOLUME (Version 6.4, May 2023) CONTENTS 1.0 MISSION CAPABILITY ............................................................................................................................... 2 2.0 TECHNOLOGY .......................................................................................................................................... 5 3.0 SYSTEM DEVELOPMENT & INTEGRATION ............................................................................................... 7 4.0 MOSA .................................................................................................................................................... 10 5.0 SOFTWARE ............................................................................................................................................ 12 6.0 SECURITY / CYBERSECURITY .................................................................................................................. 15 7.0 MANUFACTURING ................................................................................................................................. 17 8.0 RAM & SUSTAINMENT .......................................................................................................................... 19 DTRAM Paragraph Numbering Nomenclature: Area (e.g., 7.) . Factor (e.g., 7.) . Criteria (e.g., C3.) / Parent Criteria (e.g., P.) . Question (e.g., Q3) Changes to this version: Version 6.4 incorporates additional corrosion prevention and control language requested by Deputy's Management Action Group (DMAG) and required by DoD Instruction 5000.88, Engineering of Defense Systems. 1.0 MISSION CAPABILITY 1.1.P (MISSION CAPABILITY - Scope & Requirements) User expectations regarding system (initial and evolving) capability, operational employment, and the operating environment are sufficiently detailed and understood to guide development, evaluation, and delivery of integrated mission capability. 1.1.C1 Requirements (e.g. KPP, KSA, ICD, CDD, and DODAF) are clear, consistent, measurable, traceable to capability gaps, and are defined sufficiently to capture user expectations of system capability, to include addressing the evolution of capabilities to meet changing threats, technology insertion, and interoperability. 1.1.C2 CONOPS, OMS/MP, FoS/SoS relationships, scenarios, operating environment and threats (e.g. VOLT) are sufficient to guide acquisition, evaluation, and delivery of integrated mission capability. 1.1.C3 Requirements are stable, based on valid assumptions, and are achievable within program structure and timeline. 1.2.P (MISSION CAPABILITY - Design & Architecture) The program sufficiently considers integrated mission (initial and evolving) capability, operating environment, system interdependencies, and user acceptance in system design. 1.2.C1 Sufficient analysis is conducted to evaluate system design with respect to integrated mission capability to include addressing the evolution of capabilities to meet changing threats, technology insertion, and interoperability. 1.2.C2 Interdependent systems are designed, modeled and synchronized (e.g. requirements, definitions, architectures, interfaces, baselines, and system maturity) to support system development and integrated mission capability. 1.2.C3 Development strategy sufficiently incorporates mission stakeholder involvement in maturing system design and implementation with respect to mission effectiveness and operational considerations. 1.3.P (MISSION CAPABILITY - Decision & Control) The program objectively monitors and sufficiently understands progress toward realization of integrated mission capability, controls risk, and integrates user feedback and priorities in development decisions. 1.3.C1 The program employs metrics and models that sufficiently track integrated mission capability, including FoS/SoS integration, and are sufficient to manage technical risk. 1.3.C2 The program sufficiently analyzes, tracks, and mitigates mission capability risks. 1.3.C3 The program has established appropriate mission capability maturity criteria, to include interdependent systems, to determine readiness to continue scheduled product acquisition. 1.4.P (MISSION CAPABILITY - Schedule) Schedule is realistic and synchronized with interdependent programs to deliver integrated mission capability to meet program objectives. 1.4.C1 Schedule is synchronized across interdependent programs. 1.4.C2 Schedule is realistic, consistent with historical analogous acquisition programs, and sufficiently models acquisition and evaluation of integrated mission capability. 1.4.C3 Schedule sufficiently captures all program requirements for capability testing, fielding, and IOC. 1.5.P (MISSION CAPABILITY - Resources) Program staffing, skillsets, environments, and assets are sufficient to develop, and evaluate integrated mission capability. 1.5.C1 Engineering and developmental test staffing, including skillsets, organization, and funding are sufficient for mission capability development and evaluation. 1.5.C2 Integration environments and tools (e.g. data analytics tools, digital environment, warfighting labs, modeling and simulation capabilities), are available and sufficient for mission capability development and evaluation. 1.5.C3 Test assets, facilities, and ranges are available and sufficient to address integrated mission capability in operational environment. 1.5.C4 Training and support system resources are sufficient and available to achieve integrated mission capability. 1.6.P (MISSION CAPABILITY - Evaluation) Evaluation planning and activities are sufficient to evaluate and mature integrated mission capability with respect to user expectations in the intended operational environment. 1.6.C1 Evaluation plans, activities, and capacities are realistic and sufficient to deliver integrated mission capability in the projected operational environment. 1.6.C2 Evaluation methodology (e.g. lab, modeling and simulation, full system) is sufficient to demonstrate compliance with mission requirements. 1.6.C3 Test and evaluation execution is on track to support development of mission capability (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 1.7.P (MISSION CAPABILITY - Performance & Quality) Integrated (end-to-end) mission capability is on track to meet user expectations in the projected operational environment. 1.7.C1 Integrated mission capability will meet user expectations in the projected operational environment, to include the evolution of capabilities to meet changing threats, technology insertion, and interoperability. 1.7.C2 The system is on track to meet requirements and operational measures (e.g. KPPs, KSAs, MOPs, MOEs, MOSs, COIs). 1.7.C3 The system is on track to meet fielding and IOC requirements (e.g. training, support systems, and delivery quantities). 2.0 TECHNOLOGY 2.1.P (TECHNOLOGY - Scope & Requirements) Technology selection, insertion, and refresh are adequately defined to support program objectives. 2.1.C1 Critical technologies are identified and the degree of maturation necessary to support program objectives is understood. 2.1.C2 Technology insertion, refresh opportunities, and expected replacement cycle times are defined to support program objectives. 2.2.P (TECHNOLOGY - Design & Architecture) The program has a sufficient technology trade study, maturation, and incorporation approach to meet program objectives. 2.2.C1 Technology maturation activities, including activities in parallel programs, are integrated and sufficient to support system development. 2.2.C2 Technology assessments and design trades inform maturation planning and system design. 2.2.C3 Technologies for enabling systems and development (e.g. M&S, test, digital infrastructure) are identified, matured, and integrated to support system acquisition. 2.3.P (TECHNOLOGY - Decision & Control) The program objectively monitors and sufficiently understands technology maturation, controls risk, and establishes appropriate on/off-ramp decision criteria. 2.3.C1 The program employs metrics and models that sufficiently track technology maturity and are sufficient to manage risk. 2.3.C2 The program sufficiently analyzes, tracks, and mitigates technology risks. 2.3.C3 The program has established objective, time-phased criteria and events to assess technology maturity in support of acquisition decisions, including on/off-ramp decisions. 2.4.P (TECHNOLOGY - Schedule) The program schedule incorporates technology maturation, technology insertion, and technology refresh activities. 2.4.C1 Technology maturation activities are realistic, sufficiently sequenced, time phased, and integrated with the program schedule. 2.4.C2 Technology schedule reflects actual progress. 2.5.P (TECHNOLOGY - Resources) Program staffing, skillsets, funding, and investments associated with technology are sufficient to meet program objectives. 2.5.C1 Technology staffing, including skillsets and funding, are sufficient to meet program objectives. 2.5.C2 Investments are appropriate and sufficient to mitigate technology risks. 2.5.C3 Investments in enabling technologies (e.g. M&S, test, digital infrastructure) are appropriate and sufficient to support system acquisition. 2.5.C4 Primary and alternate sources for critical technologies are identified and sufficient to support system maturation. 2.6.P (TECHNOLOGY - Evaluation) Technology evaluation planning and activities are sufficient to mature capability in support of program objectives. 2.6.C1 Evaluation plans, activities, and capacities are realistic and sufficient to mature, verify, and validate technology maturity. 2.6.C2 Evaluation methodology (e.g. lab, modeling and simulation, full system) is sufficient to demonstrate technology maturity and intended capability. 2.6.C3 Test and evaluation execution is on track to support technology development (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 2.7.P (TECHNOLOGY - Performance & Quality) Each critical technology has achieved the required level of technical maturity and is likely to completely mature to meet operational effectiveness and suitability objectives. 2.7.C1 Critical technology is on track to meet maturity objectives, to include integration into the overall system, and demonstrated performance in the relevant operational environment. 2.7.C2 Results are sufficient to evaluate performance of matured technology to support program decisions. 3.0 SYSTEM DEVELOPMENT & INTEGRATION 3.1.P (SYSTEM DEVELOPMENT & INTEGRATION - Scope & Requirements) System technical baseline (e.g. design specifications, work packages, interface descriptions, use cases) is sufficient to develop and deliver intended capability. 3.1.C1 Derived requirements are adequate, consistent, measurable, traceable to user needs, and reflected in the technical baseline. 3.1.C2 Growth margins are identified and sufficient for the anticipated system lifecycle, and are consistent with historical analogous systems. 3.1.C3 Technical baseline is stable, sufficiently mature, and achievable within program structure (e.g. acquisition strategy, timeline, SECDEF goals, contract(s), and incentives). 3.2.P (SYSTEM DEVELOPMENT & INTEGRATION - Design & Architecture) System engineering plans, design, implementation, and integration of system architecture, interfaces, and components are sufficient to meet program objectives. 3.2.C1 Program sufficiently controls system configuration utilizing an authoritative source of truth, to include obsolescence and technology refreshment. 3.2.C2 Engineering considerations are sufficiently analyzed and addressed in system design (e.g. system hazards, E3, ESOH, corrosion, CBRN, force protection, survivability, spectrum management, certification, HSI, affordability, COTS/GFE). 3.2.C3 Architecture and interfaces are sufficient to meet user requirements over the intended system lifecycle. 3.2.C4 Implementation of system components and integration of subsystems are sufficient to support the technical baseline. 3.2.C5 Engineering analysis, prototyping, and experimentation are sufficient to validate cost-risk-performance relationships, design optimization (e.g. SWAP-C), and affordability. 3.3.P (SYSTEM DEVELOPMENT & INTEGRATION - Decision & Control) The program objectively monitors and sufficiently understands progress toward development of the technical baseline, controls risk, and establishes appropriate technical criteria for development events. 3.3.C1 The program employs metrics and models that sufficiently track technical progress and program execution and are sufficient to manage risk. 3.3.C2 The program sufficiently analyzes, tracks, and mitigates system development and test risks, issues, and opportunities on a timeline to meet program entrance criteria into the next phase of development and test. 3.3.C3 The program has established objective, time-phased criteria and events to assess technical baseline maturity and to determine readiness to proceed with scheduled product development (e.g. PDR, CDR, and SRR). 3.4.P (SYSTEM DEVELOPMENT & INTEGRATION - Schedule) The schedule sufficiently models the integrated development program, is achievable, and supports program objectives. 3.4.C1 Overall program schedule(s) is integrated, realistic, supported by a sound basis of estimate, consistent with historical analogous systems' schedules and sufficiently models system development. 3.4.C2 Scheduling practices and schedule health are sufficient to track progress, sufficient to accurately report EV, and manage risk. 3.4.C3 The schedule incorporates risk-related decision points, as well as the impacts of program risks and risk mitigations. 3.4.C4 The program critical path, near critical path activities, float, and schedule drivers are well understood. 3.4.C5 Program schedule is sufficient to deliver required capability, to include technical debt and defect resolution. 3.4.C6 Schedule for developmental test and evaluation is sufficient, realistic, and consistent with historical analogous systems. 3.5.P (SYSTEM DEVELOPMENT & INTEGRATION - Resources) Program staffing, skillsets, environments, assets, and funding are sufficient to develop and evaluate the technical baseline. 3.5.C1 Engineering and developmental test staffing, including skillsets and organization, and funding are sufficient to meet program objectives. 3.5.C2 Integration environments and tools (e.g. data analytics tools, digital environments, modeling and simulation capabilities), are available and sufficient for mission capability development and evaluation. 3.5.C3 Test assets, facilities, and ranges are available and sufficient to verify and validate performance to specification. 3.5.C4 Program resources accommodate resolution of technical debt (e.g. deferred, partially implemented, and deficient functionality). 3.5.C5 Data/IP rights and licensing are aligned to program plan and sufficient for system design and acquisition. 3.6.P (SYSTEM DEVELOPMENT & INTEGRATION - Evaluation) Evaluation planning and activities are sufficient to evaluate and mature the technical baseline. 3.6.C1 Developmental test and evaluation plans, activities and capacities are realistic and sufficient to support system acquisition. 3.6.C2 Evaluation methodology (e.g. lab, modeling and simulation, full system) is sufficient to demonstrate compliance and establish a basis for entry into operational test. 3.6.C3 Test and evaluation execution is on track to support program objectives (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 3.7.P (SYSTEM DEVELOPMENT & INTEGRATION - Performance & Quality) System performance and quality is on track to support program objectives. 3.7.C1 System is maturing sufficiently to meet established criteria (e.g. technical performance measures, milestone criteria) and continue acquisition on schedule. 3.7.C2 System performance, to include disposition of technical debt (e.g. deferred, partially implemented, and deficient functionality), is on track to satisfy technical baseline, entrance to IOT&E, and operations. 4.0 MOSA 4.1.P (MOSA - Scope & Requirements) Modular Open Systems Approach is incorporated to the maximum extent practicable to support user requirements and system lifecycle objectives. 4.1.C1 The program’s integrated business and technical strategy to implement (or not implement) a Modular Open Systems Approach is clearly defined, justified, and sufficient to meet objectives. 4.1.C2 Major system components and major system interfaces, that likely will be incrementally added, removed, or replaced throughout the lifecycle have been sufficiently identified in the appropriate solicitation and contractual documents. 4.2.P (MOSA - Design & Architecture) Program design, implementation, and integration of system architecture, interfaces, and components is sufficient to meet MOSA objectives. 4.2.C1 System interfaces incorporate consensus-based standards that are published and maintained by recognized organizations to the maximum extent practicable. 4.2.C2 System architecture supports evolving requirements and incorporation of new technologies over the intended system lifecycle. 4.2.C3 Systems level configuration management sufficiently ensures mission and information assurance while accommodating the program MOSA strategy. 4.2.C4 Engineering analysis and trade studies (e.g. threat analysis, functional criticality analysis, technology opportunities, evolved capability assessments) sufficiently inform the program’s technical strategy and identify system elements for MOSA implementation. 4.3.P (MOSA - Decision & Control) The program objectively monitors and sufficiently understands progress toward MOSA implementation, controls risk, and establishes appropriate technical criteria for development events. 4.3.C1 The program employs metrics and models that sufficiently track implementation of MOSA objectives and are sufficient to manage risk. 4.3.C2 The program sufficiently analyzes, tracks, and mitigates MOSA risks, issues, and opportunities. 4.3.C3 The program has established objective, time-phased criteria and events to assess MOSA implementation and to determine readiness to proceed with scheduled product development (e.g. PDR, CDR, and SRR). 4.4.P (MOSA - Schedule) The program schedule incorporates MOSA implementation activities. 4.4.C1 MOSA activities are realistic, sufficiently sequenced, time phased, and integrated with the program schedule. 4.4.C2 MOSA implementation schedule reflects actual progress. 4.5.P (MOSA - Resources) Engineering tools, labs, staffing, skills, and licensing are sufficient to develop and evaluate MOSA implementation. 4.5.C1 Engineering staffing, including skillsets and organization, support MOSA objectives. 4.5.C2 Integration environments and tools are available and sufficient for MOSA implementation. 4.5.C3 Test assets, facilities, and ranges are available and sufficient to verify and validate MOSA-enabled capabilities. 4.5.C4 Data/IP rights and licensing are sufficient for MOSA implementation. 4.6.P (MOSA - Evaluation) Evaluation planning and activities are realistic and sufficient to address MOSA implementation. 4.6.C1 Evaluation plans, activities, and capacities are realistic and sufficient to verify and validate MOSA-enabled capabilities. 4.6.C2 Evaluation methodology (e.g. lab, modeling and simulation, full system) is sufficient to demonstrate MOSA compliance with modular/open interface standards. 4.6.C3 Test and evaluation execution is on track to support MOSA implementation (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 4.7.P (MOSA - Performance & Quality) System performance is on track to meet MOSA objectives. 4.7.C1 Major system components and major system interfaces are maturing sufficiently to meet established MOSA objectives and continue acquisition on schedule. 4.7.C2 Results are sufficient to evaluate performance of MOSA-enabled capability to support program decisions. 5.0 SOFTWARE 5.1.P (SOFTWARE - Scope & Requirements) Derived software requirements, to include interfaces, are realistic and adequate to develop and deliver intended capability. 5.1.C1 Software requirements are complete, internally consistent, and traceable to program requirements. 5.1.C2 Usage scenarios, constraints, and assumptions are sufficient to guide acquisition and integration. 5.1.C3 Software scope within the technical baseline is stable and achievable, to include consideration of relevant historical benchmarks, within program structure and timeline. 5.2.P (SOFTWARE - Design & Architecture) Software development plans, design, implementation, and integration of architectures, interfaces, and components are sufficient to meet program objectives 5.2.C1 Software development methodology supports program objectives (e.g. AGILE, DevSecOps). 5.2.C2 Software development strategy (e.g. buy/build/reuse, COTS/GOTS, supportability) supports program objectives. 5.2.C3 Architecture and interfaces are sufficient to meet user requirements over the intended system lifecycle. 5.2.C4 Implementation and integration of software configuration items are sufficient to support the technical baseline. 5.2.C5 Incremental software build strategy sufficiently addresses all content (e.g. avoids technical debt) to support program objectives. 5.2.C6 Safety-critical and safety related software is sufficient to support program objectives. 5.3.P (SOFTWARE - Decision & Control) The program objectively monitors and sufficiently understands software development progress, controls risk, and establishes appropriate technical criteria for development events. 5.3.C1 The program employs metrics and models that sufficiently track software progress and performance and are sufficient to manage risk. 5.3.C2 The program sufficiently analyzes, tracks, and mitigates software risks. 5.3.C3 Software content and maturity criteria (e.g. defect thresholds) established to support acquisition decisions. 5.4.P (SOFTWARE - Schedule) The schedule sufficiently models software development, is achievable, and supports program objectives. 5.4.C1 Software related activities are realistic, sufficiently sequenced, time phased, integrated with the program schedule, and supported by a sound basis of estimate that considers relevant historical schedule benchmarks. 5.4.C2 Software schedule reflects actual progress (e.g. technical earned progress) and planned dates are adjusted with latest revised estimates. 5.4.C3 Software schedule is sufficient to deliver required capability, to include technical debt and defect resolution, and is integrated with the program schedule. 5.4.C4 Software schedule is sufficient to deliver required capability, to include technical debt and defect resolution, and is integrated with the program schedule. 5.5.P (SOFTWARE - Resources) Software tools, labs, staffing, skills, and licensing are sufficient to meet program objectives. 5.5.C1 Software staffing, competencies, and experience with the software development approach (e.g. Iterative, Agile, DevSecOps) supports program objectives. 5.5.C2 Software tools and facilities (e.g. software factory) are available and sufficient to support development. 5.5.C3 Software resources are sufficient to resolve technical debt and defects 5.5.C4 Software licenses (e.g. COTS, GOTS, FOSS), data rights, and custom developed software sufficient to support software objectives. 5.6.P (SOFTWARE - Evaluation) Software evaluation planning and activities are sufficient to mature capability in support of program objectives. 5.6.C1 Software evaluation plans, activities and capacities are realistic and sufficient to support product acquisition. 5.6.C2 Software evaluation methodologies (e.g., test automation, lab fidelity, modeling and simulation, full system) are sufficient to demonstrate compliance. 5.6.C3 Software integration and test execution is on track to support software development objectives (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 5.7.P (SOFTWARE - Performance & Quality) Software functionality and quality are on track to support program objectives. 5.7.C1 Software architecture, interfaces, and sub-system performance meeting quality and performance objectives. 5.7.C2 Results are sufficient to evaluate software performance in the intended operational environment and support program decisions. 5.7.C3 Software increments are on track to meet program objectives, including resolution of technical debt and defects. 6.0 SECURITY / CYBERSECURITY 6.1.P (SECURITY / CYBERSECURITY - Scope & Requirements) Security requirements are realistic and sufficient to support system development and operation in the intended environment. 6.1.C1 Program protection and cybersecurity requirements have been sufficiently derived from appropriate analysis (e.g. criticality analysis, vulnerability assessment, threat analysis, exportability). 6.1.C2 Security environment and operations have been considered and appropriately tailored and defined. 6.1.C3 Cyber threat environment has been sufficiently defined from mission survivability and sustainment objectives. 6.1.C4 Security scope is realistic and achievable within program structure and timeline. 6.2.P (SECURITY / CYBERSECURITY - Design & Architecture) The program sufficiently incorporates security and cybersecurity considerations into system development to meet program objectives. 6.2.C1 System security considerations to protect and preserve system design information, components, functions and CPI are sufficiently addressed in system architecture and design (e.g. countermeasures selection). 6.2.C2 System security strategy sufficiently minimizes vulnerabilities introduced by design, production, system interfaces and access points (e.g. counterfeit parts, anti-tamper). 6.2.C3 Enabling and support equipment, systems, and facilities are sufficiently addressed in system security strategy. 6.2.C4 System security design accounts for evolving vulnerabilities throughout the lifecycle (e.g. sustainment). 6.3.P (SECURITY / CYBERSECURITY - Decision & Control) The program objectively monitors and sufficiently understands security related development progress, controls risk, and establishes appropriate technical criteria for development events. 6.3.C1 The program employs metrics and models that sufficiently track security and cybersecurity progress and performance and are sufficient to manage risk. 6.3.C2 The program sufficiently analyzes, tracks, and mitigates security risks. 6.3.C3 Maturity criteria for system security are established to support acquisition decisions. 6.4.P (SECURITY / CYBERSECURITY - Schedule) The schedule sufficiently models security engineering activities, is achievable, and supports program objectives. 6.4.C1 System security engineering activities are realistic, sufficiently sequenced, time phased, and integrated with the program schedule. 6.4.C2 Security engineering schedule reflects actual progress. 6.5.P (SECURITY / CYBERSECURITY - Resources) System security staffing, facilities, test assets, tools, and trusted suppliers are sufficient to support program objectives. 6.5.C1 System security engineering staffing, including skillsets and organization, support program objectives. 6.5.C2 Sufficient secure facilities and environments are available for system acquisition. 6.5.C3 Test assets are available for integration, and evaluation of security characteristics. 6.5.C4 Automated cyber vulnerability and analysis tools are sufficient throughout the acquisition lifecycle. 6.5.C5 Sufficient trusted suppliers are available to support system acquisition and sustainment. 6.6.P (SECURITY / CYBERSECURITY - Evaluation) System security evaluation planning and activities are sufficient to mature capability in support of program objectives 6.6.C1 Security evaluation plans, activities, and capacities are realistic and sufficient to support product acquisition. 6.6.C2 Evaluation methodology (e.g. lab, modeling and simulation, full system) is sufficient to demonstrate security/cybersecurity compliance. 6.6.C3 Test and evaluation execution is on track to support security/cybersecurity implementation (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 6.6.C4 Evaluation activity sufficiently provides for both cooperative and adversarial activities to identify vulnerabilities throughout the lifecycle. 6.7.P (SECURITY / CYBERSECURITY - Performance & Quality) Security and cybersecurity performance is on track to provide protection in support of program objectives. 6.7.C1 Program has sufficiently mitigated security/cybersecurity risks to CPI, CTI, functions, and components, technologies, enabling systems. 6.7.C2 Security implementation is on track to meet program objectives. 7.0 MANUFACTURING 7.1.P (MANUFACTURING - Scope & Requirements) Manufacturing and production capability and requirements are defined, achievable, and support program objectives. 7.1.C1 Manufacturing and production requirements are realistic and achievable within program structure and timeline. 7.1.C2 Industrial base and manufacturing capabilities support program objectives. 7.1.C3 Product baseline (to include configuration items in concurrent development) is complete, stable, and traceable to requirements. 7.2.P (MANUFACTURING - Design & Architecture) Design and maturation of manufacturing capabilities support production quality and rates. 7.2.C1 Manufacturing and production processes and manufacturing technology maturation supports program requirements. 7.2.C2 Design for producibility is sufficient to meet requirements and affordability objectives. 7.2.C3 Procurement and supply chain capability support requirements. 7.2.C4 Production cut-in, retrofit, and product improvement sufficiently support requirements. 7.2.C5 Design includes implementation of corrosion prevention and control attributes during manufacturing such as selection of corrosion resistant materials, use of material manufacturing process standards, and application of protective coatings. 7.3.P (MANUFACTURING - Decision & Control) The program objectively monitors and sufficiently understands manufacturing and production progress, controls risk, and establishes appropriate technical criteria for development events. 7.3.C1 The program employs metrics that track manufacturing and production maturity, are sufficient to control manufacturing and production performance and manage risk. 7.3.C2 The program sufficiently analyzes, tracks, and mitigates manufacturing and production risks. 7.3.C3 The program has established objective, time-phased criteria and events to assess manufacturing and production maturity and to determine readiness to proceed with the production phase. 7.3.C4 Adequate entrance criteria have been set/met (for completion of system development and testing activities or for maturity of the system) in order to enter the production phase. 7.4.P (MANUFACTURING - Schedule) Manufacturing and production capability maturation and required capacity are sufficiently modeled in the program schedule, are achievable, and support manufacturing objectives. 7.4.C1 Manufacturing and production activities are realistic, supported by a sound basis of estimate that considers relevant historical schedules, sufficiently sequenced, time phased, and integrated with the program schedule. 7.4.C2 Manufacturing and production activities are sufficiently phased independent from and sufficiently decoupled from concurrent development and test activities. 7.4.C3 Manufacturing and production schedule reflects actual progress. 7.5.P (MANUFACTURING - Resources) Manufacturing and production staffing, facilities, materials, and funding are sufficient to support production quality and rates. 7.5.C1 Manufacturing and production staffing, including skillsets and organization, are sufficient to support program objectives. 7.5.C2 Manufacturing and production investments, design tools, digital environments, tooling, and facilities are sufficient to support program objectives. 7.5.C3 Manufacturing and production funding, materials, and supply chain are sufficient to support production rates. 7.6.P (MANUFACTURING - Evaluation) Manufacturing and production evaluation planning and activities are sufficient to mature manufacturing capability, quality, and rates. 7.6.C1 Manufacturing and production evaluation activities (e.g. FAI) are realistic and sufficient to accurately determine capacity yield, assembly rates and unit quality to support product acquisition and sustainment. 7.6.C2 Test and evaluation execution is on track to support manufacturing and production (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 7.7.P (MANUFACTURING - Performance & Quality) Manufacturing and production supports required product quality and production rates. 7.7.C1 Manufacturing and production capability and processes are maturing to plan sufficiently to demonstrate stable, under-control production in a relevant environment prior to production decisions. 7.7.C2 Manufacturing and production technology and capability maturing to plan. 7.7.C3 Procurement (e.g. supply chain) sufficiently supports production. 7.7.C4 Manufacturing and production meets program quality and performance objectives. 8.0 RAM & SUSTAINMENT 8.1.P (RAM & SUSTAINMENT - Scope & Requirements) R&M and sustainment requirements meet user expectations for operational employment and affordability, are realistic, and are achievable within the program structure and timeline. 8.1.C1 R&M and sustainment requirements support the Sustainment KPP, user required readiness levels, and program affordability goals. 8.1.C2 Usage scenarios, constraints, and assumptions are sufficient to guide acquisition. 8.1.C3 R&M and sustainment scope is realistic and achievable, to include consideration of relevant historical benchmarks, within program structure and timeline. 8.2.P (RAM & SUSTAINMENT - Design & Architecture) The program sufficiently incorporates R&M and sustainment considerations into system development to meet performance and affordability objectives. 8.2.C1 R&M activities are sufficiently addressed in system design (e.g. design activities for reliability, maintainability, and supportability, corrosion). 8.2.C2 Reliability growth methodology/modeling supports program objectives, and is updated based on actual performance (e.g. test results). 8.2.C3 Product support strategy meets program objectives, and sustainment models are updated based on actual performance (e.g. test results). 8.2.C4 Corrosion prevention and control planning has been addressed in technical reviews, reflected in the appropriate technical baseline, described in the Systems Engineering Plan, and described in the Life Cycle Sustainment Plan, which also includes an attached Corrosion Prevention and Control Plan. 8.3.P (RAM & SUSTAINMENT - Decision & Control) The program objectively monitors and sufficiently understands R&M and sustainment development progress, controls risk, and establishes appropriate technical criteria for development events. 8.3.C1 The program employs metrics that sufficiently track R&M, sustainment, and supportability performance and are sufficient to manage risk. 8.3.C2 The program sufficiently analyzes, tracks, and mitigates R&M risks. 8.3.C3 R&M maturity criteria established to support acquisition decisions (e.g. reliability growth curve). 8.4.P (RAM & SUSTAINMENT - Schedule) The schedule sufficiently models R&M and sustainment activities, is achievable, and supports program objectives. 8.4.C1 R&M and sustainment activities are realistic, supported by a sound basis of estimate that considers relevant historical schedules, sufficiently sequenced, time phased, and integrated with the program schedule. 8.4.C2 R&M and sustainment schedule reflects actual progress. 8.5.P (RAM & SUSTAINMENT - Resources) Sustainment, supportability and R&M staffing, facilities, materials, and funding, are sufficient to meet program objectives. 8.5.C1 Staffing, including skillsets, support R&M and sustainment objectives. 8.5.C2 R&M facilities and test articles available and sufficient to support acquisition. 8.5.C3 Tools, modeling and analysis tools, digital environments, equipment, facilities, agreements, data/IP rights, licensing, and spares are sufficient to support sustainment objectives. 8.5.C4 Product support elements (technical manuals, SE, spares, etc.) are available and sufficient for Integration, Verification, and Validation. 8.6.P (RAM & SUSTAINMENT - Evaluation) Evaluation planning and activities are sufficient to mature sustainment, supportability and R&M performance. 8.6.C1 R&M and sustainment evaluation activities, to include failure definition criteria, are realistic and sufficient (e.g. under OMS/MP conditions) to support product development. 8.6.C2 Test and evaluation execution is on track to support R&M and sustainment (e.g. capacities, scope growth, productivity) and is supplying sufficient results to support program decisions. 8.6.C3 Evaluation results update R&M and sustainment system models. 8.7.P (RAM & SUSTAINMENT - Performance & Quality) Sustainment, supportability and R&M performance are on track to meet program objectives. 8.7.C1 System tracking to the reliability growth curve. 8.7.C2 Other aspects of R&M performance (e.g., stress testing, fatigue testing, corrosion tests and environmental testing) confirms design suitability for the life cycle operating environment. 8.7.C3 System meets R&M requirements (e.g. Ao, MTBF, O&S costs), and operational effectiveness and suitability objectives. 8.7.C4 Sustainment performance (e.g. spares purchase, OEM and organic repair) is on track to meet program objectives. 8.7.C5 Design trade studies or analyses include the cost benefits of applying corrosion mitigation methods and materials in the design process to reduce the O&S cost of the system.