THE UNDER SECRETARY OF DEFENSE 3030 DEFENSE PENTAGON WASHINGTON, DC 20301-3030 RESEARCH AND ENGINEERING JUN 18 2018 MEMORANDUM FOR: SEE DISTRIBUTION SUBJECT: Framework for Risk Categorization for Use During Independent Technical Risk Assessments The National Defense Authorization Act for Fiscal Year 2017 required the Secretary of Defense to issue a framework for categorizing the degree of technical and manufacturing risk when conducting Independent Technical Risk Assessments. The attached framework provides a methodology for assessing technical risk, including technology and manufacturing risks, which could impede a program's ability to meet cost, schedule, and performance goals. This framework is in line with the approved Department of Defense Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs. The Office of the Under Secretary of Defense for Research and Engineering OUSD(R&E) will maintain this Framework, which will be published on the OUSD(R&E) system engineering website (http://www.acq.osd.mil/se/pg/guidance.html). [Signed] Michael D. Griffin Attachment: As stated DISTRIBUTION: CHIEF MANAGEMENT OFFICER OF THE DEPARTMENT OF DEFENSE SECRETARIES OF THE MILITARY DEPARTMENTS CHAIRMAN OF THE JOINT CHIEFS OF STAFF UNDER SECRETARIES OF DEFENSE COMMANDERS OF THE COMBATANT COMMANDS GENERAL COUNSEL OF THE DEPARTMENT OF DEFENSE DIRECTOR OF COST ASSESSMENT AND PROGRAM EVALUATION DIRECTOR OF OPERATIONAL TEST AND EVALUATION INSPECTOR GENERAL OF THE DEPARTMENT OF DEFENSE CHIEF INFORMATION OFFICER OF THE DEPARTMENT OF DEFENSE ASSISTANT SECRETARIES OF DEFENSE ASSISTANTS TO THE SECRETARY OF DEFENSE DIRECTOR OF ADMINISTRATION AND MANAGEMENT DIRECTOR OF NET ASSESSMENT DIRECTORS OF THE DEFENSE AGENCIES DIRECTORS OF THE DOD FIELD ACTIVITIES 2 Department of Defense Independent Technical Risk Assessment Framework for Risk Categorization June 2018 Office of the Under Secretary of Defense Research and Engineering Washington, D.C. Distribution Statement A: Approved for public release. Distribution is unlimited. Department of Defense Independent Technical Risk Assessment Framework for Risk Categorization Office of the Under Secretary of Defense Research and Engineering 3030 Defense Pentagon 3C167 Washington, DC 20301-3030 www.acq.osd.mil/se Distribution Statement A: Approved for public release. Distribution is unlimited. DOPSR Case #18-S-1828 DoD ITRA Framework for Risk Categorization 1. OVERVIEW a. Independent Technical Risk Assessment (ITRA) will assess technical risks for Major Defense Acquisition Programs as described in this framework and the Department of Defense (DoD) Risk, Issue, and Opportunity (RIO) Management Guide for Defense Acquisition Programs (https://www.acq.osd.mil/se/pg/guidance.html), including risks related to critical technologies and manufacturing. For the purposes of this discussion, the term “risk” will refer to both risks and issues, although a risk differs from an issue in that risk occurrence is probabilistic whereas an issue is certain or has already occurred. b. In general, technical risks are those events or conditions typically emanating from areas such as mission/requirements, technology, engineering, integration, test, software, manufacturing/quality, logistics, and system security/cybersecurity that may prevent a program from meeting cost, schedule, and/or performance objectives. c. ITRAs will leverage ongoing program activities whenever practical, e.g., Technology Readiness Assessments (TRA), Manufacturing Readiness Assessments (MRA), Systems Engineering Technical Reviews, and Industry Days. These assessments and activities will inform the ITRA; however, the team will provide an independent assessment of any risks or maturity concerns identified. As such, there may not be a direct correlation between external assessments or measures, such as technology readiness levels, and the team’s assessment. 2. RISK CATEGORIZATION a. The ITRA will document and characterize each risk in terms of consequence to the program and to any interdependent programs should the risk be fully realized, and the likelihood the risk will occur. If known, the cause of the event or condition also should be described. Risks will be analyzed using the likelihood and consequence criteria as established in the DoD RIO guide. b. Using these predefined likelihood and consequence criteria will provide a structured means for consistent evaluation of risks. Any deviations from these criteria will be noted in the assessment along with associated rationale. Assessors will underpin the assessment with engineering analysis and data. c. Risk consequence will be described as a potential deviation against cost, schedule, and performance in program plans or established baselines. Table 1 describes the consequence criteria. Assessments will attempt to capture all cost, schedule, and performance impacts of a given risk. The consequence rating should capture the greatest anticipated impact in cost, schedule, or performance as if the risk were fully realized, that is, without further risk reduction or mitigation efforts. Wherever possible, fully burdened costs should be used in risk assessments. 1 Distribution Statement A: Approved for public release. Distribution is unlimited. DoD ITRA Framework for Risk Categorization Table 1. ITRA Consequence Criteria Level 5 (Critical Impact): - Cost: 10% or greater increase over APB objective values for RDT&E, PAUC, or APUC; Cost increase causes program to exceed affordability caps - Schedule: Schedule slip will require a major schedule rebaselining; Precludes program from meeting its APB schedule threshold dates - Performance: Degradation precludes system from meeting a KPP or key technical/supportability threshold; will jeopardize program success 2; Unable to meet mission objectives (defined in mission threads, ConOps, OMS/MP) Level 4 (Significant Impact): - Cost: 5% - <10% increase over APB objective values for RDT&E, PAUC, or APUC; Costs exceed life cycle ownership cost KSA - Schedule: Schedule deviations will slip program to within 2 months of approved APB threshold schedule date; Schedule slip puts funding at risk; Fielding of capability to operational units delayed by more than 6 months 1 - Performance: Degradation impairs ability to meet a KSA. 2 Technical design or supportability margin exhausted in key areas; Significant performance impact affecting System-of System interdependencies. Work-arounds required to meet mission objectives Level 3 (Moderate Impact): - Cost: 1% - <5% increase over APB objective values for RDT&E, PAUC, or APUC; Manageable with PEO or Service assistance - Schedule: Can meet APB objective schedule dates, but other non-APB key events (e.g., SETRs or other Tier 1 Schedule events) may slip; Schedule slip impacts synchronization with interdependent programs by greater than 2 months - Performance: Unable to meet lower tier attributes, TPMs, or CTPs; Design or supportability margins reduced; Minor performance impact affecting System-of System interdependencies. Work-arounds required to achieve mission tasks Level 2 (Minor Impact): - Cost: Costs that drive unit production cost (e.g., APUC) increase of <1% over budget; Cost increase, but can be managed internally - Schedule: Some schedule slip, but can meet APB objective dates and non-APB key event dates - Performance: Reduced technical performance or supportability; can be tolerated with little impact on program objectives; Design margins reduced, within trade space 2 Level 1 (Minimal Impact): - Cost: Minimal impact. Costs expected to meet approved funding levels - Schedule: Minimal schedule impact - Performance: Minimal consequences to meeting technical performance or supportability requirements. Design margins will be met; margin to planned tripwires Notes: 1 Consider fielding of capability to interdependent programs as well. 2 Failure to meet TPMs or CTPs directly derived from KPPs or KSAs are indicators of potentially not meeting a KPP or KSA APB: Acquisition Program Baseline; APUC: Average Procurement Unit Cost; ConOps: Concept of Operations; CTP: Critical Technical Parameter; PAUC: Program Acquisition Unit Cost; PEO: Program Executive Officer; KPP: Key Performance Parameter; KSA: Key System Attribute; OMS/MP: Operational Mode Summary/Mission Profile; RDT&E: Research, Development Test and Evaluation; TPM: Technical Performance Measure 2 Distribution Statement A: Approved for public release. Distribution is unlimited. DoD ITRA Framework for Risk Categorization d. Risk likelihood is the evaluated probability an event will occur given existing conditions. The estimated likelihood of the risk should be tied to a specific well-defined risk event or condition. Table 2 describes the likelihood criteria the reviewers will use. Table 2. ITRA Likelihood Criteria - Level 5 | Likelihood: Near Certainty | Probability of Occurrence: > 80% to ≤ 99% - Level 4 | Likelihood: Highly Likely | Probability of Occurrence: > 60% to ≤ 80% - Level 3 | Likelihood: Likely | Probability of Occurrence: > 40% to ≤ 60% - Level 2 | Likelihood: Low Likelihood | Probability of Occurrence: > 20% to ≤ 40% - Level 1 | Likelihood: Not Likely | Probability of Occurrence: > 1% to ≤ 20% e. Based upon assessed likelihood and consequence, risks will be categorized using the risk matrix shown in Figure 1 top right. This matrix converts the combination of likelihood and the maximum of the cost, schedule, and performance consequence scores to form a risk level for each risk. Ultimately the ITRA will categorize a risk as High, Moderate, or Low in accordance with the criteria. Similarly, issues will be categorized in terms of severity of consequence as depicted in Figure 2, using the same consequence criteria in risk categorization. Figure 1. Risk Matrix Incorporating Likelihood and Consequence Criteria 3 Distribution Statement A: Approved for public release. Distribution is unlimited. DoD ITRA Framework for Risk Categorization Figure 2. Issue Consequence Matrix (Low: Green [1-2], Moderate: Yellow [3], High: Red [4-5]) f. The ITRA should also consider the effect of aggregate risk on the program and the threat that cumulative or compounding effects of multiple risks pose to successfully satisfying program objectives. The ITRA should consider and document system-of-systems and family-of-systems interactions. g. The ITRA will document existing program mitigation strategies as well as any additional recommended strategies to mitigate risks and issues. Analysis of mitigation strategies will include whether they are feasible, affordable, and timely, given program circumstances, constraints, and objectives. The assessment will include consideration of mitigation impacts to the overall program schedule and technical performance expectations. h. Key technical risks will be summarized using a risk matrix with an assessment of the estimated effectiveness of the planned risk mitigations, as shown in Figure 3. Figure 3. Sample Risk Matrix 4 Distribution Statement A: Approved for public release. Distribution is unlimited. DoD ITRA Framework for Risk Categorization ABBREVIATIONS AND ACRONYMS APB - Acquisition Program Baseline APUC - Average Procurement Unit Cost ConOps - Concept of Operations CTP - critical technical parameter DoDD - DoD directive DoDI - DoD instruction ITRA - Independent Technical Risk Assessment KPP - key performance parameter KSA - key system attribute KW - kilowatt MDAP - Major Defense Acquisition Program OMS/MP - Operational Mode Summary/Mission Profile PAUC - Program Acquisition Unit Cost PEO - Program Executive Officer PM - Program Manager RDT&E - research, development, test and evaluation RIO - risk, issue, and opportunity TMRR - Technology Maturation and Risk Reduction TPM - technical performance measure UAV - unmanned aerial vehicle 5 Distribution Statement A: Approved for public release. Distribution is unlimited.