Department of Defense Independent Technical Risk Assessment Framework for Risk Categorization

Summary

This document establishes a standardized framework for categorizing technical and manufacturing risks during Independent Technical Risk Assessments (ITRAs) for Major Defense Acquisition Programs, pursuant to the National Defense Authorization Act for Fiscal Year 2017. It defines structured criteria and matrices for evaluating risk likelihood and consequence across cost, schedule, and performance dimensions.

Memorandum: Framework for Risk Categorization for Use During Independent Technical Risk Assessments

THE UNDER SECRETARY OF DEFENSE 3030 DEFENSE PENTAGON WASHINGTON, DC 20301-3030 RESEARCH AND ENGINEERING

JUN 18 2018

MEMORANDUM FOR: SEE DISTRIBUTION

SUBJECT: Framework for Risk Categorization for Use During Independent Technical Risk Assessments

The National Defense Authorization Act for Fiscal Year 2017 required the Secretary of Defense to issue a framework for categorizing the degree of technical and manufacturing risk when conducting Independent Technical Risk Assessments.

The attached framework provides a methodology for assessing technical risk, including technology and manufacturing risks, which could impede a program’s ability to meet cost, schedule, and performance goals. This framework is in line with the approved Department of Defense Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs.

The Office of the Under Secretary of Defense for Research and Engineering OUSD(R&E) will maintain this Framework, which will be published on the OUSD(R&E) system engineering website (http://www.acq.osd.mil/se/pg/guidance.html).

[Signed] Michael D. Griffin

Attachment: As stated

Distribution List

DISTRIBUTION: CHIEF MANAGEMENT OFFICER OF THE DEPARTMENT OF DEFENSE SECRETARIES OF THE MILITARY DEPARTMENTS CHAIRMAN OF THE JOINT CHIEFS OF STAFF UNDER SECRETARIES OF DEFENSE COMMANDERS OF THE COMBATANT COMMANDS GENERAL COUNSEL OF THE DEPARTMENT OF DEFENSE DIRECTOR OF COST ASSESSMENT AND PROGRAM EVALUATION DIRECTOR OF OPERATIONAL TEST AND EVALUATION INSPECTOR GENERAL OF THE DEPARTMENT OF DEFENSE CHIEF INFORMATION OFFICER OF THE DEPARTMENT OF DEFENSE ASSISTANT SECRETARIES OF DEFENSE ASSISTANTS TO THE SECRETARY OF DEFENSE DIRECTOR OF ADMINISTRATION AND MANAGEMENT DIRECTOR OF NET ASSESSMENT DIRECTORS OF THE DEFENSE AGENCIES DIRECTORS OF THE DOD FIELD ACTIVITIES

Title Page

Department of Defense Independent Technical Risk Assessment Framework for Risk Categorization

June 2018

Office of the Under Secretary of Defense Research and Engineering

Washington, D.C.

Distribution Statement A: Approved for public release. Distribution is unlimited.

Publication Information

Department of Defense Independent Technical Risk Assessment Framework for Risk Categorization

Office of the Under Secretary of Defense Research and Engineering 3030 Defense Pentagon 3C167 Washington, DC 20301-3030 www.acq.osd.mil/se

Distribution Statement A: Approved for public release. Distribution is unlimited. DOPSR Case #18-S-1828

1. Overview & 2. Risk Categorization

DoD ITRA Framework for Risk Categorization

OVERVIEW

a. Independent Technical Risk Assessment (ITRA) will assess technical risks for Major Defense Acquisition Programs as described in this framework and the Department of Defense (DoD) Risk, Issue, and Opportunity (RIO) Management Guide for Defense Acquisition Programs (https://www.acq.osd.mil/se/pg/guidance.html), including risks related to critical technologies and manufacturing. For the purposes of this discussion, the term “risk” will refer to both risks and issues, although a risk differs from an issue in that risk occurrence is probabilistic whereas an issue is certain or has already occurred.

b. In general, technical risks are those events or conditions typically emanating from areas such as mission/requirements, technology, engineering, integration, test, software, manufacturing/quality, logistics, and system security/cybersecurity that may prevent a program from meeting cost, schedule, and/or performance objectives.

c. ITRAs will leverage ongoing program activities whenever practical, e.g., Technology Readiness Assessments (TRA), Manufacturing Readiness Assessments (MRA), Systems Engineering Technical Reviews, and Industry Days. These assessments and activities will inform the ITRA; however, the team will provide an independent assessment of any risks or maturity concerns identified. As such, there may not be a direct correlation between external assessments or measures, such as technology readiness levels, and the team’s assessment.

RISK CATEGORIZATION

a. The ITRA will document and characterize each risk in terms of consequence to the program and to any interdependent programs should the risk be fully realized, and the likelihood the risk will occur. If known, the cause of the event or condition also should be described. Risks will be analyzed using the likelihood and consequence criteria as established in the DoD RIO guide.

b. Using these predefined likelihood and consequence criteria will provide a structured means for consistent evaluation of risks. Any deviations from these criteria will be noted in the assessment along with associated rationale. Assessors will underpin the assessment with engineering analysis and data.

c. Risk consequence will be described as a potential deviation against cost, schedule, and performance in program plans or established baselines. Table 1 describes the consequence criteria. Assessments will attempt to capture all cost, schedule, and performance impacts of a given risk. The consequence rating should capture the greatest anticipated impact in cost, schedule, or performance as if the risk were fully realized, that is, without further risk reduction or mitigation efforts. Wherever possible, fully burdened costs should be used in risk assessments.

1 Distribution Statement A: Approved for public release. Distribution is unlimited.

Table 1. ITRA Consequence Criteria

DoD ITRA Framework for Risk Categorization

Table 1. ITRA Consequence Criteria

Level 5 (Critical Impact):

Cost: 10% or greater increase over APB objective values for RDT&E, PAUC, or APUC; Cost increase causes program to exceed affordability caps
Schedule: Schedule slip will require a major schedule rebaselining; Precludes program from meeting its APB schedule threshold dates
Performance: Degradation precludes system from meeting a KPP or key technical/supportability threshold; will jeopardize program success 2; Unable to meet mission objectives (defined in mission threads, ConOps, OMS/MP)

Level 4 (Significant Impact):

Cost: 5% - <10% increase over APB objective values for RDT&E, PAUC, or APUC; Costs exceed life cycle ownership cost KSA
Schedule: Schedule deviations will slip program to within 2 months of approved APB threshold schedule date; Schedule slip puts funding at risk; Fielding of capability to operational units delayed by more than 6 months 1
Performance: Degradation impairs ability to meet a KSA. 2 Technical design or supportability margin exhausted in key areas; Significant performance impact affecting System-of System interdependencies. Work-arounds required to meet mission objectives

Level 3 (Moderate Impact):

Cost: 1% - <5% increase over APB objective values for RDT&E, PAUC, or APUC; Manageable with PEO or Service assistance
Schedule: Can meet APB objective schedule dates, but other non-APB key events (e.g., SETRs or other Tier 1 Schedule events) may slip; Schedule slip impacts synchronization with interdependent programs by greater than 2 months
Performance: Unable to meet lower tier attributes, TPMs, or CTPs; Design or supportability margins reduced; Minor performance impact affecting System-of System interdependencies. Work-arounds required to achieve mission tasks

Level 2 (Minor Impact):

Cost: Costs that drive unit production cost (e.g., APUC) increase of <1% over budget; Cost increase, but can be managed internally
Schedule: Some schedule slip, but can meet APB objective dates and non-APB key event dates
Performance: Reduced technical performance or supportability; can be tolerated with little impact on program objectives; Design margins reduced, within trade space 2

Level 1 (Minimal Impact):

Cost: Minimal impact. Costs expected to meet approved funding levels
Schedule: Minimal schedule impact
Performance: Minimal consequences to meeting technical performance or supportability requirements. Design margins will be met; margin to planned tripwires

Notes: 1 Consider fielding of capability to interdependent programs as well. 2 Failure to meet TPMs or CTPs directly derived from KPPs or KSAs are indicators of potentially not meeting a KPP or KSA

APB: Acquisition Program Baseline; APUC: Average Procurement Unit Cost; ConOps: Concept of Operations; CTP: Critical Technical Parameter; PAUC: Program Acquisition Unit Cost; PEO: Program Executive Officer; KPP: Key Performance Parameter; KSA: Key System Attribute; OMS/MP: Operational Mode Summary/Mission Profile; RDT&E: Research, Development Test and Evaluation; TPM: Technical Performance Measure

2 Distribution Statement A: Approved for public release. Distribution is unlimited.

2. Risk Categorization (Continued) & Table 2. ITRA Likelihood Criteria

DoD ITRA Framework for Risk Categorization

d. Risk likelihood is the evaluated probability an event will occur given existing conditions. The estimated likelihood of the risk should be tied to a specific well-defined risk event or condition. Table 2 describes the likelihood criteria the reviewers will use.

Table 2. ITRA Likelihood Criteria

Level 5 | Likelihood: Near Certainty | Probability of Occurrence: > 80% to ≤ 99%
Level 4 | Likelihood: Highly Likely | Probability of Occurrence: > 60% to ≤ 80%
Level 3 | Likelihood: Likely | Probability of Occurrence: > 40% to ≤ 60%
Level 2 | Likelihood: Low Likelihood | Probability of Occurrence: > 20% to ≤ 40%
Level 1 | Likelihood: Not Likely | Probability of Occurrence: > 1% to ≤ 20%

e. Based upon assessed likelihood and consequence, risks will be categorized using the risk matrix shown in Figure 1 top right. This matrix converts the combination of likelihood and the maximum of the cost, schedule, and performance consequence scores to form a risk level for each risk. Ultimately the ITRA will categorize a risk as High, Moderate, or Low in accordance with the criteria. Similarly, issues will be categorized in terms of severity of consequence as depicted in Figure 2, using the same consequence criteria in risk categorization.

Figure 1. Risk Matrix Incorporating Likelihood and Consequence Criteria

3 Distribution Statement A: Approved for public release. Distribution is unlimited.

2. Risk Categorization (Continued) & Mitigation Strategies

DoD ITRA Framework for Risk Categorization

Figure 2. Issue Consequence Matrix (Low: Green [1-2], Moderate: Yellow [3], High: Red [4-5])

f. The ITRA should also consider the effect of aggregate risk on the program and the threat that cumulative or compounding effects of multiple risks pose to successfully satisfying program objectives. The ITRA should consider and document system-of-systems and family-of-systems interactions.

g. The ITRA will document existing program mitigation strategies as well as any additional recommended strategies to mitigate risks and issues. Analysis of mitigation strategies will include whether they are feasible, affordable, and timely, given program circumstances, constraints, and objectives. The assessment will include consideration of mitigation impacts to the overall program schedule and technical performance expectations.

h. Key technical risks will be summarized using a risk matrix with an assessment of the estimated effectiveness of the planned risk mitigations, as shown in Figure 3.

Figure 3. Sample Risk Matrix

4 Distribution Statement A: Approved for public release. Distribution is unlimited.

Abbreviations and Acronyms

DoD ITRA Framework for Risk Categorization

ABBREVIATIONS AND ACRONYMS

APB - Acquisition Program Baseline APUC - Average Procurement Unit Cost ConOps - Concept of Operations CTP - critical technical parameter DoDD - DoD directive DoDI - DoD instruction ITRA - Independent Technical Risk Assessment KPP - key performance parameter KSA - key system attribute KW - kilowatt MDAP - Major Defense Acquisition Program OMS/MP - Operational Mode Summary/Mission Profile PAUC - Program Acquisition Unit Cost PEO - Program Executive Officer PM - Program Manager RDT&E - research, development, test and evaluation RIO - risk, issue, and opportunity TMRR - Technology Maturation and Risk Reduction TPM - technical performance measure UAV - unmanned aerial vehicle

5 Distribution Statement A: Approved for public release. Distribution is unlimited.