SARAH HARTING, DANIEL GONZALES, MICHAEL J. MAZARR, JON SCHMID Comparative Analysis of U.S. and PRC Efforts to Advance Critical Military Technology Volume 1, Analytic Approach for Conducting Comparative Technology Assessments This publication has completed RAND’s quality-assurance process but was not edited. Research Report RAND CORPORATION For more information on this publication, visit www.rand.org/t/RRA2197-1. About RAND The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. To learn more about RAND, visit www.rand.org. Research Integrity Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/principles. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Published by the RAND Corporation, Santa Monica, Calif. © 2024 RAND Corporation RAND® is a registered trademark. Limited Print and Electronic Distribution Rights This publication and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to its webpage on rand.org is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research products for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions. About This Report U.S. lawmakers and defense policymakers need a standard set of processes and approaches for comparing efforts by the United States and the People’s Republic of China (PRC) to advance critical military technologies and to understand the implications for U.S. national security. This report outlines a methodology for comparatively assessing the capacity, progress, and impact of critical military technologies under development in the U.S. and PRC. This approach is designed to be applicable across a diverse range of technologies and is leveraged to conduct comparative analyses of U.S. and PRC efforts to develop and deploy critical technologies for military use. This approach was developed to support Section 1251 of the 2022 U.S. National Defense Authorization Act, which requires U.S. and PRC comparative technology assessments of directed energy systems, hypersonics, emerging biotechnologies, quantum science, and cyberspace capabilities. RAND National Security Research Division This research was sponsored by the Office of the Under Secretary of Defense for Research and Engineering and conducted within the Acquisition and Technology Policy Program of the RAND National Security Research Division (NSRD), which operates the National Defense Research Institute (NDRI), a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense intelligence enterprise. For more information on the RAND Acquisition and Technology Policy Program, see www.rand.org/nsrd/atp or contact the director (contact information is provided on the webpage). Acknowledgments This effort benefited from the guidance and support from the Office of the Under Secretary of Defense for Research and Engineering (OUSD R&E), Office of Strategic Intelligence and Analysis (OSI&A). We thank Eric Kim, Kaitlin Bulavinetz, and Steven Beasley for their leadership and counsel. The RAND team also benefited from collaboration with a range of stakeholders across the Department of Defense (DoD), Intelligence Community (IC), FFRDC/UARC partners, and others in industry and academia. We are grateful for their time and support. We also thank the RAND Acquisition and Technology Policy Program leadership team: Caitlin Lee, Christopher Mouton, Yun Kang, and Aaron Frank. Finally, we are grateful to our RAND colleagues, Jacob Heim and James Bonomo, for their constructive feedback, insightful comments, and encouragement throughout. Summary In this era of strategic competition between the United States and the People’s Republic of China (PRC), emerging critical technologies present both countries with the possibility of disrupting the current balance of forces to achieve significant military advantages. U.S. senior defense and intelligence officials have identified several critical military technologies—such as directed energy, hypersonic systems, emerging biotechnologies, quantum science, and cyberspace capabilities—as not only being essential for national security but also possibly providing a disruptive advantage when applied across warfighting domains. These technologies are in various stages of development in both the U.S. and PRC. While some of the technologies, such as quantum science and biotechnology, appear to be only in the “seed” stages, others—such as hypersonic technologies, directed energy, and cyberspace capabilities—may be further along in transitioning to mature military capabilities that are more-readily available. Lawmakers and defense policymakers need a standard set of processes (or a replicable methodology) for comparative assessments of the U.S. and PRC efforts to advance critical military technologies—and for assessing the implications of those efforts for U.S. national security. At the same time, understanding the complex U.S.-PRC competition underway in each critical technology area and its underlying ecosystem within each country requires the integration of diverse approaches and data sources. The outcomes of these competitions will include not just the development and deployment of military technologies within each country but also, crucial to this assessment, the underlying technological innovation and advancement that can lead to a military advantage on a battlefield. Because all these outcomes—innovation, development, and deployment—will be shaped by the interaction of technological, operational, economic, organizational, and other variables, a multidisciplinary comparative analysis is needed. This report outlines the methodology used for comparatively assessing the capacity, progress, and impact of critical military technologies under development in the U.S. and PRC. This approach, which is designed to be applicable across a diverse range of technologies, is subsequently applied to directed energy systems, hypersonic systems, emerging biotechnologies, quantum science, and cyberspace capabilities. This report and the comparative technology assessments are organized into five research tasks that align with research questions about U.S. and PRC efforts to advance critical military technologies: 1. Establishing an analytic baseline for a critical technology area. We frame why the countries are pursuing the technology, define the technology, scope the technology, identify critical technology elements essential for transitioning technology into fully operational capabilities, and assess current statuses and trends. 2. Assessing research and development (R&D) activities. We refine a generic model of defense innovation, describe and quantitatively or qualitatively measure 15 elements of innovation within R&D, and compare R&D efforts and activities across the two countries in the specified technology areas. 3. Assessing operational effectiveness. For both countries, we define operational problems, define missions/tasks or applications for emerging technologies, assess the relationships of the technologies to the missions/tasks, and assess the operational effectiveness of a critical military technology given multiple factors and considerations. 4. Analyzing countermeasures. For both countries, we identify and describe current and future countermeasures, assess the use of these countermeasures, and assess their related implementation challenges and technical risks. 5. Identifying other notable activities, uncertainties, limitations, and recommendations. In this cross-cutting analysis of the prior tasks, we account for R&D efforts outside of the U.S. and PRC that could advance the critical military technologies, discuss the uncertainties and limitations of the prior tasks, and recommend ways to address the results of the comparative analyses. Contents About This Report ... iii Summary ... iv Figures and Tables ... vii Chapter 1. Establishing an Analytic Baseline for a Critical Technology Area ... 1 Background ... 1 Establishing an Analytic Baseline for a Critical Technology Area ... 2 Framing the Technology ... 3 Defining the Technology ... 3 Scoping the Technology ... 3 Identifying Critical Technology Elements Necessary for Transitioning into Operational Capabilities ... 4 Assessing Current Status and Trends ... 4 Chapter 2. Assessing Research and Development Activities ... 5 Refining a Generic Model ... 5 Developing Metrics for the Model ... 7 Populating the Model ... 7 Comparing R&D Activities ... 7 Chapter 3. Assessing Operational Effectiveness ... 9 Defining Operational Problems and Relevant Missions/Tasks ... 9 Assessing Relationships of Critical Technologies to Missions/Tasks ... 10 Assessing Operational Effectiveness ... 11 Chapter 4. Analyzing Countermeasures ... 14 Identifying and Describing Current and Future Countermeasures ... 14 Assessing the Use of Countermeasures ... 14 Assessing Implementation Challenges and Technical Risks ... 15 Chapter 5. Identifying Other Notable Activities, Uncertainties, Limitations, and Recommendations ... 16 Other Notable R&D Activities ... 16 Caveats to the Comparative Analyses ... 16 Recommended Ways to Address the Results of the Comparative Analyses ... 16 Appendix A. Summary of Stakeholder Engagement, Data Collection, and Documentation ... 17 Stakeholder Engagement ... 17 Data Collection ... 18 Documentation ... 19 Appendix B. U.S. and PRC Variations of the NDIS Model ... 20 National Defense Innovation Process, System, and Contributions ... 20 U.S. National Defense Innovation System ... 26 PRC National Defense Innovation System ... 36 Measuring Defense Innovation by Technological Field ... 52 Abbreviations ... 59 References ... 62 Figures and Tables Figures Figure 2.1. Generic National Defense Innovation System ... 6 Figure 3.1. Expected Outputs of Operational Effectiveness Analysis ... 12 Figure B.1. A General Defense Technology Innovation Process ... 22 Figure B.2. Core Contributions of U.S. NDIS Organizations ... 27 Figure B.3. Core Contributions of PRC NDIS Organizations ... 38 Tables Table 2.1. Summary of Data and Tools Used for R&D Activities Analysis ... 8 Table 3.1. Framework for Assessing the Operational Effectiveness of a Critical Military Technology ... 11 Table A.1. Summary of Data Needs for Comparative Technology Assessments ... 19 Table B.1. Contributions and Examples of U.S. Research Organizations ... 28 Table B.2. Contributions and Examples of U.S. Production Enterprises ... 30 Table B.3. Contributions and Examples of U.S. Government Acquisition Organizations ... 32 Table B.4. Contributions and Examples of PRC Research Organizations ... 41 Table B.5. Contributions and Examples of PRC Production Enterprises ... 44 Table B.6. Contributions and Examples of PRC Government Acquisition Organizations ... 47 Table B.7. Measurement Approach for NDIS Elements ... 58 Chapter 1. Establishing an Analytic Baseline for a Critical Technology Area Background In this era of strategic competition between the United States and the People’s Republic of China (PRC), emerging critical technologies present both countries with the possibility of disrupting the current balance of forces to achieve significant military advantages. U.S. senior defense and intelligence officials have identified several critical military technologies—such as directed energy, hypersonic systems, emerging biotechnologies, quantum science, and cyberspace capabilities—as not only being essential for national security but also possibly providing a disruptive advantage when applied across warfighting domains. These technologies are in various stages of development in both the U.S. and PRC. While some of the technologies, such as quantum science and biotechnology, appear to be only in the “seed” stages, others—such as hypersonic technologies, directed energy, and cyberspace capabilities—may be further along in transitioning to mature military capabilities that are more-readily available. Lawmakers and defense policymakers need a standard set of processes (or a replicable methodology) for comparative assessments of the U.S. and PRC efforts to advance critical military technologies—and for assessing the implications of those efforts for U.S. national security. At the same time, understanding the complex U.S.-PRC competition underway in each critical technology area and its underlying ecosystem within each country requires the integration of diverse approaches and data sources. The outcomes of these competitions will include not just the development and deployment of military technologies within each country but also, crucial to this assessment, the underlying technological innovation and advancement that can lead to a military advantage on a battlefield. Because all these outcomes—innovation, development, and deployment—will be shaped by the interaction of technological, operational, economic, organizational, and other variables, a multidisciplinary comparative analysis is needed. This report outlines the five-step approach used for comparatively assessing the capacity, progress, and impact of critical military technologies under development in the U.S. and PRC. This approach, which is designed to be applicable across a diverse range of technologies, is subsequently applied to directed energy systems, hypersonic systems, emerging biotechnologies, quantum science, and cyberspace capabilities. This report and the comparative technology assessments are organized into five research tasks that align with research questions about U.S. and PRC efforts to advance critical military technologies: 1. Establishing an analytic baseline for a critical technology area. We frame why the countries are pursuing the technology, define the technology, scope the technology, identify critical technology elements essential for transitioning technology into fully operational capabilities, and assess current statuses and trends. 2. Assessing research and development (R&D) activities. We refine a generic model of defense innovation, describe and quantitatively or qualitatively measure 15 elements of innovation within R&D, and compare R&D efforts and activities across the two countries in the specified technology areas. 3. Assessing operational effectiveness. For both countries, we define operational problems, define missions/tasks or applications for emerging technologies, assess the relationships of the technologies to the missions/tasks, and assess the operational effectiveness of a critical military technology given multiple factors and considerations. 4. Analyzing countermeasures. For both countries, we identify and describe current and future countermeasures, assess the use of these countermeasures, and assess their related implementation challenges and technical risks. 5. Identifying other notable activities, uncertainties, limitations, and recommendations. In this cross-cutting analysis of the prior tasks, we account for R&D efforts outside of the U.S. and PRC that could advance the critical military technologies, discuss the uncertainties and limitations of the prior tasks, and recommend ways to address the results of the comparative analyses. Establishing an Analytic Baseline for a Critical Technology Area Fundamental to any assessment of a critical technology area is to establish a baseline from which to conduct the assessment. The first task, outlined in the remainder of this chapter, describes how to establish such an analytic baseline for each critical technology area in each country as a foundation for the comparative assessment. This first task comprises five subtasks: • framing why the U.S. and PRC are pursuing the technology • defining the technology • scoping the technology • identifying critical technology elements for transitioning to fielded capabilities • assessing status and trends. Framing the Technology The comparative analyses of any technology area should start with answering the following question: Why are the U.S. and PRC pursuing this critical technology for military use? Each country is motivated for different reasons related to their national security, and the why behind each country’s national security interest in the technology helps to explain significant differences in the technology development efforts and programs of each country. The why also helps to identify system alternatives that may be available to one side or the other to execute critical defense missions. Therefore, it is important to provide the strategic and operational framing in the context of each country’s intended goals and how the critical technology area supports the goals, because each country may measure success in different ways for specific technology areas. Understanding the why provides the basis for a comparison that is derived by objective analysis instead of unfounded assumptions that each country is conducting a race intended to meet the same goal. Defining the Technology Next, we describe how the U.S. Department of Defense (DoD) and the People’s Liberation Army (PLA) have historically defined the technology area of interest and how those definitions have evolved. To trace this evolution, we conduct a broad literature and program review to understand past work in each technology area within each country’s research and development (R&D) establishments. The U.S. literature review focuses on R&D and program activity in the DoD, the military services, and related entities, including federally funded research and development centers (FFRDCs) and university affiliated research centers (UARCs). The PRC literature review focuses on Chinese-language sources and other related sources. In aggregate, the literature review aims to ensure that the comparative assessment builds upon existing streams of research and reflects how the U.S. and PRC currently view the technology area from a strategic and national security perspective. Scoping the Technology For the comparative analyses, we leverage the literature review and resulting technology definition to scope the technology to current and planned programs or systems within the U.S. and PRC. Specifically, we answer the question: What are the current technology development plans, relevant programs, and system deployment plans for the U.S. and PRC? Identifying Critical Technology Elements Necessary for Transitioning into Operational Capabilities We identify critical technology elements (CTEs), which constitute an essential component or system necessary for transitioning the critical technology into fully operational capabilities. Identifying and understanding CTEs helps inform what critical technology challenges the U.S. and PRC must overcome to deploy these critical technologies effectively as part of a fielded military capability. Because the CTEs refer to the elements needed to deliver military capabilities envisioned by the DoD or the PLA for each technology area, the CTEs play key roles in the subsequent tasks of this analytic approach. Assessing Current Status and Trends Finally, we describe the status and trends of DoD and PLA programs and systems for each critical military technology through quantitative and qualitative analysis. We address several questions: • If programs or military capabilities exist, what is their status for each country? • What are notable differences between the programs or systems that the U.S. is developing and deploying compared with those of the PRC? • What challenges are both countries facing in the development and deployment of these critical military technologies? • What do current trends suggest for U.S. and PRC efforts in the technology area of interest? For each critical technology area, the collection of information gathered in Task 1—on the strategic framing, technology definitions, CTEs, current plans and programs and the status of those—serves as the analytic baseline underpinning subsequent analytic tasks for the comparative technology assessment of the U.S. and PRC. Chapter 2. Assessing Research and Development Activities To effectively transition technologies into fielded military capabilities requires the involvement of dozens—or even hundreds—of distinct organizations and efforts over a sustained period. This organizational R&D ecosystem, or the national defense innovation system (NDIS), is the set of actors, linkages, and the country-level contextual environment in which they operate, that work to develop new defense technology. For this task (Task 2 in our approach), we define the NDISs for the U.S. and PRC. We then focus on how each country deploys its NDIS to advance the research and development (R&D) of a specific technology area military use. We then empirically assess the activity of NDIS organizations across three dimensions: research, development, and fielding. We conclude with a comparative R&D assessment for the U.S. and PRC to understand how effective these countries have been at transitioning technologies for a critical technology area into fielded capabilities. In short, Task 2 comprises four subtasks: • refining a generic model • developing metrics for the model • populating the model • comparing R&D activities. Refining a Generic Model To assess R&D activities in the U.S. and PRC in a comparative fashion, we begin with a generic theoretical model known as the national defense innovation system (NDIS). As noted in the definition above, the NDIS can represent any network of actors that transforms and recombines resources, ideas, knowledge, and existing technology into a new military technology. We then refine conceptual models of the U.S. and PRC NDIS processes so that they are empirically grounded—and thus reflecting the way each country currently selects, develops, and fields military technologies in practice. Each country’s model, however, remains sufficiently generic to allow its application to a full range of critical technology areas. Figure 2.1 depicts a country-agnostic NDIS to illustrate the type of model we use. It comprises a set of actors, contextual factors, and linkages. The exact numbers and kinds of these components are determined by assessing how each country develops its military technologies. Each critical technology area also drives which system components are emphasized. For example, technologies such as microelectronics and mobile communications (i.e., 5G/FutureG) are developed primarily by global commercial enterprises, whereas hypersonic technology is advanced largely by key government organizations and defense contractors. Meanwhile, broad areas of technology, such as biotechnology, depend on combinations of research organizations, government laboratories, and commercial enterprises. [Figure 2.1: Generic National Defense Innovation System: Inputs include Demand Pull, Technology Push, Government Policy (Funding, Political Resources), Foreign Inputs; National Defense Innovation System actors include Research Orgs, Production Enterprises, Gov. Acquisition Orgs, embedded within Contextual Factors and connected via Linkages; Output is New Military Technology transitioning to Military End Users.] Developing Metrics for the Model The purpose of refining country-specific NDIS models is to ensure that the R&D assessment accurately reflects how each country undertakes the military technology innovation process for each critical technology area. In this subtask, we develop measurement strategies for assessing the NDIS components. Where possible and appropriate, we measure them using quantitative metrics. For a given CTE, the linkage between Research Organizations and Government Acquisition Organizations might be measured by the number of scientific publications co-authored by contributors from these organization types. Enterprises might be assessed by patent counts or market shares. In certain cases, qualitative assessments will likely be appropriate. Comparative assessments of defense acquisitions systems are likely to be based, at least in part, on qualitative assessments of system efficacy or on qualitative intelligence assessments. Populating the Model Once we have measurement strategies for all NDIS components for the U.S. and PRC, we populate each country’s model with data. For the most part, we measure at the level of a CTE. For certain CTEs, it is suitable to measure them at a higher level of aggregation (e.g., at the level of the capability or of an entire critical technology area). For example, a country’s institutional and legal context may remain constant over the whole technology area and thus may be appropriately assessed at the level of the area rather than that of the granular CTE. If necessary, we further refine the NDIS model to assure its “fit” for the technology area in question. Technologies vary regarding the testing and evaluation infrastructure required for development and the role of the commercial sector in driving technological progress. The secondary refinement process ensures that the model and its measurements reflect such heterogeneities. Comparing R&D Activities The output is a pair of data-populated models for CTEs within each technology area of interest. For certain CTEs, such as the number of active CTE researchers in each country, the metrics are commensurable (i.e., measurable by the same standard). For other CTEs, comparison depends on analytical judgment. Therefore, for each technology area, we engage subject matter experts (SMEs) through interviews, surveys, or workshops. SME engagements often reveal important caveats and qualifications. For example, both the U.S. and PRC are global leaders in quantum technology, but experts clarify that the PRC leads in quantum communications and quantum key distribution while the United States leads in quantum computing. The final stage of research on R&D activities compares each country’s NDIS measurements, as informed by analytical judgments and SME engagements, to arrive at an overall comparative assessment of each critical technology area for the U.S. and PRC. Table 2.1. Summary of Data and Tools Used for R&D Activities Analysis - Web of Science: Scientific publication data (English language) | Metric Development, Model Population - Scopus: Scientific publication data (English and Chinese language) | Metric Development, Model Population - China National Knowledge Infrastructure: Scientific Publication Data (Chinese language) | Model Refinement, Model Population - Derwent Innovation Index: Patent data (all major patent jurisdictions) | Metric Development, Model Population - World Intellectual Property Organization: Patent data | Metric Development, Model Population - Vantage Point: Data mining software | Metric Development, Model Population - Bibliometrix (R Package): Bibliometric analysis software | Metric Development, Model Population - Vos Viewer: Bibliometric analysis software | Metric Development, Model Population - Gephi: Social network analysis software | Metric Development, Model Population - RANDlex: Natural language processing software (RAND-developed) | Metric Development, Model Population - Polyglot: Chinese-language translation tool (RAND-developed) | Model Refinement, Model Population Chapter 3. Assessing Operational Effectiveness The U.S. and PRC are dedicating significant resources to developing and advancing critical technologies for military use. To understand the potential implications of these current and planned efforts relative to stated goals and objectives, we assess their operational significance and effectiveness. We define operational significance as the potential for a given technology area to make unique contributions to operational outcomes, in general or in specific missions/tasks, in cost-effective ways. At its most essential, operational significance is a function of the roles and missions a system performs as well as the effectiveness with which it performs those missions. This chapter describes how we assess the operational significance and effectiveness of each technology for each country (Task 3 in our approach). Task 3 comprises three subtasks: • defining operational problems for each country and the corresponding missions/tasks for each critical technology • assessing the relationships of the critical technologies to the missions/tasks, including consideration of alternative tools and capabilities • assessing the operational effectiveness of the critical technologies (using the framework described below in Table 3.1). Defining Operational Problems and Relevant Missions/Tasks This task is premised on specifying what operational problems the U.S. and PRC are trying to solve—that is, what motivates their pursuit of the critical military technology. Since the comparative focus is on the PRC, we limit U.S. operational problems to those relevant to the Indo-Pacific region. Once a comprehensive list of operational problems is identified for both the U.S. and PRC, the team assembles a set of relevant missions/tasks that the technologies can support. This component of the analysis is a variant of the classic strategies-to-tasks analytical approach. Assessing Relationships of Critical Technologies to Missions/Tasks For each U.S. and PRC mission-task, we identify objective measurements of the necessity and significance of the critical technologies for generating operational effectiveness. These measurements offer a repeatable approach for assessing the operational significance of many kinds of critical military technologies. The measurements incorporate four elements: 1. General Assessment: We list the criteria for assessing the utility and significance of each technology of interest for fulfilling each mission-task. We use the criteria to evaluate each application of the technology. The criteria encompass these variables: general predicted operational effects, geographic range of effects, defense industrial base (DIB) status, DIB production rates, mission implications, predicted utility and reliability over a given timeframe, survivability assumptions, targeting assumptions, and lethality assumptions. 2. Alternative Tools and Capabilities: The importance of each military system or capability associated with a critical military technology is a function, in part, of its uniqueness for solving operational problems. What missions, if any, can be performed by only these military systems or capabilities? What other systems or capabilities can perform the missions? For the United States, is it possible to develop a joint package that addresses the operational problems without relying on these critical military technologies? Is the PRC potentially more dependent on these critical military technologies than the United States? 3. Challenges in Achieving Desired Operational Effects: The team’s technical experts consult with technical experts in relevant DoD offices, FFRDCs/UARCs, defense firms, and intelligence agencies to construct quantifiable measurements of the probability of success in achieving missions/tasks by using the critical technologies. For instance, a confidence index can measure each military system’s ability to perform. 4. Cost: We assess each system’s cost, and whether the cost curve will bend over time. Table 3.1. Framework for Assessing the Operational Effectiveness of a Critical Military Technology Baseline Assessment: - What missions/tasks do the systems serve? How essential are those tasks to campaign and strategic success?: Identify purposes and missions on the strategies-to-tasks spectrum: national military objectives, campaign objectives, operations objectives, and operational tasks. - General effectiveness score of weapons relative to missions: For each mission a system will perform, can the weapons perform the mission reliably; has the country demonstrated the system in exercises or in use; does the system have the necessary personal/talent to operate it; do its testing outcomes validate effectiveness? Added Considerations: - Secondary operational effect value: Apart from the primary mission, what operational effects can be identified that affect the overall campaign; what is the weapons’ value in enhancing the holistic effect of U.S. operations? - Adversary perceptions: To what degree will a potential adversary be influenced by the presence of the weapons, either being deterred from war or shifting operational plans? Does the system allow actions in peace or crisis with signaling value? - Substitutability vs. redundancy: What other systems could fulfill the intended missions? How do those capabilities relate to a value of redundancy for certain missions? - Dependence on other systems: How reliable and resilient are the other systems on which this system will rely for its effectiveness, such as targeting? - Production and industrial considerations: Can the defense industrial base produce the necessary numbers over time; does the United States have sufficient technical personnel for the task; and is the weapons dependent on foreign technology or materials? - Vulnerability: Are the launching or operational deployed systems vulnerable to strike or other interference (cyber)? - Cost: What is the cost per item, for both development and production, relative to meeting the operational objectives? Categories of Operational Significance: • Decisive operational significance: A system that can make war-winning contributions on its own. • Critical operational significance: A system that could provide essential contributions to specific missions/tasks but serve as only one of several or many systems essential for warfighting success. • Moderate to marginal operational significance: A system that could help achieve operational success but could theoretically be substituted for other systems and/or perform missions/tasks that are not indispensable to success. Chapter 4. Analyzing Countermeasures An actor with substantial technology resources will likely develop countermeasures to a potential adversary’s current or planned capabilities. Such countermeasures to defeat these capabilities could be entirely new systems or modifications of existing systems. As weapon systems relying on new technologies become more precise and focused on specific targets, it becomes possible for an actor to develop countermeasures that defeat the precision and focus of such capabilities. Task 4 in our approach is to determine whether the U.S. and PRC have developed and implemented—or will develop and implement—countermeasures to current and planned capabilities in a technology area of interest. Task 4 comprises three subtasks: • identifying and describing current and future countermeasures • assessing the use of countermeasures • assessing implementation challenges and technical risks. Identifying and Describing Current and Future Countermeasures We use a literature review and engagements with various subject matter experts within DoD, defense and intelligence agencies, industry, FFRDCs, and UARCs to identify, assess, and understand countermeasures of interest to the U.S. and PRC in a technology area of interest. For the analysis of PRC’s countermeasures, we also survey Chinese-language publications from the PLA and state-linked academic, industry, and scientific organizations. Based on these efforts, we determine what countermeasures exist for a military technology and the status of such countermeasures pursued by the U.S. and PRC. Assessing the Use of Countermeasures We analyze how the U.S. and PRC intend to use or are using countermeasures against a critical technology. We rely on analyses and assessments from the DoD and other defense and intelligence agencies pertaining to U.S. and PRC countermeasures in each technology area. We also review Chinese-language publications from the PLA and state-linked academic, industry, and scientific journals to inform our understanding of PRC activities. If available, we include data on countermeasure status, schedules, and budgets. Assessing Implementation Challenges and Technical Risks To identify and characterize implementation challenges and technical risks, we review available operational and technical assessments from a variety of sources on DoD and PRC countermeasure activities. If possible, this includes a review of Chinese-language publications from the PLA and state-linked academic, industry, and scientific journals. Chapter 5. Identifying Other Notable Activities, Uncertainties, Limitations, and Recommendations A final cross-cutting task takes account of other notable R&D activities, caveats to the comparative analyses, and recommended ways to address the results of the comparative analyses. Other Notable R&D Activities Based on the analysis conducted in earlier tasks, this section aims to address two questions: • What significant R&D efforts outside of the U.S. and PRC could advance the critical military technologies of interest? • What are other relevant R&D efforts for the critical military technologies of interest? Our approach to answering these questions consists of the following: • collecting data on notable international R&D activities through discussions with, and data provided by, DoD R&E stakeholders in their respective critical technology areas • reviewing current and historical international agreements or partnerships by area • for the countries identified through stakeholder discussions and literature review, selectively measuring R&D activity (e.g., high-quality publications and patents) within those countries to gauge their activities in a critical technology area • identifying and assessing issues and opportunities for, and barriers to, collaboration. Caveats to the Comparative Analyses We also apply appropriate caveats to the comparative analyses by: • identifying areas of uncertainty • describing limitations, constraints, and challenges in conducting the analyses. Recommended Ways to Address the Results of the Comparative Analyses The comparative assessment is largely diagnostic in nature, providing an informed understanding of U.S. and PRC efforts to develop and deploy critical technologies for military use. However, informed by the findings from the comparative analyses and the identified caveats, we conclude the comparative assessment with recommended DoD activities. These recommended activities could include development efforts, analytic efforts, or data collection requirements to support future assessments of a specific technology area. Additionally, the recommendations could point toward opportunities for DoD to shape efforts in a critical technology area to yield greater strategic and operational advantages. Appendix A. Summary of Stakeholder Engagement, Data Collection, and Documentation Critical military technology areas are diverse, ranging from seeds of emerging opportunity (e.g., quantum, biotechnologies) to effective adoption areas where there is widespread commercial activity (e.g., artificial intelligence and autonomy, microelectronics) to defense-specific areas (e.g., directed energy, hypersonics). Similarly, strategic shocks can shape the technology or security landscape in new or unexpected ways. Efforts to understand how the U.S. and PRC advance critical technologies for military use require deep and broad expertise that accounts for the inherent similarities and differences in national R&D ecosystems, national defense and security priorities, and uncertainties or new areas of emerging development. To be sure, the outcomes of these technology competitions and of how each country develops and deploys military technologies will be shaped by a confluence of technological, operational, economic, demographic, organizational, and other factors—which need to be accounted for as part of the underlying approach. While a general analytical approach can be used for each comparative technology assessment, the specific types of data, metrics, and methods may vary. This appendix outlines a basic approach for stakeholder engagement, data collection, and documentation efforts for conducting a comparative technology assessment. Stakeholder Engagement Stakeholder engagement is required throughout the assessment to support initial scoping, data collection and analysis, and the review and coordination of the analysis and findings. To begin, each comparative assessment requires a deliberate partnership up-front between USD(R&E) and the lead research organization to understand (1) existing methods, metrics, and data that can be used to conduct the assessment, (2) gaps that can be accounted for with new methods, metrics, or data, and (3) limitations, uncertainties, and risks that may not be resolvable within the timeframe of the study but could inform development efforts or requirements to support future assessments. Based on this understanding, USD(R&E) will support the lead research organization’s efforts to obtain relevant data in a timely manner to conduct the comparative assessment. The lead research organization will work with USD(R&E) to develop a plan for engaging a broad range of experts to inform the different tasks. Such experts will include the following: • DoD and intelligence community professionals who can provide authoritative information or assessments on U.S. and PRC efforts and activities in a critical technology area • technologists who understand the fundamental science and engineering associated with the critical technology area and the technological challenges associated with developing and transitioning the technology into operational capabilities • regional experts who can identify and evaluate country interest and activities associated with the critical technology area, the organizational R&D ecosystem that underpins the ability to develop and deploy these technologies, and the challenges a country faces in developing and deploying these technologies • military experts who understand the types of missions to which these technologies can apply, particularly relative to other alternatives, and the potential counters for those technologies • R&D and industrial base experts who can identify and evaluate U.S. and PRC R&D activities associated with the critical technology area, the organizational R&D ecosystem that underpins the ability of both countries to develop and deploy these technologies, and the challenges these countries face in developing and deploying these technologies. Table A.1. Summary of Data Needs for Comparative Technology Assessments - Task 1: Baseline | Data on strategic framing (DoD strategic guidance, Congressional testimony, technology roadmaps; PLA guidance, open-source assessments of PRC primary sources); Data on current/planned programs. - Task 2: R&D Activities | Data on R&D organizations and activities for critical technologies; Assessments of development and operational T&E activities and infrastructure; R&D assessments by DoD, FFRDCs/UARCs. - Task 3: Operational Effectiveness | Data on U.S. and PRC operational use and effectiveness of critical technologies; Assessments of operational contexts and effectiveness. - Task 4: Countermeasures | Data on current/planned countermeasures (programs, technology protections, or other related activities) of the U.S. and PRC. - Task 5: International R&D Activities | Data on international agreements, partnerships, and areas of focus between the U.S., PRC, and other countries in a critical technology area. Documentation The intended output of the comparative analysis is a document that includes: • an executive summary, which summarizes findings and recommendations from the comparative assessment; • an introduction, which defines the critical technology and provides background and context related to the importance of this technology for each country; • an analytic technology baseline of current and planned efforts for each country, to include status and trends; • an R&D assessment of the organizational ecosystem and activities underway for this critical technology; • an assessment of operational effectiveness; • an assessment of countermeasures; • a description of other notable international R&D activities for this critical technology. Appendix B. U.S. and PRC Variations of the NDIS Model This appendix describes how we define and measure national defense innovation. In short, we base the definition and measurements on a country’s R&D activities and its transition of those activities to military applications. A country can attain new defense technology in two ways: by importing the technology from abroad or by developing the technology domestically. Importing technologies or subcomponents from partners and allies to fill critical gaps could offer important advantages for strategic competition. However, importing defense technology from abroad could forestall a country’s ability to attain global technological advantage for a given technology because the imported technology will already be held by the exporting country. Achieving global technological leadership in the long-term is likely to require indigenous defense technology innovation. Leveraging technology leadership to advance military capabilities requires additional organizations, concepts, processes, and resources (workforce and funding) to develop and field capabilities in operationally relevant numbers to achieve critical military advantages. National Defense Innovation Process, System, and Contributions To conceptualize the process of attaining defense technology innovation, we begin with the process output (new defense technology) and ask the question: How is this output generated? Defense technology is defined here by its relationship to its end-user; we define defense technology as technology that is intended to be used by country-level armed forces. A new defense technology can be initiated by either demand pull or technology push. In the case of demand pull, the motivation to develop a new defense technology is the result of an identified military need. In the technology push case, the discovery process precedes the identification of a military application. Before an identified military need can result in material inputs to a government-funded science and technology (S&T) effort, the need must be assessed for utility and translated into action by some government policy. The outcome is a set of three resource inputs: Funding, Political resources, and Foreign inputs. Functional Contributions to a National Defense Innovation System: - Research Organizations: The primary functional contribution of research organizations to the NDIS is to generate, scrutinize, mature, and diffuse new defense-relevant scientific or technical knowledge. - Production Enterprises: The principal functional contribution of production enterprises to the NDIS is to develop and produce new military technology and deliver it to the end user. - Government Acquisition Organizations: The principal functional contribution of government acquisition organizations is to establish the rules for participation in the NDIS, set technology priorities, communicate demand, provide oversight, and allocate resources. - Contextual Factors: Institutional conditions (e.g., norms and laws, IP regimes, contract law) under which NDIS actors operate. - Linkages: Formal and informal channels by which NDIS actors transmit and receive resources and information. U.S. National Defense Innovation System: - Research Organizations: Research universities (Stanford, MIT, Univ of Michigan, Georgia Tech), Government research orgs (NRL, AFRL, ARL, DARPA), FFRDCs/UARCs (MIT Lincoln Labs, GTRI, MITRE, RAND NDRI). - Production Enterprises: Defense prime contractors (Lockheed Martin, Northrop Grumman, Raytheon, Boeing, General Dynamics), Component producers (Federal Electronics, Eaton Metal Products, Bonney Forge), Nontraditional contractors (Palantir, Epirus, Anduril), Public manufacturers (Watervliet Arsenal, Crane Army Ammunition Activity, Norfolk Naval Shipyard). - Government Acquisition Organizations: Defense innovation intermediaries (DIU, AFWERX, Hacking for Defense), OSD (OUSD(R&E), OUSD(A&S), CAPE), Military services (Army, Navy, Air Force), Combatant commands (CENTCOM, INDOPACOM, NORTHCOM). PRC National Defense Innovation System: - Cross-cutting Actors: Chinese Communist Party (CCP) exerts ubiquitous control; State Administration of Science, Technology, and Industry for National Defense (SASTIND) regulates and directs defense S&T, administers Seven Sons of National Defense and defense labs, and manages CMF licensing. - Research Organizations: Government research orgs (Chinese Academy of Sciences / CAS, Jinan Institute of Quantum Tech, etc.), PLA-affiliated universities (Seven Sons of National Defense including Beihang, BIT, Harbin Eng, Harbin Inst of Tech, NUAA, NUST, NWPU; military universities like NUDT, AMS), Civilian research universities (Zhejiang Univ, SJTU, USTC, Fudan), State Key Labs & Defense Key Labs. - Production Enterprises: State-owned enterprises (CASC, AVIC, Norinco, CASIC), "Private" enterprises (DJI, Huawei, Tencent, Cloudwalk, Megvii, Xi'an Aisheng, AEE Technology). - Government Acquisition Organizations: Central Military Commission (CMC; Equipment Development Department / EDD, Science and Technology Commission / CSTC), Armed services (PLA Army, Navy, Air Force, Rocket Force), Theater commands (Eastern, Southern, Western, Northern, Central). Measuring Defense Innovation by Technological Field (15 Elements across 3 Dimensions): 1. Research: 1.1 Scientific output (annual WOS publications) 1.2 Organizational activity (top publishing orgs & collaboration network centrality) 1.3 Conference activity (regular conferences producing proceedings) 1.4 Superstar researchers (authors in top 0.1% H-index) 1.5 Research centrality (eigenvector & degree centrality in co-authorship networks) 2. Development: 2.1 Program activity (official military development programs) 2.2 Technological output (patent grants in focal field) 2.3 Development organization ecosystem (patent counts by org & qualitative activity) 2.4 Testing infrastructure (major testing/S&T facilities) 2.5 Critical technology element development (patents & activities per CTE) 2.6 Development funding (public/contracting spending) 3. Fielding: 3.1 Fielded systems (completed systems, operational prototypes, IOC programs) 3.2 Force structure (quantity and distribution across units) 3.3 International sales (arms exports value/quantity) 3.4 Technology transition (qualitative assessment of transition from R&D to fielded capabilities) Abbreviations: AFRL: Air Force Research Laboratory ARL: Army Research Laboratory AVIC: Aviation Industry Corporation of China BA: budget area CALT: China Academy of Launch Vehicle Technology CAPE: Cost Assessment and Program Evaluation CAS: Chinese Academy of Sciences CASC: China Aerospace Science and Technology Corporation CASIC: China Aerospace Science and Industry Corporation CCMD: combatant command CCP: Chinese Communist Party CMC: Central Military Commission CMF: civil-military fusion CMOS: complementary metal oxide semiconductor COSTIND: Commission for Science, Technology, and Industry for National Defense CSTC: Central Science and Technology Commission CTA: critical technology area / critical technology assessment CTE: critical technology element DARPA: Defense Advanced Research Projects Agency DAS: Defense Acquisition System DIB: defense industrial base DII: defense innovation intermediary DIU: Defense Innovation Unit DoD: Department of Defense DoDD: Department of Defense Directive DoDI: Department of Defense Instruction DTIC: Defense Technical Information Center ECC: equipment contractor certificate EDD: Equipment Development Department FFRDC: federally funded research and development center GAD: General Armaments Department HHI: Herfindahl-Hirschman Index IADB: international agreement database IC: intelligence community IOC: initial operational capability IP: intellectual property JCIDS: Joint Capabilities Integration and Development System KPP: key performance parameter MIT: Massachusetts Institute of Technology MOST: Ministry of Science and Technology NDAA: National Defense Authorization Act NDIS: national defense innovation system NDRI: National Defense Research Institute NETRC: National Engineering Technology Research Center NIS: national innovation system NNSFC: National Natural Science Foundation of China NPC: National People’s Congress NRL: Naval Research Laboratory NSRD: National Security Research Division OSD: Office of the Secretary of Defense OSI&A: Office of Strategic Intelligence and Analysis PLA: People’s Liberation Army PLAA: PLA Army PLAAF: PLA Air Force PLAN: PLA Navy POM: Program Objective Memorandum PPBE: Planning, Programming, Budgeting, and Execution (system) PRC: People’s Republic of China R&D: research and development S&T: science and technology SASTIND: State Administration of Science, Technology, and Industry for National Defense SKL: state key lab SME: subject matter expert SOE: state-owned enterprise STEM: science, technology, engineering, and mathematics T&E: testing and evaluation UARC: university affiliated research center UAS: unmanned aerial system USD(A&S): Undersecretary of Defense for Acquisition and Sustainment USD(R&E): Undersecretary of Defense for Research and Engineering WERDPL: Weapons and Equipment Research, Development, and Production License Catalog WERPL: weapons and equipment research and production license WOS: Web of Science References include works by Alic, Badinger & Tondl, Bellais & Guichard, Biddle, Cheung, Etzkowitz & Zhou, Fowler, Freeman, Gholz & Sapolsky, Horowitz, Isaacson et al., Lundvall, Nelson, North, Popper et al., Rosen, Schmid et al., Schmoch et al., Sullivan et al., Thaler, Van Atta et al., and Weinbaum et al.