U.S. Department of Energy Office of Inspector General Office of Audits and Inspections Audit Report Cooperative Research and Development Agreements at National Nuclear Security Administration Laboratories OAS-M-13-02 March 2013 Department of Energy Washington, DC 20585 March 15, 2013 MEMORANDUM FOR THE ACTING ADMINISTRATOR, NATIONAL NUCLEAR SECURITY ADMINISTRATION FROM: Rickey R. Hass, Deputy Inspector General for Audits and Inspections, Office of Inspector General SUBJECT: INFORMATION: Audit Report on "Cooperative Research and Development Agreements at National Nuclear Security Administration Laboratories" BACKGROUND The dissemination of technology developed by the Department of Energy's national laboratories to the general science community and the public, is one of the Department's top priorities. In 1989, the National Competitiveness Technology Transfer Act established technology transfer as a Federal mission and authorized government-owned, contractor-operated laboratories to use Cooperative Research and Development Agreements (CRADAs) to facilitate the development and transfer of technology to the private sector. In Fiscal Year (FY) 2011, the National Nuclear Security Administration (NNSA) reported it had 272 active CRADAs. NNSA site offices located at each laboratory are responsible for ensuring that laboratories obtain final reports documenting the results of research and any new inventions or technology, and forward copies of the reports to the Department's Office of Scientific and Technical Information (OSTI). Ultimately, OSTI is responsible for preserving the scientific and technical information and making this information readily available to the scientific community and to the public. In our previous audit report, Cooperative Research and Development Agreements at the Department of Energy's Office of Science Laboratories (IG-0826, September 2009), we found that Office of Science laboratories had not always received final reports from researchers nor transmitted the reports to OSTI. Management committed to address those issues. We initiated this audit to determine whether the NNSA was effectively managing the use of CRADAs at its laboratories. CONCLUSIONS AND OBSERVATIONS We found that NNSA laboratories were managing the use of selected CRADA activities that we tested in an effective manner. For example, we found that the three laboratories we visited generally met the requirements for CRADAs. Furthermore, our review found that the Joint Work Statements and CRADA Statements of Work clearly described tasks to be performed, the division of work, and the potential benefits to the Department and the U.S. economy. The work generally appeared to be collaborative in nature between the laboratories and the partners. However, we found that controls could be improved in the area of obtaining and disseminating CRADA results. Specifically, we found that NNSA laboratories had not always: • Obtained final reports from researchers for completed and terminated projects, and, • Forwarded the obtained reports to OSTI for dissemination. Meeting Reporting Requirements The Department's CRADA Manual requires the laboratories and industry partners to produce a final report as a deliverable for each completed or terminated CRADA project. The laboratories are also required to send a copy of all final reports to OSTI for dissemination to the scientific community and to the general public. Dissemination of scientific and technical information is statutorily required under several public laws, including the Atomic Energy Act of 1954 and the Department of Energy Organization Act of 1977. The Department's CRADA Manual states that the contractor must ensure that deliverables, such as the final reports, are provided to OSTI to ensure that the results of CRADAs are made known to other Department contractors for Departmental program needs. Of the 211 CRADAs completed or terminated from FYs 2008 to 2012 at the laboratories we visited, we selected 40 for review. The following table illustrates the results of our review of the laboratories' compliance with reporting requirements. Table 1: Sample of Completed or Terminated CRADAs (Fiscal Years 2008 - 2012) - Los Alamos: OIG Sample = 14, Final Reports Received = 0, Final Reports Sent to OSTI = 0 - Sandia: OIG Sample = 14, Final Reports Received = 3, Final Reports Sent to OSTI = 0 - Livermore: OIG Sample = 12, Final Reports Received = 12, Final Reports Sent to OSTI = 7 - Totals: OIG Sample = 40, Final Reports Received = 15, Final Reports Sent to OSTI = 7 Controls over Final Reports Reporting issues occurred because the NNSA site offices had not consistently overseen CRADA activities at the three national laboratories we reviewed. In particular, the site offices had not always ensured that the laboratories had implemented policies and procedures related to obtaining final reports and transmitting final reports to OSTI and that such policies and procedures were effective. Los Alamos Site Office did not ensure that the Los Alamos National Laboratory (Los Alamos) had policies and procedures to obtain final reports from researchers and transmit the reports to OSTI. Los Alamos technology transfer personnel stated that, although a final report was requested from researchers as part of the closeout procedures, they were not required to obtain the report prior to closing out the CRADA. Los Alamos personnel stated that the laboratory had no procedure for sending final reports to OSTI when a CRADA was completed or terminated. We were told that, while this may have been done in the distant past, it had not been done in recent years. As a result, none of the 52 CRADAs completed or terminated during FYs 2008 to 2012 had final reports archived at OSTI for dissemination. Similarly, prior to 2010, the Sandia Site Office was unaware that Sandia National Laboratory (Sandia) was not always obtaining final reports and did not have a policy to transmit reports to OSTI. Sandia identified that it was not receiving final reports when a 2010 self-assessment of its technology transfer programs found that waiting for these reports was delaying its closeout process. To improve the timeliness of its closeouts, in August 2010, a Sandia management official decided to waive the requirement for a final report for those projects awaiting closeout at that time, and therefore, no final reports were sent to OSTI. The Sandia Site Office concurred with this action, even though obtaining final reports was required under Department's CRADA Manual. We found that the requirement for a final report had been dropped in 11 of the 14 completed CRADAs we reviewed. In two cases, Sandia returned funds provided by the partner, $28,763 in one case, and $45,367 in the other, even though a final report had not been produced from the work. Had this funding been retained, the researchers may have prepared and provided the final report. Sandia officials told us that these two projects had been terminated before the research was completed; however, the Department's Manual requires that final reports be produced for all completed or terminated CRADAs. Sandia officials told us that since August 2010, they had required final reports to be received and transmitted to OSTI for every closed CRADA. We were provided with OSTI transmittal notices indicating that Sandia had been sending recently received reports to OSTI. However, while Sandia did maintain department-level procedures to disseminate reports to OSTI, we noted that the requirement to submit final reports to OSTI had not been codified as part of the official Sandia corporate procedures for CRADAs. Additionally, although we found that Sandia had received three final reports from previously closed CRADAs, these reports had not been transmitted to OSTI. In fact, Sandia management explained that prior to the actions taken in August 2010, Sandia did not submit any final reports to OSTI. As a result, none of the 71 CRADAs completed in FY 2008 and FY 2009 had final reports archived at OSTI, even if Sandia had received the final report. Sandia officials explained that it may not be feasible to obtain final reports for some completed or terminated CRADAs because of the lapse of time, missing researchers or the unavailability of funds. Sandia's concerns about the feasibility of obtaining final reports for CRADAs due to the lapse of time is valid and highlight the importance of obtaining timely final reports. Finally, the Livermore Site Office was active in overseeing Lawrence Livermore National Laboratory (Livermore), which may explain the performance of the laboratory in obtaining final reports from researchers. We were told that the Livermore Site Office received monthly and quarterly status reports on all CRADA projects plus the final reports published by Livermore. However, we were told that the Livermore Site Office only performs random checks of transmissions to OSTI, an area where we found Livermore still had problems. While Livermore Site Office personnel stated that they were aware of past problems in this area and had increased the frequency of checks, they stated that there was a backlog of information to transmit to OSTI. Our review of the reports that had not been sent to OSTI determined that four of the five final reports had been published in 2011, while the fifth was published in 2009. The average age of the reports was 456 days as of the date of our site visit. Livermore informed us that it had transmitted the five final reports to OSTI 3 weeks after our site visit that identified the reports. Livermore officials told us that they have made procedural modifications to ensure that CRADA final report submittals to OSTI are expedited. Dissemination of Scientific Results The Department has stated that accelerating the dissemination of research and development information serves to accelerate the pace of scientific progress itself. By not ensuring that its laboratories obtained and disseminated final reports, the NNSA has not ensured that the scientific and technical information generated by laboratory research was available throughout the Department, as well as the scientific community and the public. This process serves, as well, to document new inventions and breakthrough technologies resulting from CRADA efforts and reduce redundant research by ensuring that historical information is widely disseminated. As we stated in our previous report, these reports are not merely evidence of the work product. Instead, they serve to memorialize the technical approach and accomplishments under the CRADA, which is a vital element in the effort to transfer the technology as effectively and expeditiously as possible. RECOMMENDATIONS We recommend that the Acting Administrator, NNSA, direct all site offices to: 1. Verify that the laboratories establish policies to obtain final reports from researchers for all completed or terminated CRADAs and transmit them to OSTI; 2. Periodically review whether the laboratories are receiving and promptly transmitting final reports to OSTI; and, 3. Require the laboratories to determine whether final reports have been received on previously completed or terminated CRADAs, and ensure that any existing final reports are transmitted to OSTI. MANAGEMENT REACTION AND AUDITOR COMMENTS Management concurred with our recommendations and proposed corrective actions designed to address our concerns. Management's response identified specific actions and timeframes for addressing the recommendations that are responsive to the recommendations. As a result, a Management Decision will not be required; however, the recommendations will remain open until corrective actions are completed. Management's comments are included in Attachment 3. Attachments cc: Deputy Secretary Chief of Staff General Counsel OBJECTIVE, SCOPE AND METHODOLOGY OBJECTIVE The objective of this audit was to determine whether the National Nuclear Security Administration (NNSA) is effectively managing the use of Cooperative Research and Development Agreements (CRADAs) at its laboratories. SCOPE We conducted the audit from March 2012 to March 2013 at Sandia National Laboratory (Sandia) in Albuquerque, New Mexico; Lawrence Livermore National Laboratory (Livermore) in Livermore, California; and Los Alamos National Laboratory (Los Alamos) in Los Alamos, New Mexico. The scope of the audit covered CRADAs in effect during Fiscal Years (FYs) 2008 through 2012. METHODOLOGY To accomplish our audit objective, we: • Reviewed laws, regulations and Department of Energy (Department) directives and manuals related to CRADAs; • Identified and reviewed laboratory policies and procedures; • Held discussions with responsible Department and Laboratory personnel; • Reviewed accounting records for adequate funding and verification of licensing income distribution; and, • Judgmentally selected and reviewed a sample of 60 out of 411 CRADAs active during FYs 2008 through 2012, 20 each at Sandia, Lawrence Livermore, and Los Alamos. At the time of our review, 200 of the CRADAs were active and 211 were completed or terminated. Our sample included 40 completed or terminated CRADAs. Attributes considered in selecting our sample included the amount of CRADA funding, the type of partner, whether a small business, large corporation or educational institution, and the number of CRADAs the partner had with the individual laboratory. We tested compliance with Department requirements in the areas of competition, work descriptions, collaboration and accounting, as well as obtaining and disseminating results. Because the sample was not statistical, we could not project the sample results to the population. We conducted this performance audit in accordance with generally accepted Government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Accordingly, we assessed significant internal controls and compliance with laws and regulations necessary to satisfy the audit objective. In particular, we assessed the implementation of the GPRA Modernization Act of 2010 and found that NNSA had not established performance measures for CRADAs. Because our review was limited, it would not necessarily have disclosed all internal control deficiencies that may have existed at the time of our audit. Finally, we conducted an assessment of computer-processed data relevant to our audit objective and found it to be reliable. Management waived an exit conference. PRIOR REPORTS • Audit Report on Cooperative Research and Development Agreements at the Department of Energy's Office of Science Laboratories (DOE/IG-0826, September 2009). The audit found that although Office of Science generally managed Cooperative Research and Development Agreements (CRADAs) according to Department requirements, it did not always ensure that its laboratories received final reports and forwarded them to the Office of Scientific and Technical Information (OSTI). It was noted that site offices did not exercise adequate oversight of CRADA activities at the laboratories, and had not established goals and measures to evaluate the success of the laboratories in obtaining the final reports and forwarding them to OSTI. It was recommended that performance measures, related to obtaining final reports and forwarding those reports to OSTI, be established and monitored by the site office, and that the site office ensure that the laboratory institute policies and procedures to ensure compliance. Management agreed with our findings and generally concurred with our recommendations, but did not think it was necessary to establish performance measures because following the other recommendations would ensure compliance with Department requirements. • Audit Report on Management Controls over the Technology Transfer and Commercialization Program at the Idaho National Laboratory (OAS-M-05-07, June 2005). This audit found that the contractor at Idaho National Laboratory, had not established proper financial controls over royalty income from its licensing activities and had not properly tracked costs of technology transfer to ensure the Laboratory's spending did not exceed contractual limits. This was despite the fact that internal auditors had reported these adverse conditions several years prior. Recommendations were made to set-up a proper accounting structure to capture royalty income, to establish internal controls over the write-off of uncollectable royalty receipts and to ensure that technology transfer expenditures did not exceed administrative limits without contracting officer approval. Department of Energy National Nuclear Security Administration Washington, DC 20585 March 4, 2013 MEMORANDUM FOR GEORGE W. COLLARD ASSISTANT INSPECTOR GENERAL FOR AUDITS OFFICE OF INSPECTOR GENERAL FROM: CYNTHIA LERSTEN ASSOCIATE ADMINISTRATOR FOR MANAGEMENT AND BUDGET SUBJECT: Comments on the Office of Inspector General Draft Report Titled, "Cooperative Research and Development Agreements at National Nuclear Security Administration Laboratories" (A12CH020/2012-00414) The National Nuclear Security Administration (NNSA) appreciates the opportunity to review the subject draft report. NNSA agrees with the recommendations and NNSA will take action to correct the issues identified. The attachment to this memorandum identifies specific actions and timeframes for addressing the recommendations, as well as general and technical comments to improve the clarity and factual accuracy of the draft report. If you have any questions regarding this response, please contact Dean Childs, Director, Internal Control, at (301) 903-1341. Attachment Initial Response to Draft Report Recommendations "Cooperative Research and Development Agreements at National Nuclear Security Administration Laboratories" The Inspector General (IG) recommended that the Principal Deputy Administrator, National Nuclear Security Administration (NNSA), direct all site offices to: Recommendation 1: Verify that the [NNSA] laboratories establish policies to obtain final reports from researchers for all completed or terminated Cooperative Research and Development Agreements (CRADA) and transmit them to the Office of Scientific and Technical Information (OSTI). Management Response: Concur NNSA will issue a memorandum to all organizations (site offices, laboratories and technology transfer offices) reinforcing the existing requirements for the dissemination of scientific and technical information by providing copies of final CRADA reports to OSTI. This memorandum will also require that each laboratory capture these requirements in local policies/procedures and provide copies of those implementing documents to their respective site office and the Office of Interagency Work no later than September 30, 2013. Recommendation 2: Periodically review whether the laboratories are receiving and promptly transmitting final reports to OSTI. Management Response: Concur NNSA's Office of Interagency Work will incorporate a review of Technology Transfer activities into the annual review of Work for Others at the NNSA laboratories required by Department of Energy Order 481.1C, Work for Others (Non-Department of Energy Fueled Work). This will include an evaluation of the receipt and transmittal to OSTI of research reports for CRADAs. Based on current schedules, baseline reviews of this area at all sites will be completed by September 30, 2014. Recommendation 3: Require the laboratories to determine whether final reports have been received on previously completed or terminated CRADA's, and ensure that any existing final reports are transmitted to OSTI. Management Response: Concur NNSA will require laboratories to review previously completed or terminated CRADAs, identify any instances where final reports were not transmitted, and ensure those reports are sent to OSTI as required. Laboratories will provide a confirmation that these activities have been completed to their respective site office and the Office of Interagency Work no later than December 30, 2013. The Office of Interagency Work will monitor progress in accomplishing this milestone through scheduled Headquarters/Field Office teleconferences. IG Report No. OAS-M-13-02 CUSTOMER RESPONSE FORM The Office of Inspector General has a continuing interest in improving the usefulness of its products. We wish to make our reports as responsive as possible to our customers' requirements, and, therefore, ask that you consider sharing your thoughts with us. On the back of this form, you may suggest improvements to enhance the effectiveness of future reports. Please include answers to the following questions if applicable to you: 1. What additional background information about the selection, scheduling, scope, or procedures of the audit or inspection would have been helpful to the reader in understanding this report? 2. What additional information related to findings and recommendations could have been included in the report to assist management in implementing corrective actions? 3. What format, stylistic, or organizational changes might have made this report's overall message more clear to the reader? 4. What additional actions could the Office of Inspector General have taken on the issues discussed in this report that would have been helpful? 5. Please include your name and telephone number so that we may contact you should we have any questions about your comments. Name: _______________ Date: _______________ Telephone: _______________ Organization: _______________ When you have completed this form, you may telefax it to the Office of Inspector General at (202) 586-0948, or you may mail it to: Office of Inspector General (IG-1) Department of Energy Washington, DC 20585 ATTN: Customer Relations If you wish to discuss this report or your comments with a staff member of the Office of Inspector General, please contact our office at (202) 253-2162. The Office of Inspector General wants to make the distribution of its reports as customer friendly and cost effective as possible. Therefore, this report will be available electronically through the Internet at the following address: U.S. Department of Energy Office of Inspector General Home Page http://energy.gov/ig Your comments would be appreciated and can be provided on the Customer Response Form.