Process Change Assessment Techniques Product Overview

Summary

This report provides an overview of process change assessment techniques presented at the 2016 U.S. Space Programs Mission Assurance Improvement Workshop. It addresses methods to prevent unintended consequences and mission failures arising from unassessed or poorly vetted process changes across space hardware supply chains and internal manufacturing centers of excellence. The document outlines tools such as Process Failure Modes and Effects Analysis (PFMEA), Process Change Notifications (PCN), and targeted assessment questionnaires.

Title Page

AEROSPACE REPORT NO. TOR-2016-02121

May 5, 2016

Nelson J. Ho Communication Systems Implementation Subdivision Communications and Cyber Division

Prepared for: National Reconnaissance Office 14675 Lee Road Chantilly, VA 20151-1715

Contract No. FA8802-14-C-0001 Authorized by: National Systems Group

Developed in conjunction with Government and Industry contributors as part of the U.S. Space Programs Mission Assurance Improvement Workshop.

Distribution Statement A: Approved for public release; distribution unlimited.

THE AEROSPACE CORPORATION Assuring Mission Success

Abstract

Preventing process changes from causing unintended consequences is important to achieving mission success. Process changes, often in the name of improvement, continue to be identified as contributors to hard and expensive lessons. These lessons can be linked to inadequately assessed processes or changes in upstream processes. This presentation was given at the 2016 Mission Assurance Improvement Workshop and provides an overview of the full report, Process Change Assessment Techniques (Aerospace Report No. TOR-2016-02187).

Acknowledgments

This document was created by multiple authors throughout the government, the aerospace industry, and academia. We thank the following contributing authors for making this collaborative effort possible:

Mary Addison (Northrop Grumman Aerospace Systems)
Anthony Eary (Northrop Grumman Aerospace Systems)
George Gitchev (SSL)
Nelson Ho (The Aerospace Corporation)
Johnny Jones (Harris Corporation)
Steve Killman (The Boeing Company)
Scott Miller (Ball Aerospace and Technologies Corporation)
Thomas Reinsel (Raytheon Space and Airborne Systems)

A special thank you for co-leading this team and efforts to ensure completeness and quality of this document goes to Thomas Reinsel, Raytheon Space and Airborne Systems.

The authors deeply appreciate the contributions of the subject matter experts who reviewed the document:

Rob Bauer (Moog)
Matthew Binondo (The Aerospace Corporation)
Emanuel Bucur (The Aerospace Corporation)
Ira Johnson (Moog)
Alfred Jung (The Aerospace Corporation)
Ed Kestel (Orbital ATK)
Mark King (Micropac Industries)
Katherine Losoncy (The Aerospace Corporation)
Art McClellan (The Aerospace Corporation)
James Parke (The Aerospace Corporation)
Frank Pastizzo (SSL)
Jason Rife (Harris Corporation)
Tom Seitz (Northrop Grumman)
Michelle Settles (NOAA)
Beth Springsteen (The Aerospace Corporation)
Anthony Taconi (Lockheed Martin)
David Tong (The Boeing Company)
Colin Wilson (Ball Aerospace)

Slide 1: Title Slide

U.S. Space Program Mission Assurance Improvement Workshop

Process Change Assessment Techniques Product Overview

Tom Reinsel, Raytheon Space and Airborne Systems Nelson Ho, The Aerospace Corporation

May 5, 2016

U.S. SPACE PROGRAM MISSION ASSURANCE IMPROVEMENT WORKSHOP HARRIS CORPORATION | MELBOURNE, FL | MAY 3–5, 2016

Slide 2: Motivation for Process Change Assessment Techniques

Motivation for Process Change Assessment Techniques

• Process change, often in the name of “improvements,” can introduce unintended consequences for qualified hardware – Design changes traditionally vet through engineering change boards • Process changes are not consistently assessed for high risk – The less a supplier knows about the end use of a product, the less likely that supplier is to consider that a proposed process change could cause unwanted product impacts – Some companies have developed their own policies to manage process change, but there is no industry consensus on approach • “Late” escapes are infrequent, but can have high impact on mission success

Slide 3: Problem

Problem

Process changes may happen at outside specialty processors or in-house process “Centers of Excellence.” Problems occur when these changes manifest as unintended consequences at higher module- or unit-level builds, risking mission success.

Structure/Flow Analysis:

Prime or Tier 1 Supplier (e.g. MAIW Member): Program level (Problem discovered here)
Supplier/Subtier (e.g. SSC level): Module or Unit (Build-to-Spec: Electronics, Reaction wheels, Telescopes, Antennas, Propulsion units) / Product Line (Build-to-Spec: Electronics, Propulsion subsystem) [Scope Boundary]
Processor: Specialty processor (Coatings, Composites, Heat treat, Soldering, Welding) / “Center of Excellence” process (Composites, Soldering, Welding) (Escape occurs here)

Slide 4: Team Charter

Team Charter

• Survey data from integration and test (I&T) (failure review board [FRB], material review board [MRB], etc.) for examples of failures resulting from supplier or in-house process changes • Identify and categorize causal themes and drivers from data – e.g., facility relocations, equipment or workforce changes, breaks in procurement or manufacturing, etc. • Benchmark trade associations for methods that have addressed control of process changes • Survey/assess effectiveness of existing customer process change notification (PCN) and control requirements • Compile best practice methods and tools for effective identification and evaluation of supplier PCNs • Prioritize guidance on when and how to use existing methods and tools versus introduction of new or unfamiliar methods or tools

Slide 5: Process Escape Data Analysis

Process Escape Data Analysis

• The team analyzed data from various sources (lessons learned and root cause corrective action databases, FRB, MRB, etc.) for process changes that created hardware failures at subsequent levels of assembly, integration, test, or customer use • Each escape was assigned to a potential cause on a fishbone • Potential causes were ranked and the top three were investigated further

Included Charts:

Pareto of Potential Process Escape Causes (Counts and cumulative percentages across failure modes, requirement flow-downs, process changes, and verification)
Fish Bone Diagram for Failure to Notify Customer of Process Change (Categories: Human Factors, Methodology, System)

Slide 6: Escape Examples

Escape Examples

Example 1: • Process Change: Change of gold plating supplier for connector pins resulted in intermittent connectivity failures at cold temperatures (found in vehicle-level thermal vacuum [TVAC] testing). • Result: Reach-across to multiple programs and customers at all phases of development and integration. • Recognition of potential impact from change of source could have been mitigated earlier by perceptive screening test.

Example 2: • Process Change: A dynamic load (i.e., wiggle) test was implemented to screen for failures during manufacturing. The new screen induced unexpected latent stress failures in adjacent chip components. • Result: New failures found at higher-level printed wiring board (PWB) assemblies, necessitating more investigation and remove-and-replace (R&R). • Recognition of potential impact from “improved” screening could have been prevented with a thorough risk analysis.

Slide 7: Trade Association Summaries

Trade Association Summaries

• Conducted review of six trade associations’ policies and publications – For guidance to their practitioners to manage and control process changes • before the change was initiated • after an escape had occurred to assess the risk • These associations were selected because together they comprise the space system industry’s most well-developed sources for procedural guidance on the design and manufacture of hardware used in space applications

Reviewed Associations:

AIAA (Shaping the Future of Aerospace)
IPC
I.A.Q.G. (International Aerospace Quality Group)
JEDEC
Nadcap
SAMPE

Slide 8: International Aerospace Quality Group (IAQG)—Process Failure Modes and Effects Analysis (PFMEA)

International Aerospace Quality Group (IAQG)—Process Failure Modes and Effects Analysis (PFMEA)

• PFMEA was seen as the most effective available technique for preventing a process change-related escape • Implementation of a PFMEA (which is one element of the new AS9145 standard) can be applied specifically to evaluate a proposed process change • This methodology is particularly valuable before a process change is implemented

PFMEA Worksheet Summary Example:

Process Being Reviewed: Add glass to RTV
Potential Failure Modes: air inclusions into RTV (at ~40% by surface area)
Potential Effect(s) of Failure: reduce bond line and thermal control (Severity: 5)
Potential Cause(s) / Mechanism(s): Poor Operator Controls, no deairate operation
Current Controls (Prevention): de-airate process, Low viscosity RTV (Occurrence: 2)
Current Controls (Detection): variation in IR signature at end customer (Detection: #, RPN: 100)
Recommended Action(s): 1.) Determine acceptable void % RF thermal analysis (Resp: LaKomski); 2.) Provide supplier with IR camera system details to determine likelihood of inclusion into ATP process (Resp: B. Burk)
Action Results: Action 1 reduces severity (New Sev: 4, New Occ: 2, New Det: 10, Improved RPN: 80); Action 2 reduces detection (New Sev: 5, New Occ: 2, New Det: 5, Improved RPN: 50)

Slide 9: Process Change Notification Benchmark

Process Change Notification Benchmark

• Reviewed several companies’ flowdown requirements for PCN • One was selected as a benchmark example – Benchmark PCN policy was selectively provided to key suppliers by the contractual inclusion of a Quality Assurance Provision (QAP), also called Q-note – Program and Supplier Quality conduct a review of the Supplier Change Request Notification (SCRN) with supply chain and technical SMEs – The review considers further actions such as conducting a new first article or a re-qualification plan for the part

[Includes image preview of Supplier Change Request/Notification (SCRN) form]

Slide 10: Process Change “Improvement” Questions

Process Change “Improvement” Questions

• A set of 27 questions was developed to aid a supplier or in-house processor to identify changes that might affect the end user of the product • The questions are meant to invite the review of changes that go beyond typical form-fit-function constraints – Questions are not exhaustive • The questions help a supplier recognize that a change to an existing process could represent a risk to a product in its end use – Even if the supplier was unaware of all of the conditions of the end use of a product

Question Categories Preview:

Methods: Questions involving methods (waste reduction, alternative paths, assembly aids, shifts, execution methods)
Machinery: Labor-reducing equipment, material handling devices, standard/improved inspection or packaging equipment
Material: Expense items reduction (adhesives, tapes, wires, lubricants, coolants, inks), obsolete/unusable/alternate expense items
Environment: Cleanliness, facility rearrangement, relocation within site, different site, additional manufacturing site

Slide 11: Intended Product Use

Intended Product Use

• The purpose is – to provide awareness to the general space community for the benefit of mitigating the risks of process changes – to benchmark the most useful tools and policies currently in use • Intended for program management, procurement, and technical disciplines, including engineering, production, and quality organizations responsible for the procurement, design, manufacturing, and quality assurance of space-qualified hardware • Some of the professional associations or educational organizations that would reach the target audience are AIAA, IPC, IAQG, JEDEC, Nadcap, and SAMPE

Slide 12: Topic Follow-on Recommendations

Topic Follow-on Recommendations

• Specific recommendations for the space industry: – Enhance existing reference documents (e.g., EIA-649-1, Definition of Major (Class I) Engineering Change Proposal, and similar sources) to include language that goes beyond Class I/Class II or Major/Minor to include unintended consequences from process changes – Develop a space industry common guideline for mitigating process changes and a common structure for PCN policy that could be selectively applied between Primes -> Tier 1 and also from Tier 1 -> outside processors

• Specific recommendations for government: – Review the new AS 9145 and SAE J1739 processes from IAQG and develop a plan to flow these techniques down to suppliers depending on circumstances. May be appropriate to research a larger set of history for AS 9145 and SAE J1739

Slide 13: Team Introductions - Core Team

Team Introductions - Core Team

Company | Participant

The Aerospace Corporation | Nelson Ho*
Ball Aerospace and Technologies Corporation | Scott Miller
The Boeing Company | Steve Killman
Harris | Johnny Jones*
Northrop Grumman Aerospace Systems | Mary Addison*, Anthony Eary
Raytheon Space and Airborne Systems | Thomas Reinsel*
SSL | George Gitchev*
MAIW Steering Committee/Sponsor (Raytheon Space and Airborne Systems) | Mark Baldwin*
MAIW Steering Committee (Ball Aerospace and Technologies Corporation) | Eli Minson*
MAIW Steering Committee (Northrop Grumman) | Ed Jopson*

Attending MAIW

Slide 14: Team Introductions - Subject Matter Experts

Team Introductions - Subject Matter Experts

Company | Participant

The Aerospace Corporation | Emanuel Bucur*
The Aerospace Corporation | Matthew Binondo
The Aerospace Corporation | Alfred Jung
The Aerospace Corporation | Katherine Losoncy*
The Aerospace Corporation | Art McClellan*
The Aerospace Corporation | James Parke*
The Aerospace Corporation | Beth Springsteen
Ball Aerospace | Colin Wilson
The Boeing Company | David Tong
Harris Corporation | Jason Rife
Lockheed Martin | Anthony Taconi
Micropac Industries | Mark King*
Moog | Rob Bauer
Moog | Ira Johnson
NOAA | Michelle Settles
Northrop Grumman | Tom Seitz
Orbital ATK | Ed Kestel*
SSL | Frank Pastizzo*

Attending MAIW

Document Approvals - Part 1

AEROSPACE REPORT NO. TOR-2016-02121 Process Change Assessment Techniques Product Overview

Approved Electronically by:

Todd M. Nygren, GENERAL MANAGER, SYSTEMS ENGINEERING DIVISION, ENGINEERING & TECHNOLOGY GROUP
Jacqueline M. Wyrwitzke, PRINC DIRECTOR, MISSION ASSURANCE SUBDIVISION, SYSTEMS ENGINEERING DIVISION, ENGINEERING & TECHNOLOGY GROUP

Aerospace Corporate Officer Approval:

Catherine J. Steele, SR VP NATL SYS, NATIONAL SYSTEMS GROUP

Document Approvals - Part 2

AEROSPACE REPORT NO. TOR-2016-02121 Process Change Assessment Techniques Product Overview

Technical Peer Review Performed by:

Jacqueline M. Wyrwitzke, PRINC DIRECTOR, MISSION ASSURANCE SUBDIVISION, SYSTEMS ENGINEERING DIVISION, ENGINEERING & TECHNOLOGY GROUP

Special Programs Security Approval Granted Electronically by:

Alvania W. Thompson, SECURITY STAFF IV, CHANTILLY SPECIAL PROGRAMS SECURITY OPERATIONS & SUPPORT GROUP