Should We Improve the Process, or the System We Use to Measure the Process? Measurement Systems Analysis and The Limitations of the Gauge R &R

Should We Improve the Process, or the System We Use to Measure the Process?
Measurement Systems Analysis and The Limitations of the Gauge R &R

If you’re reading my blog, you probably are required to comply with regulations foreign and/or domestic in order to be successful.  What’s more, in order to comply with the regulations, you must understand them.  If you’re anything like me, you may have spent the better part of two decades engaged in the quest for understanding, developing the ability to apply those understanding in real life situations.

You have probably read and re-read regulatory guidance documents, both procedural and scientific, on every topic from general principles of document control to the abstract theories of ensuring sterility.   As industry members, procedural guidance, scientific guidance, international harmonization standards, industrial standardization standards have been made available to us, and our practical ability to apply those understandings can be evaluated through an ever increasing number of certification programs.
It’s likely that we are the most regulated and guided industry on this planet.
So why is it that validation of measurement systems remains such a black hole in our collective knowledge base?  
Unless you are seeking guidance on the validation of an analytical method executed in a laboratory, you have probably also spent time seeking agency guidance and, when that didn’t work, you have probably sought industry consensus – how did that work for you?
I find myself engaging in these conversations more and more frequently while interacting with device manufacturers, but the increasing frequency of discussion does not seem to be advancing consensus.  In fact, discussion frequency and consensus seem to have a decidedly indirect relationship.
There do seem to be several points on which we all can agree:
  1. Regulations, including part 820, expect Test Methods to be proven suitable for intended use  = validated
  2. The Guidance on Test Method Validation issued by FDA is limited in scope to analytical methods that involve chemical analysis
  3. Manufacturers who primarily assemble products comprised of mechanical components use Measurement Systems to characterize their products and the successful end points of manufacturing processes, not Test Methods
  4. Part 820 and the FDA expect Measurement Systems to be proven suitable for intended use
  5. Using a Measurement System that has not been thoroughly evaluated will increase business and compliance risk
We seem to universally agree on the “why?” and ”who?” but the big question, “HOW?” seems to be a source of universal confusion.
Identifying, understanding, and controlling and/or reducing the error should be the objective of any Measurement System Analysis (MSA) project.  Just as process validation begins in development and continues throughout the lifecycle of a process, MSA is a cradle to grave concept.  Developing a robust program that is capable of demonstrating suitability for use is a large scale effort that requires a large scale perspective.
Let’s begin at the beginning – Measurement Systems are a function of the combination of the following components used to quantify and/or qualify a unit of measure:
In a situation devoid of variables, a measurement system would produce the same result each time it was used to identify and quantify/qualify any and every defect introduced.
However, the real world of mechanical manufacturing involves an abundance of variables.
Every manually executed Measurement System has some level of variation – the key to proving that these systems as defined and executed are suitable for use (the identification and classification of defects in manufactured units) begins and ends with the ability to identify, measure, control, and eventually reduce that level of variation.
The assumption is that sources of variation = sources of error.   For the purposes of this discussion, types of error can be categorized in two (2) ways:

The single most common mistake that I consistently see out in the field is confusing a Gauge R&R study for a validation effort, or allowing Gauge R&R studies to become the single tool used by the MSA program.

In order to avoid this common error, remember:
  • A Gauge R&R study is limited to providing information regarding precision sources of error
  • The results of a Gauge R&R study are ratios, and offer value only when they are compared to the tolerance for error of the ENTIRE manufacturing process of which, the measurement is part.
  • Calculate the Repeatability and Reproducibility of the manual equipment used to take the measurement.  This calculation requires by its very definition, a true, and well supported (data driven) understanding of the total tolerance of error of the manufacturing process.  
Acknowledging these limitations of the Gauge R&R tool is especially crucial for any facility whose manufacturing process is largely manual, and involves many steps. In this type of environment each step of the manufacturing process has the ability to allow a unit to pass to the next step with some level of error, meaning that the total process error would be the combination of the error (difference between the specified target and the physical reality) of every step (including the measurement steps) taken.>
As the complexity of today’s Medical Devices increases, so do the number of steps required to build them.  It would not be uncommon for a single unit to have undergone dozens, or in some cases, hundreds of steps prior to completion.
The quality of the finished unit will be directly related to the sum total of allowable error of each step taken – and at some point – the quality will be impacted enough to cause rejection of the finished unit.  And as we know, the ability to detect a unit fault through finished unit testing is itself, limited.  Finished unit testing methods are developed in accordance with Failure Modes Effects Analysis, which begins and ends with our ability to predict, and seek to cause, a failure mode.  
The inherent limitations of release testing assures that finished unit testing will never be capable of identifying every defect.  This certainty is what makes analysis of measurements systems a fundamental requirement.  Because we cannot rely solely on our release testing, we must endeavor to introduce upstream measurements to optimize the probability of defect identification.  As we all know, the only alternative to upstream to release testing is downstream of release, and, above all else, we need to avoid allowing defects to enter the product marketplace.  
While no meaningful guidance for developing a robust program for the validation of test methods that are measurement systems, has been provided, do not take this as indication that the method validation programs should not be meaningful.  Demonstrating the suitability and fitness for use of a measuring system, in a complicated manufacturing environment cannot be limited to the use of a single tool.  
The Take -Aways?
  • Measurement Systems Analysis programs must qualify and quantify precision AND accuracy.
  • Gauge R &R studies can speak only to precision – they should never be used to evaluate accuracy profiles.
  • In order to appropriately demonstrate that any single measurement system is fit for its intended use, we must understand that individual steps always incorporate allowable error, and that stepwise allowable error represents a portion of total allowable error.
  • In order to determine what percentage of allowable error any one single measurement method should be allowed to consume, we must understand the relationship between the in-process manufacturing step being measured and the finished device.
  • We must resist the urge to deploy Gauge R&R tools with boilerplate acceptance criteria and instead, set acceptance criteria for each study based on the meaning of the individual manufacturing step and with respect to the percentage of the total error allowable in the entire manufacturing and testing chain.
  • To appropriately set acceptance criteria for any Gauge R&R study, the total error tolerance of the entire manufacturing process (raw material to finished device) must be known, and supported by data.  
  • Total variation in a sound and robust measurement system must be stable and within the product and process specification windows, and within all of the involved calibration ranges.  
Developing a test method program for a measurement system is challenging, requires an enormous amount of data and process knowledge, and is not likely to be summarized in a single blog.  Watch for future installment on best practice statistical methods and approaches to MSA!
Authored By:

Gina Guido-ReddenChief Operating OfficerCoda Corp USA

Connect with me on LinkedIn!

“Quality is never an accident; it is the result of high intention, sincere effort, intelligent direction and skillful execution. It represents the wisest of many alternatives.”
Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>