ROI and measurement for quality
A quality manager uses AsscherAi to investigate defect rates by asking what changed this month, on which production line, and how it compares with last quarter, then acts on the findings to adjust the intervention and improve overall quality.
A quality manager sits in a meeting, staring at a report that claims their latest intervention has improved defect rates, but they are not convinced, because the data is incomplete and the counterfactual is invisible. This is a common problem when trying to measure the ROI of quality interventions, because it is difficult to know what would have happened without the intervention.
Leading measures you can actually observe
One way to approach this problem is to focus on leading measures that can be observed, such as the number of inspections performed, or the number of non-conformance reports issued. These measures can provide an early indication of whether an intervention is having the desired effect. For example, a shift supervisor checks the maintenance log to see if the new procedure is being followed, and finds that it is, which suggests that the intervention is on track.
By focusing on these leading measures, it is possible to get a sense of whether an intervention is working, even if the ultimate outcome is not yet clear. This can help to build confidence in the intervention, and provide a basis for making decisions about whether to continue or modify it.
Why aggregate outcome metrics move for unrelated reasons
Aggregate outcome metrics, such as overall defect rates, can be misleading when it comes to evaluating the effectiveness of a quality intervention. This is because these metrics can be influenced by a wide range of factors, many of which are unrelated to the intervention itself. For example, changes in the supply chain, or fluctuations in demand, can all impact defect rates, making it difficult to determine whether an intervention is having a significant effect.
Most people get wrong about this, and think that aggregate outcome metrics are a reliable way to measure the effectiveness of a quality intervention, but they are wrong. These metrics are simply too influenced by external factors to provide a clear picture of what is working and what is not.
The quality data this depends on
The ability to measure the ROI of quality interventions depends on having access to high-quality data, including inspection records, non-conformance reports, batch genealogy, and customer complaints. This data provides a detailed picture of what is happening on the production floor, and allows for the identification of trends and patterns that can inform decision-making.
For example, a category manager compares two suppliers, using data on defect rates and inspection results to determine which one is performing better. This kind of analysis can help to identify areas for improvement, and provide a basis for making decisions about which interventions to pursue.
What it changes about
The use of data to measure the ROI of quality interventions can change the way that defects are traced and addressed. For example, by using data to identify the root cause of defects, it may be possible to trace them back to a particular batch or supplier, and take corrective action. This can help to prevent future defects, and improve overall quality.
It can also change the way that CAPA (Corrective and Preventive Action) records are used. Rather than simply writing a report and filing it away, CAPA records can be used to identify and address systemic problems, and prevent future defects. This can help to create a culture of continuous improvement, and drive ongoing quality gains.
Measured with
The ROI of quality interventions can be measured using a range of metrics, including first pass yield, defects per unit, and time to root cause. These metrics provide a clear and objective picture of what is happening, and allow for the evaluation of different interventions and approaches.
For example, a production manager uses production data to track first pass yield, and finds that it is improving over time. This suggests that the interventions that have been put in place are having a positive effect, and provides a basis for making decisions about how to build on this success.
What this does not do
This approach to measuring the ROI of quality interventions does not provide a complete picture of all the factors that influence quality. It is focused specifically on the data that is available, and the interventions that can be measured and evaluated. It is not a substitute for other approaches, such as consulting with experts or conducting further research.
It is also not a guarantee of success. Even with the best data and analysis, there are no guarantees that an intervention will be successful. However, by using data to inform decision-making, it is possible to increase the chances of success, and drive ongoing quality gains.
Frequently asked questions
How can I measure the effectiveness of a quality intervention?
Focus on leading measures like inspections and non-conformance reports to get an early indication of the intervention's impact.
Why are aggregate outcome metrics not reliable for evaluating quality interventions?
They are influenced by external factors like supply chain changes and demand fluctuations, making it hard to determine the intervention's effect.
What data is needed to measure the ROI of quality interventions?
High-quality data including inspection records, non-conformance reports, batch genealogy, and customer complaints provides a detailed picture of production floor activity.
How can data analysis change the way defects are traced and addressed?
By identifying the root cause of defects, data analysis can help trace defects back to a particular batch or supplier, enabling targeted corrective action.