• Written by Muhammed Abdulla NC, CEO

A operations manager reviewing the latest project status report is faced with a crucial decision: how to measure the return on investment of a recent process improvement initiative, when the counterfactual - what would have happened without the initiative - is impossible to observe. Judgement is required to determine the most effective measures, and it is wrong to rely solely on lagging indicators. Instead, leading measures that can be observed and influenced should be prioritized.

Leading measures you can actually observe

Leading measures are those that can be observed and influenced in real-time, allowing for timely interventions and adjustments. In the context of operations, these measures might include the rate of completion of key tasks, the number of exceptions or errors, or the level of resource utilization. By focusing on these leading measures, operations managers can identify areas for improvement and make data-driven decisions to optimize processes. A shift supervisor, for example, might check the maintenance log to identify areas where equipment downtime is impacting production, and adjust the maintenance schedule accordingly.

The use of leading measures also enables the identification of potential issues before they become major problems, allowing for proactive intervention and minimizing the impact on overall operations. This is particularly important in complex systems where small changes can have significant effects.

Why aggregate outcome metrics move for unrelated reasons

Aggregate outcome metrics, such as overall production volume or customer satisfaction, are often influenced by a wide range of factors, many of which are unrelated to the specific process improvement initiative. As a result, changes in these metrics may not accurately reflect the effectiveness of the initiative. For instance, a change in customer demand or a supplier issue could impact production volume, making it difficult to determine the true impact of the process improvement. Most people get wrong the idea that aggregate metrics are a reliable way to measure the success of an initiative, when in fact they can be misleading.

This highlights the need for more nuanced and targeted measures that can isolate the impact of the initiative and provide a clearer understanding of its effectiveness. By using operations data to inform decision-making, managers can better understand the complex interactions within their systems and make more effective interventions.

The operations data this depends on

The ability to measure the ROI of process improvement initiatives in operations depends on access to high-quality data on process and exception records, project status, budget consumption, and supplier performance. This data provides the foundation for understanding the complex interactions within operations and identifying areas for improvement. A category manager, for example, might compare the performance of different suppliers to identify opportunities for cost savings or quality improvements.

By analyzing this data, operations managers can identify trends, patterns, and anomalies that might not be immediately apparent, and develop targeted interventions to address specific issues. This data-driven approach enables more effective decision-making and resource allocation, ultimately leading to improved outcomes and increased efficiency.

What it changes about: anomalies only visible in combination and projects consuming budget faster than progress

The use of leading measures and operations data can reveal anomalies and issues that might not be visible through traditional metrics. For instance, a project might be consuming budget at a faster rate than progress is being made, indicating potential inefficiencies or scope creep. By identifying these issues early, operations managers can take corrective action to get the project back on track.

Similarly, the combination of different data sources can reveal patterns and anomalies that might not be apparent from individual datasets. This can help operations managers to identify potential problems before they become major issues, and develop proactive strategies to mitigate their impact. The ability to analyze and respond to these complex issues is a key benefit of using specialized analytics tools.

Measured with: on-time delivery, exception rate, expediting cost

The effectiveness of process improvement initiatives in operations can be measured using a range of metrics, including on-time delivery, exception rate, and expediting cost. These metrics provide a clear and objective assessment of the impact of the initiative, and can be used to identify areas for further improvement. By tracking these metrics over time, operations managers can develop a comprehensive understanding of the initiative's effectiveness and make data-driven decisions to optimize processes.

The use of these metrics also enables the comparison of different initiatives and the identification of best practices, allowing operations managers to refine and improve their approaches over time. This continuous improvement cycle is critical to achieving long-term success and staying competitive in a rapidly changing environment.

What this does not do

This approach to measuring ROI in operations does not provide a comprehensive assessment of the overall health of the organization, but rather a targeted evaluation of specific process improvement initiatives. It is not a substitute for broader strategic planning and performance management, but rather a complementary tool to support data-driven decision-making. By understanding the limitations and potential biases of this approach, operations managers can use it effectively to drive improvement and optimize their systems.