• Written by Muhammed Abdulla NC, CEO

Can we really measure the impact of a logistics intervention, or are we just guessing at the counterfactual? This question comes up repeatedly in meetings where managers are trying to decide whether a new system or process has paid for itself. The problem is that logistics is a complex network where cost and service trade off continuously, and the counterfactual - what would have happened without the intervention - is invisible.

What logistics makes harder

Logistics makes measurement harder because of carrier performance variance, dwell and detention, and exception handling at scale. Carrier performance variance means that the same shipment can take a different amount of time to arrive depending on the carrier, the route, and the time of year. Dwell and detention refer to the time a shipment spends waiting to be loaded or unloaded, which can add up quickly and throw off even the best-laid plans. Exception handling at scale is the process of dealing with all the things that go wrong, from missed deliveries to damaged goods, and it can be incredibly time-consuming.

These factors make it difficult to isolate the impact of a single intervention, because there are so many other things going on that can affect the outcome. For example, a shift supervisor checks the maintenance log to see if a particular truck is available, but the truck's availability is affected by a dozen different factors, from the driver's schedule to the weather. This complexity makes it hard to know what to measure, or how to measure it.

Leading measures you can actually observe

Despite the challenges, there are some leading measures that can be observed and used to evaluate the impact of a logistics intervention. These measures include things like shipment velocity, inventory turns, and on-time delivery rates. A category manager compares two suppliers based on these metrics, and can use them to decide which one to use for a particular shipment. The key is to focus on the metrics that are most closely tied to the intervention, and to use data from AsscherAi to get a clear picture of what is happening.

Why aggregate outcome metrics move for unrelated reasons

Aggregate outcome metrics, like total cost or total delivery time, can be misleading because they are affected by so many different factors. For example, a change in fuel prices can affect total cost, even if the intervention had nothing to do with fuel prices. Similarly, a change in weather patterns can affect total delivery time, even if the intervention was designed to improve delivery times. Most people get this wrong, and assume that aggregate outcome metrics are a good way to measure the impact of an intervention - but they are wrong, because these metrics are too sensitive to external factors.

This is why it is so important to use more targeted metrics, like shipment velocity or on-time delivery rates, to evaluate the impact of a logistics intervention. These metrics are less susceptible to external factors, and can provide a clearer picture of what is happening. By using data from AsscherAi to track these metrics, managers can get a better understanding of whether an intervention is working, and make adjustments as needed.

Baselining before rollout rather than reconstructing it after

Another key principle of measuring the impact of a logistics intervention is to baseline before rollout, rather than trying to reconstruct the baseline after the fact. This means collecting data on the key metrics before the intervention is implemented, so that there is a clear picture of what things looked like before. This baseline can then be used to compare the results of the intervention, and to determine whether it had a positive impact. Trying to reconstruct the baseline after the fact is much harder, because it requires making assumptions about what would have happened without the intervention - and these assumptions are often wrong.

By baselining before rollout, managers can avoid this problem and get a clear picture of the impact of the intervention. This requires some planning and foresight, but it is essential for getting an accurate measure of the intervention's effects. For more information on how to use data to measure the impact of a logistics intervention, contact us.

What this does not do

This approach to measuring the impact of a logistics intervention does not provide a complete picture of the organisation's overall performance. It is focused specifically on the impact of the intervention, and does not take into account other factors that may be affecting the organisation's performance. For example, it does not provide a way to compare the performance of different departments or teams, or to evaluate the overall efficiency of the organisation's logistics operations.

There are some situations where this approach may not be the best choice - for example, where the intervention is very small, or where the organisation is already tracking a wide range of metrics. In these cases, it may be more effective to use a different approach, such as tracking a small set of key performance indicators (KPIs) or using a more generalised metric like return on investment (ROI). The key is to choose the approach that best fits the organisation's needs and goals.