• Written by Muhammed Abdulla NC, CEO

The notion that AI in procurement is too complex to measure is a misconception, as the right metrics can provide valuable insights into its effectiveness, and it is the wrong objection to claim that measurement is too difficult, when in fact the challenge lies in selecting the right metrics that move for the right reason.

Leading measures: spend under management, price variance to contract

Measuring the success of AI in procurement requires focusing on leading indicators that have a direct impact on the organisation's bottom line, such as spend under management and price variance to contract, which are two key metrics that can help procurement teams evaluate the effectiveness of their AI-powered systems, and a category manager can use these metrics to compare the performance of different suppliers and negotiate better contracts.

A shift supervisor can use these metrics to monitor the maintenance log and ensure that the procurement team is getting the best value for money, and by tracking these metrics, organisations can identify areas where AI can have the greatest impact, such as reducing maverick spend or improving contract compliance, and then use this information to inform their procurement strategies and make data-driven decisions.

Why the headline metric moves for unrelated reasons

Most people get wrong the idea that a single headline metric, such as cost savings, is enough to measure the success of AI in procurement, but this metric can be influenced by a variety of factors unrelated to the AI system itself, such as changes in market prices or supplier negotiations, and therefore it is wrong to rely solely on this metric to evaluate the effectiveness of AI in procurement.

For instance, a cost savings metric may show a significant reduction in costs, but this could be due to a change in market conditions rather than the AI system, and to get a true picture of the AI system's effectiveness, it is necessary to drill down into more detailed metrics, such as spend under management and price variance to contract, which can provide a more accurate assessment of the AI system's impact.

Baselining before rollout

Baselining the current state of procurement operations before rolling out an AI-powered system is crucial to measuring its effectiveness, as it provides a benchmark against which to compare future performance, and this baseline should include metrics such as spend under management, price variance to contract, and contract compliance, which can be used to evaluate the AI system's impact over time.

By establishing a baseline, organisations can set realistic expectations for the AI system's performance and make informed decisions about its implementation and ongoing management, and a link to operations metrics can provide more information on how to establish a baseline and track performance over time.

What to record when nothing was found

Even when an AI-powered procurement system does not identify any issues or opportunities for improvement, it is still important to record the results, as this can provide valuable insights into the system's effectiveness and help to identify areas where the system may need to be refined or improved, and this information can be used to inform future procurement strategies and improve the overall effectiveness of the AI system.

For instance, if an AI system is used to monitor contract compliance and does not identify any issues, this could indicate that the system is working effectively, but it could also indicate that the system is not sensitive enough to detect certain types of issues, and by recording the results, organisations can gain a better understanding of the system's strengths and weaknesses and make informed decisions about its ongoing management and improvement.

What this does not do

This approach to measuring AI in procurement does not provide a complete picture of the organisation's overall procurement operations, and it is the wrong choice for organisations that are looking for a more comprehensive view of their procurement operations, as it focuses primarily on the effectiveness of the AI system itself, rather than the broader procurement context, and for more information on how to get in touch with a procurement expert, please visit contact us.

Additionally, this approach may not be suitable for organisations that are just starting to implement AI in their procurement operations, as it requires a certain level of maturity and sophistication in the use of AI, and organisations that are just starting out may need to focus on more basic metrics, such as the number of contracts managed by the AI system, before moving on to more advanced metrics, such as spend under management and price variance to contract.