How to measure AI in executive leadership
Measuring AI in executive leadership involves tracking leading measures like decision latency and data-driven insights, as seen when a category manager uses AsscherAi to analyse sales data and inform product decisions, allowing executives to take proactive steps to improve decision-making and organisational performance.
A common objection to using AI in executive leadership is that it is too difficult to measure its effectiveness, but this objection is misguided because it focuses on the wrong metrics. In reality, measuring AI in executive leadership is not about finding a single magic number, but about tracking the right indicators that show whether the AI is having a positive impact on decision-making. One of the most important things to get right is the choice of leading measures, which are indicators that predict future success.
Leading measures
Leading measures are crucial because they allow executives to take proactive steps to improve decision-making. Two key leading measures are decision latency, which is the time it takes to make a decision, and actions of the form "come back with numbers", which is a request for data to inform a decision. By tracking these measures, executives can see whether the AI is helping to speed up decision-making and provide more data-driven insights. For example, a category manager might use AsscherAi to analyse sales data and identify trends, which would then inform their decision on which products to stock.
A shift supervisor might check the maintenance log to see whether the AI has identified any potential issues before they become major problems, allowing them to take proactive steps to prevent downtime. By tracking these leading measures, executives can get a sense of whether the AI is having a positive impact on the organisation, even if the ultimate outcome is not yet clear.
Why the headline metric moves for unrelated reasons
The headline metric, such as revenue or profit, is often the ultimate goal of using AI in executive leadership, but it can be misleading to focus solely on this metric. The headline metric can move for a variety of unrelated reasons, such as changes in the market or economy, which can make it difficult to determine whether the AI is having a positive impact. For instance, a company might see an increase in revenue due to a new marketing campaign, rather than any changes made as a result of using AI.
Most people get wrong the idea that the headline metric is the only thing that matters, and that as long as it is moving in the right direction, everything is okay. However, this is wrong because it ignores the potential underlying issues that might be masked by the headline metric. By focusing on leading measures, executives can get a more nuanced view of whether the AI is having a positive impact.
Baselining before rollout
Baselining is the process of establishing a baseline measure of current performance before introducing a new system or process, such as AI. This is crucial because it allows executives to compare the performance of the organisation before and after the introduction of the AI, and to determine whether any changes are due to the AI or other factors. For example, a company might baseline its current decision latency and then track changes after introducing AI-powered decision support tools.
By baselining before rollout, executives can establish a clear understanding of what the organisation looks like without the AI, and then track changes over time to see whether the AI is having a positive impact. This is particularly important when introducing AI, as it can be difficult to determine whether any changes are due to the AI or other factors, such as changes in the market or economy.
What to record when nothing was found
When using AI to analyse data, it is not uncommon for the AI to find nothing of note, or to not identify any major issues. However, this does not mean that the exercise was a waste of time, as the fact that nothing was found can still be valuable information. For instance, a company might use AsscherAi to analyse customer feedback data and find that there are no major issues with the current product offerings.
This information can be used to inform future decisions, such as whether to continue with the current product offerings or to make changes. By recording the fact that nothing was found, executives can build a more complete picture of the organisation and make more informed decisions. This is an area where many people go wrong, as they often focus solely on the times when the AI does find something, rather than also considering the times when it does not.
What this does not do
While measuring AI in executive leadership is crucial, it is not a panacea for all organisational issues. There are times when it is the wrong choice, such as when the organisation is not yet ready for AI, or when the goals of the organisation are not clear. In these cases, introducing AI can actually be counterproductive, as it can distract from the underlying issues that need to be addressed. For more information on how to determine whether your organisation is ready for AI, see our article on executive and strategic leadership.
If you are unsure about how to measure AI in your organisation, or whether AI is right for you, contact us to discuss your options. We can help you determine whether AI is a good fit for your organisation, and provide guidance on how to get started with measuring its effectiveness.
Frequently asked questions
What are the key metrics to track when measuring AI in executive leadership?
Leading measures like decision latency and actions of the form "come back with numbers" are crucial to track, as they predict future success and inform data-driven decisions.
Why is it important to baseline performance before introducing AI?
Baselining allows executives to compare performance before and after AI introduction, determining whether changes are due to the AI or other factors, and establishing a clear understanding of the organisation's performance.
What happens when the AI does not find any major issues or insights?
The fact that nothing was found can still be valuable information, informing future decisions and building a complete picture of the organisation, and should be recorded to provide context and insight.
Can measuring AI in executive leadership solve all organisational issues?
No, measuring AI is not a panacea, and introducing AI can be counterproductive if the organisation is not ready or goals are unclear, requiring careful consideration and guidance to determine the best approach.