• Written by Muhammed Abdulla NC, CEO

This week, the executive leadership team must decide where to cut costs without affecting production, but their current approach is based on incomplete information about where money actually goes. The team relies on unit costs, which only tell part of the story, and do not account for the total cost of an AI workflow, including the cost of data preparation, model training, and deployment.

The difference between unit cost and total cost of an AI workflow

The unit cost of an AI workflow is the cost of processing a single unit of data, but this metric does not capture the full cost of the workflow. The total cost of an AI workflow includes the cost of data preparation, model training, and deployment, as well as the cost of maintaining and updating the model over time. For example, a shift supervisor checks the maintenance log to see how often the AI model needs to be updated, and how much it costs to do so.

This distinction is crucial because it affects how executive leadership makes decisions about cost optimization. If the team only looks at unit costs, they may miss opportunities to reduce costs elsewhere in the workflow. By considering the total cost of the workflow, the team can identify areas where costs can be reduced without affecting production.

Caching, batching and model routing as the three real levers

There are three main levers that executive leadership can use to optimize costs: caching, batching, and model routing. Caching involves storing frequently-used data in a easily-accessible location, reducing the need to recompute it. Batching involves processing multiple units of data at once, reducing the overhead of processing each unit individually. Model routing involves directing data to the most appropriate model for processing, reducing the need for unnecessary computations.

These levers can have a significant impact on costs, but they require careful consideration of the trade-offs involved. For example, caching can reduce costs by reducing the need for recomputation, but it can also increase costs by requiring more storage space. By carefully evaluating the trade-offs, executive leadership can use these levers to optimize costs without affecting production.

The executive leadership data this depends on

The ability to optimize costs using these levers depends on having access to certain types of data. Executive leadership needs to have consolidated financials, operational KPIs, sales performance, and headcount data in order to make informed decisions about cost optimization. This data can be used to identify areas where costs can be reduced, and to evaluate the effectiveness of different cost optimization strategies.

For example, the category manager compares two suppliers based on their prices and delivery times, using data from the company's enterprise resource planning system. By analyzing this data, the category manager can identify opportunities to reduce costs by switching to a different supplier or renegotiating contracts.

What it changes about cross-functional questions

One of the key benefits of using AI cost optimization is that it can help answer cross-functional questions that nobody can answer in the room. For example, the executive leadership team may want to know how much it costs to produce a certain product, or how much revenue is generated by a certain customer segment. By using AI to analyze data from different functions, the team can get a more complete picture of the company's operations and make more informed decisions.

Additionally, AI cost optimization can help resolve definition conflicts between different functions. For example, the sales team may define a customer segment differently than the marketing team, leading to conflicting reports and analysis. By using AI to standardize definitions and analyze data, the team can resolve these conflicts and get a more accurate picture of the company's operations.

Measured with decision latency, actions, and definition conflicts

The effectiveness of AI cost optimization can be measured in several ways, including decision latency, actions of the form "come back with numbers", and definition conflicts resolved. Decision latency refers to the time it takes for the executive leadership team to make a decision, and can be reduced by using AI to provide faster and more accurate analysis. Actions of the form "come back with numbers" refer to the ability of the team to quickly and easily get the data they need to make a decision.

For more information on how AI cost optimization can be used to support executive and strategic leadership, please contact us. By using AI to optimize costs, executive leadership can make more informed decisions and drive business success.

What this does not do

AI cost optimization is not a silver bullet that can solve all of an organization's cost problems. It is a tool that can be used to support cost optimization efforts, but it requires careful consideration of the trade-offs involved and a thorough understanding of the organization's operations. It is also not a replacement for human judgment and decision-making, but rather a way to support and inform those decisions.

If you are interested in learning more about how AI cost optimization can be used to support your organization's cost optimization efforts, please contact us to discuss your specific needs and challenges.