• Written by Muhammed Abdulla NC, CEO

In theory, AI cost optimisation in energy and utilities should be a straightforward process of identifying areas of inefficiency and implementing cost-saving measures. However, in practice, the complexity of asset-heavy operations, regulatory scrutiny, and geographically dispersed assets makes it a challenging task. The consequences of getting it wrong can be severe, resulting in wasted resources, inefficient operations, and ultimately, higher costs for consumers.

Unique challenges in energy and utilities

The energy and utilities sector faces unique challenges that make cost optimisation more difficult. Outage planning, for example, requires a delicate balance between maintenance, repair, and replacement of assets, while ensuring that the lights stay on for consumers. A shift supervisor checks the maintenance log to determine which assets are due for maintenance, and when, to minimise downtime and reduce costs. Regulatory reporting burden is another significant challenge, as companies must comply with strict regulations and reporting requirements, which can be time-consuming and costly.

Geographically dispersed assets also pose a significant challenge, as companies must manage and maintain assets that are spread out over large areas, often in remote locations. This can make it difficult to monitor and maintain assets, and can result in higher costs due to the need for travel and logistics.

Unit cost vs total cost of an AI workflow

One of the common mistakes made in AI cost optimisation is focusing on the unit cost of an AI workflow, rather than the total cost. While the unit cost of a single AI model may be low, the total cost of running multiple models, storing and processing large amounts of data, and maintaining the underlying infrastructure can be significant. A data scientist may develop a complex AI model that is highly accurate, but if it requires large amounts of computational resources and data storage, the total cost can be prohibitively expensive.

In contrast, a category manager compares two suppliers of AI services, and finds that one supplier offers a lower unit cost, but the other supplier offers a more efficient workflow that reduces the total cost. This highlights the importance of considering the total cost of an AI workflow, rather than just the unit cost.

Real levers for cost optimisation

So, what are the real levers for cost optimisation in AI workflows? Caching, batching, and model routing are three key strategies that can help reduce costs. Caching involves storing frequently-used data in memory, rather than re-computing it every time it is needed. Batching involves processing large amounts of data in batches, rather than individually, to reduce the overhead of processing. Model routing involves routing AI models to the most appropriate hardware, to reduce computational costs.

These strategies can be highly effective in reducing costs, but they require careful planning and implementation. A company may need to invest in new hardware or software to support caching and batching, and may need to develop new models and workflows to take advantage of model routing. However, the potential cost savings can be significant, and can help companies to stay competitive in a rapidly-changing market.

Token cost is rarely the dominant line

Token cost is often cited as a key factor in AI cost optimisation, but in reality, it is rarely the dominant line. While token cost can be significant, it is often dwarfed by other costs, such as data storage and computational costs. A company may pay a significant amount for token-based AI services, but if the underlying data storage and computational costs are high, the total cost can be much higher than expected.

In fact, the cost of data storage and computational resources can be so high that it outweighs the cost of token-based AI services. This highlights the importance of considering all the costs associated with AI workflows, rather than just focusing on token cost. By taking a holistic approach to cost optimisation, companies can identify areas where costs can be reduced, and implement strategies to achieve significant cost savings.

For more information on how to optimise costs in AI workflows, visit our website to learn more about our AI cost optimisation services.

When AI cost optimisation is not the right choice

While AI cost optimisation can be a highly effective way to reduce costs, there are situations where it is not the right choice. For example, if a company is facing significant regulatory or compliance issues, it may be more important to focus on addressing those issues, rather than trying to optimise costs. In these situations, the cost of non-compliance can far outweigh any potential cost savings from AI cost optimisation.

In other cases, a company may be facing significant operational or logistical challenges, such as managing geographically dispersed assets, or dealing with complex supply chains. In these cases, it may be more important to focus on addressing those challenges, rather than trying to optimise costs. If you are unsure about whether AI cost optimisation is right for your company, contact us to discuss your options.