What AI actually costs in production
Understanding AI cost production helps organizations allocate resources effectively by identifying areas where integration and data work dominate the spend, allowing them to plan and budget accordingly, and ultimately reducing the risk of cost overruns and failed deployments.
The objection that AI cost in production is too high to be viable is often raised, but this is the wrong objection because it misunderstands where the spend actually accumulates, which is in the integration and data work required to deploy the AI system, not in the AI model itself. This misconception leads to poor planning and budgeting for AI projects, resulting in cost overruns and failed deployments.
Integration and data work as the dominant line
The dominant line in AI cost production is not the cost of the AI model, but rather the cost of integrating the model with existing systems and preparing the data for use by the model. This includes tasks such as data cleaning, data transformation, and data loading, which can be time-consuming and labor-intensive. A shift supervisor, for example, may need to work with the IT department to integrate the AI system with the maintenance log, which can require significant resources and effort.
Most people get wrong about AI cost production is that they focus on the cost of the AI model itself, rather than the cost of integration and data work. This is wrong because the cost of integration and data work can be several times higher than the cost of the AI model. By understanding where the spend actually accumulates, organizations can better plan and budget for their AI projects.
Run cost against query volume
The run cost of an AI system is closely tied to the query volume, which is the number of questions or requests that the system receives. As the query volume increases, the run cost of the system also increases, because the system needs to process more requests and provide more answers. A category manager, for example, may use the AI system to compare two suppliers, which can generate a large number of queries and increase the run cost of the system.
By understanding the relationship between run cost and query volume, organizations can better plan and budget for their AI projects. This includes planning for scalability, so that the system can handle increased query volumes without breaking down or becoming too expensive to operate. For more information on planning for production, see our production page.
The cost of a wrong answer nobody caught
The cost of a wrong answer that nobody caught can be significant, because it can lead to incorrect decisions or actions being taken. This can have serious consequences, such as financial losses or damage to reputation. A wrong answer that nobody caught can also undermine trust in the AI system, making it less likely to be used in the future.
By implementing quality control measures, such as testing and validation, organizations can reduce the risk of wrong answers and minimize the cost of a wrong answer that nobody caught. This includes testing the AI system with a variety of inputs and scenarios, to ensure that it provides accurate and reliable answers.
Attributing spend to a process rather than a model
When attributing spend to a process rather than a model, organizations can gain a better understanding of where their resources are being used. This includes understanding the cost of integration and data work, as well as the cost of running the AI system. By attributing spend to a process, organizations can identify areas where they can improve efficiency and reduce costs.
For example, a organization may find that the majority of their spend is on data preparation, rather than on the AI model itself. By understanding this, they can focus their efforts on improving the efficiency of their data preparation processes, rather than trying to reduce the cost of the AI model. If you have questions about how to attribute spend to a process, please contact us for more information.
What this does not do
This approach to understanding AI cost production does not provide a complete picture of the costs and benefits of an AI project. It does not take into account the potential benefits of the project, such as increased efficiency or improved decision-making. It also does not provide a detailed breakdown of the costs involved, such as the cost of hardware or software.
By understanding the limitations of this approach, organizations can use it in conjunction with other methods to gain a more complete understanding of the costs and benefits of their AI projects. This includes using techniques such as cost-benefit analysis, to evaluate the potential return on investment of the project. By taking a comprehensive approach to understanding AI cost production, organizations can make more informed decisions about their AI projects and ensure that they are getting the best possible return on their investment.
Frequently asked questions
What is the primary cost driver in AI production?
Integration and data work are the dominant costs, not the AI model itself, requiring significant resources and effort.
How does query volume impact AI system costs?
As query volume increases, the run cost of the system also increases, requiring planning for scalability to handle increased queries.
What is the consequence of a wrong answer that nobody caught?
A wrong answer can lead to incorrect decisions, financial losses, and damage to reputation, undermining trust in the AI system.
How can organizations attribute spend to a process rather than a model?
By understanding the cost of integration and data work, and identifying areas for efficiency improvement, organizations can optimize their AI cost production.