• Written by Azhar Aliyar, Director

Can we really trust the answers we get from AsscherAi, or do we need to fine tune it to get the results we want, and is this just a matter of tweaking the retrieval algorithm or is it a more complex issue of changing the behaviour of the system, a question that often arises in meetings when discussing the implementation of AsscherAi, a platform that lets a large organisation query its own data in natural language and get answers in real time, built by Harns Technologies.

What each option genuinely commits you to

The decision to use RAG or fine tuning commits you to different approaches to maintaining the accuracy and relevance of the answers provided by AsscherAi. RAG, which stands for Retrieval, Augmentation, and Generation, is a process that focuses on keeping the knowledge base up to date and ensuring that the retrieval algorithm is working correctly. Fine tuning, on the other hand, involves adjusting the parameters of the model to change its behaviour and improve the accuracy of the answers. This means that if you choose to fine tune, you are committing to an ongoing process of testing and adjusting the model, which can be time consuming and require significant resources.

In contrast, using RAG means that you are committing to a process of continuous updating and maintenance of the knowledge base, which can be a significant task, especially if the knowledge base is large and complex. However, this approach ensures that the answers provided by AsscherAi are always based on the most up to date information and are auditable, which is essential for many organisations.

The case where the other one is right

There are cases where fine tuning is the right approach, such as when the model is not performing well on a specific task or when the knowledge base is not comprehensive enough. In these cases, fine tuning can help to improve the accuracy of the answers and ensure that the model is working correctly. For example, if the model is struggling to answer questions about a specific topic, fine tuning can help to adjust the parameters of the model to improve its performance on that topic.

On the other hand, there are also cases where RAG is the right approach, such as when the knowledge base is large and complex and needs to be updated regularly. In these cases, using RAG ensures that the answers provided by AsscherAi are always based on the most up to date information and are auditable. For instance, a category manager comparing two suppliers would need to ensure that the information about each supplier is up to date and accurate, which would require the use of RAG.

The cost that only shows up later

One of the costs of fine tuning that only shows up later is the risk of overfitting, which occurs when the model becomes too closely tailored to the specific data it was trained on and loses its ability to generalise to new data. This can result in the model performing well on the training data but poorly on new data, which can be a significant problem. Additionally, fine tuning can also lead to a loss of interpretability, making it difficult to understand why the model is making certain predictions or recommendations.

In contrast, the cost of using RAG is more upfront, as it requires significant resources to maintain and update the knowledge base. However, this approach ensures that the answers provided by AsscherAi are always based on the most up to date information and are auditable, which can help to avoid the risks associated with fine tuning. For more information on how to implement RAG, you can visit our website via this link.

A way to decide without a six month evaluation

One way to decide between RAG and fine tuning without a six month evaluation is to consider the specific requirements of your organisation and the type of questions you need to answer. If you need to answer complex questions that require a deep understanding of the knowledge base, then fine tuning may be the right approach. However, if you need to answer questions that require up to date and auditable information, then RAG may be the better choice.

It's also important to consider the resources and expertise you have available, as fine tuning requires significant expertise and resources to implement and maintain. You can contact us via this link to discuss your specific requirements and determine the best approach for your organisation.

What this does not do

This article does not provide a comprehensive guide to implementing AsscherAi, but rather focuses on the specific issue of RAG versus fine tuning. It's also important to note that most people get wrong about this topic is that they think fine tuning is a simple process that can be done quickly and easily, but in reality, it's a complex and time consuming process that requires significant resources and expertise.

Additionally, this article does not discuss the use of AsscherAi for purposes other than answering questions about an organisation's own data, such as using it to analyse external data or make predictions about future trends. The use of AsscherAi for these purposes would require a different approach and would likely involve a combination of RAG and fine tuning, as well as other techniques such as data preprocessing and feature engineering.