• Written by Azhar Aliyar, Director

A purchasing manager at a large retail organisation is deciding this week whether to query the purchase order table using AsscherAi, and they currently base their decision on the fluency of the answer provided, but model confidence is actually a measure of the model's certainty in its answer, which is not the same thing as the answer being correct or certain.

Definition of model confidence

Model confidence refers to the degree to which the model is sure of its answer, and it is typically calculated based on the model's internal workings, such as the strength of the signals it has detected in the data or the consistency of the patterns it has identified. The model confidence score is usually a numerical value between 0 and 1, where 1 represents complete confidence and 0 represents no confidence at all.

The model confidence score is an important aspect of AsscherAi, as it allows users to gauge the reliability of the answer provided, and it can be used to filter out answers that are not confident enough to be useful. For example, a shift supervisor checking the maintenance log may want to see only answers with a model confidence score above a certain threshold, to ensure that they are getting accurate information.

Difference from fluency

Many people confuse model confidence with fluency, which refers to how natural and smooth the answer sounds, but these are two distinct concepts. Fluency is about the presentation of the answer, while model confidence is about the reliability of the answer itself. A model can provide a fluent answer that sounds convincing, but still have low model confidence, meaning that it is not actually sure of the answer.

This distinction is important, because a model can be fluent but wrong, and relying solely on fluency can lead to incorrect decisions. Most people get wrong the idea that a fluent answer is always a confident one, but this is not the case, and it is essential to look at the model confidence score to get a true sense of the answer's reliability.

Applications in a large organisation

In a large organisation, model confidence can be applied in various scenarios, such as when a category manager compares two suppliers, they can use AsscherAi to get answers about the suppliers' performance, and the model confidence score can help them gauge the reliability of the answers. Another example is when a financial analyst is analyzing the organisation's expenses, they can use AsscherAi to get answers about the expenses, and the model confidence score can help them identify areas where the answer is not confident enough to be useful.

By using model confidence, organisations can make more informed decisions, and avoid relying on answers that are not reliable. The model confidence score can also be used to identify areas where the model needs to be improved, such as when the model is consistently providing answers with low confidence scores, it may indicate that the model needs to be retrained or updated.

Limitations of model confidence

While model confidence is a useful concept, it is not a guarantee of accuracy, and it should not be relied upon as the sole factor in decision-making. The model confidence score is only as good as the model itself, and if the model is flawed or biased, the model confidence score will not be reliable. Additionally, model confidence does not take into account external factors that may affect the answer, such as changes in the organisation or new information that is not yet reflected in the data.

For example, if a purchasing manager is using AsscherAi to get answers about a new supplier, the model confidence score may be high, but if the supplier has recently changed their business practices, the answer may not be accurate, even with high model confidence. In such cases, it is essential to use other factors, such as external research or expert judgment, to validate the answer.

When not to use model confidence

There are scenarios where model confidence is not the right choice, such as when the answer is based on a simple lookup or a straightforward calculation, in which case the model confidence score is not necessary. Another example is when the organisation has a small dataset, in which case the model confidence score may not be reliable, and other methods, such as manual analysis or expert judgment, may be more suitable.

In such cases, it is better to use other methods, such as data analysis or expert consultation, to get a more accurate answer. By understanding the limitations of model confidence, organisations can use it more effectively and make more informed decisions. Model confidence is a useful tool, but it should be used in conjunction with other factors, and not relied upon as the sole factor in decision-making.