• Written by Azhar Aliyar, Director

A procurement manager checks the purchase order table to answer a question from a supplier, and AsscherAi's natural language query system returns the relevant information. The system's ability to understand the context of the question is crucial, and this is where the concept of a context window comes in. A context window refers to the amount of text that a model can consider at once when generating a response.

Definition

The context window is a critical component of natural language processing models, as it determines how much information the model can take into account when answering a question. In the case of AsscherAi, the context window is the amount of text that the system can analyze when generating a response to a user's query. This can include the question itself, as well as any relevant context or background information.

For example, if a user asks "What is the status of my purchase order?", the context window would include the text of the question, as well as any relevant information from the purchase order table. The model would then use this information to generate a response, such as "Your purchase order is currently being processed" or "Your purchase order has been shipped".

Difference from memory

One common misconception about context windows is that they are equivalent to memory. However, this is not the case. While a model's memory refers to its ability to store and recall information over time, a context window refers specifically to the amount of text that the model can consider at once. In other words, memory is about what the model can remember, while a context window is about what the model can see.

This distinction is important, as it highlights the limitations of natural language processing models. While a model may be able to store large amounts of information in its memory, it can only consider a limited amount of text at once when generating a response. This means that the model must be able to selectively focus on the most relevant information in order to generate an accurate response.

Applications in a large organisation

In a large organisation, context windows can be applied in a variety of ways. For example, a shift supervisor may use AsscherAi to check the maintenance log for a particular piece of equipment, and the system's context window would allow it to consider the relevant information from the log when generating a response. Similarly, a category manager may use the system to compare two suppliers, and the context window would allow it to consider the relevant information from the supplier database when generating a response.

By using a natural language query system like AsscherAi, organisations can make it easier for employees to access and analyze complex information, and the context window is a critical component of this process. To learn more about how AsscherAi can be used in your organisation, visit our homepage.

Limitations

While context windows are a powerful tool for natural language processing, they are not without their limitations. One of the main limitations is that the model can only consider a limited amount of text at once, which can make it difficult to generate accurate responses to complex questions. Additionally, the model may struggle to understand the context of the question if the context window is too small.

For example, if a user asks a question that requires the model to consider a large amount of text, such as "What is the history of our company's sales figures?", the model may struggle to generate an accurate response if the context window is too small. In this case, the model may need to be trained on a larger dataset or use a more advanced algorithm to generate an accurate response.

When to use an alternative approach

In some cases, a context window may not be the best approach for a particular problem. For example, if the problem requires the model to consider a large amount of structured data, such as a database, a different approach may be more effective. In this case, the model may need to use a more traditional database query approach, rather than relying on a context window.

If you are unsure about whether a context window is the right approach for your organisation's needs, we recommend contacting our support team for more information. You can reach us through our contact page. We can help you determine the best approach for your specific use case and provide guidance on how to implement it.