• Written by Azhar Aliyar, Director

A data analyst at a large retail company is checking the purchase order table in AsscherAi, a platform that lets her query the company's own data in natural language and get answers in real time, when she notices that the answer to a previously working question is now incorrect, which is where regression testing AI comes in, a type of testing that ensures changes to the system do not break existing functionality, in this case, the ability to accurately answer questions about the company's data.

Definition of Regression Testing AI

Regression testing AI is a process that checks if changes to a system, such as AsscherAi, have introduced new errors or bugs, by re-running previously passed tests, in this case, a set of predefined questions that the system should be able to answer correctly, and verifying that the answers are still correct, which is crucial in a large organisation where data is constantly being updated and the system is being modified to accommodate new data sources or query types.

The goal of regression testing AI is to catch the change that quietly broke an answer, and to do so in a way that is efficient and scalable, which is why it is often automated, using scripts and algorithms to run the tests and verify the results, rather than relying on manual testing, which can be time-consuming and prone to human error.

Difference from Other Types of Testing

Regression testing AI is often confused with other types of testing, such as unit testing or integration testing, but it is distinct from these, in that it focuses specifically on the impact of changes to the system, rather than testing individual components or the integration of different components, and it is typically performed after changes have been made to the system, rather than during the development process.

While unit testing and integration testing are important for ensuring that individual components and the overall system are working correctly, regression testing AI is necessary to ensure that changes to the system do not introduce new errors or bugs, and that existing functionality is not broken, which is particularly important in a large organisation where the system is complex and constantly evolving.

Applications in a Large Organisation

In a large organisation, regression testing AI can be applied in a variety of ways, such as testing the impact of changes to the data sources or query types on the answers provided by AsscherAi, or testing the integration of new systems or components with existing ones, and it can be used to test the system as a whole, or individual components, depending on the needs of the organisation.

For example, a shift supervisor at a manufacturing plant can use AsscherAi to query the maintenance log and get answers in real time, and regression testing AI can be used to ensure that changes to the maintenance log or the query types do not break the ability to get accurate answers, which is critical in a production environment where downtime can be costly.

Regression testing AI can also be used to test the system's ability to handle different types of queries, such as natural language queries or structured queries, and to ensure that the answers provided are accurate and consistent, which is important for building trust in the system and ensuring that it is used effectively.

What Regression Testing AI Does Not Do

Regression testing AI does not test the functionality of new features or components, but rather ensures that existing functionality is not broken by changes to the system, and it does not replace other types of testing, such as unit testing or integration testing, but rather complements them, by providing an additional layer of testing and validation.

It is also important to note that regression testing AI is not a replacement for human testing and validation, but rather a tool to support and augment human testing, and it is not a one-time process, but rather an ongoing process that is performed regularly to ensure that the system remains stable and functional over time.

When Regression Testing AI is the Wrong Choice

Regression testing AI is not the right choice for every situation, and there are cases where other types of testing may be more appropriate, such as when testing new functionality or components, or when the system is still in the development phase, and it is also not the right choice when the changes to the system are minor or do not affect the existing functionality.

In these cases, other types of testing, such as unit testing or integration testing, may be more suitable, and it is always a good idea to evaluate the specific needs of the organisation and choose the type of testing that best fits those needs, and to contact us for more information on how to implement regression testing AI in your organisation.