To see how AI can contribute to improved processes and better software, we need to look at how software test automation can be augmented through AI. Broadly speaking, the difference between AI and simple automation is that instead of relying on following explicitly programmed rules, AI solves problems based on its understanding of an environment. In the context of testing a piece of software, the environment is the application and its current state, and the AI’s ability to understand the environment is based on a model of the application and an interpretation of the elements within. As we will see, AI algorithms can be very useful for understanding and testing an application.
For the purposes of this section, we define AI as a system that is able to gather information from an environment, determine an action to be taken, and learn from the results of taking that action. Just as Jeff Bezos describes how AI is “quietly but meaningfully improving core operations,” AI can improve the efficiency and effectiveness of software testing. In practice, this is achieved using a range of techniques.
Well-defined scenarios can be mapped out using “IF” statements, but this has the danger of becoming unwieldy and impractical for the numerous unique test flows, data, and environmental variations seen in a test framework. There are more advanced statistical approaches (variance, probabilistic), traditional machine learning (ML), or the even more advanced deep learning to build a data model than can be queried in production to help decide on an appropriate action. Finally, techniques such as unsupervised and reinforcement learning inherently change internal representations as a result of the systems with which it interfaces.
The real value of these techniques is to have a material impact on improving the effectiveness of the tests they produce. The purpose of rigorously testing an application isn’t to reach some arbitrary internal KPI; it’s to make sure that real users, whether customers or employees, don’t encounter errors that prevent them from achieving their goals. And because real users might not behave in a manner that the application’s developers had in mind, it’s important to take into account a wider set of user journeys.
AI algorithms are used to help determine whether users will be able to complete their intended task and achieve their goal. Two categories of algorithms in particular help accomplish this aim.