Quality and testing in the next 30 years

With this perspective in mind, Rik Marselis delves into the comprehensive framework of TMAP, exploring its evolution, resources, and the integration of quality engineering principles into AI-driven projects. With over 40 years of experience in systems development, Rik has assisted numerous organizations in improving their IT processes, establishing quality and testing approaches, and setting up quality and test organizations. He is an accredited trainer for TMAP, TPI, and ISTQB certification courses, and has contributed to over 20 books on quality and testing, including “Quality for DevOps Teams.” 

“Find the errors and fix them.” That’s how software testing began. The position of quality assurance and testing has changed enormously over the past 30 years. Detecting errors in systems afterwards is now a side issue. Of course, it is still very important that systems are tested for functionality, performance, user-friendliness, security and, for example, compliance with laws and regulations such as the GDPR, NIS2 and now also the AI act. Many test processes have now been automated. For example, research shows that application regression testing is a fully automated process in more than half of our country’s organizations. And where previously there was hardly any relationship between IT quality and the impact on business results, this is now also increasingly commonplace. For example, it has been proven that the responsiveness of an app can lead to 15% more sales. So-called business value-based testing is therefore also an important condition for the contemporary success of TMAP.

Our way of working has also undergone a metamorphosis. Stand-alone Testing Centers of Excellence often make way for integration into the Agile and DevOps working methods. Quality & Test Engineers are increasingly working in an Agile and DevOps environment. Quality management is thus an integral part of lifecycle management.

New era thanks to AI

With the large amount of digital products becoming available, the costs for quality assurance and testing have risen to an undesirably high level for many organizations. Endless testing is no longer an option. With the advent of artificial intelligence, testing is shifting from quality testing to predicting quality. The amount of historical data and the self-learning capacity of generative AI ensures that the right testing approach is completed in collaboration with Gen AI tools. Algorithms also provide insight into the impact on software adjustments. The so-called Large Language Model (LLM) does what you ask. Just as the calculator replaced the main calculator, the tester is now presented with test cases or test data test sets in no time with a smart prompt. A solution for dealing with the increasing amount of test and quality work in a society that is rapidly becoming more digitized and where (partly due to the application of Gen AI) the development time for software is becoming shorter and shorter.

At the same time, there is also a danger there. In this way, novice testers develop into an experienced quality and test consultant in a completely different way. Junior testers are no longer used to thinking critically because they often blindly trust AI. After all, it always goes well. Just like that calculator we have been relying on for decades. However, several incidents have now shown that AI can also fool you relatively easily. Moreover, AI can tailor everything, which only increases complexity. Finally, AI developments are moving so fast that it is hard to keep up. What seems good today may be outdated tomorrow. For example, the development of instructions to Gen AI via self-made prompts to the use of fully-fledged autonomous AI agents creates even more dependence on smart technology. Which raises the question of who will monitor and guarantee the quality of those autonomous AI agents.

Here we see the (already known to many) two-sidedness of quality and AI. On the one hand, the use of AI to build in and guarantee the quality of IT systems. On the other hand, the use of quality engineering and test knowledge to guarantee the quality of the artificial intelligence itself.

And for both, the TMAP body of knowledge offers knowledge and tools. To build in and guarantee the quality of IT systems, TMAP is used as the basis for, for example, the Gen AI Amplifier, an advanced AI-based service that is already operational. And TMAP offers a wide range of visions, techniques and tools to test and improve the quality of Gen AI, such as the application of test design techniques but also a structured framework for creating the right prompts.

In short, the use of generative AI for quality assurance and testing tasks offers enormous opportunities. The interaction between human and machine remains a considerable challenge. Humans are not being replaced by AI. But the great art is to get people to work with AI in a responsible way that does justice to the qualities of both humans and machines. With the right cooperation and support from the TMAP body of knowledge, we will certainly have another 30 great years of quality engineering and testing!!