WQR: intelligent product testing

This has created layers of complexity ranging from technological challenges to regulatory and legal ones. The pressure to create, maintain, and upgrade products requires ongoing testing, otherwise companies releasing faulty products can face an immediate backlash on social media.

Furthermore, customers’ expectations continue to rise. A faulty product not only leads to reputational damage but can quickly open up the market to new entrants.

In the recent edition of the World Quality Report, we evaluate the present condition of Quality Engineering & Testing practices (QE&T) worldwide, spanning various industries. This annual global survey entails contacting 1,750 organizations across over 30 countries to analyze trends and perspectives related to QE&T.

As testing is one of the pillars of digital success, this blog looks at the trends and expectations for intelligent product testing.

End-to-end product testing

Surprisingly, despite the nature of how interconnected products and systems are, only 36% of those surveyed thought that End-to-End (E2E) testing was important. However, as we believe in the importance of E2E testing, this will change over time.

While E2E testing has cost implications due to its complexity, you will not obtain accurate test results if you only test individual items in isolation rather than as part of an entire process. It is only with E2E testing that you will be able to refine your test cases leading to better test definition.

Test definition

A significant trend is to improve the test definition. The complexity of products alongside the hyper-personalization of such products requires potentially millions of test cases. For example, the testing of a new feature in Android must take into account thousands of different device scenarios, which in reality, is difficult to determine.

Our survey shows that the most important aspect of intelligent testing is test case selection (52%) which is followed closely by test case prioritization (50%), and in third place, root cause analysis (47%).

Test case selection allows you to efficiently reduce the number of test cases at each testing iteration, which reduces both the execution time and associated costs. Whereas test case prioritization helps to deliver value faster, as the aim is to detect bugs already in the process and to keep correcting and testing them.

Ultimately, we see the testing process shifting towards the many benefits of AI.

AI and testing

Returning to test plan definition and the automatic generation of test cases, AI and GenAI are at the heart of this, and so unsurprisingly, there are high expectations for the use of such technology.

As for the particular skills required, 44% of respondents believe that they will need to use both supervised and unsupervised Machine Learning technologies in their activities. Furthermore, to a lesser extent (42%), natural language processing techniques are also required.

While applying AI technologies to testing is clearly advantageous and there are considerable long-term savings, AI requires a specific skillset. For example, traditional programming language skills will be less important than Machine Learning capabilities.

Our survey details a 10-percentage point difference: the importance of programming languages (31%) compared to Machine Learning skills (44%) shows the direction some companies are taking.

We believe the trend towards AI is inevitable, and therefore, Machine Learning skills (among other AI skillsets) are set to keep growing in demand.

Recommendations

1. Consider an E2E testing approach to ensure seamless customer experience through the effective utilization of the abstraction of various architecture layers which includes embedded software, cloud, and connectivity.
2. Invest in AI solutions for test prioritization and test case selection to drive maximum value from intelligent testing.
3. Pay attention to your skillsets and if required, start upskilling to take advantage of AI testing capabilities.

 

Author