The evolving role of AI and data science within the Quality Ecosystem

In our latest World Quality Report – our annual global survey in which we reach out to 1,750 organisations in more than 30 countries to assess trends and attitudes in and around Quality Engineering and Testing – we wanted to understand the ‘quality ecosystem’ in depth. We wanted to identify what key trends are emerging, and what role advancements in the likes of AI and data science can play in the future…

Metamorphosis of cloud testing

With the report now in its 15th year, some significant trends have emerged, as you would expect over this period. And one of the biggest shifts we’ve seen this year is the move towards cloud testing, with 82% of respondents highlighting this as mandatory for applications on cloud.

This compares with 57% of respondents in last year’s report, and highlights a positive and decisive shift in the testing strategy that organizations are adopting on their cloud and infrastructure testing. It also demonstrates how important it is to test cloud-related features for functional and non-functional aspects of applications. We believe this change in thinking is a result of organizations realizing that movement to cloud alone does not make the system available and reliable.

Controlling the chaos

As more organizations make the inevitable leap to the cloud, this question of reliability becomes increasingly important. Applications hosted on cloud can be severely impacted by outages of the cloud services. As a result of this exposure, we are seeing an increased focus on site reliability engineering (SRE) which applies engineering to operations, including the adoption of chaos engineering to replicate disaster scenarios. The use of automation and integration is certainly emerging as a key tool in the QE landscape in this area, with 69% of respondents recognizing their usage.

Simultaneously, we are also seeing a rise in the importance of testing failure in cloud events, with 71% of respondents citing it as ‘very important.’ This use of chaos engineering within many Quality Assurance (QA) organizations helps to validate if applications hosted on cloud can handle any unplanned infrastructure outages, service, or configuration changes gracefully without significantly impacting the end-user experience. So, what we are witnessing here is an increasing need for QA organizations to adopt chaos engineering practices and tools to support their customers.

The growing importance of data and AI

Effective test data management is vital in the test ecosystem. The growing demand for end-to-end automation highlights the need for abundant, on-demand test data. This year, 69% of organizations have adopted or prepared for an enterprise-wide test data provisioning strategy— a significant leap from last year’s 31%. And in total, a whopping 99% of respondents have at the very least started to develop or prepare their strategy.

Meanwhile, when it comes to mobile application testing we see perhaps the biggest shift towards the use of AI and data-science in this area, with 76% of organizations employing AI and data science based approaches to make their mobile application testing more effective. Cloud-based automated testing with AI/ML techniques has further refined testing efforts to cater for the vast diversity of mobile devices worldwide.

In summary…

This is just a snapshot of the report’s findings, so we’d encourage you to request a copy of the report to explore in more detail. What is clear is that AI and other emerging technologies will continue to have an ever-increasing impact in the quality ecosystem as the various technologies evolve.

It is great to see that more organizations are moving non-production environments to cloud even though it is more concentrated on a hybrid strategy. We also see that organizations increasingly embracing SRE and leveraging chaos engineering and AI-data science approaches to specialized testing as a move in the right direction.

Our recommendations and summary:

• Accelerate the inclusion of cloud and infrastructure testing as part of the software development lifecycle, to improve resiliency, security, redundancy, and data recovery.
• Continue to consider intelligent integrations (including customizing specific features) to make the best use of commercial off-the-shelf and cloud native tooling – as a hybrid strategy for non-production environment provisioning.
• Adopt stringent practices to test the impact of failure events on cloud and continue to give extremely high priority to the performance and resilience of applications on cloud.
• Invest in workforce transformation in building SRE and chaos engineering capabilities within quality engineering organizations to embrace and support the emerging trends in digital and cloud migration strategies of the customers.