State of AI applied to Quality Engineering 2021-22

Section 8: Operate

by Sogeti

Business ●●●●●
Technical ○○○○○

Listen to the audio version

Download the "Section 8: Operate" as a PDF

Use the site navigation to visit other sections and download further PDF content

By submitting this form, I understand that my data will be processed by Sogeti as described in the Privacy Policy.*

Introduction

Quality Engineering is based on the premise that software quality does not occur at a fixed point in the software development lifecycle – which means that even when new capabilities are deployed, quality is not complete.

The end-user perspective raises fundamental concerns about the overall accuracy of testing. Indeed, as detailed in the 2020 Continuous Testing Report, developing, and maintaining meaningful test cases that align with end-user expectations continues to be a significant challenge.

Even when static analysis steps and tools are used, and even when exploratory testing is combined with effective test automation, quality validation will miss some of the useful and relevant information for end-to-end quality assurance purposes. To close these gaps and maximize our test coverage, we used "shift right."

"Shift right" entails leveraging the breadth and depth of production data and user behaviour, feeding it back, learning from it, and incorporating that knowledge into new iterations. It enables the extension of continuous testing, resulting in a more holistic approach to the development lifecycle and an increase in the accuracy of the validation process.

We have implemented numerous shift-right practices as an industry. We do use operational mobile analytics to determine and optimize test coverage. We do monitor the health and performance of in-production applications and services. Through synthetic monitors, we do run test scenarios in production. However, we could significantly improve our quality engineering activities by utilizing production data more effectively.

  • How to resolve a conflict in minutes before it has a negative impact on the outcome?
  • How to correlate the user journey with the model-based testing flows, thereby increasing the accuracy of our test cases?
  • How to use production logs to identify missing test cases and data scenarios.
  • How to develop more realistic automated scripts and make it easier to reproduce crashes?

Through five chapters, this section examines the applications of artificial intelligence to Operations.

In this section

Chapter 1: The Rise of Intelligent Observability in Software Development

Chapter 2: Smart impact analysis for SAP

Chapter 3: From Logs to Tests: Usage-driven Regression Testing

Chapter 4: Log mining for testing with AI

Chapter 5: Industry 4.0: from point data to business impact