Key Findings of the World Quality Report 2017/8

Disruptors, Trends & Priorities

As digitalisation continues apace, the most significant current disruptor to the QA and testing industry is the evolution of smart devices. Leading businesses are creating more mature Agile and DevOps strategies and increasing their levels of automation on their journeys towards the primary goal of continuous delivery.

In this year’s World Quality Report, created by Capgemini, Sogeti and Micro Focus responses from 1,660 executives in 32 countries show the 3 main areas that the QA and Test function should prioritise in order to remain competitive in 2018 are:

1. Intelligent test automation and smart analytics
2. The creation of smart test platforms
3. A more mature Agile approach to QA and Testing

Interestingly the Report shows that the biggest priority for test teams is getting back to basics and achieving excellent product quality. 41% rate software quality as the primary objective, up 5% from last year, and the only goal to have risen since last year’s Report. The fact that last year’s primary objective, the detection of defects before go live, has fallen 12%, shows how far testing has come in the last 12 months in terms of taking a preventative approach to bugs.

Test teams have also moved away from the trend of centralised governance via Testing Centres of Excellence (TCoEs) to enable individual teams to mature their Agile and DevOps methodologies, choose the technology that best enables their projects and create their own QA and test strategies.

Overview of #WQR17 Findings

Automation Under-exploited

The average percentage of automation being carried out in suitable test activities is 16%, which is around 50% lower than in last year’s Report. This figure shouldn’t be taken at face value as, rather than showing a decline in automation, it actually indicates a maturity in this area whereby test organisations are recognising the scope of what can in fact be automated. This has resulted in a more strategic approach to what they automate as opposed to trying to take a blanket approach and automate everything.

Cognitive automation, machine learning, self-remediation and predictive analysis are the most favoured emerging techniques to support DevOps, mobile and IoT. To realise the true value of automation, businesses need to broaden their automation strategy to include not just test plan, design and execution but also test environments and test data provisioning.

Intelligent Test Automation & Smart Analytics

Intelligent test automation and smart analytics will become essential to support testing as they enable smart decision-making, fast validation, and automatic adaptation of test suites. The scope of test automation has evolved from simply automating the test activities plan, design, and execution, to automating test environments, test data provisioning, API validation and service integration.

To see a return on their automation investment organisations need to align their automation strategy to key business objectives such as speed to market rather focussing on minor or individual project wins in cost and efficiency.

Test Platforms Get Smarter

Test environments, test data and virtualisation are reported to be the 3 biggest challenges facing the modern test organisation. In order to comply with the new General Data Protection Regulation which comes into force next year, these challenges need to be addressed. The focus therefore needs to be on smarter test platforms that provision self-aware and self-adaptive test environments, combined with intelligent lifecycle automation and an automated approach to test data generation and test data management. The first step on the journey to smarter test platforms is the creation of cloud-based environments. This “Smart QA” approach will be the defining competitive differentiator in the coming year.

Testing in Agile Development

As Agile matures the challenges of testing in an Agile development increase. 46% of respondents find lack of data and environments to be the most pressing obstacle to success, while 45% cite reusing tests across Agile sprints as the biggest challenge. On the plus side we’ve seen an improvement in establishing what to test and when, and a rise in test teams shifting left to get involved at the outset of sprint planning.

QA & Bimodal IT

The role of Testing Centres of Excellence (TCoEs) has changed to one of enablement and support as opposed to test execution and, along with the new function, comes a new name: Test Excellence Centre (TEC). Organisations are more evenly divided between those taking a centralised or decentralised approach, as they choose what works for them rather than following the latest trend.

Test Environments & Test Data

The 3 biggest challenges in this area are:

1. A lack of sufficient Agile development and test environments (56%);
2. Managing the growing requirement for different types of test environment (48%); and
3. The difficulties associated with predicting test environment utilisation and demand (47%)

A rise in containerisation is improving performance and helping to assuage the issues around provisioning test environments, while 73% of respondents have found the adoption of cloud-based environments for non-functional testing useful.

The Rise and Fall (& Rise) of Test Budgets

The proportion of the total IT budget dedicated to Testing was 35% in 2015 and 31% in 2016. We’ve seen a positive drop to 26% this year bit this figure is expected to rise again in the next 2 years as digital transformation, mobile applications, and the Internet of Things continue to advance.

To discover what recommendations the World Quality Report proposes in order overcome the challenges discussed in this blog, download your free copy of the report here.