Chapter 2 by Applitools & Sogeti
Business ●●●●○
Technical ●○○○○
Listen to the audio version
In the previous chapter, we discussed the challenges faced by modern quality engineering. We have seen that the quality requirements for product teams have increased, particularly for teams whose products are consumed via web or mobile applications. In this chapter, we will take our first steps toward examining the convergence of quality engineering and artificial intelligence. Let’s start with a personal anecdote.
Allow me to relate a recent experience with a rental car agency website. I was unable to log in late one evening during the pandemic. I was confident that my account number was correct. My password failed completely. The error message stated that I needed to contact user support regarding the loyalty program, but they were not available by phone at the time. I could have used text-message support, but that did not sound appealing. I wanted to attempt to resolve this issue on my own. I was unable to. However, rather than granting me access, the password reset process created a new account for me.
As a result, I connected with a chat agent online. After several exchanges, it was determined that my expired driver license was preventing me from logging in. My credentials worked flawlessly after the agent updated my expiration year.
So, did I encounter a bug? The system could have enabled my login and warned me that my license had expired. Instead, the system required manual intervention on their end and resulted in frustration on mine. The highest value in product development comes from creating products that delight users. When engineers have more time available, they can consider whether the current design delights customers - even if it works.
How could waste be reduced further? How could this company significantly improve their quality engineering throughput and productivity?
We can surely all relate to the previous real-world issue. Now that we are returning to quality engineering, how to get started and address any issues before they impact our end users?
You might be familiar with the periodic table of software engineering[1], which is a collection of the most critical and fundamental elements of quality engineering. It can be used as a guideline for what a software engineer should learn, understand, and most importantly, practice. They vary in depth and importance. Certain tools are capable of optimizing a complex network of actions. Several of them are capable of controlling a graphical user interface (GUI) or an application programming interface (API). Some are capable of analyzing code coverage and recommending additional actions. Some are capable of analyzing log files in search of specific behaviors. Additionally, some of them are capable of performing specialized analysis, such as visual inspection of the rendered UI to detect anomalies that make the UI effective. What about the task of writing test code? What if you could manipulate your user interface using web controls rather than web element locators. What if you could continue running your existing tests as long as the interface elements remain - even if the underlying locator code changes?
We need to assist engineers with complex tasks and reduce the need for low-value human activities. To increase our productivity, we could consider two types of tasks: routine and error prone.
With the support of modern tools, a software quality engineer might consider optimizing the following activities, each being covered by a specific section in this report.
[1] https://sites.google.com/site/markussprunck/blog-1/periodic-table-of-software-engineering-know-how
As we are about to deep dive into Artificial Intelligence applied to Quality Engineering - and you have come to the right place - caution should be exercised when conducting research in this field.
First, look for a problem statement that appeals to you. Some organizations will talk about slow, manual processes. Some will talk about error-prone processes. Do you or your team encounter these difficulties? Do you see any issues that could be automated?
Often, websites and product materials talk about the benefits or advantages of a given technology or solution. The company may have published white papers or customer success stories. Those serve as a starting point.
Realistically, AI in Quality Engineering is crossing from organizations that take technology risks to those that spend more time assessing risks. Many vendors tout their customer successes.
Reach out to their references with appropriate questions:
Beyond the customer recommendation, try to find the same answer from the company - through its website, sales team, and product literature. Take as much risk out of the process. If possible, find an implementation partner with experience deploying the product. Prepare your engineering resources for such technology adoption. They might perceive it as a threat rather than an opportunity to free them up for higher-level tasks.
Now, time for Artificial Intelligence.
Our intelligence is what makes us human, and Artificial Intelligence (AI) is an extension of that quality
says Yann LeCunn (Chief AI Scientist, Facebook).[1]
The concept of artificial intelligence is not new. Humanity has always been obsessed with developing machines capable of performing some of the tasks that humans can, culminating in the invention of the digital computer. In 1956, at a conference at Dartmouth College in Hanover, New Hampshire, the field of artificial intelligence was formally founded, and the term 'Artificial Intelligence' was coined. Between 1957 and 1997, AI experienced ups and downs, but interest soared after IBM's Deep Blue defeated Gary Kasparov in 1997. With the advent of GPU-based processing in the late 2000s, AI became accessible to the general public, and the field has continued to grow since. As Andrew Ng[2], Stanford university professor and founder of Deeplearning.ai correctly points out, the change that AI is bringing is analogous to the change that electricity brought to society.
There is no universal or legal definition of artificial intelligence. We frequently use the terms "AI systems," "AI applications," and "AI uses" for the sake of clarity. The term "artificial intelligence" is imprecise and can encompass a range of perspectives. This is because a variety of different techniques are being used to create extremely complex and (partially) autonomous technologies, all classified as artificial intelligence.
In a February 2020 white paper[3], the European commission states that: ’Simply put, AI is a collection of technologies that combine data, algorithms and computing power’. And also adds that this wording cannot be considered sufficient to define AI.
Using oversimplified terms, Artificial Intelligence (AI) attempts to simulate human intelligence through the use of computer systems. Humans are particularly good at learning and adapting to new environment based on what is learnt. AI tries to achieve these qualities using a set of mathematical algorithms which tries to simulate the way in which the human brain works. In very simple words of Yoshua Bengio[4], one of the world's leading experts in AI and a pioneer in deep learning, “AI is about making computers that can help us that can do the things that humans can do but our current computers can’t.”
In software quality engineering, what do vendors mean by "artificial intelligence" (AI) and "machine learning" (ML)?
The terms are frequently used interchangeably to refer to the use of compute engines for pattern recognition, analysis, and recommendation. In an ideal world, AI would imply the development of machines capable of emulating human learning, reasoning, and perception. As a subset, machine learning implies that machines can learn and act independently of human intervention.
In practice, AI refers to algorithms and approaches used in tools that perform human-like tasks. Which functions are comparable to those of a human? Typically, artificial intelligence makes use of a combination of sensing, learning, and reasoning. Consider reasoning. A person can reason by examining an application and determining which functions have been adequately tested and which may require additional testing. While AI may require training, once trained, it can continue to use the tests for that function as it evolves.
While this description might good enough for business profiles, we are now going to take a closer look at Artificial Intelligence. The following technical chapters will define common AI terms, such as machine learning and deep learning, and map quality engineering problems to such technology.
[1] https://research.fb.com/blog/2016/12/artificial-intelligence-revealed/
[2] https://twitter.com/AndrewYNg
[3] https://ec.europa.eu/info/sites/default/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf
[4] https://yoshuabengio.org
Moshe Milman
Moshe brings over 15 years of experience in leadership, engineering and operational roles in public and private companies. Prior to founding Applitools, Moshe held leadership roles at Wave Systems Corp., Safend, Amdocs (NASDAQ:DOX) and Sat-Smart Ltd. Moshe holds a Bachelor’s degree in Information Systems from the Technion Israel’s Institute of Technology and MBA from IDC (Wharton Business School GCP program).
Antoine Aymer (chief editor)
Antoine Aymer is a passionate technologist with a structured passion for innovation. He is currently the Chief Technology Officer for Sogeti's quality engineering business. Antoine is accountable for bringing solutions and services to the global market, which includes analyzing market trends, evaluating innovation, defining the scope of services and tools, and advising customers and delivery teams. Apart from numerous industry reports, such as the Continuous Testing Reports, the 2020 state of Performance Engineering, Antoine co-authored the "Mobile Analytics Playbook," which aims to assist practitioners in improving the quality, velocity, and efficiency of their mobile applications through the integration of analytics and testing.
Venkatesh Babu
Venkatesh Babu is a technology leader, with 22+ years of experience in JEE, .NET, Mobile, Cloud, Automation, SMAC, IoT, RPA, AI/ML, Digital technologies - architected, designed and delivered enterprise solutions for global clients. He is working in the Research & Innovation Group and passionate about Emerging Technologies, Focus areas include Cloud, Artificial Intelligence & Machine Learning IoT, Gamification, Gesture Recognition, Augmented Reality, Blockchain, Big Data, Microservices, Design thinking, Solution Architecture & Consulting, Application Integration, Product Engineering, Wearables, Crowdsourcing and Technology evangelization.
Applitools delivers the next generation of test automation platform for cross browser and device testing powered by AI assisted computer vision technology known as Visual AI. Visual AI helps Developers, Test Automation Engineers and QA professionals release high-quality web and mobile apps enabling CI/CD.
Hundreds of companies, including industries such as Technology, Banking, Insurance, Retail, Pharmaceuticals, and Media - including 50 of the Fortune 100 - use Applitools to deliver the best possible digital experiences to millions of customers across all screens.
Visit us at www.applitools.com
