Demystifying deep learning for QE

As a continuation of the previous chapter, deep learning is a subset of machine learning in artificial intelligence that utilizes neural networks to learn unsupervised from unstructured or unlabeled data. It also referred to as deep neural network or deep neural learning. Deep learning is accomplished through the use of an artificial neural network that behaves very similarly to the human brain and allows the machine to analyze data in a structure very much as humans do. Deep learning machines are self-programming and do not require a human programmer to instruct them on how to use the data. This is possible because of the massive amounts of data we collect and consume—data serves as the fuel for deep-learning models. The availability of such a large volume of training data is a necessary condition for deep learning.

Numerous artificial intelligence (AI) applications and services leverage deep learning to increase automation by performing analytical and physical tasks without human intervention. Several examples include online self-service solutions and chatbots. Driverless cars are embedding a variety of deep learning models for recognition of street signs and pedestrians.

Deep learning is used in practice for quality engineering purposes. Later in the report^[1], we'll delve into the ways in which computer vision, a subfield of deep learning, can benefit quality engineering. In early days, it was used to compare two images or even replace traditional test automation frameworks. Its scope continues to expand to image classification, restoration and segmentation, object detection, text recognition (even handwritten digits can be recognized), and more.

[1] This report’s section "Automate & See" discusses how computer vision is being applied to quality engineering.

A neural network is formed by interconnected layers of software-based calculators known as "neurons" in deep learning. The network can ingest and process input data via multiple layers that each learn increasingly complex features of the data. The network can then make inferences about the data, verify that its inferences are correct, and apply what it has learned to make inferences about newly incoming data. For instance, once it has learned how an object appears, it can recognize it in a new image.

A neural network is structured like the human brain and consists of artificial neurons, also known as nodes. These nodes are stacked next to each other in three layers: 

1. The input layer
2. The hidden layer(s)
3. The output layer

Data provides information to each node in the form of inputs. The node multiplies the inputs by random weights, calculates them, and modifies them with a bias. Finally, nonlinear functions, also known as activation functions, are applied to determine which neuron to fire.

Major Deep Learning Algorithms: 

1. Convolutional Neural Networks (CNNs) 
2. Recurrent Neural Networks (RNNs) 
3. Long Short-Term Memory Networks (LSTMs) 
4. Generative Adversarial Networks (GANs) 
5. Auto encoders 

Fundamentals

What it is

Convolutional Neural Networks (ConvNet/CNN) are a type of Deep Learning algorithm that are optimized for processing data with a grid-like topology, such as images. It prioritizes (learnable weights and biases) various aspects/objects in the image and is capable of differentiating one from the other.

When to use it

CNN's are widely used to identify satellite images, process medical images, forecast time series, and detect anomalies.

How it works

1. The convolutional neural network (CNN) receives an image as a collection of pixels and processes it.
2. It identifies the image's unique features in the hidden layers.
3. The CNN can now classify a different image if it contains the same unique features as the classified image.

QE use cases

• Object Identification on display of System Under Test:
  Identify different objects/UI elements on display of System Under Test and process them for performing Verification & Validation.
  
  
• Identify System state based on Object Detection:
  Identify current system state by monitoring indicators on a User Interface of System Under Test.
  
  
• Image Comparison:
  Compare complex images to find out similarity. E.g. Handwriting comparison.
  
  
• Defect Count Prediction:
  Time series prediction for number of incoming defects based on historical count of daily incoming defects.

Fundamentals

What it is

A Recurrent Neural Network works by saving the output of a particular layer and feeding it back to the input in order to predict the layer's output.

When to use it

RNNs are frequently used in a variety of applications, including image captioning, time series analysis, natural language processing, handwriting recognition, and machine translation.

How it works

• At time t-1, the output feeds into the input at time t.
• Similarly, at time t, the output feeds into the input at time t+1.
• RNNs are capable of processing inputs of arbitrary length.
• The computation takes historical data into account, and the model's size does not increase in proportion to the input size.

QE use cases

• Multilingual Testing:
  Translate textual test data from one language to other for multilingual application testing.
  
  
• Commit Classification:
  Identify bug fix commits to analyze commit comments. Classify commits according to their purpose, such as feature enhancement, bug fix, functional change, non-functional change, and refactoring task.
  
  
• Analyzing Defect Rate:
  The defect rate can be determined by analyzing the time series of incoming defects over a period of time.
  
  
• Image Comparison:
  Compare complex images to find out similarity. E.g. Handwriting comparison.

Fundamentals

What it is

LSTMs are a subclass of Recurrent Neural Networks (RNNs) that are capable of learning and remembering long-term dependencies. The default behavior is to recall past information for extended periods of time. They are advantageous for time series prediction because they retain information about previous inputs.

When to use it

LSTMs are used for time-series predictions, speech recognition, music composition, and pharmaceutical development.

How it works

1. To begin, they disregard irrelevant aspects of the previous state.
2. They then update the cell-state values selectively.
3. Finally, the output of specific components of the cell state.

QE use cases

• Defect Count Prediction:
  Time series prediction for number of incoming defects based on historical count of daily incoming defects.
  
  
• Release Readiness:
  Predict a product's readiness for release based on data from multiple sources in the production environment.
  
  
• Validate Visual Data by Classification:
  Validate visual data like video clips, images by Classifying them based on the visual content.

Fundamentals

What it is

A generative adversarial network (GAN) is a subclass of machine learning frameworks in which a pair of neural networks, a generator and a discriminator, compete to perform a task, resulting in improved task performance.

For instance, a GAN trained on photographs can generate new photographs that appear to human observers to be at least superficially authentic, exhibiting a variety of realistic characteristics.

When to use it

When sufficient data is not available to train an algorithmic model, GANs can generate new, synthetic data that is representative of the actual data. Additionally, they can identify new cyberattack and malware attack vectors, as well as fraudulent credit-card transactions.

Because the generator is constantly attempting to create new representations of the real data with slight variations, it may generate previously unknown types of possible attacks or fraudulent transactions.

How it works

The generator creates a synthetic representation of the underlying data (e.g., a duplicate image of a person) that is sufficiently accurate to increase the discriminator's error rate. The discriminator attempts to distinguish between genuine and synthetic data.

At first, the generator generates random patterns that the discriminator can easily distinguish from "real" data, but with each attempt, the generator generates more representative data. Over time, the generator will become extremely good at generating synthetic data, thereby increasing the discriminator's error rate.

QE use cases

• Test Data Generation:
  When given a set of test data, GANs can generate symmetric test data for a variety of data types, which is advantageous for performing additional functional testing on the System Under Test.
  
  
• System Security Testing:
  GANs can generate look-alike inputs to test the security of a facial or biometric authentication system when provided with authorized credentials.
  
  
• Identify Anomalies in Lab Results:
  GANs can be used in healthcare to identify physical anomalies in lab results that could lead to a quicker diagnosis and treatment options for patients.

Fundamentals

What it is

An autoencoder is a subtype of neural network that simply replicates its input to its output. It may be referred to as an identity function. An autoencoder consists of three main components: the encoder, the code, and the decoder.
The idea is that by copying the input to the output, the neural network picks up on useful properties of the input.
The code is the hidden layer h between the input and output layers. This represents the information about the input that is being gathered.
Encoder: Encodes the data to a hidden layer. Learn salient features of the input layer. 
Decoder: Decodes the data back from hidden layer to reconstruction of output layer. 

When to use it

Perfectly performing the copying task would simply duplicate the signal, which is why autoencoders are typically constrained in ways that force them to approximate the input, preserving only the most significant aspects of the data in the copy. Autoencoders are extremely useful for dimensionality reduction and feature learning because they retain only the most important features of the input data.

How it works

1. Autoencoders are structured in such a way that they can take an input and transform it into another representation. They then attempt to reconstruct the original input with the highest degree of accuracy possible.
2. In the absence of a clearly visible input image, it feeds an autoencoder neural network.
3. Autoencoders encrypt the image first, then compress it into a smaller representation.
4. The autoencoder then decodes the image to create the reconstructed image.

QE use cases

• Rectify Noisy Images in Medical Systems:
  Reduce noise in an image to analyze it more accurately for verification & validation purpose.
  
  
• Production Line Quality Control:
  Autoencoders can be trained to identify defective products on the Production Line, using the camera feed of the production line.