by Sogeti
Business ●●●○○
Technical ●●○○○
Listen to the audio version
As the amount of textual data we generate across the application lifecycle grows, processing and accessing that data becomes increasingly critical to make more informed decisions. We require assistance in analyzing, categorizing, and prioritizing the data at our disposal. The constraint is the human language's puzzling and diversified nature. We have distinctions in our syntactic, semantic, and regional languages. Additionally, comprehension of our QE domain expertise is required. This section discusses how artificial intelligence – more precisely, Natural Language Processing (NLP) – enables us to comprehend unstructured information and derive meaningful insights. Allow us to investigate the context of two different case studies before we introduce you to NLP in the next chapter.
Test case design and production are mostly manual activities that occupy up to 70% of the time over the lifecycle of a software test. Test cases prepared by inexperienced testers frequently do not adequately satisfy functional requirements. Additionally, requirement changes reveal the inability to reuse manually generated test cases, resulting in schedule slippage and expense increases. Numerous software projects employ some type of behavior-driven development to define requirements as they go from business stakeholders to plain language user stories.
User requirements must eventually be converted to natural language—so that they are easy for humans to comprehend and implement. Natural language test cases have limits due to the possibility of improper interpretation resulting in verification mistakes.
Traceability for user requirements and test cases can be established and utilized for requirement to test coverage analysis. End user technical documentation, which often covers features and functions, is never examined, and there is no mapping between test case techniques and technical documentation. To get understanding about high-end and sensitive systems that are controlled by professional operators, it is recommended to go to the end user technical paper. This highlights the critical nature of ensuring that the different areas/workflows stated in the technical specification are covered by the test procedures designed and identifying any gaps that may exist.
For any company developing software, quality control should stand out as a differentiating factor in ensuring the product's quality, stability, and resilience. To ensure optimal coverage of software features, it is necessary to write and implement suitable test procedures against the feature set. Traceability between product characteristics and test procedures is a demanding and complex task.
This context was the one faced by a medical device OEM firm situated in North America. End-to-end traceability between product features and test processes was established using Natural Language Processing and advanced deep learning techniques, ensuring that product features received the maximum amount of test coverage. It proved extremely beneficial in terms of product quality assurance and cost reduction by reducing the difficult process of developing test strategy and design. With the assistance of the solution built, features with sparse or no test coverage were found, helping test managers enhancing their test strategy, planning, and design. The underlying technology stack was IBM Watson, Java, Cloudent DB, IBM Bluemix.
When a software project grows in size, managing the workflow and development process gets more difficult. As a result, it is critical for the management team and lead developers to comprehend the nature of the work performed by software developers.
In simpler terms, any developer, at any point during the project's various phases, can write code that seeks to accomplish one of the following:
The goal here is to define some broad categories of work that span the vast majority of development tasks.
Understanding how the development team's work is allocated throughout the four categories above will assist management in making more informed decisions about how to manage the software's growth while continuously enhancing its functionality. If the majority of the development team's time is spent resolving anomalies, management should direct lead developers to place a higher premium on quality.
To accomplish this, we can make use of the commit comments, which are provided in natural language by developers. These comments can be labeled and used to train classifiers that will subsequently classify future changes automatically.
This is the methodology taken by Capgemini engineering in the healthcare, automotive, and independent software vendor (ISV) sectors. The underlying technology is Python, Sklearn, NLTK, Numpy, Pandas, MySQL.
The method built for classifying commits using natural language processing achieved greater than 90% accuracy and assisted in classifying incoming commits into one of the categories indicated above. It became easier to assess the software development process and workflow of a very large team using this software engineering solution. The allocation of effort and the project's momentum were easier to track and optimize.