In many respects, mobile testing is not all that different from other types of IT testing. From methodology all the way through to most of the actual testing tools, what quality assurance organizations and experts have learned in non-mobile testing can be applied to mobile testing.
So, where are the differences and how should you accommodate them? We have identified ten areas that are highly relevant to mobile testing and require particular attention but if they are handled correctly will mean a successful mobile testing effort.
Most mobile solutions depend on some kind of network connectivity. Solution design and testing are needed to address variable band-width, offline and flight mode scenarios, and validate user sessions moving between different network conditions.
Both automated network simula-tions and manual testing in real network conditions are required to ensure consistent behavior.
Understanding Physical Characteristics
The physical characteristics in mobile solutions differ significantly from non-mobile solutions. Mobile testing needs to take several of these characteristics into consideration such as:
Some testing scenarios related to these physical characteristics can be automated, for example, orientation changes; but some scenarios do require manual testing, for example, synchronization of gestures and sounds.
An ever increasing number of solutions utilize location data and GPS integration. In these solutions, features are designed to depend on location or distance to other locations. Successful mobile testing ensures quality across different types of GPS implementations and needs to utilize efficient location simulation.
The ability to automate tests using scripting or recording makes mobile testing significantly more efficient. Smart test automation utilizes parameterized scripts for both device types and user interaction, and is enabled to run on devices in-house and in the cloud. Some sophisticated user interactions involve touch, gestures, and sensors which cannot be fully automated, and, as a result, will require manual testing.
Dealing with Fragmentation
The market fragmentation for both operating systems and device types continues to challenge solution design and testing. Most solutions, both internal and external, need to support hundreds of device types and several versions of operating systems. A relevant mobile testing tool box includes:
User experience is a main criterion for success. Testing user experience is an art that consists of both objective and subjective measurements. Key elements include:
Engaging with Third Parties
Most apps are distributed through open and public app stores, each with its own set of guidelines. Apps that fail to adhere to guidelines may be rejected. And, given the third party review process, there is a bug fix latency inherent in updates. Also, when operating systems are updated, the new releases can break existing apps. This means that successful mobile testing must consider:
End-to-End Integration Testing
As mobile solutions and apps become increasingly more business relevant, the scope and depth of transactional features expand and so the need for well-designed and validated system integration solutions has to be addressed as well. When specific systems are needed to remotely manage a fleet of handheld mobile devices (Mobile Device Management or MDM), the mobile solution should bevalidated and tested against the main func-tionalities of the MDM systems:
Mobile solutions can overload servers, through the rapid growth in the number of users and an increase in the average life span of transac-tions due to variations in bandwidth and latency. This can have a performance impact on all users. Also, there are significant performance differences across different types of devices. An app or responsive web may run well on a high-end device but may not be acceptable on a low-end device.
Successful mobile testing focuses on network capabilities, system integration and back end layers, as well as the app itself.
Mobile solutions are implicit in an increasing number of scenarios where sensitive systems are accessed and private data is in transit or at rest, that is, stored in mobile devices. And given the physical nature of mobile devices, they are more easily forgotten, lost, or stolen. Mature mobile testing aims to:
We are mobile testing experts and understand these central success factors and know how to leverage them. Our fine-tuned capabilities are summarized in this document, along with a description of our market-leading testing methodology, TMap NEXT®, which supports the complete testing process. We will explain in detail how our services can help your business achieve its mobile testing goals.