Why Software Testing for mobile applications is different? Top 10 reasons.

By on March 21, 2013 Comments

We have witnessed transition from desktop to web and are witnessing another transition from web to mobile. I have been thinking about a blog series around testing mobile applications for a while and this is the first blog post in the series. In the coming few weeks, I will try to cover various topics / products / approaches related to testing mobile applications. I will focus on Android to start with and will move on to other platforms.

Before I dwell deeper into the subject - it is important to understand how testing mobile applications is different from testing browser / desktop applications. If we understand the distinction and challenges of testing mobile apps, it will be a bit more easier to tackle them.

1. Supported platforms & devices - you have more combinations to test

Desktop apps were usually targeted for specific platforms and it was relatively easy to access those platforms. Web based applications made it a bit more challenging by adding another dimension - browsers.

Mobile applications take complexity of supported platforms to the next level by adding devices. Ensuring that mobile apps are working on all type of devices (SmartPhone, Tablets and Phablets) supplied by major brands (various models from Samsung, Sony, Nokia, HTC, Apple etc) and on all the platforms (iOS, Android, Windows, BlackBerry etc) is challenging. On top of that, new devices are hitting market so often that it becomes impossible to cover all the major devices.

In the mobile world, it is important to create something on the lines of graded browser support used by Yahoo to ensure that major platforms are covered.

2. Adaptability & Limited space - Screen size is changing constantly

Pretty much all the major players are changing screen sizes of their phones, tablets and phablets to figure out what works or in response to the competition. How applications adapts themselves for various screen sizes, layout and configuration is a challenging task.

Apart from adaptability to different screen sizes, mobile applications have to deal with the limited screen size. Limited screen size means that user can not be given 30 different options on a single screen - usability, similar experience, on-screen help, inability to use search or other applications easily etc poses different challenges and as a tester we need to think beyond what is developed and always think of who will use it and in what circumstances.

3. Complex user interaction - More than one way to do everything

User interaction in desktop and browser based applications was pretty much limited to mouse and keyboard. Mobile applications on the other hand are trying to make user interaction as fluid as possible. We had touch screen and with new phones from Samsung, you can just wave your hand to give commands. Siri is becoming more and more advanced and gives us a glimpse of future that voice commands may become part of every application in future. Devices are smart enough to understand complex gestures, eye movement, direction, tilt, movement, acceleration, GPS coordinates, surroundings, sound and so on.

As a tester, we need to ensure that application works as expected when user interacts with the app in different ways.

4. Application Type - HTML5, Native or Hybrid?

In the desktop and browser world, applications were straightforward. They were either desktop or web based applications. However, with the adoption and support of HTML5 - applications are merging. On all the mobile devices, it is not difficult to find HTML5 applications, Native applications and hybrid applications. Testing for hybrid application would be different from testing native applications and it is important to understand that difference.

5. Dependency on emulator / simulator - Get devices

For the desktop and browsers, developers always had access to the platform or browsers they were targeting with their applications. Also, virtualization has become more or less commonplace and can be trusted for desktop and browsers.

Mobile devices on the other hand relies on emulator and simulators. However, they are still not true representation of the devices. It is also not possible to replicate advanced user interaction on these simulators. As a tester, we have to be aware of the capabilities and limitations of these emulators / simulators and figure out what can be tested (reliability) on them and what can not.

6. Security & Privacy - You can’t touch me but I can.

Though most mobile applications live in their own sandbox but many platform features are accessible to them. For example, applications such as phone book, pictures and videos are accessible to many other applications. These are all personal user data - and any defect around the misuse (unintentional) of this data can jeopardize trust of the application.

In mobile world, it is important to ensure that applications are secure from the intruders, and it is equally important to ensure that applications are not intruding or accessing data unintentionally.

7. Dependency on Network / carrier - more variations

In desktop and web world, most users were either on LAN or Wireless. These network were not predictable, but compare to mobile networks, they were very predictable. Many connected mobile applications rely on the network - how application responds to 3G, 4G, weak signals, no signals, powerful signals, Switching from cellular to wireless and vice-versa or when user is moving at different speeds etc can affect how application will behave. It is often not possible to come up with or simulate real life situations for mobile applications.

Apart from the variation in signal strength and type, mobile apps can respond differently to different carriers. As a tester, it is important to understand if there are any difference or not and whether application works for all the major carriers or not.

8. Installation, removal and upgrade - Would you come back?

Mobile apps are installed, removed or updated more frequently than desktop applications. Also, underlying OS and platform is updated more frequently as well. As an app developer and tester in the mobile world, you have to be on top of what changes are coming in the next revision of OS / Platform and how it might affect application.

Usually for most of the applications, user data is stored on the servers and not on the devices. It makes installation a bit tricky - what if user has multiple devices, what if multiple devices have different version of applications and so on.

Things like backward compatibility, simultaneous support for multiple versions, data preservation, restoring state and data, ability to install / upgrade multiple times etc are all part of important checks for mobile application testing.

9. Session Management & Interruptions - Who’s calling?

Handling Interruptions are the way of life for mobile applications. Apps and users are constantly interrupted by calls, SMS, push notifications and so on. How applications handle these interruptions, how they maintain their state etc are important, but it is also important to see how much interruptions application is generating and what triggers those interruptions.

As a tester, it is important to ensure that application behaves properly when it is interrupted and it is also important to ensure that application does not interrupt unnecessary and works according to the boundary defined by platform or users.

10. Mobile specific Non-functional testing - and you thought it’s over.

Mobile applications add many more dimensions to the non-functional testing. Old school performance of the application is the obvious one, but there are many other factors as well which should be considered.

How much data your application is consuming? How much it would cost user (data usage) to use this application? How much battery is consumed by applications? Does it behave differently in high battery and low battery conditions? How much space it is occupying? How much trail it is leaving? How it is clearing the trails / logs etc are important non-functional factors which should be considered as part of mobile testing strategy.

So these are my top 10 reasons - and I am sure there are more reasons. What else makes testing mobile applications different from testing desktop or browser based applications? Let’s discuss.

blog comments powered by Disqus
Finished reading? Browse all posts »