As soon as an app is completed, the next step is app testing. The app is tested for functionality and usability as well as its compatibility with different devices and operating systems.
Most developers want of course want to publish their application as soon as possible after developing it and setting the final app design – but before the software is offered in the app store, it is important to test its suitability across various categories. Common methods for determining this are performance, compatibility, and usability tests. These and other testing methods are grouped together under the term “mobile application testing” (or “mobile app testing” for short). Different app testing tools are used throughout the various stages of this process.
In order for an app to be successful, it not only needs to offer impressive content and be well-marketed, but above all must work flawlessly and be user-friendly. Therefore, fundamental testing before the release of an application is vital – whether it is a mobile app or a desktop application.
Testing apps is, however, becoming more difficult because of one increasingly complex factor: adaptation to different user devices is considerably more elaborate for apps than for other software. In the early years of app development this adaptation was easier to manage due to the relatively small selection of smartphones on the market. Since then, a huge selection of mobile devices has emerged with a variety of different features. Most smartphone and tablet models function on roughly the same quality of hardware, but installed components such as the processor (CPU), memory (RAM), screen size, and screen resolution are the main factors determining the rendition of an application.
In addition to its hardware, the software of a device also plays a role. Software mainly contains the different operating systems of a platform, which influence app rendition. Older operating systems can make app usage more difficult due to missing updates, and the prevalence of old Android and iOS operating systems should not be underestimated. The variety of operating systems in use is particularly evident when looking at Android systems: On the whole, the older versions of Google’s operating system are used more commonly than the current version. This has to do with the fact that many Android manufacturers either deliver updates late, or don’t deliver them at all. With iOS the new version of the operating system is generally delivered more quickly, but some of the oldest devices no longer receive updates from Apple, therefore old iOS versions can still be found.
The variety of hardware and software for mobile devices is called fragmentation and it presents many challenges when it comes to app quality assurance. This is where mobile app testing comes into play: This checks whether an application is even capable of running under different conditions, and if so, whether the conditions present any difficulties for playback of the app. Detecting and correcting such issues is necessary to ensure an optimal user experience for as many users as possible. Only when an app offers good performance and ease of use for as many devices as possible can it reach its maximum potential.
App testing is not all done the same way. There are many different approaches to quality assurance. The two main questions to address are who should check the quality of the application (either manual or automated tests) and how the tests should be conducted (tests directly on the respective devices or tests via a simulator or emulator).
Manual app tests (carried out by users)
The manual test is the classic approach to mobile app testing – in this case, real people test the application in question. Since the testing of applications can be quite extensive (especially with Android apps), the development team usually cannot do this by themselves. Consequently, test subjects are used. A group is assembled (representative of a larger group, in the best case scenario) which then uses the app according to predetermined test scenarios and subsequently evaluates it. Outside test subjects have the advantage that they do not have any prior knowledge of how the application should work, and therefore can evaluate with judgment close to that of a potential user.
In addition to the manual mobile app testing, crowd testing is another classic approach. Crowd testing services offer a pool of testers, which allows developers to choose targeted test persons to test the app on their own personal devices. Crowd testing does not take place in a special testing environment, but rather in the natural environment of the users, which can increase the validity of the test results. For iOS apps (as well as watchOS and tvOS apps) developers can use the TestFlight Beta Testing service to invite up to 2,000 users to test the application and organize their own crowd tests. Google also offers software developers various options for private and public beta testing of Android apps.
Automated app tests (carried out by app testing tools)
In contrast to manual tests, automated controls are used in areas where certain processes are continuously tested under different conditions. For automated app tests, developers need to use a special software. The main advantage is how much time it saves – test procedures can carry out programs in a fraction of the time that a user would need. However, automation is not suitable for all test fields.
Tests on physical mobile devices
Testing apps on individual mobile devices is the ideal solution for being able to understand the app in combination with specific software and hardware. However, this is quite expensive and time-consuming, because a large number of smartphones and tablets are required for comprehensive tests, and the behavior of the app has to be constantly be checked in detail on each device. The easiest way to test using this method is by using various mobile app testing vendors, which provide access to a variety of mobile devices (often called “device lab” or “test lab”) that can be used to test the app.
Tests via Simulator/Emulator
Simulators and Emulators are programs that imitate particular devices and allow applications to be tested virtually. However, the results obtained from these tests are not always 100% reliable, as not all user interactions can be authentically simulated. In addition, some tests should be carried out on physical devices as the simulation can reach its limits in certain areas.
Simulators and emulators can admittedly not replace tests on physical devices, though they are a good addition and particularly suitable as an automated testing tool. With their help, it is possible to quickly test the functionality of an app with different hardware and software combinations. Many simulators and emulators are also available for free – for example, those contained within the Android Studio and Apple’s Xcode programs.
In addition to the various methods for testing an app, there are also various areas of testing. Apart from the tests mentioned in this section, there is also safety testing, security testing, and localization testing, as well as other test types and areas.
By using functionality tests, a developer can check that the app is really working the way they want it to. Ultimately, all elements of the app will be checked here for correct operation – every function, screen, and button will be inspected. Functionality testing focuses on the following central questions:
In addition to the app itself, tests should check the behavior of the application in the certain following scenarios:
App usability testing checks how user-friendly the operation of the application is. Automated tests are not an option here – after all, only users can assess whether the how well they think an app works. The use of a simulator or emulator is also generally not helpful for usability testing, as app operation can only truly be evaluated by usage on a mobile device; only when using the actual application can control procedures such as swiping, scrolling, and pressing buttons be felt and their reaction times and behaviors be evaluated. Other aspects that are examined in usability testing are as follows:
Usability testing should discover any problems with the operation of the app or shortcomings that detract from its ease of use. In order for the results to be conclusive, the usability test must be carried out by a sufficient number of test users. It is also advantageous if the test users (at least a majority) are people from the target group – that way the operation and structure of the app can be optimized for expected users. The resulting improvements allow the developer to offer the best possible user experience.
The compatibility of an app in regard to the hardware and software of mobile devices should always be tested. In terms of hardware, one important example is testing the rendering of the application on different displays. Because there are different screen sizes and resolutions, display and customization errors can occur relatively quickly, especially if several layouts of the application for different screen formats have not already been created. In addition to the display format, CPU and RAM vary from device to device. Keep in mind that older mobile devices are not a rarity. Apps that are designed not only for current models, but also for older smartphones and tablets, automatically increase their market range.
In regard to device software, the smooth playback of an application is largely controlled by the different versions of the operating system – Android app testing on various versions of the operating system is especially important, as this platform has significantly more versions with more variation of use than Apple’s iOS platform.
Compatibility testing allows developers to check which hardware or software processes cause immediate problems with rendition. If the requirements for the device components are very high, it may be helpful to reduce certain system requirements in order to avoid shrinking the target group.
Performance tests show how many system resources an application consumes during intense use. Does the app continue to run smoothly, or does it experience lapses? Beyond that, performance tests can determine if a device’s battery will become overloaded or if the device itself will overheat after a certain period of use. As with compatibility testing, tests on the performance of the applications are controlled under certain conditions – for example, how the playback of the app changes if there is hardly any space left on the device’s hard drive or if the battery is almost empty. Performance testing is intended to reveal the extent to which certain circumstances negatively impact the app’s performance.
If one or more components are going to be subject to high utilization, then a load test (or stress test) is used. One example of this is the server stress test, where the app’s behavior is tested in reaction high server demand. For applications that require an internet connection, a network load test is conducted to see how a weak internet connection affects app functionality. Stress tests are designed to show how much stress certain components can take before their performance becomes restricted.
The above tests don’t necessarily have to be carried out manually. Diverse testing tools can make the work considerably easier in many areas of app testing. Seeing as this article series discusses mobile application development for both Android and iOS, the following app testing tools are those which are consistently available across both platforms.
Appium is a cross-platform open source program for test automation thatis available for download free of charge. This tool doesn't only test native Android and iOS apps, but also supports test automation of hybrid apps and web apps.
Appium uses the same programming interface for both iOS and Android, so it is easy to reuse previously written code. The created tests can be carried out on physical devices, simulators, or emulators. Appium is incredibly helpful software for the execution of automated app tests, however, the program does require the user to have a certain level of pre-existing know-how.
On TestFairy’s website, developers can upload their app and invite people to participate in app testing. These people can then download and use the application – essentially, TestFairy allows for self-organized crowd testing. The tool has an error reporting feature and can record video of app test usage. TestFairy is available as a free download and also a paid enterprise version.
Ubertesters is a platform that helps with beta testing for apps. Developers have complete control over the tests and receive detailed reports. On the platform it is easy to work together as a team, detect programming errors using various tools, create and test different versions of the app, and perform crowd tests (for example, in the form of functionality or usability tests). Ubertesters is offered in various packages, each with different features – the package with the fewest features is available for free download.
If an emulator discovers an error, it should be reassessed with a physical device before troubleshooting begins. Not all errors that occur in emulators also occur in real usage of the application. If the errors still occur on a mobile device, then steps should be taken to fix them. But what is the cause of the malfunction? An exact error diagnostic cannot be provided through app testing. Some app testing tools may help, but often app problems and lapses have to be analyzed independently. Typically, app errors are a result of faulty programming or certain device components not being 100% compatible with the application.
When the reason behind a faulty application function is eventually found out, it must be decided how important solving the problem is, and to what extent resolution is possible without affecting the functionality of the app. General program errors, of course, should not be shown in the final version of the application – allfundamental features (such as navigation or mapping of application content) need to function at all times.
The decision for or against correcting an error becomes more difficult when the root of the problem is not in poor programming, but in a conflict with certain hard- or software. This mostly affects outdated operating systems or old or low-performance mobile device components, which slow down or obstruct one or more app functions. In such cases, the results following an app test should be carefully analyzed to find out which hard- or software element could be responsible for the function not being performed correctly. The trial-and-error approach is often helpful here.
Obviously, the reach of an app can be extended by designing it to run on many older devices. The adaptation of an app to the largest possible number of devices, though, has its limits. CPU-intensive applications, for example, whose functions require a stronger processor and/or more memory, can sometimes not be prepared optimally for older hardware. Another example: if an app requires a powerful internet connection, it should absolutely not be adapted to any usage situation in which there is only a weak internet connection.
Some applications simply require certain technical prerequisites. For example, some applications require a particular screen size in order to run. Another example is the YouTube app, which can only function with access to a sufficiently fast data transfer rate; yet it is still one of the most downloaded applications ever. Adjustments and reductions of functionality in favor of a large app range should only be made if the app functionality is not reduced more than desired.
Extensive testing of applications is the last step in app development. Of course it takes a lot of work, but in the end it is worthwhile in every respect: Mobile app testing contributes to the assurance that as many users as possible can optimally use the app. It is important to ensure a balance in the tests between the use of physical devices and simulators and/or emulators.
If the app has been scrupulously tested and contains no serious unresolved errors or inconsistencies, then it’s official: The app concept has successfully been implemented and the application can finally be published in the app store. However, there are still some tasks to be done.
For one thing, some of the previously performed tests should be repeated at regular intervals. Constant monitoring ensures that the app continues to run smoothly in the future and that the user is still able to use the app in the best possible way – most notably, the compatibility and performance tests should continue to be conducted.
Additionally, setting up the application in Google and Apple’s app stores also involves a lot of effort. First the app needs to be signed up for distribution in the stores, which leads to the fifth article in this series: subdivided into registration in Apple’s App Store and registering with the Google Play Store. Additionally, after the app is published work can be done to improve its rank in the Google Play Store and the Apple App Store – the final part of this article series, which includes all important information about the handling of the app stores of both Apple and Google.
Are smartphones and tablets the beginning of the end for stationary computers? Many developments confirm this, not just the increasing number of mobile search queries on Google. The mobile first approach follows this trend and places primary importance on creating mobile websites. A slimmer scope and more dynamic layout yield more than just satisfied smartphone users — they boost search result...
The internet is becoming increasingly more mobile, which is proving challenging for website operators. There are still many websites which aren’t properly optimized for smartphones and tablets and can only be partly displayed. Since Google’s mobile update, these sites can expect significant visibility losses. Separate mobile websites, adaptive and responsive designs, and mobile plugins for content...
If you want to create your own native mobile app, you’ll need to think about the development environment. When it comes to classic programming, integrated development environments are useful tools to help make the process easier. But since this requires prior knowledge of software development, many beginners prefer to find an alternative option. The answer for inexperienced users usually lies in...
Provide powerful and reliable service to your clients with a web hosting package from IONOS.
View packages
