Testing mobile performance can seem tricky and time consuming, but it’s an aspect of the mobile user experience that developers and testers can’t overlook in 2020.
According to Google, 70 percent of users abandon an app because it takes too long to load. Similarly, App Samurai has found that 70 percent of app uninstalls are due to app crashes. These are both negative consequences related to poor mobile app performance that brands must avoid if they want their apps that they’ve poured so much time and energy into making to be a success.
In this post we will see how we can do mobile performance Testing using Apptim.
Apptim is a new, free tool that can be used during development and testing to catch critical bugs faster and identify new performance issues in an app. The tool automatically measures app render times, power consumption, and resource usage while capturing crashes and more on Android and iOS devices. Apptim consists of a desktop application that can be used on Windows to test native Android apps and on MacOS to test native iOS and Android apps. It also has a web-based workspace where you can save your reports and share them with team members that you’ve invited.
This makes it easy for manual testers, developers, and even product owners to run a test.Once you’ve installed Apptim, you can run your first test and obtain performance results in as little as five minutes.
- Native and hybrid app testing: Apptim evaluates the performance of native mobile apps or hybrid apps (but it only captures data from the native part).
- Performance and bug reports: After each test session, the tool generates a report regarding the performance metrics and another report for each bug that you may find during your test session with Apptim.
- JIRA Integration: Through an API key, it integrates with Jira, so you can publish and track bugs directly in this project management tool.
- Test on real devices: In order for this tool to do its own thing, you must connect your mobile device with a USB cable to the PC where Apptim is installed.
- Easy to use: Apptim’s user interface is elegant and intuitive, making it a simple tool to use. In addition, on its website you can find detailed documentation that will help you through your first steps with the tool and there is the option to ask for support every step of the way.
- Compare two test sessions: If you repeat a test for a particular feature once a new version of your app is ready, you can compare the results to see if there are any big differences in the performance.
What’s unique about Apptim is that you don’t need an SDK in order to start capturing performance information. This makes it easy for manual testers, developers, and even product owners to run a test. Once you’ve installed Apptim, you can run your first test and obtain performance results in as little as five minutes.
These are the steps we’ll follow to test mobile performance with Apptim:
1 – Install Apptim Desktop
2 – Install the demo app
3 – Run a test
4 – Apptim Reports
1 – Install Apptim Desktop
There’s a simple two-step process to configure Apptim Desktop.
First, you can create an account on the Apptim website and then download the Windows or iOS desktop application. Apptim requires installing some dependencies, which it will list for you, and after clicking “Install Dependencies,” it will begin to install them for you if they are not already installed on your computer.
After the installation process is completed, you will see Apptim’s home screen.
2 – Install the Demo app
You can download the Android demo app from here. Then, connect your Android phone to your PC or Mac using a USB cable. Make sure to follow these instructions to correctly allow Apptim to access the device later in the test.
If you were to test an iOS app, you would follow all these same steps, except instead of connecting an Android device, you would use an iPhone or iPad and your Mac.You can install the demo app in the Android device by executing the following command from the Terminal in Mac or Command line in Windows: adb install <apk_path>.
Make sure to run this command from inside the ../android-sdk/platform-tools/ folder or add the Android SDK directory to your path.
3 – Run a test
Once you have the demo app installed in your device and the device is connected to your PC or Mac through USB, you are ready to start a test with Apptim!
Go to Apptim and click “Start a new test,” it will ask you to select your mobile device and then you can choose the app on the device that you want to test or search for it by name.
Here you will search for “ReferenceApp” and then choose the app from the list.
Then, it will ask what type of test you’d like to run. Currently, only exploratory tests are supported, but in the future, there will be the option to run a bot test.
Next, it will ask you to add a name to your test and press “Start” to begin your exploratory testing session
During the Test
At any moment during your testing session, you can choose to report a bug or take a note. You can end your testing session anytime by clicking on “End Session.”
Report a bug
When you click on “Report bug,” you will be prompted with a new screen where you can give the bug a title and description, add a tag, and attach a video and screenshot that were automatically captured. You also have the option to edit the screenshot.
Take a Note
When you click on Take a note, you will be prompted with a new screen where you can write a Note, a Risk identified, an Idea, or just a To do as a reminder for later. You can also attach a screenshot.
Once you end your testing session, you will see a summary with bugs, ideas, risks, and notes taken.
Once you’ve finished your test session, Apptim generates a report with a summary of all of the performance data that it has automatically captured for you while you were testing the app’s functionality.
You can publish the test session report to your workspace in the Apptim Cloud and then share it with other team members. You have the option to share it privately, by adding team members to the workspace, or publicly, by generating a link.
In each Apptim report, you’ll find:
- A screen recording of the device during the execution of the exploratory session
- Alerts and warnings related to the user experience (nested layouts, screen controls, rendering times, etc )
- Alerts and warnings related to the usage of device resources (CPU, Memory, Threads, Power Usage, Network Data), with reference thresholds, shown in yellow or red depending on their severity
- Test environment data (device, app, version info, etc)
- Crashes/exceptions details with associated screenshots, and the exact minute of the session in which it happened in order to locate the specific moment in the video
- Detail of resource usage with associated graphs
- Logs and downloadable artifacts collected during the test
Here are some screenshots of what an Apptim Report includes:
Here you can also see an example of an iOS Apptim report.
mobile performance is more crucial than ever, but it’s also a challenge to gain visibility into all of the factors that impact it. Apptim provides easy-to-read reports for developers to check mobile app performance, review logs, and crash details. Teams can now release new versions of their app with confidence by getting full visibility on app performance trends.