In most of the cases User Testing is not considered a top priority and it is only considered after the design and development process of the mobile app is over. But the real thing is that there should be evaluation and validation of your app throughout the app creation process if you want the best results. This will allow the app developers to iterate on what they have learnt, make sure that the app is attractive for the users and it meets all of the expectations even before its final launch. If you don’t do the user testing, then there is no way to know if you are on the right track.
In this guide, I will tell you about the fundamentals and strategies which will help you to create your own testing plan during the Mobile App Development process which are specific to your business needs. We will discuss about defining your goals, selecting between remote and in-person testing, recruiting your participants, preparing your testing environment and materials and also finding the type of data to collect.
After going through this article, you will understand each of the core user testing steps and the special considerations you need to get started.
What is user testing?
User testing is a process that identifies product issues by testing it with real users. Earlier user testing was used to improve military equipment but now it is also being used to test so many things including mobile applications.
Why do we need to do user testing?
You ask any Mobile App Creator and he will tell you the important of user testing. User testing is needed as catching the potential problems early within the product development life cycle of the app is cheaper and it is much easier to resolve them. For this tech companies follow an Agile Workflow. In this, the product managers start by defining product requirements that are passed along to the UX team. After finalizing design, the product moves through development, QA, UAT and the final launch. This cycle repeats itself in an iterative manner until the final product is complete.
User testing was developed as a quick and cheaper solution to eliminate some of the risks. Using it, you can validate assumptions, steer product and design direction more easily and find the hidden issues using actual user feedback.
Defining the User Testing Goals for Your App
First you have to decide exactly what you want from the Mobile app testing which is a part of your app development process. Your goals determine everything from your testing methods to the exact environment, tasks and questions you will ask to the participant users. Hence it is important to determine your goals before proceeding further. Your user testing goals will change depending on where your app is in its lifecycle. For example, User testing goals during the development tend to lean more towards validation and evaluating. Means finding out if users understand and enjoy your app and whether the features meet their requirements.
Also remember that the testing doesn’t stop once your app is live in app stores. It is important to continue testing to make sure your app is aligned with the evolving user expectations. Hence your goals in the post-launch phase will mainly focus on further validation. Means, how your app is performing, options for optimization and whether your user experience is liked by the users. For this you have to give your participant users your actual app and try A/B testing to find out what is best for your users.
Selecting Between in-Person and Remote Testing
User testing can be done either in person or remotely. For both of these testing, you have to observe participant users using your product and find out any usability issues. They also have some differences. With in-person testing, you will get the opportunity to establish a real connection with your participants and see their reaction closely. In-person testing also makes it easier to read the participant users’ body language. But these in-person interactions are costly as they take more time because it requires travelling to a location, setting up a testing environment and being present for each user session. You may also have to rent a physical space for testing. Remote testing generally works well for unmoderated sessions and when the budget is low. This testing is ideal for tighter timeframe as there is no need to travel or arrange testing facilities and there is less work for test moderators. But in this testing, you cannot interpret the body language and it is more difficult to know about the participants’ feedback.
Moderated or Unmoderated Remote Testing
In remote testing, you can go for either moderated or unmoderated remote testing. Moderated sessions often offer more insights because you have the chance to ask questions in real time. You can get as much information from the users as you want. But there is high burden on time and effort as you have to be present for each session.
In unmoderated remote testing, you cannot ask follow-up questions. You also cannot troubleshoot the technical issues. Hence this particular testing could result in less fruitful insights. Although unmoderated sessions have one benefit that you are able to test your app in a unique context or environment. This is useful for apps used under specific circumstances.
Select Your Participant Users
This is important as if you select the wrong participant users or if you don’t have enough users then your testing will not generate good results. Hence first you need to identify your app’s target audience and based on that select the participant users. Also you should select the participant users based on their behavior rather than demographics. Before selecting them, be sure to determine how technically proficient your participant users should be and whether they need access to a specific platform like Android or iOS. Select only the unbiased people.
Number of Participant Users You Require
This depends on your testing goals. Research says that you should use atleast 5 participant users to uncover 85% of a product’s issues. I recommend to use 7 users for best results.
Get Ready with your User Testing Materials and Script
When you get ready to prepare for your user testing sessions then it is generally done the same way for in-person and remote testing. At first, you have to write down the tasks which you want your participant users to do during the testing session. After that create a script to follow throughout the test.
Always create your testing tasks, materials and script based on your user testing goals. You can also include metrics question to collect more insights. For example, note how long it will take for a user to complete a task. By this you can estimate the issues with your user experience and navigation. Also ask the participant users whether the task was easy or difficult. Testing works best when you expose your participant users to the same variables. Means you have to script everything you want to say to the user and you have to follow the script as it is.
Creating and Writing the Script
While you are writing your script, make sure that it matches your research questions. You can do this by translating your research questions into simple tasks that represent realistic user goals. Hence for each of the task, create a user story to make it more realistic for the users. Your script should also include any interview questions that you want to answer, prompts for follow-up questions and debriefing questions. Also include clear and detailed instructions on how to get to the app. Therefore, instead of telling the participants to install your app from the App Store, write down each and every step in the process.
Set Up Your User Testing Environment
Generally, your testing environment and the tools depend on your testing method. Hence in-person testing provides you full control over your testing environment while your participant’s remote testing experience depends more on the quality of the tools you choose to use.
While doing in-person testing, your area should be free from distraction like closed door conference room. By this the participant users can focus on their tasks easily.
In remote testing, it is important to select the right tools which are easy to use. These tools need to be reliable.
Test Your Test
You should do a dry run before your first real session begins. By this you can fix red flags or issues that might come up. Normally, running a pilot test with a co-worker help remove out any issues in the test script and tasks and gives you a chance to evaluate the flow of the test, add or remove any tasks and forecast any potential difficulties. Hence you will be more prepared and confident for your sessions as you have already resolved possible issues. After that at the fixed time, you can start conducting your user test after debriefing your participant users.
Analyze On Your User Testing Results
Now watch the recordings and take note of whatever you see and hear. Have a glance on all of your observations together and try to find why some particular things happened and create theories about the causes of any issue, frustration or any problem. Analyze the direct feedback of your participant users and examine measurable data points to reach at the conclusions about your app. After you observe, record and discuss your findings with your team members, start creating an action plan of recommendations for improving your app experience.
Conclusion
If you have done properly, user testing will make sure that your app matches your user’s expectations, needs and requirements. Without conducting this you cannot find out whether you are on the right track throughout the design and development process. This user testing will leave you with actionable next steps for your mobile app. Apps reach the final stage of development only when usability issues are addressed and general feedback is positive. If feedback is neutral or negative, then revise your prototype and run another round of testing. Revise, test, repeat – do it again and again till you get a final satisfied version of your app which gets positive feedbacks. I hope this guide will help you to build a good foundation from where you can build your own successful user testing strategy.
In this guide, I will tell you about the fundamentals and strategies which will help you to create your own testing plan during the Mobile App Development process which are specific to your business needs. We will discuss about defining your goals, selecting between remote and in-person testing, recruiting your participants, preparing your testing environment and materials and also finding the type of data to collect.
After going through this article, you will understand each of the core user testing steps and the special considerations you need to get started.
What is user testing?
User testing is a process that identifies product issues by testing it with real users. Earlier user testing was used to improve military equipment but now it is also being used to test so many things including mobile applications.
Why do we need to do user testing?
You ask any Mobile App Creator and he will tell you the important of user testing. User testing is needed as catching the potential problems early within the product development life cycle of the app is cheaper and it is much easier to resolve them. For this tech companies follow an Agile Workflow. In this, the product managers start by defining product requirements that are passed along to the UX team. After finalizing design, the product moves through development, QA, UAT and the final launch. This cycle repeats itself in an iterative manner until the final product is complete.
User testing was developed as a quick and cheaper solution to eliminate some of the risks. Using it, you can validate assumptions, steer product and design direction more easily and find the hidden issues using actual user feedback.
Defining the User Testing Goals for Your App
First you have to decide exactly what you want from the Mobile app testing which is a part of your app development process. Your goals determine everything from your testing methods to the exact environment, tasks and questions you will ask to the participant users. Hence it is important to determine your goals before proceeding further. Your user testing goals will change depending on where your app is in its lifecycle. For example, User testing goals during the development tend to lean more towards validation and evaluating. Means finding out if users understand and enjoy your app and whether the features meet their requirements.
Also remember that the testing doesn’t stop once your app is live in app stores. It is important to continue testing to make sure your app is aligned with the evolving user expectations. Hence your goals in the post-launch phase will mainly focus on further validation. Means, how your app is performing, options for optimization and whether your user experience is liked by the users. For this you have to give your participant users your actual app and try A/B testing to find out what is best for your users.
Selecting Between in-Person and Remote Testing
User testing can be done either in person or remotely. For both of these testing, you have to observe participant users using your product and find out any usability issues. They also have some differences. With in-person testing, you will get the opportunity to establish a real connection with your participants and see their reaction closely. In-person testing also makes it easier to read the participant users’ body language. But these in-person interactions are costly as they take more time because it requires travelling to a location, setting up a testing environment and being present for each user session. You may also have to rent a physical space for testing. Remote testing generally works well for unmoderated sessions and when the budget is low. This testing is ideal for tighter timeframe as there is no need to travel or arrange testing facilities and there is less work for test moderators. But in this testing, you cannot interpret the body language and it is more difficult to know about the participants’ feedback.
Moderated or Unmoderated Remote Testing
In remote testing, you can go for either moderated or unmoderated remote testing. Moderated sessions often offer more insights because you have the chance to ask questions in real time. You can get as much information from the users as you want. But there is high burden on time and effort as you have to be present for each session.
In unmoderated remote testing, you cannot ask follow-up questions. You also cannot troubleshoot the technical issues. Hence this particular testing could result in less fruitful insights. Although unmoderated sessions have one benefit that you are able to test your app in a unique context or environment. This is useful for apps used under specific circumstances.
Select Your Participant Users
This is important as if you select the wrong participant users or if you don’t have enough users then your testing will not generate good results. Hence first you need to identify your app’s target audience and based on that select the participant users. Also you should select the participant users based on their behavior rather than demographics. Before selecting them, be sure to determine how technically proficient your participant users should be and whether they need access to a specific platform like Android or iOS. Select only the unbiased people.
Number of Participant Users You Require
This depends on your testing goals. Research says that you should use atleast 5 participant users to uncover 85% of a product’s issues. I recommend to use 7 users for best results.
Get Ready with your User Testing Materials and Script
When you get ready to prepare for your user testing sessions then it is generally done the same way for in-person and remote testing. At first, you have to write down the tasks which you want your participant users to do during the testing session. After that create a script to follow throughout the test.
Always create your testing tasks, materials and script based on your user testing goals. You can also include metrics question to collect more insights. For example, note how long it will take for a user to complete a task. By this you can estimate the issues with your user experience and navigation. Also ask the participant users whether the task was easy or difficult. Testing works best when you expose your participant users to the same variables. Means you have to script everything you want to say to the user and you have to follow the script as it is.
Creating and Writing the Script
While you are writing your script, make sure that it matches your research questions. You can do this by translating your research questions into simple tasks that represent realistic user goals. Hence for each of the task, create a user story to make it more realistic for the users. Your script should also include any interview questions that you want to answer, prompts for follow-up questions and debriefing questions. Also include clear and detailed instructions on how to get to the app. Therefore, instead of telling the participants to install your app from the App Store, write down each and every step in the process.
Set Up Your User Testing Environment
Generally, your testing environment and the tools depend on your testing method. Hence in-person testing provides you full control over your testing environment while your participant’s remote testing experience depends more on the quality of the tools you choose to use.
While doing in-person testing, your area should be free from distraction like closed door conference room. By this the participant users can focus on their tasks easily.
In remote testing, it is important to select the right tools which are easy to use. These tools need to be reliable.
Test Your Test
You should do a dry run before your first real session begins. By this you can fix red flags or issues that might come up. Normally, running a pilot test with a co-worker help remove out any issues in the test script and tasks and gives you a chance to evaluate the flow of the test, add or remove any tasks and forecast any potential difficulties. Hence you will be more prepared and confident for your sessions as you have already resolved possible issues. After that at the fixed time, you can start conducting your user test after debriefing your participant users.
Analyze On Your User Testing Results
Now watch the recordings and take note of whatever you see and hear. Have a glance on all of your observations together and try to find why some particular things happened and create theories about the causes of any issue, frustration or any problem. Analyze the direct feedback of your participant users and examine measurable data points to reach at the conclusions about your app. After you observe, record and discuss your findings with your team members, start creating an action plan of recommendations for improving your app experience.
Conclusion
If you have done properly, user testing will make sure that your app matches your user’s expectations, needs and requirements. Without conducting this you cannot find out whether you are on the right track throughout the design and development process. This user testing will leave you with actionable next steps for your mobile app. Apps reach the final stage of development only when usability issues are addressed and general feedback is positive. If feedback is neutral or negative, then revise your prototype and run another round of testing. Revise, test, repeat – do it again and again till you get a final satisfied version of your app which gets positive feedbacks. I hope this guide will help you to build a good foundation from where you can build your own successful user testing strategy.
