App store optimization for better UX: how to test your listing

Equally as important as fine-tuning your mobile app is its storefront—the app store listing. From being the first impression of your product or brand to giving potential customers an idea of what you’re all about, your app store listing is the bridge between your product and a potential new customer. 

While it can be easy to get caught up in perfecting your mobile app, from evaluating how it stacks against competitors to making it appealing for a variety of users, letting its listing fall to the wayside can be costly. A user won’t be able to see the utility and benefits of an app (and be willing to give up precious storage space) without an engaging listing.

Meanwhile, an appealing listing becomes devalued if the actual app doesn’t live up to expectations. And if the two don’t align in terms of messaging and branding, this may cause confusion or hesitation among users that can be difficult to recover from. 

Studies show that 44% of organizations use mobile apps as their primary marketing technique for customer retention, while there were 230 billion mobile app downloads in 2021 alone. As the data shows us, mobile apps carry a lot of weight. Whether you’re prioritizing the growth of your mobile app or giving it a refresh to keep the experience consistent with its web counterpart, there are plenty of benefits to optimizing your app store listing, including improved customer retention and adoption.

Here’s how to optimize your app store listing, from start to finish, including creating and testing your listing. 

Table of Contents 

• What is app store optimization (ASO) and why is it important? 
• What do I need to create an app store product page? 
• How do app stores organize their rankings? 
• How to optimize and test your app store listing

What is app store optimization (ASO) and why is it important? 

You’ve probably heard of search engine optimization (SEO), and if your organization has an app, you should become just as familiar with app store optimization. ASO is a method of increasing an app’s visibility in an app store’s search results, with the goal of boosting traffic, downloads, and app conversion rates. 

When you look at some of the most popular app downloads globally, you'll see household brands like Instagram, TikTok, Facebook, and WhatsApp. And while your app probably won’t dethrone Instagram (at least not yet), you do want your mobile app to appear—and remain—at the top of app store search results when users type in a relevant keyword. This is what ASO can help you achieve. 

Even if you have a near-perfect mobile app, it won’t mean much if it’s not being marketed the right way or not being found by your target users. And even if your product doesn’t yet have direct competitors, it undoubtedly does in the app market—with the iOS App Store and Google Play Store housing 2 million and 4 million apps respectively, according to a 2021 App Annie guide. These numbers will likely only grow, making it all the more important that you have a competitive optimization strategy.

App store optimization is an ongoing effort and process. Whether you’re trying to boost your app store ranking, or you’re lucky to be highly ranked on the app store, growing or maintaining that ranking is a continuous process that requires regular feedback from your current and prospective users. And whenever you overhaul or update your mobile app, whether with new features or versions, you need to test and update your listing to reflect those changes. According to App Annie, top apps update their apps up to 44 times annually—which means they’re constantly focused on their ASO, and so should you. 

What do I need to create an app store product page? 

Before you can dig into what you need to create your listing, you first need to decide where you’ll list your app. The iPhone-or-Android debate has been polarizing for what feels like decades, which can be influenced by everything from gender, education, and income, to occupation. That means your listing needs to be inclusive of both iOS and Android users, which requires tailored, separate ASO strategies.

Or, think ahead about having some kind of reasoning why you’re leaning one way over the other, which can be as simple as time and budget. For instance, Google Play is known to offer a less time-consuming submission process. By honing in on one listing first, you can give a more focused effort, and give yourself room to make mistakes before your second listing. 

While it’s ideal to launch your product on both the iOS Store and Google Play, it makes sense to initially hone in on one, especially for the purposes of testing. Think about your app’s target audience, and even consider running a test to find out the demographics, before making a decision. This could be as simple as setting up a test to gauge who’s most interested in your app, and making note of their demographic information and smartphone preference. 

If it seems like your audience is split down the middle on their phone preference, and you prioritize a large market share, launching on Android first may be the better choice. If your organization primarily uses Apple products, then it’s a no-brainer to first pursue the iOS Store. You may also weigh in factors like app store commission fees, consumers’ app spending habits, submission process speed, competition, and language support. 

Now that you’ve decided where to list your app, here’s what you’ll need to set up your product page in the app store.

Apple App Store requirements

When crafting your iOS store product page, you’ll need to create: 

• App name
• Icon 
• Subtitle
• App previews
• Screenshots
• Description 
• Promotional text 
• Keywords 
• In-app purchase 
• What’s new 
• Ratings and reviews 
• Categories 
• Localization 

Google Play Store requirements

And when creating a Google Play store listing, you’ll need: 

• App icon 
• Short description 
• Long description 
• Feature graphic 
• Screenshots
• Preview video 

How do app stores organize their rankings? 

Once your app listing is set up, you’ll probably start checking your app’s ranking and metrics and might wonder what they mean or how they’re calculated. 

Whether you’re testing your mobile app for either the Apple App Store or Google Play Store, or both, listed below are ranking factors that you should evaluate (and examples of what you can test for). However, keep in mind that app store algorithms are consistently evolving, so these factors are subject to change. 

The iOS store prioritizes: 

• Accurate keywords 
• Compelling app name and description 
• Correct category assignment 
• Positive ratings 
• In-app events 
• In-app purchase promotion 

Google Play lists several factors as key to how they organize and rank apps, including: 

• User relevance 
• Quality of the app experience 
• Editorial value 
• Ads
• User experience 

Your app store ranking is also influenced by a variety of factors, from the device being used (e.g., a TV versus a smartphone), where on the app store one is looking, and user preference. By establishing an evergreen strategy and consistently keeping your app’s information up-to-date, you’ll be ahead of the curve by the time the next (and the next) update comes around. 

How to optimize and test your app store listing 

As always, it’s better to test early and often. However, depending on your timeline and needs, you may opt for testing when your product is undergoing a redesign, after a new launch, or during prototype development efforts. And as you can see from the ranking factors above, there’s no shortage of what you can test. As a best practice, we recommend focusing on one element at a time so test participants don’t get fatigued. Below, we’ll hone in on testing screenshots, icons, and reviews. 

Before you start testing, you’ll need to find the right audience, which you can do with the help of screener questions. When planning your screener questions, opt for users who match your target audience and if you’re hoping for unbiased feedback, you might even screen out those who’re already familiar with your app or its competitors.

You can even think outside the box. If your app has undergone a significant change recently, you may consider targeting users who used to subscribe to your app but no longer do—and learn whether or not they’d be convinced to return (and why they left to begin with). 

1. Testing app store screenshots

An app store’s screenshots (images that offer a glimpse into the app) are one of the most eye-catching elements. When a user scrolls through the app store or search results, the screenshots take up most of the screen and can be what convinces them to stop scrolling and click on the listing. In fact, an optimized screenshot has the ability to increase conversion rates by 28%, which may be attributed to the fact that typical App Store users are 10x more likely to look through a listing’s screenshots over its description. 

Understanding this impact, Rovio Entertainment turned to SplitMetrics before the release of Angry Birds 2 to test app screenshots and icons, analyzing everything from vertical and horizontal orientations to color. Through A/B testing, they discovered that portrait app screenshots boosted conversion rates and found valuable, new insights about their top users. The result is that the organization was able to boost conversion rates by 13%, which translates to 2.5 million extra downloads during its first, post-launch week. 

Source: SplitMetrics

Apple’s App Store sets a limit of three screenshots total for the search results, and 10 total screenshots within the product page. For more detail, view their requirements here. Meanwhile, on Google Play, a maximum of eight screenshots are offered, and you can find more of their guidelines here. 

You should pick carefully when deciding how to use this precious real estate—and take advantage of the amount you’re given. You can use this space to highlight any signature branding or offer a glimpse of the app experience when downloaded. For an extra layer of customization, pick between vertical or horizontal screenshots, though the vertical layout seems more common.

Keep in mind that within search results, your screenshots’ text (if any) appears smaller than how it would in the product page. A best practice would be to enlarge your font or keep any copy short and sweet. 

To implement user feedback into this process, consider a preference or A/B test, to gauge reactions from your ideal audience on multiple designs. User testing and A/B testing go hand in hand. While user testing provides the qualitative feedback, A/B testing mobile apps takes care of the quantitative. For more thorough feedback, or an opportunity to probe your test participants, consider a Live Conversation—which may be preferred if you’d rather share your screen over a screenshot. 

Here are examples of what you can ask: 

• Which design did you prefer?
• Which design did you find the most visually appealing? Why?

2. Testing app store icons

Your app store icon is an integral part of your brand that helps distinguish your organization from others and makes it easy to find. As a design principle, remember that the simpler, the better, especially to make it compatible across different devices. 

Remember that your icon isn’t set in stone, as Apple’s App Store and the Google Play logos have evolved over the years, along with many others like Uber and Instagram. As customer preferences change, so do logos, which means testing your icon is a step that can’t be skipped in your ASO strategy. 

Source: 9to5Mac

Like with screenshots and other visuals, you may consider conducting a preference test to test several different design options. If you’re making a drastic change from an existing app icon, be sure to get user feedback on the new design concepts early in the process. This early feedback will help you more effectively iterate your design because it’s based on real user feedback. When you land on your final design, you can launch it with confidence knowing your target audience has already shared their perspectives on the design. 

Here are some sample questions you can ask when testing your app’s icon:

• What adjectives come to mind when you see this icon? 
• Does the icon match what the organization is trying to achieve? 
• What would you change about the icon? Why? 
• How does the new icon compare to the original? 

3. Testing app store reviews

We’ve all had that moment of being in an app and receiving a pop-up notification to leave a review or rating—and sometimes, this may have been more unwelcome than others. To increase your chances of receiving a review, conduct a test beforehand on the ideal moment for this message to appear (like when a user may satisfied with the app), and hear user feedback on any contributing factors. 

Additionally, you can rely on the existing reviews you already have to help guide your future tests and pinpoint areas needing improvement. You may also find it useful to evaluate your competitors’ reviews to target common complaints or pitfalls which you can strengthen within your own app. 

Understandably, consistent negative user reviews can decrease an app listing’s ranking. And with the app rating being one of the first few elements users can see in search results, which can make or break a first impression, this makes it an area worth prioritizing. 

Potential questions to ask include: 

• What motivates you to leave an app review? 
• What do you think is an acceptable amount of times to be asked to leave a review? 
• How much weight do reviews and ratings have when you consider downloading an app? 

How Krikey and Your.MD leveraged UserTesting to improve their app store listings 

Still not convinced? Take it from our customers, who have successfully used UserTesting’s Human Insight Platform for their challenges. Augmented reality gaming app Krikey conducted tests on their Google Play store listing, where the majority of their users make a decision to install or not. Test participants gave valuable feedback on everything from product descriptions, imagery, and their preference for certain terms over others. 

After implementing user feedback from their initial tests, Krikey found that their Google Play Store install conversion rates increased from 5% to 40% in just six months. And that’s not all; combined with tests on their mobile app itself, Krikey has since reached over 1 million users. 

For more on their story, check out the video below. 

Remote video URL

 Krikey is just one example of a customer who has successfully tested their app store listing. Healthcare app Your.MD, also known as Healthily, set aims to help anyone find reliable and customized support for their health. Looking to enhance their product’s UI and UX, they found recruitment agencies too time-consuming for their needs, turning to UserTesting. While their main focus was assessing their app experience, they also found that the core feature’s description misled and confused users. After removing it from the US app store, this resulted in an increase in both US customers and retention rates. 

Looking ahead 

While your app store optimization efforts are just one aspect of your mobile strategy, they can make a huge impact on the success—or failure—of your app. Using human insight as your north star to evaluate and optimize your listing will validate your data metrics, and get you on track to improving them with real, customer-centered feedback. Combine this with a mobile app that lives up to what its listing promises, and you have an app store force to be reckoned with.