Early Testing Strategies to Maximize Your Hyper-Casual Game’s Potential

Editors note: This article was written by Vivian Dong, Senior Publishing Manager at JoyPac. This is based on their most recent webinar, which you can watch here.

Testing is the most important part in the whole process of hyper-casual game development.

When you have an exciting new game idea, you need to quickly answer the most fundamental questions: Will your idea be appealing enough to draw in players? When they download and play it, will they keep playing it? Will it make money? To answer these questions, you’ll need to start testing as early as possible.

We recently hosted a webinar on early testing, along with Gabriel Stürmer, CEO at Clap Clap Games, and Kristina Truvaleva of Business Development & Global Partnerships at ZeptoLab.

Here are ten lessons we took away from the webinar.

Lesson 1: Test your idea’s marketability before you even start making the game

You need to know as soon as possible if your game idea has marketing potential. And to find out, you’ll want to run a click-through rate (CTR) test.

You can test something as simple as a ‘creative’ – a short video that displays the basic concept of the game. You display it as an ad on Facebook and see what percentage of users click on it.

“You need to think: is the game going to be attractive? Can you make the first five seconds of the video grab people’s attention? That’s crucial for CTR testing as you’re not making the game – you’re making creatives. We try to make creatives as clean as possible, with no additional texts. Additional add-ons won’t save a game if the core mechanic isn’t attractive.” – Gabriel, Clap Clap Games.

“We try to do videos that are very focused on gameplay. We found that the simpler the video, the better the results. You don’t need to over-polish the game because it doesn’t affect the results and can sometimes even affect them negatively.” – Kristina, Zeptolab.

Lesson 2: Have a clear process and know what results you’re looking for

When you run CTR tests, you need to use the same process for each idea you test, so you can accurately compare the results. And you need to know what CTR would be good enough to justify developing the concept further.

At Clap Clap Games, Gabriel runs CTR tests using this process:

1. Create four or five videos, lasting 10-15 seconds.
2. Upload them to Facebook as a ‘link click’ campaign in US iOS for four or five days. Spending 50USD per day.
3. Check to see if the CTR is 4% or higher. If it achieves that result, continue developing the game. If it doesn’t, consider abandoning it and moving onto another concept.

Different developers will have different methods and different targets. You need to adapt the process to your style.

Kristina, for example, does CTR tests slightly differently at ZeptoLab: “For us, it’s enough to run the CTR test for only one day. And if the result is above 2.5% then it’s usually a good enough indicator that we should work further on the idea.”

Lesson 3: You can create new concepts much faster than you think

Gabriel works in only a two-person team at Clap Clap Games. But they can create roughly 14 ready-to-test concepts in just a week.

How do they achieve this incredibly fast output? It starts with generating lots of new ideas. Gabriel takes inspiration from simple, day-to-day tasks – the two examples he showed us in the webinar are based on popping bubble-wrap and sorting luggage on conveyor belts. But he also looks at existing, successful concepts and looks for ways to put a fresh new spin on them.

“Sometimes we find interesting objects and rebuild them in different settings and use them in different ways. By doing that, you increase your chances in finding one concept that works well and also you can do a whole lot more by building on top of something that you already have.” – Gabriel.

Gabriel also tells us that the more concepts you create, the faster you can create them – because you build up a big store of assets from ideas you’ve worked on before: “We have a huge Unity file with lots of assets and scripts. So if we want to do a 3D runner, for example, we already have a script for the movement – we already have the assets we need to make this idea work.”

Lesson 4: When a user clicks on the ad, where they end up matters

When you’re running a very early CTR test, you’re running an ad for a game that doesn’t exist yet. It begs the question: where does the link actually take you?

“In the very early tests, we linked to just a random App of ours, like another game. But that’s not ideal because Facebook sees that people aren’t actually downloading the game – which is what they optimize for. We switched to link click campaigns, so we don’t point to a specific game anymore. We point people to a page, either on the app store or our website, saying ‘game is coming out soon – sign up here to get notified’. – Gabriel, Clap Clap Games.

Lesson 5: Target a broad audience

While running early tests, keep your audience as wide as possible and don’t try to narrow it down. Both Gabriel and Kristina test on US Facebook – the biggest and most representative market – for all their games, apps, advertising, etc.

Lesson 6: If your CTR results are good enough, move onto testing cost per install (CPI) and retention rates

With good CTR results, you’ve proved your game concept has the potential to succeed. Now it’s time to develop a basic version of the game. (Gabriel says this basic version would usually be ten or so levels.)

Once you have this basic build of your game, you can upload it to the app store and start a new round of testing. You want to find out how profitable the game can be (CPI) and how long it can hold players’ interest (retention rates).

At ZeptoLab, they run CPI tests for one to five days on US iOS. Kristina says their target CPI is less than 0.30USD. If the CPI is higher, they usually abandon the concept, because they optimize the project only on the later stages, during the soft launch.

At Clap Clap Games, Gabriel tells us they aim for 40% day one (D1) retention and around 13% for day seven (D7). If the results fall just short of these targets, they might test a new iteration of the concept. But otherwise, they just kill the idea and switch to something new.

Lesson 7: There are several factors that can mess with your CPI

If you have a great CTR with great D1 and D7 retention rates, don’t jump to conclusions if your initial CPI turns out to be disappointingly high. It could be the result of something unrelated to the viability of your game.

Look at your store conversion rate. Sometimes users can click through to the app store and be discouraged from downloading the game by something minor and easily fixable. Maybe the screenshots are too different to what they saw in the trailer. Maybe something in the description is putting them off.

It could also be that the CPM (cost per 1,000 impressions) is unusually high on Facebook at the time. It could be higher because of the audience Facebook optimizes for, or a result of competing with a lot of other advertisers.

Lesson 8: Facebook is pretty much the only good platform for early testing

Throughout our webinar, no one even mentioned other advertising platforms. And there’s a good reason for that.

“When we’re talking about the early testing, we use only Facebook at the moment. It has all the settings we need and it’s easy to optimize.” – Kristina.

“Same for us. Facebook is standard for the whole industry and it’s the only one that allows you to compare the numbers you get. For example, if you get 50 cents CPI for one game and 37 CPI for the other, you have a direct comparison. While if you go with a different ad network, usually, you don’t really know if you’re getting lower or higher CPI because of the different network or because of the actual game.” – Gabriel

Lesson 9: A/B testing is great for diagnosing problems

This didn’t come up in the webinar, but we talked about it off-screen and it’s a useful lesson. If there’s something causing a problem for your game, an A/B test is a good way to confirm the cause – and often settle some in-house disagreements. Here’s an example:

Let’s say your retention is failing because the ads start appearing too early. Your analytics tell you that players are leaving your game after level seven, but you don’t know the cause. It could be to do with level structure, difficulty, ads, items or any number of things.

You discuss it and decide it could be that the difficulty spikes too early. So you run an A/B test – some players get the easier version, some get the normal version. When the retention rates for the easier version show no improvement, you now know the difficulty isn’t the problem.

So next you run another A/B test with a version that holds back the ads until level 12, when players are more invested in the game. Sure enough, that version has much improved retention. And voila – you’ve diagnosed your problem.

And finally, lesson 10: Know when to kill a concept

All testing can do is show you how much potential a game has. If it’s showing results that are almost good enough, you might want to play around with it and find ways to boost the numbers.

But the whole idea of testing is to sink as little time and effort as possible into games that aren’t destined to be successful. If you spend ages tweaking and retesting a concept simply because you really love the idea of the game, you defeat the purpose.

If a concept clearly isn’t showing the right results in early testing, you need to kill it and move onto something else – no matter how much you like the idea.

Watch the full webinar online

You can watch this full webinar on our website. And you can also catch up with all our previous webinars.