User metrics — and the insights product teams derive from them — can work as the foundation for feature upgrades users crave. But watching the wrong metrics or failing to effectively synthesize all the data from various sources can lead to acting on inconclusive conclusions.
That’s to say, there’s no one-size-fits-all method for successfully acting on user data.
For instance, the product team at gaming company MobilityWare uses the data management platform Leanplum to get real-time data on their player-counts. On the other hand, dating platform Tinder built its own platform, “Phoenix,” to host its user data-driven experiments and monitor their results.
Professionals from these two LA companies discussed why aspects like ease of use and performing real-time versus static data analysis helps them choose the perfect tools for their needs.
MobilityWare Product Owner Eden Simpson said her team tests feature usage in smaller games, and if user data deems the features successful, they port the new features to games with bigger audiences.
What tools or technologies are you using to capture and analyze user behavior data?
In mobile gaming analytics, we are always balancing the trade-offs between the speed at which we get data, and how complete it is when we act on it. So we use tools that excel on each side of this spectrum. To interact with our data quickly — during a live ops campaign, for example — we use data management platforms like Leanplum or Treasure Data. For strategic analysis that requires highly accurate data, we unify data from all our sources into Snowflake, our preferred cloud-based data warehouse. From there, our analysts can write SQL-like queries for analysis and build dashboards in Tableau.
The behavior we watch most closely is how many games people are playing.”
Why did you decide to use these tools over the other options on the market?
We choose data solutions based on a couple of areas of focus, and the first is feature set. The more functions a specific tool does well, the more likely we are to use them. For example, does a data management platform only track in-app message campaigns, or can it execute A/B tests and visualize KPIs as well?
The second focus is ease of use. Does leveraging features require a degree in computer science, or are they intuitive enough for product owners to use them?
The last focus is on implementation and documentation. Vendors will try to wow users with how effective their tool is, but how easy is it to integrate? Engineering resources are precious so we factor in the cost of incorporating and maintaining a given solution.
What specific behaviors are you keeping an eye out for?
We earn most of our revenue from advertising, so we don’t have to search for ways to encourage players to spend money on in-app purchases. When we look at data and adding new features, we are always focused on how to make our games more enjoyable. Due to this methodology, the behavior we watch most closely is how many games people are playing, which are in the billions each day across our titles.
Beyond user behavior analysis, what’s the most effective method you’ve used for improving user retention?
To improve retention, we focus on new game features that give players a reason to come back. Luckily, we have games of varying sizes. This diversity of size allows us to test new features in smaller games and move them to larger ones if the data shows players enjoy them. Adding features this way allows us to try new things and innovate without upsetting our most loyal players.
Our most successful feature is a daily challenge feature, which we were the first to release in card games. It increases the number of days the players interacting with the feature logs in. It was picked up by many of our competitors and is now seen as an expected feature in card games.
Product Manager Jona Cho said Tinder’s user metrics revolve around helping people make real-world connections. Cho said her team uses machine learning and Tinder’s custom-built data platform to act on metrics leading to that end.
What tools or technologies are you using to capture and analyze user behavior data?
We use an experimentation platform we built in-house called Phoenix to both run our experiments and monitor results. For more in-depth analyses, we use Sisense and Mode. Sisense is great for time-series analysis while Mode, with it being a Jupyter Notebook-type tool, provides a lot of flexibility around collaboration and sharing across the organization.
We look for the events that proxy long-term value from the ecosystem of our product.”
What specific behaviors are you keeping an eye out for?
We look for the events that proxy long-term value from the ecosystem of our product — connecting in real life with another person — and keep an eye on leading indicators of that behavior. We iterate on the product to increase the value we deliver to our members to improve long-term retention.
Beyond user behavior analysis, what’s the most effective method you’ve used for improving user retention?
Tinder members can have up to nine photos in their profiles. However, the photos might not be in the best order to increase their chances of receiving likes. By using machine learning, we reorder photos within profiles to optimize the ranking and make sure that the best photo is featured first. This experiment not only helped members receive more likes but also improved retention.
