· 4 min read
Data Retention and Event Ingestion
Shariq Manji
Senior Product Manager at GameAnalytics
During this year, our tech teams have been working hard on updating our tool to bring some significant improvements to GameAnalytics, such as A/B Testing and Benchmarks+. As our platform increases in popularity and usage (we now have over 70,000 developers using our tool), a key focus of ours continues to be improving the performance for all of our users.
To do this, we’ll be making some changes to our data retention policy, querying capabilities, and event ingestion rules (for legacy SDKs), in mid-April 2020.
Here’s an overview of what we’re changing and how this affects you.
Data retention policy
One of the most important changes we’ve made is our data retention policy. As always, you can query Historical Data for the last 13 months across all of our features (except for exporting your data, you can do this for only six months).
But now, all filters will be available for data of less than six months old. And a subset of filters will be able to use data for up to 13 months.
How does this effect usage of the platform
Querying data < 6 months old
There are no changes here, you will continue to be able to query data for metrics with all of the below filters and splits:
- Dimensions
- Configs
- A/B Tests
- Cohorts
- Events (All Ad, Business, Error, Progression, Resource events)
Querying data > 6 months old
You will still be able to query data for all metrics, however with a subset of filters and splits:
- Dimensions
- Configs
- A/B Tests
- Cohorts
- Events
- Ad: all events
- Business: all events
- Error: all events
- Progression: Status events (Start, Fail and Complete) only
- Resource: Flow Type (Sink / Source) and Currency events only
In short, we’ll be removing the ability to filter and split by the following events (mentioned below). Mainly, if you run a query with any of these filters for over six months, you won’t see any data.
- Design: all events id’s
- Progression: progression01, progression02, progression03 events
- Resource: Item Type and Item ID events
If you need a refresher on our different event categories and their structure, please check out our documentation for Event Types, Metrics & Dimensions here.
Why are we making these changes
After speaking to a lot of our users, and looking at the usage of our platform, we noticed that there aren’t many queries for data older than six months. And those of you that do, rarely filter on specific events.
The last thing we want to do is limit how you can use our tool. We want to keep GameAnalytics free to use, and optimize the platform so that you can get the information you need as fast as possible, with excellent user experience.
These changes will provide better querying performance across the board while reducing our costs. Which, in turn, lets us focus on building new features and making further improvements.
In the meantime, we’re looking into ways to increase our data retention, so that we can provide data storage and querying for longer than 13 months. We’ll let you know once we have more updates on this.
We will no longer be supporting V1 event ingestion
V1 events are legacy events, sent to our servers by SDKs that we released before 2015. If you’re still integrated with GameAnalytics via our legacy SDKs, you’ll need to update it to the latest version.
Essentially, we’re updating our data collection infrastructure to only allow for the ingestion of V1 Session Events. What this means is that our platform will no longer support most metrics for games with the old SDKs. This accounts for less than <0.7% of events we receive, thus only affecting a few studios (we’ll reach out to you directly if you fall under these criteria).
If you haven’t done so already, please update your SDKs here.
Thank you for understanding
To improve our service, we sometimes need to make changes that we wouldn’t otherwise want to do. For this situation, we know that this won’t have a significant impact on your usage of the platform, and we hope you understand that this will let us make some important improvements to the tool moving forward.
If you have any questions or feel like we’ve missed something, feel free to get in touch, and we’ll get back to you right away. And make sure to check out our latest product update here.