Not Evil Sudoku recently crossed 50,000 installs and it seemed like the perfect time to build an analytics service for it.
Press enter or click to view image in full size
A Few Notes
- This is not a “How to build your own analytics service” article, although if this article inspires you to do so, feel free to ask me for more details.
- This is by no means “the best way” to go about this, every business and person has their own needs and requirements. This is just how I did it, feel free to suggest improvements.
- In order to pull this off I also had to use Amazon Certificate Manager and Route 53 but those services are not directly relevant to this project and so I’ve left them out.
- I’m not an expert, I’m just a guy with a laptop! I’m able to experiment because this service doesn’t come in contact with any personal/sensitive data.
Introduction
Not Evil Sudoku is a sudoku app for iPhone and iPad that I released almost a year ago (April 2022) after becoming increasingly frustrated with existing sudoku apps. A few days into January 2023, the app crossed 50,000 installs. Users can turn analytics off with a single toggle switch in the Settings.
Why Build Your Own?
There is an abundance of analytics services and it’s not too difficult to configure the non-privacy focused ones to be private.
I built my own simply because I thought it would be fun (it was fun). Building and deploying an analytics service at scale (even if the scale is relatively small) is a cool opportunity and I’ve always been interested in this kind of thing.
Building my own analytics service was never going to be time or cost-efficient (atleast in the short-run) but it comes with some perks. My analytics service is highly tailored to what I require, the app collects only exactly what I need and can also track things like game quality.
Cloud Providers
Why Amazon Web Services? I’ve used parts of Azure and Google Cloud before in projects and other things so I thought it would be interesting to try AWS. In addition, I was planning to use serverless and AWS Lambda seemed like the tried and trusted option. Cloudflare Workers, Google Cloud Functions, and Azure Functions are all alternatives that would’ve also worked.
Why serverless? AWS EC2 and similar options seemed like overkill for my needs.
Process
Goals
Build a private, cost-effective analytics service that would help me prioritize future development for Not Evil Sudoku.
Privacy
- The identifier isn’t linked to personal information
- No device metadata is linked to the identifier
- All data is collected in aggregate
The app uses Apple’s identifier for vendor API which allows the app to identify a device without collecting any personal data.
On top of that, the app doesn’t even relate the device metadata to the identifier. All metadata is just aggregated. This means that if your device tells the analytics service that the iOS version is 16.2 then in the “Global Aggregated Analytics table” the field 16.2 will be incremented by 1.
This just tells me that x devices are on 16.2 which is all I need to know anyway.
Cost
I’ve already spent more than $60 because I misconfigured AWS and I was keen not to lose more money. Lambda and DynamoDB both had fairly decent free tiers and that definitely played a role in my decision to use them.
Devices only send an analytics update once every 30 days which should also reduce costs. As opposed to most “real” analytics services which send real-time updates, sending data as soon as a user does something. But for my purposes this isn’t necessary.
In the future, I can always make it send updates more frequently but this seemed best until I get a better idea about costs.
Game Quality
I’ve always felt like an imposter on the App Store and the app becoming more popular has only intensified that feeling. I’ve had many a sleepless night filled with doubt about whether the app’s puzzles are good enough. Are my hard puzzles actually hard?
To investigate this, I’d been playing sudoku on a variety of other apps and websites to compare difficulties. I found that many of my hards were actually really hard but there were also some that felt too easy.
That’s where the Game Quality table comes in. Every month the user’s device tells the service which games they played and how long each game took them. Of course, this is also aggregated and I couldn’t tell you which games a user played even with their identifier.
This allows me to track the average time a user takes to solve a particular puzzle and now I can make my levels more consistent by booting out the ones that are too easy/hard for their difficulty level.
Track Usage Over Time
Finally, I also wanted to see usage over time, hopefully, that would be a strong signal of whether the app was getting better or worse.
This would also allow me to track the number of Monthly Active Users, (atleast the number of monthly users with analytics enabled) and compare from month to month.
To do this, I set up EventBridge to run a Lambda function to save all the aggregate global data to the historic data table at the end of the month. This also resets all the aggregated fields to 0.
Conclusions
This is still verison 1.0 and there’s lots of scope for expansion and improvement.
There are tons of optimizations I could make inside each of these services but overall I’m pretty happy with the architecture.
I had a lot of fun building this, it’s not often that one gets the opportunity to build an analytics service for a slightly popular app. If you have any suggestions, feel free to leave a comment or tweet at me: @selvarajashish.
I hope this was at least a mildly interesting read.