Progress Update and Roadmap

In our recent post about Accrescent’s financial future and sustainability, we announced that to build trust and transparency with our community, we’d be publishing a follow-up blog post about what we’ve been working on lately and where we’ve been setting our sights, i.e., what our roadmap is. This is that post.

If you haven’t already read our post about Accrescent’s future, we highly recommend reading through it for context. The items planned on our long-term roadmap cannot be accomplished unless we reach our funding goal, so any time you spend reading or sharing our post is deeply appreciated.

Beside our long-term roadmap we also provide a timeline of what we plan to accomplish in the next 3 months. We hope this will encourage our supporters and demonstrate our commitment to delivering a first-class Android app store which not only is private and secure, but has the modern features users and developers have come to expect.

Without further ado, here is what we have accomplished in the past many months.

Development progress update

Because we believe in the open source ecosystem, most of our development happens in the open on GitHub and is published under an open source license. However, when creating new projects or experimenting with large changes, we tend to keep some work private temporarily until it’s ready for release¹. We’re publicizing and open sourcing our most recent work² (described below) today to share with our supporters what we’ve been accomplishing. This doesn’t mean it’s all ready to be used in production, but most of it is close or already deployed.

There are 4 major projects we want to talk about:

1. Parcelo, our server backend
2. The directory service, a major addition to our server architecture
3. Ina, our delta update library
4. Our server infrastructure

Server backend (Parcelo)

Parcelo was already a public project, but the changes we’ve made recently are significant enough to warrant a mention here especially since they relate closely to our other projects.

Decoupling for scale

First, we’ve made a number of changes to Parcelo which make it easier to reason about, deploy, and scale. Originally, Parcelo only ran as a single instance on a single server. Its database and all files it stored and served were only accessible on one machine. This approach was sufficient for a while but eventually began to crack at the seams. Server operating system limitations³, VM configuration drift, and manual maintenance tasks⁴ made maintenance difficult and time-consuming. Besides these problems, Accrescent app developers are distributed worldwide, meaning those far away from our server experience higher latency than those close to it, worsening their experience.

In the end, we decided to decouple Parcelo from its dependencies to enable it to:

• Run decoupled from its deployment environment, removing OS limitations
• Scale to support multiple instances worldwide, bringing them closer to developers
• Deploy automatically almost anywhere, reducing our maintenance burden

In more technical terms, we rewrote core parts of Parcelo and our deployment scripts to:

• Use a central networked database (PostgreSQL) rather than a local database (SQLite)
• Store files in object storage (S3, GCS) rather than the local filesystem
• Read configuration from the environment rather than a local file
• Deploy Parcelo in a container rather than on a VM
• Deploy our developer console frontend in the cloud rather than on a VM behind a reverse proxy

This migration required extensive changes in Parcelo’s codebase, but in the end, it made Parcelo much easier to maintain and update. To be specific:

• We are no longer limited by the operating system we choose
• It is much easier for third parties to deploy Parcelo in an environment of their choosing
• We can automatically deploy to the cloud much easier than before
• Parcelo can run with multiple instances⁵

all while maintaining very reasonable costs.

We’ll go over our cloud usage and how it’s changed in more detail when we get to our server infrastructure.

The directory service

The directory service is a new server we’re introducing to Accrescent’s architecture. It represents the largest overhaul to how Accrescent works since its inception, completely revamping how Accrescent finds apps, calculates app compatibility⁶, fetches app listings, and checks for updates. Its introduction also enables a swath of new features and improvements and makes implementing our most-demanded features finally possible.

Features and features and features, oh my!

To be specific, integrating the directory service will immediately make the following improvements:

• Drastically improve APK resolution⁶
  • Add support for telling Accrescent whether an app is compatible before installing it
  • Add support for previously unsupported technical compatibility specifiers⁷, e.g.:
    • Minimum and maximum Android version
    • Required device features
    • Texture compression format (commonly required by games)
    • Minimum OpenGL ES version
  • Add support for install-time asset modules / asset packs (commonly required by games)
  • Add support for install-time feature modules
• Enable easy backward-compatible API changes through a future-proof gRPC API⁸
• Increase network bandwidth efficiency and reduce network latency⁹
• Provide a consistent and stable API for other clients (such as our future web client)

Additionally, the directory service will make the following features very simple to add shortly after release:

• Support for localized app listings (i.e. non-English app descriptions)
• Support for long-form app descriptions
• Support for screenshots in app descriptions
• On-demand app publishing / listing and unpublishing / de-listing
• Scheduled releases / publishing

Finally, the groundwork the directory service lays will make it finally possible and feasible to implement these important major features:

• Viewing app usage metrics
  • Downloads
  • Active installation estimates
  • Listing views
• Release channels (e.g. beta releases alongside regular stable releases)
• Delta updates
• Private / internal apps and paid apps¹⁰

How it works

Accrescent requests all app metadata from the directory service. This includes:

• Which apps are available in the store
• Whether an app is compatible with the device
• App descriptions and icons
• How large an app is
• Whether updates are available
• Where to download an app’s APKs
• etc.

Accrescent can then download an app’s files from the app repository as before. The key difference from our previous architecture is that app metadata is dynamic. Thus, it can adapt to users' different devices and languages on-demand.

Quarkus ClickHouse extension

As part of developing the directory service (which uses the Quarkus framework), we’ve developed a Quarkus extension¹¹ for using the ClickHouse database. This makes using ClickHouse from Quarkus significantly easier as a developer, so we hope the community finds it helpful.

Source code

Today we’re open sourcing our work on the directory service and the Quarkus ClickHouse extension and releasing it to the public. You can find the directory service’s source code at the embedded link.

Delta updates (Ina)

Delta updates are a method of reducing software update sizes by reusing existing code from the old version of an app when it hasn’t changed in the new version. Google Play has had delta updates since 2012, but they are noticeably absent from alternative Android app stores. Because we believe the alternative app store experience shouldn’t lack the features of the big players, we created our own implementation of delta updates for Accrescent early last year to bring dramatic bandwidth savings to our tens of thousands of users.

Our implementation is designed to have:

• A memory-safe implementation
• Android support out-of-the-box
• Comparable performance to existing solutions
• Low memory requirements
• Streamable patches

We’ve been waiting to integrate delta updates into Accrescent and open source them until after the directory service is deployed since it will be significantly easier for us to do so then. While we still need to deploy the directory service before adding delta updates to Accrescent (which we can’t do without your support), we’re open sourcing Ina (our delta update library) today. You can find Ina’s source code at the embedded link on our GitHub and test it out for yourself by installing Rust and then Ina with

cargo install ina-cli

The ina and sufsort crates are also available on crates.io for any Rust developers who want to use them.

Server infrastructure

We’ve made massive changes to our server infrastructure this year to handle our increases in global traffic and to ensure we (the Accrescent developers) can spend our time focused on improving Accrescent instead of managing and deploying servers. For some numbers: we serve over 17 TB per month in apps, respond to over 28 million requests per month, and have an estimated 63,000+ active installations today¹².

The two biggest changes we’ve made in this category are 1) migrating to the cloud and 2) managing our infrastructure as code. These two changes combined have made our server infrastructure nearly zero-maintenance and increased our capacity for traffic with only a slight increase in costs compared to previously.

For an idea of the size of this effort, we currently have 80 managed resources including:

• Object storage buckets
• Containerized applications
• Certificates
• Databases
• API keys
• DNS records
• Application configurations
• VPC networks
• Firewall rules

and many, many more. While some of these are infrastructure boilerplate, almost all of them represent resources we previously had to manage manually, so this migration lifts a huge maintenance burden off of our shoulders.

Managing our infrastructure as code also means deployments are (semi-)automated. We used to use Ansible playbooks to deploy new configurations or versions of Parcelo, but the process was slow and error-prone. Now, we only have to update a few values and run a deploy to upgrade server applications or update configurations, which has drastically reduced the amount of time we spend on deployments and server maintenance

Roadmap

Accrescent is a novel Android app store focused on security, privacy, and usability. Building an app store is an enormous project, and it’s especially difficult to build an app store with built-in security and privacy guarantees beneath a clean, modern, and usable interface. We have big plans for Accrescent, so here we outline some of the major features we plan to implement in an estimated timeline for the next 3 months along with a coherent long-term roadmap.

Short-term timeline

The above image summarizes the features we want to implement (or start implementing) in the next 3 months of Accrescent’s development. Here’s what they mean:

• Directory service: Developing, deploying, and integrating Accrescent with the directory service, a huge overhaul to how Accrescent works which enables a swath of new features and makes implementing further features drastically easier. This is mostly an under-the-hood change.
• Long descriptions: Developing and releasing long-form (up to 4,000 characters) descriptions for app listings. Developers will be able to add long descriptions to their apps, and users will be able to see them inside Accrescent.
• Localized listings: Developing and releasing support for localized app listings. Until now, Accrescent has only supported creating a single listing per app, meaning information about the app (e.g. its description) could only be distributed in a single language. This feature will allow developers to create app listings for multiple languages and enable Accrescent to automatically select the best-matching app listing language based on the user’s settings.
• Listing screenshots: Developing and releasing support for screenshots in app listings. A highly requested feature, this will allow developers to add screenshots of their app to app listings and enable users to see them from within Accrescent, boosting visibility and giving users an opportunity to preview apps before installing them. Screenshots will support localization.
• Download metrics: Starting development on support for app download metrics. Another highly demanded feature, this will enable developers to see how many downloads their apps are getting, how popular their listing pages are, and how many active installations they have all while preserving user privacy.

This timeline is only an estimate and could change based on the difficulties (or welcome surprises!) we encounter.

Long-term roadmap

As helpful as a short-term timeline is for the immediate future, we know it’s just as important to have a big-picture vision and a plan for the long haul. That’s why here we list a pool of major features we hope to implement after our short-term roadmap is complete.

These features are not yet set to any specific timeline. As a developing project, we want to stay flexible to changing priorities and incorporate feedback from our users. However, we have ordered the below features in the approximate order we would like to implement them from top to bottom.

• Miscellaneous features
  • Release channels: Allowing developers to create (and users to opt-in to) beta development channels of their app for testing new or unstable app features.
  • Delta updates: Updating Accrescent’s architecture to generate delta (i.e. patch) updates using Ina, our delta update library and utilize them in the Accrescent app to save dramatic amounts of bandwidth for users and us.
• Developer-focused features
  • App management teams: Enabling multiple Accrescent console users to manage a single app with fine-grained permission support.
  • App publishing API: Developing a stable API to enable developers to publish app updates, listing changes, etc. programatically instead of exclusively via the Accrescent developer console.
  • Scheduled releases: Allowing developers to schedule when app updates or listing changes are published instead of publishing them only on-demand.
  • Staged rollouts: Supporting staged app update rollouts (i.e. updates which are rolled out to only a percentage of users which developers can increase over time).
• New projects
  • Web store frontend: Creating a web store frontend to Accrescent so that people can browse Accrescent’s app catalog without installing Accrescent.
  • Accrescent CLI: Developing an official CLI for app developers to package, upload, and manage their Accrescent apps with ease from an interactive terminal or CI/CD pipeline.

There are many other features we hope to implement in the background, such as improving the robustness of our server applications, enhancing our infrastructure security, refining store policy, and improving the user experience of our developer console. But the above includes the major features we currently have in our sights.

One small step, one giant leap

It is absolutely incredible for us to see how people have surrounded and supported Accrescent. What began as a passion project with a vision has grown into a an app store globally trusted by tens of thousands of people unified in their desire for a secure, private, and easy-to-use Android app store. We have made leaps in our goal of providing a modern, open source alternative to the big tech app stores, and we hope to make bounds more by pushing the traditional quality standards of free and open source software.

But we can’t do that alone. All of the above planned long-term work is conditional on us reaching our funding goal, which is $5,800/month in recurring donations to cover full-time development. Without funding, we cannot afford to continue advancing Accrescent, so we need your help. If just 2% of our users each gave $5/month, we’d have enough to keep Accrescent actively developed far into the foreseeable future. Our expenses are very lean, and we hope this post demonstrates the progress we can make and hope to make with the donations we receive.

If you want to help us develop Accrescent into the featureful app store it was meant to be, please visit our donation page and make a recurring donation with GitHub Sponsors (all donation forms are appreciated, but recurring is the most helpful for ensuring we can continue development). If you can’t donate but still want to help, share this post with others to spread the word. You can always email sponsor@accrescent.app with donation questions or for other sponsorship arrangements.

We’ll be sharing funding progress updates on our social media accounts and you can view the progress directly on our GitHub Sponsors page. Until then, we’ll ensure Accrescent continues running smoothly in the background: freely, securely, privately, and open source as ever.