Planning a new, modern and stable NewPipe · TeamNewPipe NewPipe · Discussion #10118

Hey guys. You might have noticed the somewhat slower and haphazard updates to NewPipe for the last few releases. The thing is, NewPipe was started, like so many others in FOSS, as an enthusiast project. It's been over 7 years since then, and the age of the codebase is starting to show through spaghetti code, outdated UI, bad design choices and instability.

It has gotten to the point that we're having difficulties even maintaining the current code and fixing bugs, let alone adding new features, due to complex interconnections and lack of proper modularity. Progress has been slow, since developers were focused on fixing nasty bugs. Often users requested features that would make sense, but nobody had the time (and, sometimes, knowledge of the codebase) to implement them.

To mitigate the issues we initially tried to do incremental refactors of various components. Take for example the player, which has often been deemed "a hot pile of garbage": it underwent a couple major Pull Requests that restructured it while maintaining all features. Although now the code is a bit better, new crashes and unwanted behaviors have arisen, because the interaction between the old and the new code is not optimal. We tried again and again to debug the problems, without luck.

At the moment NewPipe is relying on libraries and components that have become outdated, which might not only drive users away from the app, but also push away developers wanting to get some experience on new technologies. We need to migrate to Material Design 3, Kotlin and Jetpack Compose, and unfortunately doing so basically requires rewriting everything.

So, we've discussed and discussed this ad nauseam. We're very sure the old code needs to go, no matter what, to be replaced by a modern and professional codebase. But we can't decide on the approach to get there.

Anyone who's been in the FOSS community for long has come across at least one or two failed rewrites or refactors, and we don't want to repeat such tragic history. We believe NewPipe really has the potential to be a staple app in people's devices, if only we could get across this massive hurdle and come out the other side relatively intact.

What we need from you guys is advice. Preferably, experienced advice. Do we just dump the old code into maintenance mode and start afresh? Or replace it piece by piece like the Ship of Theseus? A combination of the two? Or something else entirely?

Please let us know. Also, if you don't have any directly helpful advice, please, please restrict yourself to just upvoting or leaving a thumbs up instead of commenting, so that there is less noise and more info in the thread. We all know how fantastic walls of text are at putting us off from reading, and we'd like to avoid that.

Also, if you are experienced in Android mobile development and would like to help us out with the rewrite, be sure to reach out to us! We will be using the most modern components whenever possible, as detailed below, so it is sure going to be a good learning experience for everyone involved. As a first step, please join the IRC from the ReadMe.

By the way, don't worry, all currently open PRs that introduce big features will be merged in NewPipe 0.26.0 or future releases of the current app. See the related Project and also take a look at the plans we have for the current app below.

Thanks! The NewPipe team.

What approach do we want to use?

• migrate the existing codebase of the app:
  • => risks:
    • strange issues between new and old code;
    • keeping an architecture which could be bad in some places;
    • going nowhere;
    • lack of interest from external contributors.
  • => benefits:
    • staying on one repository only;
    • better handling of issues.
• create a new app and reuse good code:
  • => risks:
    • a lot of time before reaching feature parity with the current app;
    • abandonment of devs due to the huge work required.
  • => benefits:
    • creating a modularized app in an easier way, and following good architecture and dependency models, hence making migration processes easier too;
    • potential increase of interest for NewPipe projects, which could lead to more (quality) contributions and to new team members;
    • embedding more community input in the app.

Similar or common issues are being raised increasingly in the Extractor repo, so these questions could be considered for the Extractor too.

Key elements we should use and not forget during development

• Good design thinking: possibility of contacting UI/UX designers and using community feedback to create new app designs:

  • => use Material 3 design system

  • => think about accessibility and form factors

• Asking feedback from the community and taking it into account where that's relevant: results of a subreddit poll returns around 80% in favour;

• Better organization: good use of Projects, which we would use to plan what should be in the next releases, what is in progress, what's remaining

  • => we've started to do this in the recent release, and we should grow our usage of these tools

• Try to get new and experienced people into the team: most of us are students passionate about technology, but:

  • we have other things to do in our life, and some or all of us may leave the NewPipe project at some point;

  • we can't know everything, and we lack knowledge in some domains.

• Apply more agile principles such as doing smaller but more frequent release cycles.

• For the development by itself:

  • Jetpack Compose;
  • ViewModels;
  • Hilt for dependency injection;
  • modules for app components (player, downloader, channel, search, settings, ...) with usage of interfaces for communication between them;
  • use Kotlin everywhere in the app to reduce code and bugs;
  • Jetpack Media3 for the player.
  • use https://penpot.app to create designs (it's an OSS alternative to Figma)

Decide what we want to ship in the current app

• highly requested features, mostly the ones which are blockers for content accessibility (also see the related Project):

  • comment replies (almost ready);
  • channel tabs (almost ready);

  => work of these features probably needs to be done by the team due to code and/or approaches from external contributions not quite meeting our standards

  • multiple audio tracks support (already merged, released in 0.25.2);
  • advanced search filters (could be a blocker for channel tabs due to the reworking of filters in the extractor);
  • better quality of images, by introducing multiple images support (a pull request was opened).

• bug fixes and regressions, in particular the most annoying ones (we can't guarantee we will be able to debug everything though, for the aforementioned reasons):

  • fullscreen issues on some devices since the player UI rewrite;
  • player "App Not Responding" and crashes on some devices and in some cases;
  • ViewHolders not attached crash;
  • fullscreen button with autorotation enabled.