I have been a fangirl of the container ecosystem since 2014, when I started working at this new startup named Docker, but it's time to talk about where we are today, how we got here, and what matters most going forward.
Docker was born as DotCloud in the PaaS era of Heroku and CloudFoundry. The philosophies espoused by Heroku’s 12 factor app and Pivotal’s haiku “Here is my code. Run it in the cloud. I do not care how.” and 5 S’s still hold the imaginations of many technology leaders. These platforms provided a great developer and operator experience – one without the headaches of building and maintaining the platform but simply focused on using it to get the application outcome you wanted.
A disturbance in the force awakens a new ecosystem
In 2013, Docker was born as the DotCloud team extracted some of the underlying technology in their PaaS and made it accessible to developers. Containers became a key pivot in the timeline. A year later Kubernetes was released and a year after that the CNCF was formed.
The massive developer adoption of containers raised new questions about how to handle these new apps beyond the developer’s laptop, including…
How do you group multiple containers together to form an app?
How do containers communicate with each other?
Where are we deploying to? How do we configure and manage that?
How to schedule and orchestrate containers? and at scale? Can it be automated?
How to secure containers along the SDLC and at runtime?
…and many more…
Like virtualization was a paradigm shift from physical compute that created a new set of tools for virtualized environments; containers required us to rethink how we addressed all of the concerns in dev and production to work in an ephemeral and dynamic environment and in this new era, we also need to make it open source.
And a decade later, we’ve ended up here.
Today, we have thousands of individual technologies with loads of options within each category and I can’t help but wonder if we may have lost sight of what the point of all this was. We are all still in search of that platform experience.
Completely bespoke and completely complex
So where did we go wrong? Or did we? We got so excited about what a couple of new components unlocked that it inspired us to chase every single component in the stack. The great innovation that the CNCF brought on was that so much of the experimentation and innovation was out in the open. You would see tipping points in areas like storage, networking, or security that would drive vendors to create a host of new projects and then standards would emerge to agree upon the interfaces. The many SIGs and project maturity stages helped to provide a structure for big changes and setting expectations for end users.
Even with that the rate of new projects introduced and the sometimes seemingly VHS vs. BETA type debates between technology approaches made it difficult for end users to decide which technology to choose and bet their enterprise architecture on. In addition to weighing an ever growing and evolving list of technology choices, the pressure to be truly “engineering-led” or “innovative” led many an IT organization to piece together their own platform from the open source puzzle pieces, support it themselves, and engage publicly with the upstream. This alone requires an engineering culture that many companies were prepared for.
While interesting from a technology point of view, this led to a complexity that often bogged teams down with lifecycle and integration management and maintenance of the system more than supporting getting apps built and deployed to production.
Our excitement into the ingredients led us to over shop for the latest and greatest without planning ahead for the final meal. Too many end user RFPs were seen with project names or technology buzzwords sprinkled in without a mention of new or different capabilities to be delivered that will improve business initiatives and application outcomes.
The question is really, does anyone remember what they were building and who they were building for? How differentiated is your platform heavy lifting?
Coming full circle to the return of PaaS
Don’t call it a comeback and especially don’t call it a PaaS but the idea is back and today it goes by Internal Developer Platform or Platform Engineering now. Regardless of the name, the spirit remains that the idea we had with PaaS was right. We still want that experience.
This past decade’s boom in operations and lifecycle tooling, we completely complicated the interface for developers. It is one thing for the architects and IT to evaluate multiple infrastructure vendors, it is not advisable to make developers do that – mostly because their choices may just get pushed back when the apps are headed for deployment anyways. And in the quest to improve this developer experience, more new projects have emerged.
The rise of platform engineering as the buzzword du jour coincides a bit with the end user shift to more abstracted solutions like CloudRun and Serverless as a way to get to the outcome faster and not worrying about the guts that make it work. On some level, we expect that the underlying stack powering these services are based on Kubernetes and conforms to standards, but more and more end users are not as fixated on the details. Even those going down a DIY or composable path are looking for more prescriptive patterns for the most common scenarios. Why waste on undifferentiated heavy lifting?
The return of PaaS is the return to platform thinking instead of component thinking - the container ecosystem is on the long journey home to its origins.
Focusing on the purpose
Containers and Kubernetes ushered in a new era of computing with a vibrant ecosystem and community. Even with that, it alone is not the point. The missions that led teams to create containers and Kubernetes were about solving problems facing developers, operators, and specific types of applications they were building and running.
We (the collective we) need to re-focus on the purpose of the platform; for who and what application patterns are we building for to have what kind of characteristics? This is often the harder part because it deals with people and nebulous things like expectations and goals, but the groundwork in identifying your application patterns and their needs will help you match the right platform, services, or mix of technologies needed to get the job done.
Want to keep the conversation going? Reach out at hello@bettyjunod.com