Buildpacks Go Cloud Native
blog.heroku.comThis is Heroku's post, the HN discussion about the CNCF blog post is at https://news.ycombinator.com/item?id=18131378
Pivotal's post for reference: https://content.pivotal.io/blog/peace-of-mind-for-developers...
Aren't they just shell scripts? Last time I looked inside one anyway. Heroku makes a big deal about them, but most Unix people have similar things floating around in ~/bin.
> Aren't they just shell scripts?
They can be written in any language. I have personally worked on buildpacks written in bash, golang, ruby, python and I forget what else.
> Heroku makes a big deal about them, but most Unix people have similar things floating around in ~/bin.
I don't know that I agree. Could you elaborate a bit?
All the ones I've looked at have been shell, as Buildpacks are used to build systems. Python would work too.
They're useful, but they're not an 'invention' or a 'technology' any more than any other bootstrap shell script is.
Nothing is new under the sun and any Turing complete system can do what any other Turing complete system can do.
There's more than "can it run a command?" here. For example, I find that vanishly few operators managing fleets with tens of thousands of containers are prepared to go along with "just run this random bash script on the VM; I promise to keep it up to date".
Yes, Buildpacks are trusted maintained bash scripts. In that way, as you mention, they're better than random bash scripts. That doesn't make them novel. I know you said nothing is new, and I presume you mean completely new, but there's not a single novel thing about this.
Another poster on this thread is using the term 'invented'. Another is describing them as a 'technology' that needs to be understood. Describing a bootstrap script this way sets an incredibly low bar for DevOps.
I think you're dramatically underestimating what a buildpack is.
The Java buildpack is a Ruby application with a modular dependency download/caching subsystem to enable both internet-connected and air-gapped enviroments, hundreds of unit and integration tests, support for runtime injection and configuration modification for TLS certificates, support for several languages including Scala , Groovy, Clojure, support for several package types, several app runtimes, etc. https://github.com/cloudfoundry/java-buildpack
The PHP buildpack is a Python application with a modular dependency download/caching subsystem to enable both internet-connected and air-gapped environments, hundreds of unit and integration tests, support for easily configuring many different PHP extensions, libraries and modules, and extension points to customize the container staging behavior. https://github.com/cloudfoundry/php-buildpack
For developers, this saves a lot of time, effort, and variance for staging their containers.
For operators, this enables a control point that simplifies CI/CD pipelines dramatically and standardizes a means of containerization across projects.
It might not be for everyone, but I think it's a useful alternative to Dockerfiles.
Yep it's a Ruby script that deploys Java. It can run offline. And run tests. And install certs. Nice script.
> For developers, this saves a lot of time, effort, and variance for staging their containers.
Yes. This is scripting. Scripting is good. Sharing your deploy scripts with others is good too. Stop trying to pretend this is a new technology.
Hadn't heard of buildpacks before.
Where do they fit into packaging picture?
How do they compare to (1) VMWare images (2) Docker images (3) Debian packages? And when would you use them instead of one of these.
Buildpacks focus on turning sourcecode into a runnable container. Essentially you just throw code at it and It Just Works.
I think the Onsi Haiku is still the best description:
Heroku invented this, it's been picked up by lots of other groups (including for Cloud Foundry -- I've worked on buildpacks for Pivotal in that regard).Here is my source code. Run it on the cloud for me. I do not care how.> (1) VMWare images (2) Docker images
You have to build these yourselves. The boundary is your CI system, Dockerfile etc -- which you have to maintain. Buildpacks provide a curated, managed, easily upgraded system that does this for you.
> (3) Debian packages?
These are, in some sense, on the far side of where Buildpacks sit. But also slightly different. A debian package is intended to land in a pluripotent environment, possibly under direct human supervision. A Buildpack is meant to build completely self-contained runnable images, starting from sourcecode.
> Buildpacks focus on turning sourcecode into a runnable container
So buildpacks are like Debian source packages?
It's a package of my .c files, and the consumer runs $(CC) to compile and run those and It Just Works?
Not quite. The source package has to be packaged by someone, then dpkg knows what to do with it.
A Buildpack instead takes source as it is. You use Maven? The relevant buildpacks recognise pom.xml and know what to do. Bundler? Ruby buildpack sees the Gemfile and takes responsibility. And so on.
I as a developer can send the same sourcecode I have in my git repo or on my local filesystem, with no additional effort to prep it. The buildpack does all the work.
Putting it another way: it makes sense to create a debian source package buildpack. There are apt buildpacks in the wild, I expect they'll be revisited.
> The source package has to be packaged by someone
Same with buildpack-packager.
But I get the idea. Thanks for explaining!
Non vendor answer:
It's a script. It might use packages, it might not. It builds a box with whatever runtime is necessary to run your app. It's not a new technology. The resultant image could be a container or any other kind of imaging format.
Here's the node one. It's a shell script. It sets up node. You probably already had it before you knew what a build pack was:
https://github.com/buildpack/samples/blob/master/nodejs-buil...
You're looking at the sample v3 buildpack. It's simple because it's a sample.
A current v2b NodeJS buildpack can be found here: https://github.com/cloudfoundry/nodejs-buildpack
A current work-in-progress Java buildpack: https://github.com/projectriff/riff-buildpack
I've worked on Buildpacks fulltime twice now for Pivotal.
I've seen things you people wouldn't believe. Package systems on fire off the shoulder of Orion. I watched C-compilers glitter in the dark near the obscure Nokogiri-interacting-with-stray-environment-variables-if-you're-using-a-slightly-too-old-SQLite-and-the-symlink-is-broken-due-to-an-upstream-bug-that-wasn't-fixed-because-nobody-realised-this-would-happen gate.
But unlike the shell script, these moments will not be lost in time like rain. They will be shipped reliably, repeatably into production by two professional organisations prepared to assign more than a dozen engineers, watch hundreds of upstream dependencies, invest in millions of dollars of automation, test against thousands of distinct applications with tens of thousands of running containers serving billions of requests per day across multiple customers in multiple countries.
I invite you to maintain a nodejs buildpack yourself. Prove us all wrong.
I maintain cloudinit generators already that are about the same complexity and have seen things you can only dream of.
https://twitter.com/boringenormous/status/104757133137253990...
I don't have anything to prove though, since I'm not claiming I invented bootstrap scripts.