Vault 0.4 released – a tool for managing secrets
hashicorp.comCan somebody tell me how Hashicorp makes money? They seem to employ people, and they have a page that sends traffic off to technology and solutions partners, but where do they make their own money?
We use Vagrant at work and I'm considering whether and how we could use more of their tooling. But I always want to know about the business model behind the tools I recommend before I recommend them.
Anyone?
As the founder of HashiCorp, I feel qualified to answer this.
The adjacent comment about Atlas is correct. Atlas itself is admittingly poorly marketed (something we're working on right this very moment) right now. The best way to describe how we make money is: we target the enterprise user and give them the features, integrations, and support they need in their regulated, legacy-encumbered, etc. environments. Atlas in particular is focused on giving enterprises a flexible application delivery pipeline that is opinionated in a certain way, but flexible enough to support their diverse environments (all at once: containers, VMs, physical machines. Windows, Mac, Linux. Etc.)
Due to our adoption we have the rare benefit of primarily letting open source adoption happen organically, and having large companies approach us as they get more serious about our software.
Vault, in particular, is a good example of this. Vault suffers from an issue where most impressive people who use Vault can't actually publicly say they use Vault. So we have a hard time talking about how widely adopted it is. The best I can give you is from what I said at our HashiConf keynote: "If you traded stocks, used a credit card, or did anything involving a bank, then you've interacted with Vault-secured data." As specific as I can be as to how widely spread Vault is right now.
Anyways, a digression from the original question: enterprises have interesting needs they're willing to pay for. We address those needs in enterprise products (Atlas, others that aren't public yet) and support. We don't have any SaaS-like "enter your credit card and pay us" (other than the Vagrant VMware plugin which predates all of this), and instead do deals with larger companies in a way that most of HN would likely perceive as old school in some sense. :)
Note when I say "enterprise" I'm not trying to exclude anyone. If you're a relatively small company (I didn't click your name to find out), then still feel free to email us any of your concerns and we should be able to either help you ourselves or route you in the right direction.
> Can somebody tell me how Hashicorp makes money?
> As the founder of HashiCorp, I feel qualified to answer this.
And that's why I stick with HN. Really good citations :-)
On a slightly more serious note, how do you build your sales pipeline? Is it really as lusciously simple as build great OSS stuff, promote the hell out of it, get a call from Dev lead at Mega Corp? do you have traditional enterprise sales people?
Mega Corp has engineers who know what's up. When they're ready, Mitchell gets a call.
This works in specialties where an enterprise solution is very hard and only a very few are tackling it well.
Well, OK then.> As the founder of HashiCorp, I feel qualified to answer this.Thanks for taking the time to write down such a thoughtful reply. We are a relatively small company, but a lot of the products we make are enterprise-focused. I'll be sure to check out Atlas here in the near future.
I think they make the bulk of their money* off of Atlas, which is a subscription service that ties together and automates a lot of their free products. They also have a paid add-on to Vagrant for VMware support.
* I would also bet they do consulting & support for companies that are using their products, and/or other automation advice. Whether that accounts for more or less revenue than Atlas, I have no idea :)
Someone correct me if I'm wrong but Vault on its own (without an agent) would be quite difficult to use in a simple web application setup, no?
For example let's say I store an API token in Vault and want to use that in my Node.js application.
That means I can't do "var api_token = MY_API_TOKEN;" because the secret needs to come from vault and get refreshed, etc...
I'd imagine you will need some agent to manage the secret lease/expiry and for that to reload your entire application to ensure you don't end up with old secrets hanging around in the memory.
This topic is not addressed anywhere in the Vault documentation, I looked everywhere I could.
We plan on augmenting the documentation with a user-friendly "guides" section. A big rework of a lot of the docs is actually very high on the near term TODO list, so I'm sorry for the current state of it.
As for how you can more easily use it, we recommend [the now weirdly named] consul-template or envconsul. The former (https://github.com/hashicorp/consul-template) will put secrets automatically into files and watch for changes, update the file, and refresh the process. If you put the files onto a non-swappable ramdisk, then it is reasonably secure (relative to most things, less secure than deeply integrating with Vault).
Envconsul, on the other hand, injects secrets as environment variables to a process. This is also reasonably secure but users have to be aware of the various ways that env vars can be read out of process (/proc for example).
These are the two easiest ways to get started that allow Vault to be used with brownfield software. If you greenfield something, integrating Vault 1st class is the way to go for the most security and is what we're seeing bigger users go for.
In the past I've thought about using the an ec2-meta-data style endpoint for secrets that would accessible by an agent with some sort of key. that way the application always dynamically requests the secret as oppose to setting it statically at launch.
It took me a while to run across envconsul and consul-template after discovering Vault also. But the simplicity of the tools together is much easier than any other configuration software option I've tried. So much more straightforward than an ops tool like puppet or chef (though those have their place).
There sure are a lot of these systems these days. While I'm always happy to see innovation in this area, I'm personally beginning to get confused as to why I might prefer to use (and probably then contribute to) one of these projects over another. After all, there's this, and Lyft's Confidant [1], and Square's Keywhiz [2], and plenty more that don't come to mind right now. They all have nice documentation about what they do, but none of them sufficiently explain to me what architectural differences they have, their pros and cons. I think it would be great to see that added to these products' pages at best, or at least some guy write a blog post about it.
[1] https://lyft.github.io/confidant/ [2] https://square.github.io/keywhiz/
> none of them sufficiently explain to me what architectural differences they have, their pros and cons.
Have you looked at https://vaultproject.io/intro/vs/index.html ?
This page does not adequately address many of the practical questions that orgs with secrets should be asking.
Examples:
* Does it integrate with my existing infrastructure security policies? * Does it introduce a new set of principals for me to track and manage? * Are the configurations stored in a cleartext manner suitable for storing in version control? * Will the audit logs integrate with my existing audit log tooling? * Does it help or hinder continuous deployment? In other words, will my code be littered by conditionals to support non-prod jobs? * Can my devops team run this or does it require additional staffing? * How many layers is this adding between me and the security primitives offered by my platform (Windows AD, AWS IAM, etc) * Is it serverless, or does it require me to run additional servers? * How do I ensure continuity if the software is no longer actively developed? * Can it deploy secrets to dev and test environments in the same way? * What's the revocation story?
The answers to most of these questions for most of the tools that have made it to HN recently are not promising. For AWS the best choice seems to be using KMS directly and integrating your app with the declarative configuration tools AWS offers (IAM, CF, MFA).
Outside of AWS, Vault is the only one that I've seen which handles real-world end-to-end in a responsible way. For example, Vault is the only one that doesn't focus strictly on storing static values: Vault can integrate with services such as Postgres and SSH to dynamically provision time-limited credentials (https://vaultproject.io/docs/secrets/postgresql/index.html).
+1 to what whisk3rs said, as that's how I feel, as well.
For example using Vault v. Keywhiz [1] since I made that comparison earlier, this page feels like it's written to say "Vault does X. Keywhiz does too." That's not really what I'm looking for; I can diff the two projects' features in that way as well, but I want to know why it is that Vault is better, i.e., why is it that it exists, what sets it apart from other solutions I can use? Why should I invest my time and port my infrastructure into this project? Should I expect it to be around or abandoned in a year if there is no adoption?
I probably should have clarified this in my original post; sorry for the confusion.
I'm one of the Keywhiz developers, and we've been considering writing such a page, though I've been hesitant to do so because I'm not totally familiar with the alternatives: I have no operational experience with Vault or alternatives, and haven't put a ton of thought into this comparison.
From my limited perspective: Keywhiz is a system to take secrets, stored in a database, decrypt them with an HSM backed key, and expose them as files on a server. Because the secrets are exposed as a fuse filesystem, they require zero special code in your applications. Most other secrets stores just provide you with an API that requires custom integration work. You could probably even modify keywhiz-fs to talk to a Vault server.
On the other hand, Keywhiz doesn't know anything about the secrets inside it, so there's not much out of the box for secrets revocation, and other operations like that. There's also minimal secrets generation: We just provide an API to handle that. The biggest sources of secrets for us are service-to-service TLS certs, generated from our deploy system, which just calls the automation API to put the certs it gets from our internal CA. Keywhiz has a plugin system for generating secrets, but it's quite simple compared to what Vault has.
Vault also has some operational decisions that seem scary to me. They have a scheme for "unsealing" a vault with a set of secrets an operator has to input. This seems quite likely to cause an outage, while defending against an attacker you've noticed but who hasn't managed to steal the secrets out of your unsealed vault. While operating, your vault is unsealed, so an attacker who gets root has your secrets in both Keywhiz and Vault, but a service restart causes an outage. If you could somehow (via physical access, perhaps) boot a malicious OS onto a Keywhiz server, and your HSM doesn't defend against that, you could read secrets. For Vault, you also need to steal the keys to unseal. But those are accessible to humans, so that's probably easy.
As for lifetime, we don't expect Keywhiz to go anywhere. It's a crucial part of running Square. We've been using it for years now (though only open-sourced it earlier this year), and will probably continue to exist as long as the company does.
I'm happy to answer any other questions, too.
I don't want to make a big comparison right now, so just addressing your specific concern about Vault:
Unsealing doesn't cause an outage since you run them in HA-mode (side by side, leader elected).
As you said, if you gain root, both in Keywhiz and Vault, you can extract secrets at will. I can't comment to Keywhiz's capability (I'm sure some exists), but Vault will send audit logs to multiple backends so at least you can see it happening. The root token in theory can disable audit backends, but that action itself is also audited before it is disabled. That scenario in particular (root token compromise) is the worst case scenario. Ideally you throw that root token away into a N-key area as soon as possible. This feature is coming to Vault soon (expanding the shamir to arbitrary paths/secrets in Vault to support N-key operations).
Vault itself is used in places with very very high requirements of uptime (one user in particular can measure their overall downtime in seconds in the last quarter century). So we've had to go through some rigorous due diligence in such places to make sure it can withstand that. So, I assure you operationally that it won't cause an outage if it is properly configured in HA-mode. :)
I hate to nit, but do the applications really not need special code? Most applications require secrets in one of their own config files, or through some other mechanism that is not some file in some format. If you need to write the code to get this secret off the file system and to where the app can use it.. What's the difference between using a simple client to fetch from vault vs fetch from the file system?
I don't currently use either vault or keywhiz(we rolled our own solution using KMS and dynamodb) so I'm not really knocking either method. The current state-of-the-art with configuration management and secret distribution(particularly for container stuff) still requires a lot of customization code specific to each wrapped application, unless you wrote it all yourself to suite.
At the risk of sounding like a fanboy though, I will say that my own solution and others that have come out feel very itch-scratchy compared to vault. From what I have read about vault it has a much more thought out and productized vibe. Lots of cool features beyond delivering an encrypted blob from point A to B. I get this impression from a lot of Hashicorp components in general, and feel it's a shame larger open source projects like Kubernetes aren't integrating them while letting scaling to 200 hosts be a blocker to their next release... But I digress.
Well, our #1 use case is TLS certs, and basically every app supports reading them from a file.
For config files with secrets, there's often a way to include another file, or sometimes we just drop the whole config file into Keywhiz.
Here's an example from the Keywhiz codebase, of a file that's loaded from disk, but in prod could be swapped for a properly secured secret https://github.com/square/keywhiz/blob/master/server/src/mai...
In development, you don't need to run Keywhiz, so you can just write a file on disk, so that's a nice advantage: Less things to depend on.
There's probably some strategy tax at work here since Hashicorp is trying to build a self-contained ecosystem of devops/microservices tools.
Vault fits in with my existing stack very well. Though a simple tool it solves so many of the common security issues with modern architecture (secrets distribution, multi-factor auth, compatible with multiple public/private networks simultaneously, has an API and GUI...). It's clear the authors have run into the problem time and time again.
And yesterday I got it to build and run on SmartOS too, for extra security and scalability. Thanks to Hashicorp team for their work, and their commitment to open-source.