Launch HN: Infracost (YC W21) – open-source cloud cost estimator
Hi, we’re Ali, Hassan and Alistair and we co-founded Infracost (https://www.infracost.io). Infracost is an open-source cloud cost estimator for your pull requests. When you change your infrastructure code (Terraform), Infracost posts a comment in the pull request, which tells you the impact of this change to your cloud bill, e.g. “this will increase your bill by 25% next month”.
Existing cloud cost management products focus on post-bill analysis and target finance and management teams via charting dashboards. We built one of these back in 2013. They are all missing an important piece - the people who are responsible for purchasing cloud resources are not shown costs upfront, so they don’t know how much the resources will cost before launching them. We want to make cloud costs simpler to understand for developers and DevOps so they can make better decisions, which we believe will lead to more cost-efficient systems.
In 2011 Ali and Hassan started a cloud cost forecasting company based on Ali’s PhD research. They applied to YC and got through to the interview round. RightScale acquired them in 2012. I read about their YC interview experience on HN, reached out and ended up joining them. We went on to form the team that built RightScale’s cloud cost management product (now called Flexera Optima).
In our most recent startup (which failed) we were launching cloud stacks for users on-demand and we wanted a way to work out the cost of each. We hacked together something by building a GraphQL-based cloud pricing API and a CLI that parsed our Terraform code and output a cost breakdown.
We released the code on GitHub as Infracost and discovered that others had similar problems. We got requests to support more cloud services and integrate it into pull requests. At the moment, Infracost supports Terraform for AWS and Google Cloud (we’re adding new resources every week). It can be integrated into GitHub, GitLab, CircleCI, Bitbucket and Atlantis, or can be used anywhere through the CLI. In the future we plan to add support for more cloud vendors and infrastructure-as-code tools (Azure, CloudFormation, Pulumi, etc).
We now spend a lot of our days trawling through the cloud pricing pages working out how pricing works for different cloud services. We’re grateful for the contributors who have helped us with this. AWS currently has over 2 million price points and this is constantly increasing. Users are requesting better support for usage-based services like data transfer, S3 and Lambda. Currently we allow for usage estimates to be passed into the tool, and are looking at other methods, i.e. based on last month’s actual usage. We’ve also learned, the hard way, the importance of UX in CLI and workflow tools.
So far we are seeing a few use-cases for Infracost. Some enterprise users have integrated it into their “self-service” cloud catalog to set cost expectations before provisioning. Other users have integrated it into their CI pipeline as a safety net to catch unexpected costs. And some users are running it at design time to compare options and model usage.
We’ve talked to Sid Sijbrandij (CEO of GitLab), and Ian Tien (CEO of Mattermost) about when and how to monetize. Currently we are thinking about a buyer-based open core approach, in which the individual contributor edition will always be free, and enterprise paid features will include multi-team support, management reports and private cloud support.
We’d really appreciate it if you try it out and give us feedback. You can check out the repo at https://github.com/infracost/infracost. We’d love your thoughts on our approach, and anything that has worked, or hasn’t worked for you when it comes to managing cloud costs. I love the idea but one of the challenges I’ve found at quite a few companies is a difficulty across the organisation in understanding the true cost of infrastructure: I’ve experienced situations in which putting the $ upfront causes people to focus on the $ amount. Do you have a guide for how this information should be used most effectively? I’m thinking of co-workers who would request a change that reduces the monthly cost by $10 but the time it takes to go through the code review and make the change and test it... by that point any $ cost saving has been spent on the time it spent in code review. My instinct is that perhaps rather than reporting the $ or % change, instead projects could have alarms / limits on cost increases, but I’m not so sure if that is really tackling the problem. You're right. It has to be a balance between the cost of the infrastructure and the developer's time. One feature we have for this is to only show comments in PRs if the cost increase is above a certain threshold. Another idea we have is to allow developers to set alerts based on their actual project/IaC concepts instead of configuring alerts based on tags and services. Do you think this would help? I wish you would add support for Azure sooner and also VMWare VMC cloud. I literally have to deliver cost estimate comparison of deploying our micro services to AKS vs VMC cluster. I have a tedious excel spreadsheet open that I am currently plugging in numbers to come up with estimates and comparison. edit : just adding that this is the suckiest part of my job and I fully expect to be grilled by management for every little cell in the spreadsheet Azure support is in the works - it's also our most upvoted issue (https://github.com/infracost/infracost/issues/64). Are you using VMC on Azure as well? >Are you using VMC on Azure as well? No. we are using VMC cloud , which I believe is its own offering. Do you know if they have public pricing available? Or are you using negotiated rates? If so, we're looking to add support for custom price books in the future. we are using negotiated rates. Also, we have a huge infrastructure group that adds a markup and conditions to the published pricing. For example, my line of business pays a different rate for using a Azure VM than the price published by Azure for the VM. Being able to plugin "lob" price as opposed the prices posted by the vendor is very useful to us. Great, thanks for the details - supporting these negotiated rates and any custom price books is definitely on our roadmap. once you guys have it workable for Azure, email me at manish dot sharan at tee dee dot com. I will get you in front of the architecture group -- cost projections is a contant pain in the ass for us. (not tee dee literally .. ) I will delete this post later. Thanks got it, will contact you when it's live. This is a great project but the fact that cloud pricing has led down this road is insane. The only thing more complicated than cloud billing at this point is medical billing. Yeah! When we started in the cloud cost space about 9 years ago I remember AWS had about 10,000 unique price points... now it's over 2 million! The level of services/options available now is incredible, but that just means the pricing complexity keeps increasing. Off-topic, but it leads me to think what an open crowd-sourced data for medical billing would be, that would estimate the cost of procedures/treatment plans over time... the crowd can plug in their costs (EOBs) they receive along with their insurance information, and an open source engine would crunch the numbers to provide these estimates? that's a cool idea, who would benefit? consumers I guess? They'd be able to compare or even just understand what they're about to pay for (either direct or via insurance). How is this different from Terraform Cloud's cost estimation please? https://www.terraform.io/docs/cloud/cost-estimation/index.ht... We are free and open source and support more AWS and Google services (https://www.infracost.io/docs/supported_resources). We also fit in with existing toolsets - we think it’s important that users don’t need to change their workflow, so we integrate with different ways of running Terraform (Terragrunt, Terraform Cloud) different source control systems (GitHub, GitLab, etc) and CI/CD systems (Atlantis, CircleCI). We also have additional features, like the ability to specify resource usage (for data transfer and Lambda, etc) and generate HTML and JSON reports so it can be integrated into your own systems. How does this compare to the FOSS aws cost estimator for TF that's already available? We cover 70+ resources across AWS and GCP (the tool you mentioned only covers 12 AWS resources) and we support a number of integrations including GitHub Actions and Atlantis (https://www.infracost.io/docs/integrations/) so you can see diffs in pull requests. We're seeing a lot of users who are using multiple tools for deploying their services, for example Terraform + Kubernetes or CloudFormation + Serverless. In the future we want to work across multiple tools so we can support these use-cases as well. I'm hoping that Infracost doesn't need the entire state file shipped to an outside service/server, like the one mentioned here does. State files should be considered secret.... They have a jq script[0] that removes everything but the necessary data for the cost estimation. [0] https://github.com/antonbabenko/terraform-cost-estimation/bl... Agree, state files can contain sensitive information. We parse the Terraform client side and only send the details to the API that are needed to determine the price, i.e. the instance type, region, etc. I'm sure I'm not the only one who's accidentally incurred a $1K+ charge on a personal AWS account because of confusing pricing docs. This is awesome - looking forward to using this ASAP Thanks! Yeah we've been in the same situation. Looking forward to your feedback. > We’d really appreciate it if you try it out and give us feedback. The landing page is gorgeous. Re the service: IDK, on the one side it's amazing, on the other hand, key to significant cost savings is to use Google and AWS only for must-have services, otherwise avoid them as much as possible and deploy the rest on self-managed k8s-clusters on vps farms. Latter might not work for huge deployments like Fortnite's server park but creates a very competitive performance-to-cost-ratio for small to medium deployments.[1] Not sure how this would fit in your design but just my two cents. Also, once you fully locked-in yourself into a cloud provider real cost-savings get harder and harder. If your tool is just about forecasting for finance and controlling then it's sufficient. [1] If your business model is so good (like Fortnite's) you shouldn't obsess about cloud costs, going all-in with AWS and Google might be the better choice. Thanks, running on self managed clusters or bare metal can save costs in some circumstances, but comes with trade-offs and overheads as well. In our previous company we were managing Kubernetes on bare-metal and what we found was every team started using more and more resources until we ended up with a sprawl of over-provisioned and under-utilized services. We want to support this use case as well by allowing price books to be set up for custom environments so each developer can see a "cost" for these cases as well. > can save costs in some circumstances [...] and what we found was every team started using more and more resources until we ended up with a sprawl of over-provisioned and under-utilized services This can happen but already for medium deployments the costs are so much lower that you can hire a bunch of devops managing a proper centralized RBAC-based k8s cluster which isn't over-provisioned nor under-utilized. And even if, it should be still significantly cheaper than anything from Google and AWS. But again, if a company runs a strong-margin business model cloud cost optimizations are the wrong thing to think about. Whatever, your solution is definitely a big step in the right direction. Congratulations on the launch! Keeping track of our AWS costs and attributing those costs to each team was a huge pain at my previous job, a tool like this would have come in real handy then. Thanks! Yeah we think that making the costs visible to engineers will help remove the pain here. We're seeing some big companies like Netflix, Slack and Segment implement dev-focussed approaches to cost-management and think this will become more of a standard in the future. This is cool! We accidentally spun up a Private CA as part of ACM. It was a $400/month ulcer causing click. hah! I remember reviewing the PR for that resource (https://github.com/infracost/infracost/pull/377). I was surprised it cost that much This makes a lot of sense. I've been telling people building something in the cloud is like trying to get healthcare: You can't make good decisions up front because it's impossible to figure what something will cost and somebody else ends up paying the bill anyway. And getting cloud-bill shock is like going to emergency department :D Seems like a very useful tool, and I really dig the look of your website (except the missing tab focus effects). Congrats on the launch! Haha thanks, great point - will fix! this is very cool! pricing will be key here - you need to figure out some per-repo, per-pull request pricing. Yea, we will have to strike a balance so we don't contribute to the cost complexity too! haha We talked about if we should approach it with user-based pricing, or usage-based pricing. Do you think usage based pricing would be preferred here? Or should we keep it in a way that users can run it as many times as they like? Our current thinking is buyer-based open core with per user pricing, maybe like SYNK. per-user is tricky. I can have a super duper complex terraform repo with just 1 user and a 10 liner repo with 5 people. you should do a "build minutes" kind of pricing. or the number of times i triggered a re-run. From your perspective, it can only be better than per-user, since the number of commits will be minimally proportional to the number of users. Good point. I wonder as the world goes more towards serveless, if users will slowly become used to per call (or maybe per comment, or as you say per re-run) pricing its already normal right ? Github Pipelines or CI/CD bill this way.
Actually the best example of this abstraction was Heroku with its "dynos". Yea! Very true. and exactly as you say, CI/CD is going that way as well and we fit in there, so it would make sense for our pricing to fit that. Really nice design of the home page! Do you folks use a template for that or you built it in-house? Thanks! The site design was custom done by a friend of ours. This is so smart! Love how this can even become a selling point for switching to Iaac setups! Thanks! Yeah we're big proponents of IaC. Excited to try it out :) Thanks, looking forward to your feedback! Fantastic. We are planning on doing a POC in our infra projects. Thanks, feel free to reach out if we can help with anything. Awesome project. We started looking into this area at https://www.brainboard.co and was happy to found that infracost already started the journey. Keep going! That's cool, I had seen https://cloudcraft.co but BrainBoard seems to be taking things to the next level. We're seeing the need for cloud price and cost information to be available in other products and are happy to explore integration opportunities. My email is ali@infracost.io Great job. This is a wonderful project!
$ terraform state pull | curl -s -X POST -H "Content-Type: application/json" -d @- https://cost.modules.tf/