Nitric Is Terraform for Developers
medium.comMy opinion is that these tools for most companies are the perfect footguns.
How much time a developer will think about the right instance size, fine tuned IAM permissions, naming conventions or tags? I would wager a substantial amount that most will just copy-paste and get done with it, which eventually leads to the creation of a devops/sre/platform team to do the refactor and cleanup and we are back to step one.
Okay but what's the alternative? Seems like the real problem is not the tool, but the thing the tool operates on.
The alternative is making it possible for developers to only think about code, not permissions, or at least specify the permissions in terms of what you want to do, not what permissions you need. Think iOS, you write "I need fine grained location access" into the manifest, you don't configure the permission system to allow you to call the API.
Another poster touched on another important point: it's important for this to be changeable independent of the code. The reason for this is actually kind of subtle. Obviously, you don't wanna have to need to rebuild in order to regenerate permissions. But the real reason, imo, is that it should be easy to parse for a human, easy to locate for a human, and also easy to parse and adjust for a machine, that might determine a permission is no longer necessary, or who is trying to build a dependency graph in order to determine who to wake up during an incident. That means it should go into configuration that is versioned and deployed alongside the code, but not in the code.
If you make this hard to understand and change, people will just copy it, and the you're back to square one. It's gotta be the easiest thing to do the right thing, because at scale, people are gonna do the easiest thing.
I feel like I'm kinda going on at length about this, so instead I'm gonna leave you with a link to a blog I wrote about the same concepts, if you wanna read more. It's about Kubernetes network policies, but really the same concepts apply to all kinds of access.
https://otterize.com/blog/network-policies-are-not-the-right...
How you're describing iOS is similar to how nitric works. Developers indicate in code "I'm reading from this bucket", it's a request not an order, they're not actually configuring the permissions system. That request is collected into a graph of other requests (for resources, permissions, etc.) and passed via an API to a provider to fulfill.
If you want to change what "read" means you're free to do that in the provider without changing a single line of application code. But you also get the benefit on the Ops side of not needing to read the application code to try and figure out what permissions it needs to work, that part it generated so you can't miss anything.
If you want to output Terraform or config files or something else like you do today, to enable audits and keep it alongside the code, you can do that easily.
That's actually something nitric helps with, it provides an API for deployments, separating it from application code. Those other teams like devops/sre/etc. are free to customize that process how they see fit. They can enforce naming standards, tagging, sizing, etc. all through automation.
Ops type person here. We have these tools now, Terraform and cloud specific tools (Bicep/CloudFormation). What devs tend to miss is benefit to this approach puts up guardrails against common dev mistakes. Stuff like Nitric and Pulumi can end up with some really nasty deployment because there is multiple nesting, dead ends and other things that result in difficult to debug infrastructure deployment. Also JS unfortunately is not a common language for us DevOps types. We lean towards Python and Go because they are most useful for our line of work.
Nitric is a layer on top of Pulumi/Terraform. It also supports Python, Go and Dart. Other language support is also possible.
The Ops side of nitric (the providers) can be written in any language, but the out of the box providers are all built with Go, interacting with Pulumi or CDK for Terraform. You can just as easily write the deployment part of nitric with Terraform HCL to avoid the nasty issues you mentioned.
It'd better make it possible to change permissions apart from the code.
Interesting take, what would be the benefit here? If the application code writes to a resource and you change the permissions externally without updating the code you've essentially broken the application.
It’s step one except now you actually have a record of what’s deployed
Developers are inevitably going to grant themselves full admin access to everything.
const imagesBucket = nitric.bucket('images').allow('read', 'write');Yep, developers have been making those mistakes for quite a while already (even if just with CloudFormation, CDK or TFCDK). The problem isn't really "HCL is icky", it's the specialisation and knowledge required to make safe, secure and reliable systems.
In a way, the public cloud and their APIs are what zero trust is to classic corporate moat networks. You used to be safe 'inside' your code, and now you're not. On top of the security issues, you're now also getting real time billed for your mistakes.
This is of course where platform engineering (and the gradient of development and operations) comes in: give developers the self-serve resources they need so they can own their stuff. That also means safe-to-consume pre-made sets of cloud resources. That way you're not expecting everyone to bear the same additional cognitive load, while still having major benefits from the resources that are available.
Essentially the same thing the industry has been refining since the time of sending a deck of punchcards to the mainframe operator who will deliver the results of your batch compute once it's done. No cognitive load added, only the job at hand on your mind.
I think this is a great take. I'm on board on the HCL hate - it really reminds me of shoe-horning a templating solution into a complex, code level concern - but just removing YAML is not enough. My one caveat to this is a lot of the "platform engineering" products out there right now seem to want to abstract away all concerns behind a UI, which is just replacing YAML with UI problems. What we really need are open-source, cross-language sdks that allow self serving by developers using the tools they already know, but with additional configuration being able to color that settings by more specialized folks in their areas. For instance, an sdk that lets developers say ram is light, medium, heavy, etc. in whatever language they operate in; followed by a review of the sdk and some additional configuration layered on by the ops folks who monitor the entire company's spend and define what light/medium/heavy is. Too many of the platform engineering "solutions" seem to be about vendor lock-in.
That's essentially the idea with nitric, you ask for cloud resources in an abstracted request. How that request is fulfilled is determined by the provider used. What "read" access in nitric means can be up to the platform engineers.
To me, nitric seems to be a combination of TFCDK and the multi-provider abstraction (which limits you to the common denominator cloud resources -- generally not a good idea), which already exists, in multiple programming languages too. Even nitric itself already exists in the shape of pulumi.
Common denominator implies you're restricted to some subset of the cloud's features. I'm curious what restriction you see, since nothing about nitric limits combining nitric code with existing cloud libraries or extending providers to change the cloud interactions behind the abstractions.
You are indeed restricted to some subset, which is the only reason multi-cloud abstractions exist to begin with. The problem isn't punching through the abstractions, it's the pointlessness of the abstractions as a whole. Generic public cloud abstractions don't really work because the value is in the specificity. If you don't need that, don't use the public clouds since those are really expensive and impractical versions of resources for generic consumption.
If your project or BU is too small to make use of the specifics of a cloud, just don't use them. And if you are big enough to do so, you're also big enough to not be helped by some application specific IaC flavour of the day.
The reason for the abstractions is that the behavior of equivalent services from different clouds is functionally the same, but offered through subtly different APIs. If you think the value of each cloud is how unique their object storage (S3, Cloud Storage, etc.) is, you’ve missed the point entirely - the value is being able to store and retrieve data. The same is true for all the services nitric abstracts.
Saying they’re expensive and impractical because they’re not unique is obviously wrong. They’re valuable because they’re easy to use, safe, scalable, etc. while requiring almost no maintenance from the user. Storing data or sending messages, etc. aren’t uniquely valuable components worth spending time on for most applications. They’re basic building blocks, so why not minimize the effort and time to use them as much as possible?
There are unique and valuable services offered by cloud providers, those aren’t what nitric is focused on. I’d rather spend my time dealing with those unique, valuable tasks instead of wasting it on the basics.
Honestly? This is fine.
Just design your infrastructure around it. Segregate data using AWS accounts into granular "buckets".
One IAM table aka AWS subaccount, or in GCP another "project", for one app or small collection of apps that go together is the right approach in most cases. Yes.
Wait. So who is Terraform for?
Yes I had the same question. Maybe I haven't worked at a big enough company but I cannot understand this paragraph:
> The point here is that while Terraform and similiar technologies have existed for the past decade, providing Ops and Infrastructure teams with the tools they need to cleanly document their infrastructure, application development teams have had access to no such tools in this time. This is what the emergence of IfC is all about.
What does a company structure look like where "application development teams" are not allowed to access the Terraform code? It seems like Nitric is a wrapper around Terraform, which is fine and I can see how that could be useful, but the framing of the article doesn't make any sense.
The article isn't discussing replacing Terraform or suggesting Terraform isn't for developers. It's stating that Nitric brings the ability to create executable documentation for Applications, the same way that Terraform does for Infrastructure provisioning.
Well, that was as clear as mud.
I consider myself a software developer and I use it. We have tooling like Tenable to help ensure we’re using least privilege and adhering to security and compliance standards.
CDK is literally terraform for developers:
We really like CDKTF and Pulumi, they're two options nitric uses for deployments, but it's also a layer on top that helps decouple application code from the target cloud(s).
Ok that makes sense, it’s like an ORM for terraform for developers, cool
Pulumi > CDK's (TF or CF) > HCL
Nitric may work if you fit within its guardrails or golden path, but the stuff I work on never does.
I'm curious what you think wouldn't work. Nitric doesn't cover every use case, but in those cases you just continue to do what you would have already.
it reminds me of encore
i feel like simplyifing is great for early stage, but will be a tedious pain to scale out when you need to
It's very similar to encore, with the exception that the deployment engines are fully open source and extensible.
So, Pulumi?
It appears to be an abstraction on top of Pulumi/Terraform (it's clearer on their homepage, which refers to both: https://nitric.io/) that abstracts over the underlying cloud resources (which tend to be cloud-dependent) with higher-level concepts like "buckets" and "services".
Just wanted to take the time to appreciate this comment. It is a rare thing when someone takes the time to read beyond the headline, and decide to dig deeper to understand something. Thank you.
We already have structure + style + logic in the same file, now we want to add cloud provisioning and cloud configuration? At what point is it too much?
Software built with nitric ends up with considerably less cloud related code in the application. The bulk of it is split into the provider, enabling separation of concerns.
For example, instead of AWS client libraries, environment variables for ARNs, etc. existing in the code you instead have 1 line defining a resource using the SDK. That other code is separated into the provider, enabling testing in isolation and separation.
Frontend developers have been making great tools to render text for a long time. Why are backend developers so slow on adopting the same ideas?
> The point here is that while Terraform and similiar technologies have existed for the past decade, providing Ops and Infrastructure teams with the tools they need to cleanly document their infrastructure, application development teams have had access to no such tools in this time.
...Don't they have access to the same tools, though? And if they don't, it sounds more like an internal policy issue or operational inefficiency of whatever org they're in? Terraform is Terraform for developers.
Besides, I believe the promoted approach is fundamentally misguided.
Application code has no business getting involved in the infrastructure it's running on. They layers of abstraction should be separated.
It's fine if your team manages their own Dockerfiles, k8s schemas, helm charts, terraform/CF templates or whathaveyou. Cohost them in the same git repo even, if you must. Use all the platform-specific APIs and integrations you want.
The application itself should be competetely agnostic to all of that, though, and has no business interacting with any of those APIs. You use common interfaces like environment variables, filesystem, and CLI flags to communicate between upper and lower layers. You do not have your services go and read k8s secrets over API, interact with the container runtime, or make direct API calls to ECS[0], for example. You keep those layers abstracted away from application code. That has nothing to do with the "who".
On a language level, there are _very_ good reasons to prefer a declarative DSL over any turing-complete general-purpose language. Not saying you should use HCL specifically but scripting your infrastructure provisioning in JS doesn't seem like a step forward compared to ye olde bash scripts...
[0]: I'm sure you have a valid counter-example. The point still stands in general.
The application is agnostic to that with nitric, more so than usual. It could be worth reading the docs on how it works, I don't think you got it.
I did browse the docs and repos[0]. I'm saying that "IfC" as formulated here[1] is an anti-pattern and what I refer to as misguided.
> IaC tools like Pulumi, Terraform, AWS CDK, Ansible and others bring repeatability to infrastructure by giving you scripts that you can use across different environments or projects to provision infrastructure. This code/config is in addition to your application code, typically with a tight brittle coupling between the two.
If there is a tight and brittle coupling, this is not an issue stemming from either the IaC approach or any of those tools. It's an issue of poor design.
What am I missing?
[0]: Props on great examples repos, fwiw: https://github.com/nitrictech/examples https://github.com/tjholm/multiboy
I'm curious what designs you use to avoid the issues. For example, if your code needs to access a resource (e.g. making a call to send an event to a cloudwatch event bus or SNS topic on AWS), how do you deal with things like:
- Consuming AWS client libraries (or APIs) in your application code
- Avoiding writing cloud specific mocks/etc. for testing that same code
- Avoiding env vars with resource names/ARN/etc. to use with that code, that then need to duplicated in the Terraform/other IaC without typos etc.
- The code not knowing whether it has the permissions needed to make those calls, so it's can't be guaranteed to be correct before deploying and testing in a live environment
- No longer needing that topic in future, but it lingers in the IaC because the two are unaware of each other
To me those are all examples of the application code "getting involved in the infrastructure it's running on", which I agree it has no business doing.
Nitric deals with these things by separating that code into another module that's sole responsibility is dealing with the cloud, exposed by a common interface for any cloud/environment.
What other designs/tools do you recommend?
This is an aside, but I find it remarkable when I find anyone still writing on Medium in 2024.
So its kinda like cdk(tf)
Or like Pulumi?
Basically, Pulumi
Not quite, actually more like Pulumi's old cloud framework https://www.pulumi.com/docs/clouds/aws/cloudfx/
Resources are documented using abstract cloud concepts in code, and then provisioned by a deployment engine. The nitric deployment engines are actually build on IaC such as Pulumi/CDKTF.