Engineering with Enclaves
onefootprint.comEnclaves are really useful, but I wish AWS would support an SDK in Go or another language. The current one is written in C: https://github.com/aws/aws-nitro-enclaves-sdk-c
There are some third party implementations available, but we'd worry too much about maintenance of those to use them. Something officially supported would really be helpful.
yes agree! we use Rust and ffi into this C library, but wish there was a native Rust SDK as well
My team and I are deploying a zero trust independently verifiable secret forwarding system on AWS Nitro Enclave, please ask me anything.
The idea is basic secret forwarding - you want to send a secret to many destinations, but find it too cumbersome to encrypt it with each destination’s public key, or you might not have all the public keys in hand.
To address this, we provide you with a code base running inside a Nitro Enclave. You have a KMS account, and you configure it to allow access only to 1) our code base hash AND 2) only when that signed code is running inside a certified Nitro Enclave.
The enclave bootstraps itself on first run by generating a key pair and encrypting the private key with a data key from KMS.
You can then send a secret to the enclave encrypted with its public key, and verifiably know exactly what’s going to happen. Only trusted code running inside the secure confines of the enclave would be able to decrypt and operate on the secret.
There’s lots of gotchas - if you make a server in the enclave remember to use a server that binds on the VSOCK AF and port; on the host run socat or a systemd socket unit to translate HTTP/TCP calls into the enclave’s VSOCK. Make sure you enrich all requests passed into the enclave with all the data and encrypted artefacts it needs to work. Make sure to send the encrypted artefacts out from the enclave and store them safely.
We do this for cinema movie distribution - filmmakers just send their movie’s encryption key to this system, and it sends it out to the public key of each projector in each screen in each theatre in the world. And big studios can verify that their billion dollar movies aren’t going anywhere other than their chosen rules.
How did you verify and validate that a AWS Nitro Enclave actually provides isolation and attestation guarantees?
Do they provide certifications or audits confirming conformance to multi-level security guarantees? Or do they provide you detailed specifications allowing you to evaluate that yourselves? Did you run red team exercises that resulted in no detected deficiencies as would be required by a multi-level security claim?
Trust in AWS and Nitro HyperVisor system is taken for granted here, if that’s what you mean. We don’t necessarily examine AWS’s code or hardware, we’ll just have to take their word for it.
Which is what we and our customers do already, when we run our workloads on it.
We know this is a problem, but this is where we’re ok with drawing the line. If you’ve seen Reflections on Trusting Trust you’ll know we have to draw the line somewhere.
Do they provide any third party certifications with teeth, legally binding guarantees, or contractual guarantees supporting the claims you are relying on?
Not right now, no. From what I've seen from our previous suppliers (HSMs from Thales) no one will take on liability (if that's what you mean). The trust instead relies on the FIPS-140 certification process - certified devices are sold as-is.
In the case of the cloud I haven't seen FIPS certification of any of the internal systems from cloud providers yet.
This need for certification needs to be balanced against usability - it's relatively easier to certify a hardware design that has no compute, or a physically isolated CPU and memory like a SEE Machine, but it's also much less useful.
We prefer general purpose machines that have the same strength barriers that we already know are being deployed to protect customers from each other, that are being probed and tested all day every day both by AWS internally and by bad actors trying to steal other people's data. The lack of known pending exploits, continuous patching by AWS and being battle tested on a massive scale is more important to us than a FIPS certification at this point. Although if AWS managed to get FIPS to certify the Nitro system it would be a big plus.
Thank you for the replies. It helps confirm my belief that AWS is just peddling the same security bullshit as everybody else and people keep falling for it.
Literally everybody has been promising general purpose machine isolation for decades and basically every one of them has failed. Actually designing such a system is a extraordinary claim demanding extraordinary evidence. Actual standards that can actually distinguish such a property such as the Common Criteria at EAL 6 and 7 require rigorous verification work such as formal proofs of correctness to actually positively assert such properties. It is ridiculous that people keep believing such claims without any guarantees, verifications, or audits when everything uncertified is so catastrophically bad at achieving isolation.
To quote Theo de Raadt:
“You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.
You've seen something on the shelf, and it has all sorts of pretty colours, and you've bought it.“
Nobody that knows what they are talking about is making claims like "system x does not have any security holes". It's all about threat models, probabilities, and ultimately tradeoffs.
Consider a scenario where you need to compute some a function of secret data, but unfortunately that function isn't of the shape that any CC- or FIPS-certified solution (HSM or otherwise) supports out of the box, and ordering one to be built against your specifications would eat through your budget for the next 15-20 years. Your choices roughly are:
- Use a HSM anyway, but only use it to encrypt the data (or its encryption key), and have your application servers decrypt, evaluate the function, return the result and delete the data from memory every time they need to? Then you're arguably just doing fancy encryption at rest.
- Use enclaves. Even though they are much less hardened than a CC certified and audited solution, let's say you assign some probability to the event that somebody compromises them and burns the corresponding 0-day on you. Would you assert that that probability is 1? If not, why not still do it?
And you can still store the actual data encryption keys in a HSM and any higher-level features that it does support! In fact, if you use your HSM in a pretty low-level way (i.e. with an interface like "decrypt data x using key k"), access to the HSM needs to be guarded carefully, and enclaves can be a candidate for that.
- Don't evaluate the function, or possibly don't even store the data in the first place. The only problem with enclaves, in my view, is if they would make people disregard this option due to the misunderstanding that "enclaves are perfectly secure". (This applies to any security technology, but the fancier/more magical a solution looks like or is marketed as, the higher the temptation.)
- Do none of the above, but still store the data and evaluate the function on your systems. This is arguably the norm today in most organizations.
Having seen the FIPS certification process happening for a popular brand of HSM (as an outside observer on the attack side), I’d put a fair bit of trust in these systems.
Very good isolation at all layers, formal verification, highly resistant to tampering, fuzzing, and so on. Every single part was locked down in a very thoughtful way.
There may be no legal liability with the purchase contracts, but these manufacturers certainly seem to approach the problem with the level of consideration that I’d hope for given the importance of the technology.
Except they were discussing AWS which does not have FIPS certification. In fact, it has no third party certification of any security value, they did no first party verification, there are no public binding guarantees, and no private guarantees. There is nothing but Amazon’s marketing literature and wishful thinking backing the claims of its security. Amazon would never stand by the claim that it encourages people mindlessly propagate, that AWS provides the multi-level security actually required for general purpose shared tenancy as they are 100% incapable of delivering it and they know it.
Apologies I misread the criticism and thought Amazon had FIPS, but that people weren't happy with that.
It sounds like Amazon should be getting that! I wonder if perhaps it's just ill-suited to service providers, and only really workable for hardware.
I don’t think there’s any specs to certify against for hypervisors. Think other AWS offerings are FIPS certified, like the CloudHSM, but Enclaves is a different system and use case.
There are no off the shelf specs you can certify against for hypervisors in particular. However, the certification of general purpose isolation in a shared tenancy environment is structurally similar to certification against the Separation Kernel Protection Profile (SKPP) [1] as done for INTEGRITY-178B [2].
Unfortunately certification against that standard directly is no longer possible in the normal course because as part of the security assurance requirement (SAR) AVA_VLA_EXP.4 it requires a NSA penetration test and evaluation that identifies no deficiences to cross-verify the proofs of correctness.
However, with the amount of money the US government is spending on AWS it should be trivial for them to get such a certification done if they were competent to actually do it. In addition, you could always just use a comparable standard that replaces AVA_VLA with the standard AVA_VAN which just validates against a “generic nation-state attacker” similar to what PikeOS [3] did. Given that PikeOS is made by a relatively small company, especially compared to something like Amazon or even the just the AWS division, it would again be easy for Amazon to demonstrate such capability if they were competent to do so.
Instead Amazon is 100% guaranteed to fail such a evaluation like how Microsoft failed their evaluations because, like Microsoft, nobody in their organization has ever designed, worked on, or probably even seen a actual high security system and they have no incentive to learn when they can just make up bullshit that people believe. Their organizations and management are structurally incapable of developing secure systems without a complete management and technology replacement.
[1] https://www.niap-ccevs.org/profile/Info.cfm?PPID=65&id=65
[2] https://www.commoncriteriaportal.org/files/epfiles/st_vid101...
[3] https://www.commoncriteriaportal.org/files/epfiles/1146a_pdf...
It seems like everything you have described could be done with TPM: creating a signing key for TLS mutual authentication (against the secret store) with policy that allows using that key only if system configuration did not change (PCR values stay consistent). Additionally TPMs allow remote attestation (via quotes and endorsement keys).
So I'm wondering what's the advantage of Nitro Enclaves? Better out of the box tooling?
I’ll check out the TPM, but our big draw with the Enclave system is being able to run general purpose code (Python / Go) inside a secure isolated environment. The earlier system we had was SEE machine on a HSM and required a special compiler to run - I haven’t seen the Nitro TPM just yet, but I doubt I’d be able run a container inside the boundaries of the TPM.
In case it's helpful: I'm maintaining a tool kit that makes it possible to run unmodified, general-purpose code inside Nitro enclaves: https://github.com/brave/nitriding
I'm not OP, but AWS introduced Nitro Enclaves about two years before Nitro TPM, which is a relatively recent addition. It's likely that TPM wasn't available in AWS at the time of OP's original development. Nitro TPM is not available at all on Graviton 1 or 2; only the newest Graviton 3.
That clarifies some matters - thanks!
Last I looked the NitroTPM product didn't have anything like an Endorsement Key certificate or any mechanism for authenticating a public Endorsement Key. Discrete TPM chips usually have an EKcert. GCP's vTPMs do not have an EKcert but Google provides a facility for looking up a guest's EKpub. It'd be nice if NitroTPM also had this.
Also, it's passingly strange to see PCRs mentioned with no mention of TPMs.
I'd appreciate pointers to adversarial attack models on nitro. I find papers leveraging nitro to build higher order processing models, I think it looks good, but where's the work to certify it in something analogous to FIPS? Nitro+FIPS searches suggest its hand-off to a card, not innately in the s/w system itself so its the usual key leakage issue: the real key might not leak, but ability to operate the key may in some circumstances be as bad as leaking it: if a Nitro instance can be subverted, it can securely sign to the end of time for bad purpose.
Confidential computing is really exciting in terms of software workload identity! As mentioned in the article, the AWS Nitro Enclaves PCR0 is a runtime measurement of the enclave image file, which contains all the code that is running - in other words, a representation of "something you are" rather than "something you have" (a token, a certificate, etc.).
Side note - I work on confidential computing at Anjuna, would love to talk more.
Nitro Enclaves seem super neat. For security focused people, it seems like a no-brainer. Great article!