This is the second of a series of two blog posts to talk about how to easily enable secure boot and sign firmware images for ESP32 series of chips.
In the first blog post, we talked about enabling secure boot V2 (SBV2) on ESP32 platforms in development, and released the developer tools to make firmware signing easy for secure boot enablement in development. In this post, we will talk about how to use the Thistle Web app to enable secure boot (V2) on ESP32 platforms in production, where the firmware signing keys are managed with the Google Cloud Key Management system (Cloud KMS).
Firmware Signing Key Management in Production
Recall from the previous blog post that for secure boot enablement, Thistle recommends to distinguish two scenarios: key management in development and in production.
If one would like to enable SBV2 on production devices, it's a good idea to perform firmware development and testing on SBV2-enabled development units. Ideally, production firmware should be a nearly faithful mirror of development firmware, where the two differ only in bits associated with public keys and signatures. Production and development firmware should be similar, so that after development testing is completed, a minimum level of additional testing would be needed on production firmware. Production and development firmware should be signed with different keys, because the security and operational requirements on them are different.
Development signing: Development firmware is typically flashed onto only a handful devices during earlier phases of hardware development. Signing key confidentiality in this case is usually not crucial, because development hardware is supposed to be physically more tightly controlled, and not not plugged into production systems. The development signing process should be easy to allow fast feedback loops.
Production signing: Production firmware runs on production devices, which usually ship in large volumes, and are used in the real world by end users. For this reason, production signing keys are typically highly confidential, and shall not be exposed to software or human operators. The production signing process in many cases should be easy to allow fast firmware releases.
The development tools Thistle released in the GitHub project thistletech/esp32-devenvs facilitate building and signing ESP32/ESP32-S2/ESP32-S3 images using file-based private keys, which satisfy development signing requirements. We learned from an earlier blog post that although storing a private key in a computer's filesystem is simple for development and testing, there are many reasons it may not meet production requirements. More specifically,
The private key file can be easily copied, and subsequently leaked, by an entity, e.g., careless or malicious personnel, a backup program, or malware running on the computer.
Even if the private key is password protected, the password can be brute-forced offline.
It isn't a straightforward task to securely back up a private key file. On the other hand, not backing up a private key file used for production signing can easily be a production continuity risk - imagine the consequences of a disk failure.
Leaking firmware signing keys can lead to severe impacts - remember the leaked MSI UEFI signing key incident not too long ago. It's not impossible, but usually difficult and costly to create and maintain a release process that can mitigate the above risks associated with private key management. Hence, a proper key management system is desired for secure boot signing.
ESP32 SBV2 Enablement with Thistle
Thistle provides a secure boot enablement Web service for production signing of ESP32* (we currently support ESP32 ECO3 onward, ESP32-S2 and ESP32-S3 series) firmware images, where the secure boot signing keys are created, used, and managed through Google Cloud KMS. This Thistle service has the following properties.
Signing key security. A customer's production signing key is created in the Cloud KMS, and is never exposed outside of the Cloud KMS. No entity, including Thistle employees, Thistle's users, and any Thistle software, has access to plaintext production signing keys, and thus cannot leak them, deliberately or by accident. In general, a signing key's lifecycle is managed in the Cloud KMS, allowing us to leverage Cloud KMS provided features, such as customer control, identity and access management (IAM), key access control and monitoring, and key backups.
Ease-of-use. Production signing key creation is transparent and simple. Production signing of firmware images is done in a few button clicks (or API calls).
Robustness. As long as locally signed firmware images (inputs to the Thistle service, produced using tools in thistletech/esp32-devenvs) work on development-fused ESP32* units, it's guaranteed that production signed firmware images (outputs of the Thistle service) will be readily flashable and work on production-fused units.
The following video demonstrates how to use the Thistle Web app to obtain production signed ESP32 firmware images from development images that were created using tools in thistletech/esp32-devenvs, and signed with non-security-critical, dummy keys.