X509-limbo: testvectors and tooling for evaluating X.509 path validation

NIST did something like this many years ago, the NIST PKI test suite. Turned out that several of their gold-standard reference tests were wrong, and a number of others were both right and wrong depending on how you interpreted the standards, and which part of the conflicting and contradictory sections of the standard you took as definitive.

The specific problem with the limbo tests is that they're not a set of test cases like the NIST work was but a gigantic pile of rubber bands and duct tape mixed up in C, Python, Rust, and Go that you have to plumb into each new implementation that you want to test. The NIST suite OTOH was a pile of cert chains and an indication for each one whether it should validate or not, and if so why not. Want to test the Windows cert handling? Click on the .p7s. Want to see what OpenSSL does? Feed it onto 'openssl pkcs7 -verify ...'. I don't know why they've chosen to make it so difficult to apply.