freenode
AnalysisSecurity & Cryptography

When one CA's weak P-384 keys force everyone else to purge trust

Flawed eMudhra intermediates triggered root removals, Python toolchain freezes, and a sharper fight over who should own the CA bundle.

A certificate authority's private keys should be the one part of the TLS stack nobody ever recovers by guessing. In mid-October 2026 that assumption failed for eight eMudhra intermediates: their P-384 scalars were so weak that researchers could reconstruct them almost instantly, and the cascade that followed ran from Mozilla root-program advice through certifi, pip, uv, interpreter installers, and ad-hoc Fedora blacklists. The technical failure was local. The operational pain was ecosystem-wide, and it reopened an old argument about whether language runtimes should keep shipping their own copies of the world's roots.

Dennis Jackson of Mozilla told trust-store maintainers the facts in blunt terms. Eight intermediate private keys corresponding to certificates trusted by Mozilla, Microsoft, Apple, and Chrome programs had been compromised. Six were for TLS; two for code signing. eMudhra revoked on the 18th of October and Sectigo on the 19th. Clients that reliably check current revocation and fail closed could ride that out. Everyone else needed to act. Once Mozilla opened its tracking bug, the mechanism became public: the eight keys came from a April 2025 batch of twenty-four intermediates spanning RSA-4096 and P-384. All of the broken ones were the P-384 keys in the B and C series. Jackson's preliminary write-up was unsparing. "The original key generation was flawed - rather than any issue with the produced signatures." P-384 private keys are supposed to be random integers in the full group order. "The 8 impacted keys all used integers less than 256. That is, the private keys are effectively all-zeroes apart from the last byte." Recovery was trivial with baby-step giant-step or simple enumeration.

Community reconstruction of the ceremony filled in the human side. Fedora discussion and eMudhra's own statement pointed at a key wrap/unwrap step that emitted a second, weak set of keys beside the strong production material; the weak set was what landed in browser root stores. The mistake sat undetected for roughly eighteen months and through two independent audit cycles. One intermediate apparently saw production use. The rest did not, which did nothing to shrink the blast radius for clients that still trusted the issuing roots.

Mozilla's mitigation path was the usual layered one: revoke, restrict, and where necessary distrust intermediates or roots for clients that do not consume CRLs or OCSP in a fail-closed way. Downstream stores had to choose how aggressive to be. certifi's answer was a CRITICAL advisory and a 2026.10.19 bundle that dropped emSign roots entirely. Seth Larson warned Python users that failure to upgrade could enable monster-in-the-middle attacks against TLS clients, including traffic to python.org and pypi.org. The upgrade list was long for a CA incident: certifi itself, pip 26.2.2, uv 0.13.4, sigstore, conda, pixi, and expedited CPython releases that rebundled the fixed pip. macOS python.org installs needed explicit certifi refreshes. Container images had to be rebuilt with layer caches disabled so stale CA bundles would not linger. Virtualenvs everywhere needed a secure bootstrap of the new pip before they could even pull the fixed certifi.

That is where the second debate took over from the first. Valtteri Vuorikoski, after a day of chasing copies, argued the structural point: packages other than certifi should stop vendoring roots. "Users should install their distro's ca-certificates package or similar." If that is impossible, expose certifi as an optional dependency, but make the system trust store the default. "This sort of easter egg hiding to serve users with broken installations reduces the security of everyone else." Eli Schwartz noted that distro packaging already does better when it can: many Linux certifi packages are patched or rebuilt against a system-store bridge, and pip is unbundled to follow. Sam Bull observed that the Python ecosystem was already drifting toward truststore for exactly this reason, and that the incident would raise the priority of that move. alex_Gaynor, speaking for certifi's tracking of Mozilla, said the project would keep following Mozilla's root decisions; more removals could still arrive.

Collateral damage made pure root deletion uncomfortable. Stuart Henderson pointed out that distrusting "emSign ECC Root CA - G3" also hits InCommon intermediates used far outside eMudhra's customer base, and that the long-lived Comodo AAA Services root sits in the same graph because it cross-signs the compromised intermediates. That root had already lost its TLS trust bit in browser programs in 2025 over age policy, yet it still appeared in some stacks and still mattered for S/MIME. Fedora users who did not want to wait for a ca-certificates update posted manual blocklist recipes for both the emSign ECC root and the Comodo AAA root, accepting the breakage as the price of not being an open relay for a recovered CA key.

The deeper technical embarrassment is that the weakness class was both extreme and, in hindsight, scannable. Solar Designer asked why every CA root and intermediate public key is not continuously audited the way weak PGP RSA keys once were: "We need it automated, not something done occasionally when someone is in the mood." Hanno Böck replied that his badkeys project already scans CCADB material for known problems, but "the 'all the known weaknesses' part is tricky if you don't know what to look for." He had been told to watch for small ECDSA scalars and had not yet implemented the check. Peter Gutmann noted that Let's Encrypt has long applied Fermat factorization and similar filters to public keys it sees. None of that caught eight intermediates whose secrets fit in a single byte until a researcher recovered them after the fact.

What remains open is not whether the affected intermediates should be untrusted. They should. The open questions are distributional and architectural. How many other language ecosystems and appliance images still embed a Mozilla-derived bundle frozen at an arbitrary date? How quickly can pip, requests, botocore, and peers stop carrying private root lists once truststore and system stores are viable defaults on the platforms that matter? Will root programs and the CA community mandate continuous weak-key scanning that includes tiny ECC scalars, dual-ceremony artifacts, and other generation footguns, or will detection stay dependent on the next curious outsider? Until those answers settle, a single flawed HSM ceremony will keep turning into emergency releases, manual blocklists, and MITM risk for every client that cannot fail closed on revocation.