RSA Private Exponent Generation according to FIPS 186-4 in openssl v1

Question

I guess this is more of a math problem in a cryptography context so I apologize beforehand if it is not the right place to ask. Basically I have to check whether a certain implementation of RSA key-pair generation adheres to FIPS 186-4. More specifically, Appendix B-3-1. FIPS 186-4 necessitates that $d$ (the private exponent) be created like so:

$d = (e^{-1})\bmod(\text{LCM}(p-1, \space q-1))$

The library in question(openssl v1.0.1) calculates $d$ like so:

$d = (e^{-1})\bmod((p-1)(q-1))$

I can't prove or disprove whether these two create the same set of answers for $d$.
The condition for generation of $p$ and $q$ is that $(p-1)$ and $(q-1)$ are both relatively prime to $e$ (the public exponent) so both formulas have answers.
Also since $p$ and $q$ are both prime, $(p-1)$ and $(q-1)$ will both be even numbers and from $a \times b=\text{GDC}(a, \space b) \times \text{LCM}(a, \space b)$ we know that $\text{GCD}(p-1, \space q-1) \geq 2$ so $\text{LCM}(p-1, \space q-1) \neq (p-1)(q-1)$.

My question is are they the same or different?

I would also appreciate if you could point me in the right direction math-wise so that I could potentially solve this myself.

P.S.: I understand for openssl v1, there is a FIPS module and also that openssl v3.0 will try to apply for a FIPS 140-2 certificate. I am unfortunately stuck with the version I mentioned and I cannot change that(it's not up to me).

Answer 1

FIPS 186-4's $d_1=e^{-1}\bmod m_1$ with $m_1=\operatorname{lcm}(p-1,q-1)$, and OpenSSL's $d_2=e^{-1}\bmod m_2$ with $m_2=(p-1)(q-1)$, are different with probability $>1/2$ for random choice of $p$ and $q$ and either fixed $e$ adding further constraints on $p$ and $q$ (as common), or $e$ chosen somewhat randomly after $p$ and $q$.

Justification: it holds $m_2=g\,m_1$ for integer $g=\gcd(p-1,q-1)$. That $g$ is an integer at least $2$ (and noticeably often a larger even integer). It follows $d_2\bmod m_1=d_1$. It's well-verified that $d_2$ is roughly uniformly distributed on the interval $[0,m_2)$ within the constraint of being coprime with $m_2$. Thus the modular reduction from $d_2$ to $d_1$ causes a change with probability next to $1-1/g$, which always is at least $1/2$. It's easy to make an example where that occurs. The bound $1/2$ for the probablity that $d_1\ne d_2$ could be improved (increased, a little over $2/3$ actually) by considering the distribution of $g$.

Such frequent discrepancy is not as bad as it sounds, because

• What's really needed for $d$ to work (when and if used as RSA private exponent, or to compute or check $d_p$ and $d_q$ ) is that $e\,d\equiv1\pmod{\operatorname{lcm}(p-1,q-1)}$ (assuming $p$ and $q$ are distinct primes). Both $d_1$ and $d_2$ match this, and conform to PKCS#1 which additionally requires $0<d<n$ (that follows from $d_1<m_1<m_2<n$ and $d_2<m_2<n$).
• In practice $d$ is seldom used, because private key operation is faster with the Chineese Remainder Theorem, which typically only uses $(n,e,p,q,d_p,d_q,q_\text{inv})$ or a subset of that, in which case the only possible issues with $d_1$ or $d_2$ is when it first checked, and that's bound to be detected by key import and a single use.
• Any FIPS 186-4 RSA key is accepted by any version of OpenSSL. I would not bet the house in the other direction, but then it's rare to import a key from OpenSSL into a FIPS 140 device. That might even be prohibited in FIPS mode, and a FIPS device (at least in non-FIPS mode) would be allowed to accept any mathematically valid $d$ including $d_2$, or ignore the given $d$.

Answer 2

To my understanding, the requirement in FIPS 186-4 to use for the calculation of the private key $d$ the least common multiple of $p-1$ and $q-1$ instead of the their product has the purpose to prevent attacks against small $d$ like Wiener's attack. The common believe among experts is that one is secure against any improvement of Wiener's attack, if the length of $d$ is at least half of the length of the modulus $m$.

As all (known) variants of attacks against small private key $d$ work depending on the size of the smallest possible $d$ (the one calculated using the least common multiple), NIST insists on using the least common multiple, so that one can check that the smallest possible $d$ is big enough.

As NIST requires that $p$ and $q$ have about the same size, you can verify the length requirement for $d$ also instead by simply checking that $d_p$, $d_p+p-1$, $d_q$ and $d_q+q-1$ are pairwise different. If side-channel attacks are an issue for you, this test can be protected much easier than the calculation of the greatest common divisor of $p-1$ and $q-1$, which became in the last years the target of several published attacks. Which $d$ you use in your calculations shouldn't matter much, as adding a multiple of $(p-1)(q-1)$ to the exponent is commonly used countermeasure against side-channel attacks.