Is RSA of a random nonce with no padding safe?

Question

Consider the following protocol: Bob has a private RSA key $B_{priv}$, and Alice knows the public key $B_{pub}$. Alice wants to send confidential messages to Bob (no integrity intended). To send a message, Alice randomly generates a single-use symmetric key (say, an AES key) $K$, encrypts the message with this symmetric key, and sends this encrypted message alongside with $K$ encrypted with RSA with Bob's public key.

Suppose we use no padding for $\textrm{RSA}_{B_{pub}}(K)$ (in other words, to pad with zero bytes). The obvious (to me) problems with a lack of padding are that an attacker can guess the plaintext (not possible here, it's random) and that encrypting the same plaintext twice gives the same result (again, not happening here). What are the known weaknesses of this scheme (or is it safe)?

Answer 1

If you pad a 128-bit value $K$ with zeroes to the left, and interpret the value numerically with big-endian convention (as happens in PKCS#1 in general), you end up with a value which is no greater than $2^{128}$. If the public exponent is $e = 3$, then the "encrypted value" is no greater than $2^{768}$, and no actual modular reduction takes place. Consequently, a simple cube root extraction on plain integers reveals $K$.

So, that can be quite unsafe.

What is safe is the following:

• you generate a random integer $x$, between $0$ and $n-1$ ($n$ is the RSA modulus);
• you encrypt $x$ by simply raising it to the power $e$ modulo $n$;
• the encryption key you will use with AES is $h(x)$ for some hash function $h$.

It is still relatively limiting, in that this is safe as long as you do one encryption only of $x$. If you want to "send" your symmetric key $K$ to several recipients (a common model, e.g. for email encryption), then the scheme above does not work, and, more generally, you need some per-encryption randomness inserted (as PKCS#1 does for encryption).

Answer 2

Thomas Pornin's answer is almost (but not quite) Victor Shoup's RSA-KEM-DEM construction and fails to recognise that authenticating a message is required for the security proofs that are necessary due to the lack of padding.

Pick an $r \in \{0, 1, 2, \dots, n-1\}$, compute and send $r^e \bmod n$, and derive the key $k = \text{kdf}(r \mathbin\| (r^e \bmod n))$.

Notice that $k$ is derived from both $r$ and $r^e \bmod n$ instead of just $r$; this eliminates the RSA-side of malleability attacks and significantly reduces the control an adverarial client has over the resulting $k$.

Recall that DEM only requires a single-message authenticated encryption. For multi-message authenticated encryption we need more proofs; however, this is conjectured to be safe when using an AE[AD] such as AES-CTR-PMAC (or better an MRAE such as AES-CTR-PMAC-SIV).

With the above tweak we've gone from the proven single-message RSA-KEM-DEM to the unproven (conjectured safe) multi-message RSA-KEM-AE.