Have any Hybrid Encryption Schemes been defined for Quantum Resistant algorithms?

Question

Now that several quantum-resistant asymmetric algorithms are in the process of being standardized (or indeed are already standardized), and Google has integrated the recently standardized ML-KEM algorithm into their BoringSSL library (which I understand is used by Chrome desktop) - have any Hybrid Encryption Schemes been formally (or informally) defined which implement them?

I've been experimenting with ML-KEM and I understand it as follows...

1. Alice generates a ML-KEM key pair, resulting in aliceSecretKey and alicePublicKey. And makes alicePublicKey available to anyone who wants it.
2. Bob wants to send a private message to Alice, so he starts be obtaining alicePublicKey
3. Bob encapsulates alicePublicKey, which produces mlKemcipherText, and bobSharedKey
4. Bob encrypts his message to Alice, using some AEAD scheme like AES-GCM, with bobSharedKey as the symmetric key, and prepending mlKemcipherText to the output, as well as specifying mlKemcipherText as the AAD (additional associated data) when encrypting.

So the final pseudocode would look something like:

cipherText = AESGCMEncrypt(data: message, key: bobSharedKey, nonce: randomNonce, aad: mlKemcipherText)

And the final output would be something like:

finalCipherText = mlKemcipherText | nonce | cipherText

Alice then receives finalCipherText from Bob:

1. Alice removes mlKemcipherText and the nonce from the leading bytes of finalCipherText (based on the length of each)
2. Alice decapsulates mlKemcipherText with aliceSecretKey, which produces aliceSharedKey (which matches bobSharedKey)

Alice then decrypts the bytes which remain of mlKemcipherText, with:

clearText = AESGCMDecrypt(data: mlKemcipherText, key: aliceSharedKey, nonce:nonce, aad: mlKemcipherText)

Alice knows that the message hasn't been tampered with, although she can't be sure that it came from Bob (but that's beyond the context of this question).

Does this scheme seem sound? Is there something I've overlooked? Is there a similar scheme which has been formalized which I can refer to?

Answer 1

Is there a similar scheme which has been formalized which I can refer to?

Well, there is HPKE (RFC 9180), which has been extended to include ML-KEM (draft-connolly-cfrg-hpke-mlkem). It works pretty much as you have outlined, except that there is a KDF step between the ML-KEM and the AEAD cipher. That KDF is there because HPKE is generic - some KEMs don't produce uniform shared secrets (ML-KEM does), and sometimes there's a size mismatch between the shared secret generated by the KEM, and the key size of the AEAD cipher.

Answer 2

TL;DR: I think you get the gist of how KEM's work but not about the security surrounding hybrid encryption. This is kind of a separate issue though, you'd face the same issue when using (EC)DH-IES or even direct RSA encryption.

---

First of all, using the term "hybrid encryption" has become a bit of an issue because "hybrid" can have at least two meanings. "Hybrid" can be used to indicate that both hybrid quantum- and classical algorithms such as RSA are used.

On the other hand it can also mean that hybrid encryption schemes such as IES are used (IES is a hybrid encryption scheme for Diffie-Hellman). In the latter case - which you are referring to - the hybrid part is about asymmetrical and symmetrical algorithms.

This is a bit of a bugger-up by the cryptographic community. I'd be sure to indicate clearly what is meant when creating a "hybrid" protocol.

1. Alice generates a ML-KEM key pair, resulting in aliceSecretKey and alicePublicKey. And makes alicePublicKey available to anyone who wants it.

Sure, but in that case you are using a static key pair. I guess that's OK for this purpose especially if you want to establish trust for that public key. Many transport protocols such as TLS use ephemeral keys to provide transport security though, with separate key pairs used for entity authentication.

2. Bob wants to send a private message to Alice, so he starts be obtaining alicePublicKey

Yeah, OK. You'd have to trust that the public key belongs to Alice of course, even if you forgo authenticated encryption. Because if you don't establish that trust you may be encapsulating with Eve's public key instead, enabling man-in-the-middle and other attacks. In other words, it would not even offer confidentiality if no trust is establihshed.

3. Bob encapsulates alicePublicKey, which produces mlKemcipherText, and bobSharedKey

Yes, but the shared key or in this case shared secret is - of course - property of both parties - but that's just nagging about the name. And it is the key or secret that is encapsulated using alicePublicKey (i.e. alicePublicKey is the encapsulating key, not the encapsulated key). Again, that's just about using the right jargon.

4. Bob encrypts his message to Alice, using some AEAD scheme like AES-GCM, with bobSharedKey as the symmetric key, and prepending mlKemcipherText to the output, as well as specifying mlKemcipherText as the AAD (additional associated data) when encrypting.

Yes, although normally you'd maybe derive a key from the shared secret key for particular messages. Yes, this is about using a key derivation function or KDF.

Alice knows that the message hasn't been tampered with, although she can't be sure that it came from Bob (but that's beyond the context of this question).

No, without authentication of Bob anybody could be sending messages, so this is not a tamper resistant scheme. OK, if the public key of Alice can be authenticated then an attacker cannot decapsulate and read the ciphertext, but they can still simply replace the message by anything else.

GCM offers message authentication, but it assumes that the secret key is established between trusted parties.