Fast cryptographic hash function for short inputs

Question

I am looking for a cryptographic hash function optimized for speed on short inputs, in order to implement a pseudorandom generator with expansion factor 2 (e.g. takes 16 bytes of input and outputs 32 pseudorandom bytes).

Here are some natural candidates I tried:

• SHA256: good baseline
• Blake2: designed for speed on large inputs, does not perform as well on short inputs
• AES-CTR: faster than SHA256 when the hardware supports AES-NI. The input is used as a key for AES to encrypt a predefined byte array of the desired output length. However, re-initializing the cipher for each call to the hash function is costly.

I also found this interesting construction: STHash. It is a keyed cryptographic hash function optimized for speed on large inputs. I don't mind having a keyed hash function instead of a general-purpose one.

Is there any analogous construction for short inputs, or a more efficient way to leverage AES-NI than AES-CTR?

Some informal benchmarks

For each hash function, I hash an array of 16 bytes into a 32 bytes array, and I repeat 10 million times. For stream ciphers like AES and ChaCha, I create a new cipher at each iteration with the input as key on a public fixed plaintext and nonce. If the cipher needs a 32-bit key, I just pad the input with 0. If the hash function does not produce enough bits (e.g. SipHash outputs only 128 bits), I run it several times.

I am running Rust Nightly on an Intel® Core™ i7-1065G7 CPU @ 1.30GHz × 8, the experiments run on a single thread.

• SipHash 1-3: 476.9ms
• Chacha8: 590.4ms
• SipHash: 670.3ms
• AES-128: 665.3ms
• SHA256: 780.4ms
• Blake2s: 1413.9ms

For information, some results about Haraka (using a not well-known optimized implementation):

• Haraka-v2 256-5: 55.2ms
• Haraka-v2 256-6: 69.9ms

Answer 1

[For devices with the AES-NI instruction set]

I highly recommend looking at the Davies-Meyer construction based on fixed-key AES. By now, it has been used in dozens of different works, and though it is of course not a perfectly standard assumption on AES, it is quite widely accepted as a plausible one. In particular, it provably yields a PRG when modeling AES as a random permutation.

The construction is as follows:

• Fix two keys $(K_0, K_1)$ once for all. They are public and will never change, hence you will only have to do the key schedule twice.
• On input $x$ of size 16 Bytes, output $(\mathsf{AES}_{K_0}(x) \oplus x,\mathsf{AES}_{K_1}(x) \oplus x)$.

Note the crucial XOR with $x$ (the construction would of course be insecure without it, as AES is invertible given the keys). A similar variant of the above, which requires a single key schedule and provides the same security (when modeling AES as a permutation) is:

• Fix one key $K$ once for all.
• On input $x$ of size 16 Bytes, output $(\mathsf{AES}_{K}(x) \oplus x,\mathsf{AES}_{K}(x\oplus 1) \oplus (x\oplus 1))$.

To read more about this approach, this is the proper starting point. In particular, you will see discussions on related constructions achieving different security properties. I don't think the paper directly proves that the exact constructions I sketched above are PRGs (when modeling AES as a random permutation), but the proof techniques that they use (using Patarin's H-coefficient technique) can be easily adapted to prove it (I worked out this exact analysis over the summer).

There are dozens of papers using this construction to implement the PRG required for the GGM pseudorandom function. This is especially common for implementing function secret sharing, or for building MPC-in-the-head post-quantum signatures (I can provide a sample of pointers upon request).

As for speed, fixed-key AES with the AES-NI assumption can be as fast as 1.3 cycles per Byte, which seems very hard to beat (XORs are of course super fast as well, especially using vector instructions). See e.g. here for a reference that indicates this number.

=== EDIT (answering OP's comment below) ===

As far as I know, Davies-Meyer and Matyas-Meyer-Oseas are two slightly incorrect ways of referring to the same construction (the one I described above). Originally, both Davies-Meyer and Matyas-Meyer-Oseas are compression functions to be used in hash function constructions from block ciphers:

• Davies-Meyer: $H_i = E_{m_i}(H_{i-1}) \oplus H_{i-1}$ (i.e. the next block is computed from the previous block using the current message block $m_i$ as the key)
• Matyas-Meyer-Oseas: $H_i = E_{g(H_{i-1})}(m_i) \oplus m_i$ (i.e. the next block is computed from the message block using the some function of the previous block as the key)

In our context, the key is always fixed anyway and there is no compression function - we're building a PRG. But the design is somewhat analogeous to either of the above and can be seen as either with a slight abuse of definitions of what counts as the previous block and what counts as a message block.

I think the terminology Matyas-Meyer-Oseas was the one originally used here, but later works (e.g. this one) switched to Davies-Meyer, probably because it felt a slightly more accurate naming (but both are debatable).

Answer 2

These days, on CPUs with AES hardware acceleration, Areion512 looks like a perfect fit and is likely to beat other options from a performance perspective.

Areion512 is a very efficient 512-bit permutation based on the AES round function.

Since introducing a key is an option, you can use $P(m \oplus k) \oplus k$ to efficiently expand 16 bytes into 32 bytes ($P$ being the Areion512 permutation and $m$ padded to 32 bytes).

Answer 3

Did you try SipHash, especially the reduced-round version SipHash-1-3?

It was explicitly designed for short input, doesn’t require key expansion, is fast on pretty much all kind of architectures, and can output 64 or 128 bit.

The name might be confusing, though: a key is required, but since you mentioned that it wasn’t an issue for your use case, give it a try.

Answer 4

You can use HighwayHash. It is a fast SIMD-based keyed hash function (5x faster than SipHash) with security claims and suitable for hashing short inputs.