As long as the first 256 bits of the SHAKE256 output are identical, the remaining bits must also be identical？

Question

According to the official Keccak data, the second preimage resistance of SHAKE256 is given as min(d, 256), where d is the output length. Based on the definition of second preimage resistance, consider the following cases:

1. Given an input m₁ that produces a 255-bit output out₁, finding another input m₂ (with m₂ ≠ m₁) such that SHAKE256(m₂) = out₁ requires, on average, 2²⁵⁵ attempts (since the security level is 2²⁵⁵).

2. Given an input m₁ that produces a 256-bit output out₁, finding another input m₂ (with m₂ ≠ m₁) such that SHAKE256(m₂) = out₁ requires, on average, 2²⁵⁶ attempts (since the security level is 2²⁵⁶).

3. Now, suppose we have an input m₁ that produces a 257-bit output, out₁₍₂₅₇₎. If we truncate out₁₍₂₅₇₎ to its first 256 bits (call this output₁₍₂₅₆₎) and then try to find another input m₂ (with m₂ ≠ m₁) such that SHAKE256(m₂) produces a 257-bit output out₂₍₂₅₇₎ whose first 256 bits (output₂₍₂₅₆₎) equal output₁₍₂₅₆₎, the security level remains 2²⁵⁶—so on average, it should take 2²⁵⁶ attempts. The question then is: In this case, must out₁₍₂₅₇₎ equal out₂₍₂₅₇₎? In other words, do the two 257-bit outputs necessarily have the same final (257th) bit? If not, would the probability of the final bits matching be 1/2? And if it is 1/2, then one might argue that, given an input m₁ that produces a 257-bit output, finding a second preimage m₂ (with m₂ ≠ m₁) such that SHAKE256(m₂) equals m₁’s 257-bit output would require, on average, 2²⁵⁶ × 2 = 2²⁵⁷ attempts—contradicting the official Keccak data.

4. Based on the reasoning in point 3, it appears that as long as the first 256 bits of the SHAKE256 output are identical, the remaining bits must also be identical.

Is this reasoning correct? If not, what is the error in this derivation?

Answer 1

In other words, do the two 257-bit outputs necessarily have the same final (257th) bit?

No, not necessarily. You'd find two states where the same output is produced. In the case of SHAKE the state is represented by the capacity. Generally the state is double the size of the security provided - in the case of SHAKE256 the capacity is 512 bits. The chance that for 256 bit output the state is identical is therefore astronomically low.

In other words, do the two 257-bit outputs necessarily have the same final (257th) bit?

So that's a solid no.

If not, would the probability of the final bits matching be 1/2?

Yes, the output of most secure hash functions cannot be distinguished from random (assuming you don't know the state or the input, of course).

And if it is 1/2, then one might argue that, given an input m₁ that produces a 257-bit output, finding a second preimage m₂ (with m₂ ≠ m₁) such that SHAKE256(m₂) equals m₁’s 257-bit output would require, on average, 2²⁵⁶ × 2 = 2²⁵⁷ attempts—contradicting the official Keccak data.

No. The trick is to find a method to create the same internal state. Note though that the security claim is - by definition - a minimum of the security provided. It isn't said that an adversary can always generate the same 257 bit output using $2^{256}$ operations.

Based on the reasoning in point 3, it appears that as long as the first 256 bits of the SHAKE256 output are identical, the remaining bits must also be identical.

That's definitely not the case due to the state, as mentioned before.

Is this reasoning correct? If not, what is the error in this derivation?

The errors are in misunderstanding what a security claim is, and in treating the SHAKE function as a black box, without taking the algorithm design (and its state in particular) into account.