Is there a benefit to "double encryption" when it comes to attacks analyzing attempted decryption results?

Question

This is a beginner's question related to general security, but it fits cryptography exchange because it's more specific to the action of cryptographic attacks and how they work.

Assume the following theoretical situation:

1. I do not know how attacks of all types work programmatically. From the aspect of this question, assume theoretically (hypothetical quantum computing algorithm for example) there is a working attack against AES that can break it or any encryption by whatever means.

2. BUT we've "double encrypted" our data with two separate symmetric algorithms by means of hybrid encryption. We've also NOT sent keys with the initial message but by some other secured channel.

3. Attacker (after however many attempts, by whatever means) guesses the correct key for the first layer of encryption and successfully decrypts the first layer.

Questions:

1. At this point, how does the attacker's system know it has broken the encryption, especially if it has attempted many previous attempts & at insane speeds? Corollary: how can an attacker know they have decrypted successfully since the decrypted text would look exactly like the encrypted text of the next layer of encryption?

2. Assuming various agencies, countries, or hackers, are capable of breaking various encryption techniques, would such a system (from a high-level viewpoint, based on question one logic) add any level of security?

3. Would reverse engineering by decompiling such a system enable all layers of encryption to be broken easily?

Answer 1

At this point, how does the attackers system know it has broken the encryption, especially if it has attempted many previous attempts & at insane speeds? Corollary: how can an attacker know they have decrypted successfully since the decrypted text would look exactly like encrypted text of the next layer of encryption?

It doesn't. But note that to verify that the encryption is broken only takes that the other cipherstream is distinguishable from random. That may be less costly than a full attack that requires decryption. Furthermore, depending on the construction, meet-in-the-middle attacks may be feasible.

Assuming various agencies, countries or hackers, are capable of breaking various encryption techniques, would such a system (from a high level viewpoint, based on question one logic) add any level of security?

Yes, but how much depends on the details. Meet-in-the-middle attacks were already mentioned. However, it could also be that both ciphers have a common structure, which means that if one can be broken then the other might get broken as well. In that case the effort is doubled, if that. If Grover is used for quantum cryptanalysis then we can assume that doubling the number of qubits would get you pretty far from breaking the double-encryption system.

Would reverse engineering by decompiling such a system enable all layers of encryption to be broken easily?

You cannot "decompile" a cryptosystem, that's term is not defined within cryptography. The algorithms are considered known, while the keys are considered secret.