7

I'm not quite sure how's the decoder output is flattened into a single vector. As from my understanding, if we input the encoder with a length N sentence, it's output is N x units (e.g. N x 1000), and we input the decoder with a length M sentence, the output of the decoder will give us M x units output. M is not fixed (M should be the length of the decoder's raw input) and will change during the different steps of inference. How do we go from here to a single vector? enter image description here Screen shot from "Attention is all you need"

Ian
  • 73
  • 5

1 Answers1

4

I'm not quite sure how's the decoder output is flattened into a single vector

That's the thing. It isn't flattened into a single vector. The linear transformation is applied to all $M$ vectors in the sequence individually. These vectors have a fixed dimension, which is why it works.

Valentin Calomme
  • 6,256
  • 3
  • 23
  • 54