| 1 | Let's compare the encoder and decoder side by side. The encoder has two sub-layers per block: multi-head self-attention, then a position-wise feed-forward network. The decoder has three: masked multi-head self-attention, then encoder-decoder attention, then the feed-forward network. Around every sub-layer the same wrapper applies — output equals LayerNorm of x plus Sublayer of x, the classic residual connection plus normalization. All sub-layers and embeddings emit a 512-dimensional vector so the residual addition is well-defined. The three asymmetries on the decoder side are the third sub-layer, the causal mask that preserves auto-regression, and the one-position shift of the output embeddings. |