| 1 | <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1280 720" width="1280" height="720"> |
| 2 | <defs> |
| 3 | <filter id="figFrame" x="-5%" y="-5%" width="110%" height="110%"> |
| 4 | <feGaussianBlur in="SourceAlpha" stdDeviation="6"/> |
| 5 | <feOffset dx="0" dy="3"/> |
| 6 | <feFlood flood-color="#1A365D" flood-opacity="0.08"/> |
| 7 | <feComposite in2="SourceAlpha" operator="in"/> |
| 8 | <feMerge><feMergeNode/><feMergeNode in="SourceGraphic"/></feMerge> |
| 9 | </filter> |
| 10 | </defs> |
| 11 | |
| 12 | <g id="bg"> |
| 13 | <rect width="1280" height="720" fill="#FFFFFF"/> |
| 14 | </g> |
| 15 | |
| 16 | <g id="header"> |
| 17 | <rect x="60" y="56" width="6" height="40" fill="#3182CE"/> |
| 18 | <text x="84" y="88" font-family="Georgia, "Microsoft YaHei", serif" font-size="34" font-weight="700" fill="#1A365D">Encoder–Decoder, stacked self-attention, point-wise FFN</text> |
| 19 | <text x="84" y="118" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="13" font-weight="700" fill="#4A5568" letter-spacing="1.5">MODEL ARCHITECTURE · FIGURE 1 FROM THE PAPER</text> |
| 20 | </g> |
| 21 | |
| 22 | <g id="paper-figure"> |
| 23 | <rect x="80" y="150" width="382" height="540" rx="12" fill="#FFFFFF" stroke="#E2E8F0" stroke-width="1.5" filter="url(#figFrame)"/> |
| 24 | <image href="../images/attention_p3_0.png" x="92" y="162" width="358" height="516" preserveAspectRatio="xMidYMid meet"/> |
| 25 | <text x="271" y="710" text-anchor="middle" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="11" font-style="italic" fill="#4A5568">Figure 1 · The Transformer — model architecture</text> |
| 26 | </g> |
| 27 | |
| 28 | <g id="legend-1"> |
| 29 | <circle cx="510" cy="170" r="16" fill="#1A365D"/> |
| 30 | <text x="510" y="176" text-anchor="middle" font-family="Georgia, "Microsoft YaHei", serif" font-size="16" font-weight="700" fill="#FFFFFF">1</text> |
| 31 | <text x="544" y="166" font-family="Georgia, "Microsoft YaHei", serif" font-size="20" font-weight="700" fill="#1A365D">Inputs + Positional Encoding</text> |
| 32 | <text x="544" y="194" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Token embeddings get a deterministic sinusoidal position vector</text> |
| 33 | <text x="544" y="216" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">added — order is injected, not learned (see P11).</text> |
| 34 | </g> |
| 35 | |
| 36 | <g id="legend-2"> |
| 37 | <circle cx="510" cy="262" r="16" fill="#1A365D"/> |
| 38 | <text x="510" y="268" text-anchor="middle" font-family="Georgia, "Microsoft YaHei", serif" font-size="16" font-weight="700" fill="#FFFFFF">2</text> |
| 39 | <text x="544" y="258" font-family="Georgia, "Microsoft YaHei", serif" font-size="20" font-weight="700" fill="#1A365D">Nx Encoder block</text> |
| 40 | <text x="544" y="286" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Multi-Head Self-Attention → Position-wise Feed Forward, both</text> |
| 41 | <text x="544" y="308" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">wrapped by Add & Norm. N = 6 in the base configuration.</text> |
| 42 | </g> |
| 43 | |
| 44 | <g id="legend-3"> |
| 45 | <circle cx="510" cy="354" r="16" fill="#3182CE"/> |
| 46 | <text x="510" y="360" text-anchor="middle" font-family="Georgia, "Microsoft YaHei", serif" font-size="16" font-weight="700" fill="#FFFFFF">3</text> |
| 47 | <text x="544" y="350" font-family="Georgia, "Microsoft YaHei", serif" font-size="20" font-weight="700" fill="#1A365D">Nx Decoder block</text> |
| 48 | <text x="544" y="378" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Masked Self-Attn → Encoder-Decoder Attn → FFN, each with</text> |
| 49 | <text x="544" y="400" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Add & Norm. The mask enforces auto-regression.</text> |
| 50 | </g> |
| 51 | |
| 52 | <g id="legend-4"> |
| 53 | <circle cx="510" cy="446" r="16" fill="#1A365D"/> |
| 54 | <text x="510" y="452" text-anchor="middle" font-family="Georgia, "Microsoft YaHei", serif" font-size="16" font-weight="700" fill="#FFFFFF">4</text> |
| 55 | <text x="544" y="442" font-family="Georgia, "Microsoft YaHei", serif" font-size="20" font-weight="700" fill="#1A365D">Add & Norm — residual + LayerNorm</text> |
| 56 | <text x="544" y="470" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Every sub-layer output is LayerNorm(x + Sublayer(x)). Enables</text> |
| 57 | <text x="544" y="492" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">deep stacks; d_model = 512 throughout (see P10).</text> |
| 58 | </g> |
| 59 | |
| 60 | <g id="legend-5"> |
| 61 | <circle cx="510" cy="538" r="16" fill="#3182CE"/> |
| 62 | <text x="510" y="544" text-anchor="middle" font-family="Georgia, "Microsoft YaHei", serif" font-size="16" font-weight="700" fill="#FFFFFF">5</text> |
| 63 | <text x="544" y="534" font-family="Georgia, "Microsoft YaHei", serif" font-size="20" font-weight="700" fill="#1A365D">Linear + Softmax — output probabilities</text> |
| 64 | <text x="544" y="562" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Decoder output is projected back to vocabulary size and</text> |
| 65 | <text x="544" y="584" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">softmaxed to next-token probabilities.</text> |
| 66 | </g> |
| 67 | |
| 68 | <g id="takeaway"> |
| 69 | <rect x="494" y="618" width="700" height="60" rx="8" fill="#F5F7FA" stroke="#E2E8F0" stroke-width="1"/> |
| 70 | <rect x="494" y="618" width="4" height="60" fill="#3182CE"/> |
| 71 | <text x="514" y="644" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="13" font-weight="700" fill="#3182CE" letter-spacing="1">SAME SKELETON AS PRIOR SEQ2SEQ</text> |
| 72 | <text x="514" y="666" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="14" fill="#1A202C">What is new is what fills the boxes: only self-attention + FFN.</text> |
| 73 | </g> |
| 74 | </svg> |
| 75 |