Translations:Attention Is All You Need/16/en
The encoder consists of six identical layers, each containing a multi-head self-Lua error: Internal error: The interpreter exited with status 1. sublayer followed by a position-wise feed-forward network, with residual connections and Lua error: Internal error: The interpreter exited with status 1. around each sublayer. The decoder adds a third sublayer that performs multi-head Lua error: Internal error: The interpreter exited with status 1. over the encoder output, and masks future positions in the self-Lua error: Internal error: The interpreter exited with status 1. to preserve the autoregressive property.