Since the model contains no recurrence, positional encodings are added to the input embeddings using sinusoidal functions of different frequencies:
