Translations:Softmax Function/17/en: Difference between revisions
(Importing a new version from external source) |
(Importing a new version from external source) |
||
| Line 1: | Line 1: | ||
A naive implementation of softmax can overflow when logits are large (e.g., <math>e^{1000}</math> is infinite in floating point). The standard fix subtracts the maximum logit: | A naive implementation of softmax can overflow when {{Term|logits}} are large (e.g., <math>e^{1000}</math> is infinite in floating point). The standard fix subtracts the maximum {{Term|logits|logit}}: | ||
Revision as of 19:42, 27 April 2026
A naive implementation of softmax can overflow when logits are large (e.g., $ e^{1000} $ is infinite in floating point). The standard fix subtracts the maximum logit: