In a vocabulary of $ V $ words, a one-hot vector for the $ i $-th word is a $ V $-dimensional vector with a 1 in position $ i $ and 0 elsewhere. This representation has two critical shortcomings:
