The answer
One-hot vectors identify vocabulary entries but are sparse and do not directly encode learned relationships. Dense embeddings provide a practical representation for subsequent neural computations.
Why are text inputs converted into embeddings before entering a neural network?
Why not use one-hot vectors throughout the entire network?
More FAQs in Why are text inputs converted into embeddings before entering a neural network?
Usually it is part of the model itself. Tokenization is a separate preprocessing stage, while the learned embedding lookup participates in the neural network’s computation.
A basic token lookup vector does not. Later contextual layers incorporate the available sequence, so the same initial vector can contribute to different representations in different sentences.
Still have questions?
Browse all our FAQs or reach out to our support team
Want to upskill yourself?
Your learning library
Small steps. Big possibilities.
Find your next course. Build something great.