The answer
Not necessarily. Tokens can be shorter than words or include punctuation and other text units. The tokenizer determines which units the model predicts.
How does an LLM predict the next token in a sequence?
Does the model predict the next whole word each time?
More FAQs in How does an LLM predict the next token in a sequence?
The prediction depends on the broader available context, including earlier tokens and instructions. Different contexts can give the same word different roles and meanings.
A logit is an unnormalized score produced by the model. A normalization operation such as softmax converts the set of scores into probabilities.
Still have questions?
Browse all our FAQs or reach out to our support team
Want to upskill yourself?
Your learning library
Small steps. Big possibilities.
Find your next course. Build something great.