The answer
You can plot attention heatmaps to see exactly which words the model focused on when generating a specific answer.
Understanding Attention Mechanism in Deep Learning
How does attention help debug model outputs?
More FAQs in Understanding Attention Mechanism in Deep Learning
An attention calculation that divides scores by the square root of the vector dimension to prevent gradients from exploding.
Global attention looks at all words in the input, while local attention only looks at a sliding window of neighboring words.
Because it relates positions of a single sequence to calculate a representation of the same sequence.
Still have questions?
Browse all our FAQs or reach out to our support team
Want to upskill yourself?
Your learning library
Small steps. Big possibilities.
Find your next course. Build something great.