Attention Mechanism - What are Query, Key, and Value?
Explain Query, Key, Value and Output in Attention Mechanism with an analogy of Information Retrieval System. Discuss different Attention Mechanisms, including Dot-Product, Scaled Dot-Product Attention, and Self Attention. Talk about the Query, Key, Value, and Output choices in the standard Transformer.
