GaAN

Gated Attention Networks (GaAN) is a new architecture for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head’s importance.

Image credit: GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs