Algebraic Invariants of Lightning Self-Attention

Yulia Alexandr, Hao Duan, Guido Montúfar

We study the polynomial coefficients of lightning self-attention as coordinates of an algebraic variety. We identify linear and nonlinear families of algebraic invariants, including Chow-type, low-rank, Veronese-type, and Sylvester resultant-based constraints.

Read on ELI