Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane
Published in International Conference on Machine Learning (ICML), 2026
Spiral RoPE encodes spatial relationships along multiple directions by rotating groups of embedding channels using projected patch positions. This extension improves vision transformer performance on image classification, segmentation, and generation tasks.
Authors: Haoyu Liu, Sucheng Ren, Tingyu Zhu, Peng Wang, Cihang Xie, Alan Yuille, Zeyu Zheng, Feng Wang