Large World Model and Ring Attention

### Model description Ring Attention is a new kind of attention that can expand the context size reliably (tested with precise needle search) up to several _million_ tokens. See https://arxiv.org/abs/2310.01889. One of the models trained with RingAttention is multimodal Large World Model https://largeworldmodel.github.io/ I believe this attention replacement can be crucial for future LLMs (and not only), and should be included with the Transformers library ### Open source status - [X] The model implementation is available - [X] The model weights are available ### Provide useful links for the implementation LWM checkpoints https://huggingface.co/LargeWorldModel Ring Attention in PyTorch by Lucidrains https://github.com/lucidrains/ring-attention-pytorch LWM in Jax https://github.com/LargeWorldModel/LWM