Request for Flash Attention 2.0 Support in GPNRoFormerForMaskedLM

Hello, I trust this message finds you well. I am currently attempting to run the GPN-MSA model, which utilizes AutoModelForMaskedLM, and I am keen on parallelizing the computation across multiple GPUs. To optimize the model's performance, I would like to request the integration of Flash Attention 2.0 support into GPNRoFormerForMaskedLM. As I explore this avenue for parallelization, I envision that many others within the community could benefit from this enhancement. Thank you for your time and consideration. Best regards.