[RFC] Add `modeling_xxx_fusion.py` to support kernel fusion

## Introduction I am an engineer currently working on 3D model parallelism for transformers. When the tensor model parallelism (https://github.com/huggingface/transformers/pull/13726) is done, I am going to introduce [kernel fusion](https://stackoverflow.com/questions/53305830/cuda-how-does-kernel-fusion-improve-performance-on-memory-bound-applications-on) feature to transformers.  For this, I want to create a new modeling file called `modeling_xxx_fusion.py`. This work is currently being discussed with @stas00 and @RezaYazdaniAminabadi (DeepSpeed team). ## Kernel fusion API ```python from transformers import BertForMaskedLM # create model model = BertForMaskedLM.from_pretrained("bert-base-cased") # 1. fuse_modules # `fuse_modules` is function level fusion, It supports a wide variety of models. # all arguments is `True` as default model.fuse_modules() # fuse selective modules model.fuse_modules( word_embedding=True, scale_mask_softmax=True, layer_norm=True, bias_act=True, bias_dropout_residual=False, cross_entropy=True, ) # 2. fuse_layers # `fuse_layers` is block level (attention & mlp) fusion, only a few models are supported. # argument (`inference`) is `None` -> `not self.training` of `torch.nn.Module` as default. model.fuse_layers(inference=None) # fuse layers for inference model.fuse_layers(inference=True) # fuse layers for training model.fuse_layers(inference=False) ``` ## Implementation The internal module of each model will be re-implemented using kernel fusion method, and the existed module will be replaced with the fused module. The following example is an example of `BertOutput(nn.Module)`. ```python # transformers/models/bert/modeling_bert.py class BertOutput(nn.Module): def __init__(self, config): super().__init__() self.dense = nn.Linear(config.intermediate_size, config.hidden_size) …