Training GPT2 with run_clm.py exceeds the described memory amount .

### System Info - `transformers` version: 4.40.0.dev0 - Platform: Linux-6.5.0-28-generic-x86_64-with-glibc2.17 - Python version: 3.8.19 - Huggingface_hub version: 0.22.2 - Safetensors version: 0.4.2 - Accelerate version: 0.29.2 - Accelerate config: not found - PyTorch version (GPU?): 1.10.0+cu111 (True) - Tensorflow version (GPU?): not installed (NA) - Flax version (CPU?/GPU?/TPU?): not installed (NA) - Jax version: not installed - JaxLib version: not installed - Using GPU in script?: <fill in> - Using distributed or parallel set-up in script?: <fill in> ### Who can help? @ArthurZucker and @younesbelkada ### Information - [X] The official example scripts - [ ] My own modified scripts ### Tasks - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction python run_clm.py \ --model_name_or_path openai-community/gpt2 \ --dataset_name wikitext \ --dataset_config_name wikitext-2-raw-v1 \ --per_device_train_batch_size 8 \ --per_device_eval_batch_size 8 \ --do_train \ --do_eval \ --overwrite_output_dir \ --output_dir /tmp/test-clm ### Expected behavior The example in the script mentions training with a K80 GPU at a batch size of 8, noting that the K80 has 24GB of memory. However, when I use an RTX 3090 with a batch size set to 4, it consumes 20GB of memory without modifying any settings. Why is this the case?