NotImplementedError: ggml_type 3 not implemented

### Description When trying to use the model `Qwen/Qwen2-7B-Instruct-GGUF` with the gguf file `qwen2-7b-instruct-q4_0.gguf`, I encountered a NotImplementedError about ggml_type 3 not implemented. ### Who can help? _No response_ ### Information - [ ] The official example scripts - [X] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [X] My own task or dataset (give details below) ### Reproduction Install the required libraries: `pip install transformers==4.42.3 gguf==0.6.0` Load the model and tokenizer: ``` from transformers import AutoTokenizer, AutoModelForCausalLM model_id = "Qwen/Qwen2-7B-Instruct-GGUF" file_name = "qwen2-7b-instruct-q4_0.gguf" tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file = file_name) model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file = file_name) ``` ### Expected behavior The code should process ggml_type 3 without raising an error, or provide an alternative implementation. ### System Info - transformers version: 4.42.3 - gguf version: 0.6.0 - Platform: Ubuntu 20.04 - Python version: 3.9.16 - PyTorch version (GPU?): 2.2.2(True) - Using GPU in script?: Yes - Using distributed or parallel set-up in script?: No