LLaMa 3 8B - offloaded_static cache - layer_device_map TypeError

### System Info Transformers Patch release v4.45.2 PyTorch 1.10.1 Python 3.8.0 cuda 11.1 NVIDIA V100 ### Who can help? @gante @zucchini-nlp @Rocketknight1 ### Information - [ ] The official example scripts - [X] My own modified scripts ### Tasks - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction Stack trace: ``` --------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[19], line 1 ----> 1 outputs = pipe( 2 messages, 3 max_new_tokens=3000, 4 eos_token_id=terminators, 5 do_sample=True, 6 temperature=0.6, 7 top_p=0.9, 8 # cache_implementation="static", 9 cache_implementation="offloaded_static", 10 ) 11 assistant_response = outputs[0]["generated_text"][-1]["content"] 12 print(assistant_response) File python3.8/site-packages/transformers/pipelines/text_generation.py:267, in TextGenerationPipeline.__call__(self, text_inputs, **kwargs) 262 if isinstance( 263 text_inputs, (list, tuple, KeyDataset) if is_torch_available() else (list, tuple) 264 ) and isinstance(text_inputs[0], (list, tuple, dict)): 265 # We have one or more prompts in list-of-dicts format, so this is chat mode 266 if isinstance(text_inputs[0], dict): --> 267 return super().__call__(Chat(text_inputs), **kwargs) 268 else: 269 chats = [Chat(chat) for chat in text_inputs] # 🐈 🐈 🐈 File python3.8/site-packages/transformers/pipelines/base.py:1268, in Pipeline.__call__(self, inputs, num_workers, batch_size, *args, **kwargs) 1260 return next( 1261 iter( 1262 self.get_iterator( (...) 1265 ) 1266 ) 1267 else: -> 1268 return self.run_single(inputs, preprocess_params, forward_params, postprocess_params) …