Tuning generation_config in Trainer hyperparameter_search (Optuna backend)

### Feature request Adding generation configurations to the parameters that can be tuned in a `Trainer`. ### Motivation When defining the Optuna hyper-parameter space, I would like to investigate whether or not different generation configurations can affect performance. For example, something as simple as: is beam search with groups better than standard beam search? Example of implementation: ```python def optuna_hp_space(trial): # Define default generation parameters generation_params = { "max_length": 512, "max_new_tokens": 512, 'top_k': 20, } # Define the generation strategies and pick one with Optuna # REF: https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/generation/configuration_utils.py#L71 generation_strategy_params = { "greedy": {"num_beams": 1, "do_sample": False}, "contrastive_search": {"penalty_alpha": 0.1, "top_k": 10}, "multinomial_sampling": {"num_beams": 1, "do_sample": True}, "beam_search_decoding": {"num_beams": 5, "do_sample": False}, "beam_search_multinomial_sampling": {"num_beams": 5, "do_sample": True}, "diverse_beam_search_decoding": {"num_beams": 5, "num_beam_groups": 5, "diversity_penalty": 1.0}, } gen_strategy = trial.suggest_categorical("generation_strategy", list(generation_strategy_params.keys())) generation_params.update(generation_strategy_params[gen_strategy]) # Update the generation params with the temperature temperature = trial.suggest_float("temperature", 0.5, 1.1, log=False) generation_params["temperature"] = temperature # Instantiate a GenerationConfig object to pass to the Trainer arguments generation_config = GenerationConfig(**generation_params) # Setup learning rate warmup ratio warmup_ratio = trial.suggest_float("warmup_ratio", 0.0, 0.1, step=0.01) # Setup learning rate scheduler type and its fixed kwargs lr_scheduler_type = trial.suggest_categorical("l…