Export to ExecuTorch

## Feature request Unlock a new workflow for on-device use-cases via [**torch.export**](https://pytorch.org/tutorials/intermediate/torch_export_tutorial.html) and [**ExecuTorch**](https://pytorch.org/executorch/main/intro-overview.html). So ideally the users can have an e2e experience by loading a pretrained transformer model from HuggingFace, export and lower it to `ExecuTorch` and get reasonable performance out-of-the-box. For example: 1. Load a model with StaticCache: ``` model = AutoModelForCausalLM.from_pretrained( hf_model_repo, config=config, attn_implementation="sdpa", cache_config={ "use_cache": True, "cache_implementation": "static", "max_cache_length": 128, }, # Mandatory field to set ONLY for "Export to ExecuTorch" workflow, optional in other use-cases ) ``` 2. Then export the model with StaticCache. ``` exported_program = convert_and_export_with_cache( model, args=(model_inputs,), kwargs={"position_ids": <val>, "inputs_embeds": <val>, "cache_position": <val>} ``` and then further lower the exported program to `ExecuTorch` with delegates for performance: ``` executorch_m = lower_to_executorch( model, recipes="xnnpack_fp32", # Delegate to XNNPACK backend ) # The lowered artifact can be saved into a `.pte` binary format for integration and distribution. ``` With that you may get a model for on-device with reasonable performance to start with. From there and still within `ExecuTorch` stack, you can easily tailor the experience for your use-cases, of course, with better performance! Note that `ExecuTorch` supports delegatation to [XNNPACK backend](https://pytorch.org/executorch/main/native-delegates-executorch-xnnpack-delegate.html), [Apple Core ML](https://pytorch.org/executorch/main/build-run-coreml.html) and [MPS](https://github.com/pytorch/executorch/tree/main/examples/apple/mps), [Qualcomm QNN](https://pytorch.org/executorch/main/build-run-qualcomm-ai-engine-direct-backend.html), …