POET

## Train BERT and other large models on smartphones ### Model description `POET` enables the training of state-of-the-art memory-hungry ML models on smartphones and other edge devices. POET (Private Optimal Energy Training) exploits the twin techniques of integrated tensor rematerialization, and paging-in/out of secondary storage (as detailed in our paper at ICML 2022) to optimize models for training with limited memory. POET's Mixed Integer Linear Formulation (MILP) ensures the solutions are provably optimal! approach enables training significantly larger models on embedded devices while reducing energy consumption while not modifying mathematical correctness of backpropagation. We demonstrate that it is possible to fine-tune both `ResNet-18` and `BERT` within the memory constraints of a Cortex-M class embedded device while outperforming current edge training methods in energy efficiency. ### Open source status - [X] The model implementation is available - [ ] The model weights are available ### Provide useful links for the implementation [Implementation](https://github.com/ShishirPatil/poet) [Paper](https://arxiv.org/abs/2207.07697) [Author's Website](https://shishirpatil.github.io/poet/)