Track for flaky batching tests

### System Info We added a test for comparing model outputs when the input is batched or not batched in [this PR](https://github.com/huggingface/transformers/pull/29297). The assumption is that batching should not have impact on the results, disregarding numerical precision errros. Currently, all the models pass the test, but there are 3 flaky tests. This issue is to keep track of them. The models that are flaky: MobileNetV2, DPT hybrid, VIT hybrid. I cannot give any more information except that it is flaky, so it needs more exploration. ### Who can help? @amyeroberts for tracking ### Information - [ ] The official example scripts - [ ] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction NA ### Expected behavior NA