Annotations not working when toggling on and off

### What happened? When annotations are on and the dashboard is refreshed, the annotations wont disappear when toggling them off. When annotations are off and the dashboard is refreshed, the annotations disappears as expected. https://github.com/user-attachments/assets/bade9988-36bd-428e-ace6-baf407db79b1 It works as expected if I add &scenes=false to the url or adding dashboardScene=false to the [feature_toggles] section. The bug is only on dashboards where the Data source is "--Dashboard--". If the Data source is e.g. "Microsoft SQL Server" it works fine. ### What did you expect to happen? When toggling annotations off I expect them to disappear. When toggling annotations on I expect them to appear. I expect the above to work no matter if the annotations are enabled on dashboard load or not. Note, the issue is the same nomatter if "newVizTooltips" is enabled or not. ### Did this work before? The bug was introduced in Grafana 11.3.1. Latest Grafana version where it worked was 11.3.0. All versions after 11.3.1, including 12.01, contains the bug. ### How do we reproduce it? 1. Toggle annotation on. 2. Refresh dashboard. 3. Toggle annotation off. Problem: Some annotations are not turned off. ### Is the bug inside a dashboard panel? _No response_ ### Environment (with versions)? ### Grafana platform? A downloaded binary ### Datasource(s)? _No response_