status code 500 for cannot delete with provided provenance '', needs 'api'

### What happened? From our log we see error message: cannot delete with provided provenance '', needs 'api' for PUT /api/v1/provisioning/alert-rules/c171e2a1-df51-4fb0-a55b-bee351b17854 The status code is 500. ### What did you expect to happen? If this is caused by user error, 500 is not an appropriate status code. ### Did this work before? current version 9.5.6 ### How do we reproduce it? Cannot see from log ### Is the bug inside a dashboard panel? _No response_ ### Environment (with versions)? Grafana: OS: Browser: ### Grafana platform? None ### Datasource(s)? _No response_