Add support for prefix_allowed_tokens_fn to maintain a state throughout decoding

### Feature request Add an optional argument in `prefix_allow_tokens_fn` to allow for state to maintained throughout decoding or add a stateful alternative to `prefix_allowed_tokens_fn`. ### Motivation `prefix_allowed_tokens_fn` is great but has one major downfall which is that you cannot maintain a state throughout decoding. This is inefficient because at each step you must go through your past `inputIds`, build up your current "state", and then figure out which tokens are allowed to appear next. Instead, there should be a class we can subclass that gets passed the next token ID at each step of decoding (`Constraint` does not achieve this as `update` does not get every token ID). For example if you are trying to create a function to output json format (https://gist.github.com/BorisTheBrave/969f303a082c9da1916d04ee1eb04452), then you could track where you currently on in the json as each token ID is being received instead of going through everything on each new token. ### Your contribution Unfortunately can't make a PR.