Structured Output doesn't work with ChatOpenAI via proxy LiteLLM

### Checked other resources - [x] This is a bug, not a usage question. - [x] I added a clear and descriptive title that summarizes this issue. - [x] I used the GitHub search to find a similar question and didn't find it. - [x] I am sure that this is a bug in LangChain rather than my code. - [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package). - [x] This is not related to the langchain-community package. - [x] I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS. ### Package (Required) - [x] langchain - [x] langchain-openai - [ ] langchain-anthropic - [ ] langchain-classic - [ ] langchain-core - [ ] langchain-model-profiles - [ ] langchain-tests - [ ] langchain-text-splitters - [ ] langchain-chroma - [ ] langchain-deepseek - [ ] langchain-exa - [ ] langchain-fireworks - [ ] langchain-groq - [ ] langchain-huggingface - [ ] langchain-mistralai - [ ] langchain-nomic - [ ] langchain-ollama - [ ] langchain-perplexity - [ ] langchain-prompty - [ ] langchain-qdrant - [ ] langchain-xai - [ ] Other / not sure / general ### Reproduction Steps / Example Code (Python) ```python from langchain_openai import ChatOpenAI from langchain.agents import create_agent from langchain.output_parsers import ToolStrategy # Create ChatOpenAI instance pointing to LiteLLM proxy llm = ChatOpenAI( api_key="dummy", base_url="http://<litellm-proxy>", model="<model-id>", ) tools = [... any tools] agent = create_agent( llm, tools, response_format=ToolStrategy(QueryOutput), ) response = agent.invoke({"messages": [HumanMessage(content="What is the capital of France?")}}) ``` ### Error Message and Stack Trace (if applicable) ```shell - No errors , this is the problem, the output message just looks like this without the structued response {'messages': [HumanMessage(content='What is the capital of France? Use the Query Response format for your answer {\n "query": "W…