CUDA error: out of memory - Phi-3 Mini 128k prompted with 20k+ tokens on 4GB GPU

### What is the issue? I get a CUDA out of memory error when sending large prompt (about 20k+ tokens) to Phi-3 Mini 128k model on laptop with Nvidia A2000 4GB RAM. At first about 3.3GB GPU RAM and 8GB CPU RAM is used by ollama, then the GPU ram usage slowly rises (3.4, 3.5GB etc.) and after about a minute it throws the error when GPU ram is exhaused probably (3.9GB is latest in task manager). The inference does not return any token (as answer) before crashing. Attaching server log. Using on Win11 + Ollama 0.1.42 + VS Code (1.90.0) + Continue plugin (v0.8.40). The expected behavior would be not crashing and maybe rellocating the memory somehow so that GPU memory does not get exhausted. I want to disable GPU usage in ollama (to test for CPU inference only - I have 64GB RAM) but I am not able to find out how to turn the GPU off (even though I saw there is a command for it recently - am not able to find it again). Continue settings log: ``` Settings: contextLength: 24000 maxTokens: 4000 model: phi3:3.8-mini-128k-instruct-q4_0 stop: <|end|>,<|user|>,<|assistant|> log: undefined ``` The memory error: ``` CUDA error: out of memory current device: 0, in function alloc at C:\a\ollama\ollama\llm\llama.cpp\ggml-cuda.cu:375 cuMemSetAccess(pool_addr + pool_size, reserve_size, &access, 1) GGML_ASSERT: C:\a\ollama\ollama\llm\llama.cpp\ggml-cuda.cu:100: !"CUDA error" ``` Full Ollama server log: ``` time=2024-06-11T20:39:29.457+02:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=3 memory.available="3.2 GiB" memory.required.full="12.7 GiB" memory.required.partial="3.0 GiB" memory.required.kv="8.8 GiB" memory.weights.total="2.0 GiB" memory.weights.repeating="1.9 GiB" memory.weights.nonrepeating="77.1 MiB" memory.graph.full="1.5 GiB" memory.graph.partial="1.5 GiB" time=2024-06-11T20:39:29.470+02:00 level=INFO source=server.go:341 msg="starting llama server" cmd="C:\\Users\\username\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\cuda_v11.3\…