Inconsistent Embedding Results with Non-Power-of-Two Context Sizes

### What is the issue? When using different context sizes (`num_ctx`) with the Ollama embedding model, I noticed big differences in the cosine similarity of the embeddings. Specifically, when I set the context size to a non-power-of-two (like 513), the similarity scores drop significantly compared to powers of two (like 512 or 1024). This suggests that the model might be optimized for powers of two, leading to inconsistent results with other values. In contrast, other embedding providers like FastEmbed and Sentence Transformers produce stable results even with context sizes like `2^x + 1` (e.g., 513). The similarity between FastEmbed and Sentence Transformers embeddings is nearly perfect, regardless of context size, indicating that this issue seems specific to Ollama. ### Steps to Reproduce 1. Run the code below to generate embeddings with Ollama using different context sizes (512, 513, and 1024). 2. Compare the cosine similarity of these embeddings with those from FastEmbed and Sentence Transformers. 3. Observe that Ollama’s similarity scores vary a lot with non-power-of-two context sizes, while FastEmbed and Sentence Transformers stay consistent. ### Code ```python from ollama import Client from fastembed import TextEmbedding from sentence_transformers import SentenceTransformer import numpy as np target_data = """Text data, should be something big.""" fe_nomic = TextEmbedding(model_name="nomic-ai/nomic-embed-text-v1.5", cache_dir="fastembed_cache") model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True) ollama = Client(host='http://localhost:11434') ollama512 = ollama.embed( model="nomic-embed-text:v1.5", truncate=True, options={ "num_ctx": 512 }, input=target_data ) ollama513 = ollama.embed( model="nomic-embed-text:v1.5", truncate=True, options={ "num_ctx": 513 }, input=target_data ) ollama1024 = ollama.embed( model="nomic-embed-text:v1.5", truncate=True, options={ …