Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

PSA: DO NOT USE OLLAMA FOR TESTING.

Ollama silently (!!!) drops messages if the context window is exceeded (instead of, you know, just erroring? who in the world made this decision).

The workaround until now was to (not use ollama or) make sure to only send a single message. But now they seem to silently truncate single messages as well, instead of erroring! (this explains the sibling comment where a user could not reproduce the results locally).

Use LM Studio, llama.cpp, openrouter or anything else, but stay away from ollama!



I looked around to get confirmation, and I did find some related issues. Seems like it works properly when context is defined explicitly. There also appears to be a warning logged about "truncating input prompt", so it isn't an entirely silent failure. https://github.com/ollama/ollama/issues/2653 + https://github.com/ollama/ollama/issues/4967 + https://github.com/ollama/ollama/issues/7043 + https://github.com/ollama/ollama/issues/8144




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: