fe05b700dcThe streaming loop used response.iter_text() with timeout=None, which blocks until the next chunk arrives from llama-server. On large models like Qwen3.5-27B where each token takes seconds, pressing Stop in the UI would not take effect until the next token was produced. Fix by using a 0.5s read timeout and a new _iter_text_cancellable() helper that checks cancel_event between timeout windows and explicitly closes the response when cancelled. Applied to both the regular chat completion and tool-calling streaming paths. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| audio_codecs.py | ||
| defaults.py | ||
| inference.py | ||
| llama_cpp.py | ||
| orchestrator.py | ||
| tools.py | ||
| worker.py | ||