fe05b700dcThe streaming loop used response.iter_text() with timeout=None, which blocks until the next chunk arrives from llama-server. On large models like Qwen3.5-27B where each token takes seconds, pressing Stop in the UI would not take effect until the next token was produced. Fix by using a 0.5s read timeout and a new _iter_text_cancellable() helper that checks cancel_event between timeout windows and explicitly closes the response when cancelled. Applied to both the regular chat completion and tool-calling streaming paths. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| export | ||
| inference | ||
| training | ||
| __init__.py | ||