Roland Tannous 5cedd9a5b2 Fix httpcore asyncgen cleanup in /v1/messages and /v1/completions ... The earlier fix in 985e92a9 was incomplete: it closed aiter_lines() explicitly but still used `async with httpx.AsyncClient()` / `async with client.stream()` inside the generator. When the generator is orphaned (e.g. client disconnects mid-stream and Starlette drops the StreamingResponse iterator without explicitly calling aclose()), Python's asyncgen finalizer runs the cleanup in a DIFFERENT task than the one that originally entered the httpx context managers. The `async with` exits then trigger httpcore's HTTP11ConnectionByteStream .aclose(), which enters anyio.CancelScope.__exit__ with a mismatched task and raises RuntimeError("Attempted to exit cancel scope in a different task"). That error escapes any user-owned try/except because it happens during GC finalization. Replace `async with` with manual client/response lifecycle in both /v1/messages passthrough and /v1/completions proxy. Close the response and client in a finally block wrapped in `try: ... except Exception: pass`. This suppresses RuntimeError (and other Exception subclasses) from the anyio cleanup noise while letting GeneratorExit (a BaseException, not Exception) propagate cleanly so the generator terminates as Python expects. Traceback observed in user report: File ".../httpcore/_async/connection_pool.py", line 404, in __aiter__ yield part RuntimeError: async generator ignored GeneratorExit ... File ".../anyio/_backends/_asyncio.py", line 455, in __exit__ raise RuntimeError( RuntimeError: Attempted to exit cancel scope in a different task		2026-04-13 12:09:46 +04:00
..
data_recipe	Let recipes use the model loaded in Chat (#4840)	2026-04-08 03:48:22 -07:00
.gitkeep	root studio folder	2026-02-02 09:13:49 +00:00
__init__.py	feat(studio): training history persistence and past runs viewer (#4501)	2026-03-25 00:58:55 -07:00
auth.py	Studio: add API key authentication for programmatic access	2026-04-10 13:33:14 +00:00
datasets.py	fix: subprocess crash during map operation on Windows (#4507)	2026-03-22 05:21:09 -07:00
export.py	Final cleanup	2026-03-12 18:28:04 +00:00
inference.py	Fix httpcore asyncgen cleanup in /v1/messages and /v1/completions	2026-04-13 12:09:46 +04:00
models.py	fix(studio): custom folder scan fails to find GGUF variants when pointing directly at a model directory (#4860)	2026-04-06 08:31:07 -07:00
training.py	[Studio] multi gpu finetuning/inference via "balanced_low0/sequential" device_map (#4602)	2026-03-30 02:33:15 -07:00
training_history.py	feat(studio): training history persistence and past runs viewer (#4501)	2026-03-25 00:58:55 -07:00