ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend
History
Roland Tannous 06144affdb fix(studio): close aiter_bytes/aiter_lines explicitly in passthroughs 
The httpcore asyncgen cleanup fix in 5cedd9a5 is incomplete on Python
3.13 + httpcore 1.0.x: it switched to manual client/response lifecycle
but still used anonymous `async for raw_line in resp.aiter_lines():`
patterns in all three streaming paths. Python's async for does NOT
auto-close the iterator on break/return, so the aiter_lines /
aiter_bytes async generator remains alive, reachable only from the
surrounding coroutine frame. Once `_stream()` returns the frame is
GC'd and the orphaned asyncgen is finalized on a LATER GC pass in a
DIFFERENT asyncio task, where httpcore's
HTTP11ConnectionByteStream.aclose() enters anyio.CancelScope.__exit__
with a mismatched task and prints "Exception ignored in: <async
generator>" / "async generator ignored GeneratorExit" / "Attempted
to exit cancel scope in a different task" to the server log.

User observed this on /v1/messages after successful (status 200)
requests, with the traceback pointing at HTTP11ConnectionByteStream
.__aiter__ / .aclose inside httpcore.

Fix: save resp.aiter_lines() / resp.aiter_bytes() as a variable and
explicitly `await iter.aclose()` in the finally block BEFORE
resp.aclose() / client.aclose(). This closes the asyncgen inside the
current task's event loop, so the internal httpcore byte stream is
cleaned up before Python's asyncgen GC hook has anything orphaned to
finalize. Each aclose is wrapped in try/except Exception so nested
anyio cleanup noise can't bubble out.

Applied to all three streaming passthrough paths:
- _anthropic_passthrough_stream (/v1/messages client-side tool path)
- _openai_passthrough_stream (/v1/chat/completions client-side tool
  path, new in this PR)
- openai_completions (/v1/completions bytes proxy from PR #4956)
2026-04-15 18:30:23 +04:00
..
assets Add Gemma 4 model sampling defaults (#4838) 2026-04-03 13:57:15 -07:00
auth Studio: Expose openai and anthropic compatible external API end points (#4956) 2026-04-13 21:08:11 +04:00
core fix(studio): forward OpenAI tools/tool_choice to llama-server (#4999) 2026-04-15 17:18:35 +04:00
loggers Final cleanup 2026-03-12 18:28:04 +00:00
models fix(studio): forward OpenAI tools/tool_choice to llama-server (#4999) 2026-04-15 17:18:35 +04:00
plugins Bump Data Designer to 0.5.4 (removes litellm dependency) (#4569) 2026-03-25 02:01:43 -07:00
requirements studio: pin peft to 0.18.1 to fix export subprocess issues (#5015) 2026-04-14 20:16:30 +04:00
routes fix(studio): close aiter_bytes/aiter_lines explicitly in passthroughs 2026-04-15 18:30:23 +04:00
state Final cleanup 2026-03-12 18:28:04 +00:00
storage feat: custom scan folders for GGUF model discovery (#4723) 2026-03-31 06:40:31 -07:00
tests fix(studio): surface httpx transport errors from OpenAI passthrough 2026-04-15 17:59:36 +04:00
utils Studio: support GGUF variant selection for non-suffixed repos (#5023) 2026-04-15 15:32:01 +04:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Fix Studio crash on Anaconda/conda-forge Python (#4484) 2026-03-22 05:36:55 -07:00
colab.py Fix/studio colab button message: Add fallback message for Colab Studio button when proxy URL fails (#4866) 2026-04-05 21:57:45 -07:00
main.py studio: fix api-keys access + refresh (#5005) 2026-04-13 23:48:51 +04:00
run.py Studio: Expose openai and anthropic compatible external API end points (#4956) 2026-04-13 21:08:11 +04:00
startup_banner.py studio: unify Windows installer/setup logging style, verbosity controls, and startup messaging (#4651) 2026-03-30 00:53:23 -07:00