ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/studio/backend
History
Roland Tannous 16d5fbda71 Fix httpcore asyncgen cleanup in /v1/messages and /v1/completions 
The earlier fix in 985e92a9 was incomplete: it closed aiter_lines()
explicitly but still used `async with httpx.AsyncClient()` /
`async with client.stream()` inside the generator. When the generator
is orphaned (e.g. client disconnects mid-stream and Starlette drops
the StreamingResponse iterator without explicitly calling aclose()),
Python's asyncgen finalizer runs the cleanup in a DIFFERENT task than
the one that originally entered the httpx context managers. The
`async with` exits then trigger httpcore's HTTP11ConnectionByteStream
.aclose(), which enters anyio.CancelScope.__exit__ with a mismatched
task and raises RuntimeError("Attempted to exit cancel scope in a
different task"). That error escapes any user-owned try/except
because it happens during GC finalization.

Replace `async with` with manual client/response lifecycle in both
/v1/messages passthrough and /v1/completions proxy. Close the
response and client in a finally block wrapped in
`try: ... except Exception: pass`. This suppresses RuntimeError (and
other Exception subclasses) from the anyio cleanup noise while
letting GeneratorExit (a BaseException, not Exception) propagate
cleanly so the generator terminates as Python expects.

Traceback observed in user report:
  File ".../httpcore/_async/connection_pool.py", line 404, in __aiter__
      yield part
  RuntimeError: async generator ignored GeneratorExit
...
  File ".../anyio/_backends/_asyncio.py", line 455, in __exit__
      raise RuntimeError(
  RuntimeError: Attempted to exit cancel scope in a different task
2026-04-13 11:43:56 +04:00
..
assets Add Gemma 4 model sampling defaults (#4838) 2026-04-03 13:57:15 -07:00
auth [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-04-10 13:33:51 +00:00
core Split /v1/messages into server-side and client-side tool paths 2026-04-12 14:52:42 +04:00
loggers Final cleanup 2026-03-12 18:28:04 +00:00
models Wire min_p / repetition_penalty / presence_penalty through /v1/messages 2026-04-12 23:25:54 +04:00
plugins Bump Data Designer to 0.5.4 (removes litellm dependency) (#4569) 2026-03-25 02:01:43 -07:00
requirements Pin transformers and huggingface-hub in main Studio venv (#4788) 2026-04-02 09:21:30 -07:00
routes Fix httpcore asyncgen cleanup in /v1/messages and /v1/completions 2026-04-13 11:43:56 +04:00
state Final cleanup 2026-03-12 18:28:04 +00:00
storage feat: custom scan folders for GGUF model discovery (#4723) 2026-03-31 06:40:31 -07:00
tests Rename test_studio_run.py -> test_studio_api.py 2026-04-13 09:31:21 +04:00
utils Add AMD ROCm/HIP support across installer and hardware detection (#4720) 2026-04-10 01:56:12 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Fix Studio crash on Anaconda/conda-forge Python (#4484) 2026-03-22 05:36:55 -07:00
colab.py Fix/studio colab button message: Add fallback message for Colab Studio button when proxy URL fails (#4866) 2026-04-05 21:57:45 -07:00
main.py Add AMD ROCm/HIP support across installer and hardware detection (#4720) 2026-04-10 01:56:12 -07:00
run.py Add /v1/completions, /v1/embeddings, /v1/responses endpoints and --parallel support 2026-04-10 14:44:22 +00:00
startup_banner.py studio: unify Windows installer/setup logging style, verbosity controls, and startup messaging (#4651) 2026-03-30 00:53:23 -07:00