unsloth/unsloth_cli/commands/studio.py
Daniel Han c608649552
feat(studio): run chats in parallel in the Chat tab (#7455)
* feat(studio): run chats in parallel in the Chat tab

New Chat used to cancel whatever the current conversation was generating.
It now leaves it running, like switching to the Train or Export tab: the
sidebar shows which chats are still going, and Stop is per conversation.

Plain `unsloth studio` launched llama-server with one decode slot, so the
admission queue serialised every chat regardless of what the UI did. Both
entry points now default to the same slot count as `unsloth studio run`.

A model swap still ends every running chat, since they all decode on one
llama-server. /load and /unload now refuse with 409 and name those chats
unless the caller passes force_cancel_active, and the UI asks first.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): scope the composer tool badge to its own conversation

The green "Running Python: ..." badge above the composer read a single
global store value, so one chat's tool call showed above every other
chat's composer, including a brand-new empty one. Its elapsed counter
also restarted at 0 on every thread switch, and a run ending anywhere
cleared the badge everywhere.

Key the status by thread and store the moment it started, so each
conversation shows only its own tool call and the counter resumes rather
than restarts. Also adds a test that every conversation gets its own
tool sandbox directory, which parallel tool calls depend on.

* Fix stalled tool calls while awaiting approval for PR #7455

Three problems, all from the approval prompt behaving as though only one
chat could ever run.

Arguments were not streamed for a gated call, so the chat stayed blank for
as long as the model took to write the payload, which for a large file is
minutes. Nothing runs before the decision either way, and the code is what
is being approved, so python and terminal now stream their card while
gated. render_html stays suppressed: its card renders the payload.

The status read "Running ..." with a climbing timer while the call had not
started. It now reports that it is waiting for approval, then switches to
running once allowed.

The admission lease was held across the wait, so four unanswered prompts
held all four decode slots and no other chat could start while llama-server
sat idle. A parked run keeps its lease but no longer counts against
capacity.

Measured with four prompts left open: every gated call streamed its code,
none reported running, and a fresh chat answered in 0.4s where it
previously waited 290s and never did.

* Fix duplicated and truncated tool cards for PR #7455

A gated tool call rendered two cards: the provisional one that streams the
arguments, plus a second one keyed by the approval id. Only the second ever
got its tool_end, so the first spun "Running" for the rest of the chat.
Reuse the open part when the approval prompt arrives.

The terminal card also showed nothing but a 60-char trigger label, so a long
heredoc read as no progress at all. It now renders the command the same way
the Python card renders its script, and neither is capped at 10k chars.

Both cells moved inside the collapsible, so one chevron hides the code with
the output and Copy / Download exist only while the card is open. A card
parked on the prompt says so instead of counting up "Running".

* Fix review findings on the parallel-chat gate for PR #7455

Backend:
- /unload rechecks active generations under the lifecycle gate, like /load,
  and lets its 409 through the catch-all instead of rewriting it as a 500.
- /load gates only once _load_model_impl has decided this is a real reload,
  so an Apply on the already-loaded model no longer refuses, and the retry it
  asks for no longer cancels every chat before returning already_loaded.
- The direct /v1/responses stream registers in the cancel registry, so a
  non-forced unload can no longer tear llama-server down under it.
- run_server defaults to the same slot count as the CLI. colab.py calls it
  without the argument, so Colab was still serialising every chat.

Frontend:
- Cancelling a backgrounded chat aborts its own request rather than only
  posting a cancel id, which is the only thing that ends an external-provider
  or audio run.
- The model-swap dialog counts local runs only, and falls back to the backend
  when this tab's map is empty, so a reload or a second tab still gets asked.
- Context usage and the diffusion canvas are scoped to the chat that produced
  them; a compare row reads activity from its member threads.

Tests:
- The extracted-source cancel harnesses supply the active-generations module,
  which the tracked-cancel class now depends on.

* Fix the swap confirmation scope and cancel timing for PR #7455

A forced load cancelled every chat before the model identifier, GPU selection,
training coexistence and download checks had run, so a load that then failed
those checks stopped the chats and replaced nothing. The refusal still happens
early, but the destructive cancel now sits immediately before the teardown it
is paying for, and rechecks under the gate like /unload does.

The swap dialog only reconciled with the backend when this tab looked idle, so
one local chat was enough to hide a second tab's runs. Confirming then sent
force_cancel_active, which cancels every backend run, including the ones the
dialog never mentioned. The backend snapshot is now merged in every time, so
the dialog names what will actually stop. External-provider runs are never
registered there, so the union stays local-only.

Also drops the active-generations docstring claim about restoring sidebar
spinners, which nothing consumes.

* Defer destructive cancels and track every local stream for PR #7455

/unload cancelled the running chats before it had resolved that it unloads
anything. A stale model_path, which a second tab produces routinely, killed
every chat and then no-opped, leaving the resident model up. It now refuses
early and cancels only at each teardown, matching /load.

The swap dialog also stopped every chat locally the moment the user confirmed,
which threw away the two-phase backend behaviour: a load that then failed
identifier resolution, GPU validation or the training guard had already
truncated the replies. The backend now owns the cancel.

Three local streams decoded on llama-server without registering, so a
non-forced unload counted zero generations and tore the server down mid
response: /v1/completions streaming, and the plain and server-tool Anthropic
streams, the first of which is the default /v1/messages path. Note this makes
a non-forced load return 409 during those runs rather than draining quietly,
the same trade the /v1/responses fix made.

The safetensors tool loop still announced a gated call as running while it
waited on a human; only the GGUF loop had been fixed. A source-level parity
test now pins both.

Also drops stopAllChatThreads, which has no callers left.

* Studio: close three load/unload gate races found in review

Re-check the in-flight load guard after the stop-running-chats confirm.
The confirm always GETs active-generations before its zero-running
early-out, so the guard no longer sits atomically ahead of the
reservation and two picks in that window both reached performLoad over
the same refs. ejectModel had the same shape and gets the same re-check.

Reject a sidecar swap immediately before the forced cancel in both load
branches. The previous check was back at the top of preflight, so an
install reserving during identifier resolution, the tier probe, the
training guard or the download check made the post-drain recheck 409 a
load whose chats had already been stopped.

Enter the Anthropic passthrough's cancel tracker inside its body
generator. It was entered eagerly and returned through
_sse_streaming_response, which sets no unstarted_cleanup, so a response
whose body never started left the run registered forever and 409'd every
later non-forced load and unload.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim comments across the files this PR touches

Tightens the comments and doc blocks in the backend, CLI, tests and frontend
files changed by this PR: collapses multi-line explanations to a single line
where they still read clearly, and drops the ones the code already says.

No code changes, verified by an AST comparison against the previous commit.

* Studio: defer the destructive cancel and close two gate gaps

Move the forced cancel behind every check that can still reject a swap.
The drain now runs first with the runs it is about to cancel discounted,
so it waits only for inference the cancel cannot end, then the sidecar
check decides, then the cancel fires, then a second drain lets those runs
unwind before teardown. A sidecar install reserving during the drain no
longer 409s a load whose chats have already been stopped.

Track the non-streaming /v1/completions proxy. It was the last local
decode path missing from active_generations, so an unload, which runs no
drain, tore llama-server down under it and force_cancel_active could not
signal it. It now uses the same tracked cancel event and dedicated client
as the OpenAI pass-through.

Skip the client's preliminary unload while chats are generating and let
/load evict at its own post-preflight point instead. Forwarding
force_cancel_active there truncated replies before identifier
resolution, the GPU and training guards and the download check had run.

Keep per-thread context usage so returning to a chat whose background run
finished restores its bar instead of leaving it blank until the next turn.

Make the running-flag clear run-specific. Every run without a resolved
thread id shares the "__default" key, so concurrent compare panes could
clear each other's flag and strand a live stop handle.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: register the embeddings proxy with the swap gate

/v1/embeddings proxied straight through the pooled client with no tracked
cancel event, so it never appeared in active_generations. /unload runs no
idle drain, so a concurrent non-forced unload counted zero generations and
killed llama-server mid-request, and force_cancel_active had no event to
signal. Mirrors the completions proxy: tracked event, dedicated unpooled
client closed by a cancel/disconnect watcher, unregister in a nested
finally so a close failure cannot leave a phantom generation behind.

* Trim comments on the newest changes in this PR

Comments only, no code changes: shorten the ones added by the load-gate
ordering, embeddings and per-thread usage work down to the same density as
the rest of the diff.

* Studio: register the legacy generate stream with the swap gate

/generate/stream built a cancel event but never entered the tracker, so it
was invisible to active_generations. Being in the keep-warm middleware's
inference suffixes only covers /load, which drains; /unload does not, so a
non-forced unload passed the 409 gate and then blocked on the standard
backend's generation lock, and a forced swap had no event to signal.
Registered inside the body generator under a nested finally so a teardown
failure cannot skip the unregister.

The AST contract test asserted the cleanup finally by overwriting its flag
per Try node, so a nested try made the last one win. Accumulate instead,
which is what the existence claim meant.

* Studio: three more swap-gate gaps found in review

Register /audio/generate with the gate. TTS holds the model for the whole
request and /unload runs no drain, so unregistered a non-forced swap counted
zero generations and tore the model down mid-generation; the orchestrator
path only waits 15s for the generation lock, which real TTS exceeds. No
cancel keys: no backend takes a cancel_event for audio, so the event has no
observer and a forced swap still cannot interrupt audio already in flight.

Thread the tracked cancel event into the /v1/responses admission wait. It
was the only admission caller passing None, so a queued run could not be
reached by cancel_all() and a plain /inference/cancel could not stop it at
all. Same omission fixed at the upstream send there and on /v1/completions.

Let an unforced unload of a stale model path reach the no-op check. Before
this PR that request returned 200 and did nothing; the new gate refused it
with 409 for a request that reaches no teardown branch. Gate both refusal
passes on the disjunction of the route's own teardown conditions, including
not is_loaded, so a mid-load GGUF still refuses.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: register the remaining non-streaming decode paths

stream defaults to false on all three of these, so they are the ordinary
shape of their routes, and each holds a local backend for the whole
request. /unload runs no idle drain, so with no registry entry a non-forced
swap counted zero generations and tore the backend down mid-request instead
of returning 409, and a forced one had no event to signal.

Non-streaming /v1/messages: all three helpers ran with an empty registry,
since only the streaming siblings were tracked. Registered at the call site
because the pass-through takes no cancel_event of its own, and with no
cancel keys, matching those siblings.

Non-streaming standard chat and audio-input chat: the trackers in this route
sit inside their `if payload.stream:` arms, so neither else branch was
covered. The GGUF sibling already registers its own non-streaming branch.

Each exit is in a finally on the branch's existing try, so the except arms
are covered too: a leaked entry 409s every later swap until restart.

* Studio: tighten the swap-gate comments

Comment-only pass over the newest swap-gate registrations: collapse the multi-line rationales in /unload, the legacy generate stream, audio generation and the non-streaming chat branches, and the matching test preambles, to the shortest form that still carries the reason. No code changes.

* Studio: stop the reselect dialog promising a stop that never happens

Picking an external provider leaves the local model resident and stops the
status poll mirroring it, so reselecting that model showed the stop-chats
dialog, and /load then answered already_loaded ahead of its cancel hook.
Confirmed with the live backend: the same pick with force_cancel_active set
still returned already_loaded and the chat kept streaming. Not stopping
those chats is right, since the load never interrupts them, so remove the
prompt rather than honour it. Blanket-skipping is unsafe, because the same
id and variant with one sampling setting changed is a real reload and 409s,
so the branch only fires when a status fetch confirms the resident
checkpoint and variant match, and then adopts it without calling /load.

Redact native model paths from the active-generations response. Registering
/generate/stream recorded backend.active_model_name verbatim, which is an
absolute path for a native local model, and this route is the only place
that serialises it. Redacting at the response covers every tracker rather
than the one that surfaced it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep hydrated context usage in the per-thread map

The history loader restores a saved conversation's usage through
setContextUsage only, and it runs once per mount, so switching away and
back left the bar blank for a hydrated chat even after the per-thread map
landed. setContextUsage now writes the value through to the visible
thread's own entry and clears that entry when passed null, which covers
both hydration call sites and any future writer.

* Studio: unblock load cancellation and share unresolved thread keys

Run the two stop-loading fast paths ahead of the unload route's pre-gate
refusal. _unload_may_evict returns True for exactly the model being
cancelled, so the refusal was blocking the branch that cancels a load which
has replaced nothing and can interrupt no chat. The client made that
unrecoverable: cancelLoading sends the unload without force, drops the
result, and its abort never reaches /load, which takes no signal, so the
load ran on and could later cancel those chats and swap the model. Nothing
else is exempted; an unload that would tear down a serving model matches
neither fast path and still 409s. The comment claiming the client lets that
409 surface is corrected, since it discards it.

Hold every owner behind a shared thread key. Runs with no resolved thread id
share "__default" (concurrent compare panes, since startCompare clears
activeThreadId), so a single owner slot let a second run replace the first's
token and then delete the shared entry while it was still generating, and
the server-cancel map lost the older handle the same way. Both now hold a
list, the running and local flags survive until the last owner clears, and
stopChatThread stops every handle under the key.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: carry a confirmed swap into the sidecar install, key restored usage by thread

Picking a model that needs a newer transformers while chats generate raised the
"stop N chats" prompt, but the answer never reached the install that runs before
the load: /install-latest-transformers refused on those same chats and took no
force flag, so Retry hit the same 409 and nothing in the flow stopped them.

Carry force_cancel_active through the consent dialog into the installer. Only
the pre-gate fast path is skipped: the recheck under the lifecycle gate still
has to pass, so an unconfirmed caller is refused as before. The cancel runs last
inside the gate, after every check that can still reject the install, and the
drain behind it is bounded since it holds the gate and the sidecar reservation.

Also key restored context usage by the thread the loader read. history.load()
captures remoteId before two awaited round trips, so a switch inside that window
filed one thread's usage under another and setActiveThreadId kept re-applying it.

Preserve sibling owners when a run key is cleared without an owner: the image
rejection gate now uses its own token, and the reducer leaves owned runs alone.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: bound the post-cancel drains, and make cancellation reach the paths that ignored it

A forced swap cancels the chats it interrupts, then waits for them to unwind.
That wait had no deadline while holding the lifecycle gate, and TTS on the
subprocess backend observes no cancel event at all, so one audio generation
could pin every load, unload and new request for its whole duration. Bound both
post-cancel drains. Pre-cancel drains stay unbounded: the swap can still be
refused there, so shortening them would weaken what they protect.

/unload had the opposite problem and no drain at all, cancelling and tearing
down on the next line, which turned a clean stream end into a dropped
connection. Give it the same bounded wait, gated on the cancel having cancelled
something so an idle Eject pays nothing.

Make the cancel actually land where it can. GGUF TTS now takes a cancel_event
and a watcher closes its client to break the blocking POST. The Anthropic
non-streaming pass-through did the same thing the completions and embeddings
paths used to: register with the gate, then run both POSTs on the pooled client
that cannot be closed. It now uses a per-request client like they do.

Also: park and unpark the admission queue the reservation actually holds, since
queues are keyed by base_url and a reload mints a new port; key tool output by
remoteId on both sides, so the first turn of a New Chat stops writing under one
key and reading another; and give tool status a run owner, so a finishing run
cannot blank the badge a concurrent one is still showing.

Clamp --parallel to 1 on a llama-server without --kv-unified. The new default of
4 would otherwise split -c four ways on such a build, quartering the context
window for a feature it cannot serve.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: scope a chat's Stop to its own generation, and clear the way before a confirmed install

Safetensors generation is serialized on _gen_lock and the worker has a single
cancel event, so a chat still queued on that lock owns no generation. Its Stop
handler called reset_generation_state() anyway, which set the shared event and
ended whichever conversation was actually running. Parallel chats is what makes
that reachable.

_generate_inner now records its cancel_event as the current holder once it takes
the lock, and reset_generation_state drops a reset from anyone else. Every route
call site passes its own request event. A reset with no event stays global, so
unload and model switch cannot leave a generation alive, and a reset while
nothing runs still resets, so an error path before generation is not a no-op.
The other two backends take the argument too, or the standard one raises
TypeError on every cancel.

The sidecar install had the mirror of the /load ordering problem: it cancelled
the chats first and drained second, so an unrelated counted request the cancel
cannot reach (a count_tokens, say) was still there for the recheck, which then
refused an install that had already stopped every chat for nothing. Drain the
unreachable remainder first, discounting the registered chats, then cancel.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: close the windows the previous round's fixes left open

Three follow-ups, two of them holes in the fixes just before them.

The worker claim went in after _send_cmd, so the command was already running
unclaimed and a queued chat's Stop in that window still reset it. Claim first,
with the send inside the same try, so a failed send releases it too.

Tool status kept one entry per key with an owner. That stops a foreign clear but
not an overwrite: under the shared unresolved-thread key the second run replaced
the first's entry, and its own clear then removed the only one while the first
tool was still running. Keep per-run entries and render the newest.

/unload gated its drain on having cancelled something, so a request that passed
the keep-warm middleware but had not reached its tracker yet was invisible to it
and the teardown landed on an already-admitted request. Drain on the middleware
count instead, which covers that window as well as the cancelled runs, then
re-cancel whatever registered while waiting. Bounded, not a refusal: an unload is
deliberate, and on expiry it proceeds exactly as before.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the parallel-chats comments to their reasons

Compress the multi-line rationales added by this branch into shorter forms and drop
restatements of the code below them. The reasons behind the drain bounds, the deferred
cancel, the per-request generation ownership and the thread-scoped tool and usage keys
are kept, just said in fewer lines.

* Studio: own the worker per generation, and make a resumed chat requeue for its slot

Ownership was a single lock holder, so dispatched runs (compare mode bypasses
_gen_lock by design) never claimed it and the guard fell straight through to the
global reset: a Stop on one of them ended its siblings. Track the generations
actually running instead, claimed before the send and released in the same
finally on both paths. A reset still proceeds when nothing is running, so an
error path ahead of generation is not swallowed.

park() hands the freed slot to a waiter, so a chat resuming from a tool approval
could take it back while that waiter was still decoding, putting two holders on
a one-slot server and sending the resumed tool loop past the admission limit.
unpark_async waits for room; the plain unpark stays for a holder tearing down,
which will not decode again.

Audio only observed its cancel event on a forced swap. An explicit Stop just
aborts the fetch, and this route has no cancel id, so llama-server ran on to the
request timeout after the chat reported it stopped. Watch the disconnect.

Also read tool status by remoteId, matching the key the adapter writes and the
fix already made for tool output, and stop an unresolved run from writing its
usage into whichever conversation the user moved to.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: let only the generation the worker is running speak for it, and hold a slot for a resuming chat

The ownership list recorded admission, but the subprocess runs generations one
at a time, so a dispatched request queued behind another counted as an owner and
its Stop signalled the shared cancel event, ending the request that was actually
running. Keep admission for release bookkeeping and gate ownership on execution
instead, promoted when the worker first answers that request. Nothing executing
still permits a reset, so an error path ahead of generation is not swallowed.

The worker has one cancel event and no per-request cancellation, so this decides
who may pull the lever rather than making the lever per-request.

A resuming chat also polled for a slot it could never see: release() grants to
the next waiter under the same lock, so later arrivals overtook an approved chat
indefinitely. A pending unpark now reserves the next slot and they queue behind
it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: cover the prefill window, and keep a first turn's tool output readable

Gating worker ownership on execution left the interval between the send and the
first response uncovered: nothing is executing then, and the empty case admitted
anyone, so a queued chat's Stop still ended the one in prefill. Split the empty
case. Nothing claimed at all still permits a reset, so an error path ahead of
generation is not swallowed; claimed but unanswered resolves to the oldest
claim, which is what a FIFO command queue is working on.

Putting both sides of the tool-output scope on remoteId left the first turn of a
New Chat writing under the unresolved scope for its whole life while the readers
recomputed the moment the autosave assigned an id, so the card blanked mid-run.
The readers now fall back to the unresolved scope, which only an unpersisted
first turn can occupy.

* Studio: order the parked approvals, and tie a worker claim to its enqueue

The reservation added for admission fairness was a bare count, so every approved
holder counted against every other: park two chats, approve both, and once the
last decoder released, nothing could ever satisfy the check again. That is a
deadlock where the problem it fixed was only unfairness. Make it a FIFO ticket
so a pending unpark blocks the ones behind it and no others.

_owns_worker reads claim order to decide which request the worker is prefilling,
which only holds if claiming and enqueuing cannot interleave. Hold one lock
across both on the dispatched and the locked path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: follow a first turn's run when its thread gets an id, and key the denoising canvas per chat

A run started before its thread existed filed every handle under "__default". Nothing
moved them once autosave assigned the real id, so the sidebar row showed no spinner and
Stop could not reach the generation, which kept holding a slot.

adoptDefaultThreadRun re-keys the run maps onto the real id from the thread adapter's
initialize(), where the id first exists; anything already filed under that id wins, since
that is a later run. The adapter captures its key once at run start, so it now resolves
the live key per use through runKeyForOwner, looking its own serverCancel up in the owner
map. Without that the migrated entries are stranded and the spinner never clears.

The denoising canvas was one global slot, so two diffusion chats overwrote each other and
the ownership tag then hid the visible preview until that thread emitted again. It is now
activeDiffusionCanvasByThreadId, written and cleared per thread, and the frame no longer
carries a threadId of its own. The bubble reads threadListItem.remoteId, dropping the dead
threadListItem.id arm: the writer tags unstable_threadId, which is exactly remoteId.

Two existing backend tests needed the same treatment. _bare_orchestrator skips __init__, so
it now sets the claim bookkeeping the worker ownership check reads. The Anthropic
passthrough gate test anchored on comment prose that a rewrap had broken; it anchors on the
code instead.

* Studio: hand the worker over cleanly between generations, and stop unresolved runs sharing each other's state

Worker ownership moved off the consumer and onto the dispatcher. Consumers read their
mailbox whenever they get around to it, so a request whose gen_done had been routed still
owned the worker while the next one ran, and a late Stop for it cancelled that one. The
dispatcher is the only place responses arrive in the order the worker produced them: it
now retires a request at its terminal response and promotes the next one, and answering a
request makes it the sole executor, since the subprocess runs one generation at a time.

reserve()'s immediate path ignored the unpark tickets that _grant_waiters_locked already
honours, so a request arriving between a slot freeing and an approved chat's next poll
took it, repeatedly. It applies the same reservation now.

Three places let concurrent first turns share state through the "__default" key. Nothing
links a run filed there to the id its thread later receives, so rather than guess, each
now declines when the key is ambiguous: adoption only re-keys a lone run, the composer
badge only claims a lone status, and the tool-output fallback only applies to a thread
that is still running. That leaves two concurrent first turns where they were before
adoption existed instead of handing one thread the other's handles.

A first turn's usage was never filed, because its key stayed null for the whole run while
autosave moved activeThreadId to the real id, so the context bar went blank after the
first reply. It resolves the adopted key like the cleanup handles do.

Cancelling a forced load left the UI with no model: the previous one stays resident until
/load's teardown, and the cancel path cleared the checkpoint without rolling back. It now
resyncs from the backend, which is right whether or not the load got that far.

The sidecar install drain is weighted 1:4 rather than halved, total unchanged. Only the
second half benefits from patience, and cutting it short refused installs whose chats had
already been stopped for nothing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: give a first turn its real thread id before the run starts

A first turn filed every run handle under a shared unresolved key because
assistant-ui binds unstable_threadId before the thread is persisted. Two of them
overlapping there is unresolvable afterwards, and the last round's migration could
only decline rather than guess, which left neither sidebar row showing its run.

The id is available earlier than I claimed. append() already tracks
threadListItem.initialize() by the user message id, and createPersistedRunAdapter
already awaits that promise before invoking the adapter, so the thread is persisted
by the time the run begins. It was only being discarded: the tracked promise resolved
to void. It now resolves to the assigned id, and the wrapper hands it to the adapter
when assistant-ui had none. An id that is already set is never replaced, since that
would move a running chat's handles out from under the row watching them. The
existing unresolved-key guards stay as a safety net but should no longer carry weight.

The sidebar counted running thread ids rather than rows, so one compare conversation
read as two chats. It folds ids into rows through the same threadIds the row spinner
uses, and still counts a running id that matches no row.

_TrackedCancel always registered kind="chat", so an embeddings or raw completions
request appeared in the model-swap prompt as an unnamed conversation and confirming
cancelled it while calling it a chat. The non-conversation routes now pass their own
kind, and the prompt says "requests" whenever the snapshot is not all chats.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: withhold the shared worker cancel from a request the worker has left

Moving ownership to the dispatcher fixed reset_generation_state, but the token loop
signals the shared worker event directly and did not carry the same rule. A dispatched
consumer runs with mark_started off and can still be draining tokens buffered before
its gen_done was routed, so stopping it there ended whichever request the worker had
started next.

It now signals only when _owns_worker agrees, the same predicate reset_generation_state
uses. The local drain and return are unconditional, since those touch nothing but this
stream. The remaining _cancel_generation callers are deliberately global: subprocess
shutdown, the pre-load kill and unload_model.

* Studio: add the AGPL-3.0 header to the first-turn identity test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: stop the dispatcher and a _gen_lock stream fighting over the response queue

Nothing stopped the dispatcher starting under a _gen_lock generation, so once compare
was opened while an ordinary chat was still streaming, both consumed _resp_queue and
whichever response the dispatcher took without a mailbox was dropped, gen_done included.
That chat truncated or hung. This PR is what makes it reachable, since navigating into
compare no longer ends the chat behind it.

Delaying the dispatcher would serialise compare behind whatever chat happens to be
streaming, so the direct readers get a mailbox instead. _direct_reader returns a reader,
a cancel drain and a release, and files the mailbox under _direct_mailboxes rather than
_mailboxes, which means "compare requests are in flight" to the unload and distributed
paths and must not count an ordinary chat.

Both directions close. The dispatcher finds the direct reader's mailbox instead of
dropping. And this reader can already be blocked on the queue when a compare request's
dispatcher starts, so a response that is not ours goes to its own mailbox rather than
being consumed, which would have corrupted the chat and hung the pane. All three
_gen_lock readers use it, and the cancel drain goes through it too.

The sidebar's return target still picked a raw pane id while the count grouped by row,
and /chat addresses compare with `compare`, not `thread`. It resolves through the same
items now, so a running compare row returns to its pair.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep worker ownership honest across audio, API traffic and a replaced worker

The audio-input send got a mailbox last round but stayed unclaimed, so a compare request
queued behind it looked like the oldest owner and stopping that queued request signalled
the shared event into the audio chat. It claims under the send lock and releases in the
finally, like _generate_inner.

Ownership is keyed on cancel-event identity with nothing tying it to a worker generation,
so a consumer still blocked on its mailbox when the process was replaced stayed recorded
as the executor, and a generation on the fresh worker could not be stopped.
_shutdown_subprocess clears that state once the process is confirmed dead, mailboxes
included: nothing routes to them again, and a stale one reads as compare activity to the
unload path. Not on the survived-SIGKILL path, which keeps its handle on purpose.

The four public /v1/messages trackers were registering as chats. The distinction is a
Studio thread, not the protocol, and those branches already say "No thread_id: public API
surface" while the Studio path passes payload.thread_id separately. They carry their own
kind now, so the swap prompt stops calling an external request a chat.

The swap confirmation still counted raw pane ids, so a compare conversation asked to stop
two chats and listed its title twice. It folds panes onto pairId and lowers the count by
what it collapsed, leaving a first turn the backend can count but not name.

Deep Research set runningByThreadId but registered no server-cancel handle, and that map
is how Stop, archive and delete reach a thread that is no longer active. Leaving the
outgoing thread running is this PR's doing, so the run was left unreachable while its
supervisor kept working against a conversation the user could delete.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten the parallel-chats comments

* Studio: replay a Deep Research stop that arrived before the run existed

The handle is registered before createResearchRun resolves because the thread can be
stopped while that request is in flight, but it had no id to act on and dropped the stop.
The supervisor then followed a run the user had already stopped, archived or deleted.

It latches instead: a stop with no id yet sets a flag, and the adapter replays it against
the id the moment creation returns rather than starting to follow.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix worker ownership on a raced reroute, and the stop-chats prompt

Four review findings on the parallel-chats work, all reproduced first.

- _direct_reader hands a foreign response to its own mailbox, but skipped the
  ownership move the dispatcher makes. A _gen_lock reader already blocked on
  resp_queue can beat the compare dispatcher to that request's first response,
  and the compare consumer opts out of marking, so nothing promoted it: the
  direct request stayed the recorded executor, its late reset cancelled the
  compare generation, and the compare chat's own Stop was ignored.
- A chat stopped while queued on _gen_lock was still claimed and sent once the
  lock freed. Cancellation is only checked on a token, so a long prefill, or a
  generation reaching gen_done without one, occupied the worker after Stop.
  Same hole in the audio-input path, which shares the lock.
- The stop-chats prompt counted generation handles, not conversations. One chat
  holds several while a tool continuation registers its next leg before the
  previous unwinds, so it offered to stop two chats and listed one title.
- Ejecting a model confirms through that dialog, which told the user
  "Unloading the model reloads the model" and offered "Stop and reload".
  Confirming calls /unload and leaves nothing loaded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: name the TTS run's thread so the stop prompt counts it once

The audio branch registers its run locally under the thread key but sent no
thread_id, so the backend tracker filed the same generation under no thread.
The stop-chats prompt then had a named local run and an unnamed backend one and,
since e8e7594 started adding unnamed entries to the named ones, counted a single
TTS chat as two requests. The backend already reads payload.thread_id, so
sending it lines both registries up on the same run.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 04:40:38 -07:00

2946 lines
118 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import importlib.util
import hashlib
import hmac
import json
import os
import platform
import re
import secrets
import sqlite3
import subprocess
import sys
import tempfile
import time
import types
import urllib.error
import urllib.request
from datetime import datetime, timezone
from pathlib import Path
from typing import List, Literal, Optional
import typer
from unsloth_cli import _studio_deps
from unsloth_cli.commands import _password_prompt
studio_app = typer.Typer(help = "Unsloth Studio commands.")
def _enable_verbose_access_logs() -> None:
"""Restore every per-request access log by disabling the burst dedup and the
quiet-poll heartbeat. Inherited by the spawned/re-exec'd server via the env."""
os.environ["UNSLOTH_STUDIO_ACCESS_LOG_DEDUP_MS"] = "0"
os.environ["UNSLOTH_STUDIO_ACCESS_LOG_POLL_DEDUP_MS"] = "0"
# Resolve install root: UNSLOTH_STUDIO_HOME, then STUDIO_HOME alias, then
# sys.prefix inference (so a direct call to <root>/bin/unsloth resolves after
# the installer's env var has expired), then legacy ~/.unsloth/studio.
# UNSLOTH_STUDIO_HOME wins when both env vars are set.
def _looks_like_installer_managed_studio_home(candidate: Path) -> bool:
"""Sentinel check (studio.conf or bin shim) so a dev venv named
unsloth_studio is not misidentified as a custom Unsloth root.
"""
shim_name = "unsloth.exe" if platform.system() == "Windows" else "unsloth"
return (candidate / "share" / "studio.conf").is_file() or (
candidate / "bin" / shim_name
).is_file()
def _resolve_studio_home() -> tuple[Path, bool]:
override = (os.environ.get("UNSLOTH_STUDIO_HOME") or "").strip()
if not override:
override = (os.environ.get("STUDIO_HOME") or "").strip()
if override:
try:
return Path(override).expanduser().resolve(), True
except (OSError, ValueError):
return Path(override).expanduser(), True
try:
prefix = Path(sys.prefix).resolve()
if prefix.name == "unsloth_studio":
inferred = prefix.parent
legacy = (Path.home() / ".unsloth" / "studio").resolve()
if inferred != legacy and _looks_like_installer_managed_studio_home(inferred):
return inferred, True
except (OSError, ValueError):
pass
return Path.home() / ".unsloth" / "studio", False
STUDIO_HOME, _STUDIO_HOME_IS_CUSTOM = _resolve_studio_home()
def _ensure_studio_env_exported() -> None:
"""Re-export UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH only for real
custom roots so subprocesses inherit the right install. Called from each
studio subcommand entry rather than at import time, to avoid leaking env
state into unrelated importers (tests, --help, CLI introspection).
"""
if not _STUDIO_HOME_IS_CUSTOM:
return
# Truthy-check (not setdefault) so a blank UNSLOTH_STUDIO_HOME= does not
# suppress the inferred custom root.
if not os.environ.get("UNSLOTH_STUDIO_HOME"):
os.environ["UNSLOTH_STUDIO_HOME"] = str(STUDIO_HOME)
# When override == legacy default, llama.cpp stays at ~/.unsloth/llama.cpp.
try:
_legacy_studio = (Path.home() / ".unsloth" / "studio").resolve()
_is_legacy = STUDIO_HOME.resolve() == _legacy_studio
except (OSError, ValueError):
_is_legacy = STUDIO_HOME == (Path.home() / ".unsloth" / "studio")
if _is_legacy:
_llama_dir = Path.home() / ".unsloth" / "llama.cpp"
else:
_llama_dir = STUDIO_HOME / "llama.cpp"
if not os.environ.get("UNSLOTH_LLAMA_CPP_PATH"):
os.environ["UNSLOTH_LLAMA_CPP_PATH"] = str(_llama_dir)
BOOTSTRAP_PASSWORD_FILE = ".bootstrap_password"
DESKTOP_SECRET_FILE = ".desktop_secret"
DEFAULT_ADMIN_USERNAME = "unsloth"
DESKTOP_SECRET_PREFIX = "desktop-"
API_KEY_PBKDF2_SALT_KEY = "api_key_pbkdf2_salt"
DESKTOP_SECRET_HASH_KEY = "desktop_secret_hash"
DESKTOP_SECRET_CREATED_AT_KEY = "desktop_secret_created_at"
PBKDF2_ITERATIONS = 100_000
_START_API_KEY_MARKER_ENV = "_UNSLOTH_START_API_KEY_MARKER"
def _consume_start_api_key_marker_env() -> bool:
"""Consume the one-shot readiness marker passed across a Studio re-exec."""
return os.environ.pop(_START_API_KEY_MARKER_ENV, None) == "1"
# __file__ is unsloth_cli/commands/studio.py -- two parents up is the package root
# (either site-packages or the repo root for editable installs).
_PACKAGE_ROOT = Path(__file__).resolve().parent.parent.parent
def _should_hide_windows_subprocesses() -> bool:
"""Hide child console windows only for non-interactive Windows launches."""
if platform.system() != "Windows":
return False
try:
return not sys.stdout.isatty()
except (AttributeError, OSError, ValueError):
return True
def _windows_hidden_subprocess_kwargs() -> dict[str, object]:
"""Return Windows-only Popen kwargs that suppress transient console windows."""
if not _should_hide_windows_subprocesses():
return {}
kwargs: dict[str, object] = {}
create_no_window = getattr(subprocess, "CREATE_NO_WINDOW", 0)
if create_no_window:
kwargs["creationflags"] = create_no_window
startupinfo_factory = getattr(subprocess, "STARTUPINFO", None)
startf_use_showwindow = getattr(subprocess, "STARTF_USESHOWWINDOW", 0)
sw_hide = getattr(subprocess, "SW_HIDE", 0)
if startupinfo_factory is not None and startf_use_showwindow:
startupinfo = startupinfo_factory()
startupinfo.dwFlags |= startf_use_showwindow
startupinfo.wShowWindow = sw_hide
kwargs["startupinfo"] = startupinfo
return kwargs
def _stream_for_subprocess(stream):
"""Return *stream* if it has a real OS file descriptor, else None.
subprocess.run on Windows refuses to inherit std handles unless
they're passed explicitly (otherwise close_fds=True forces
bInheritHandles=False, and a CREATE_NO_WINDOW child ends up with
no stdio at all). When sys.stdout / sys.stderr is a real fd-backed
stream we want to hand it through; when it's been captured by a
test harness (pytest's capsys, an in-memory wrapper, etc) we fall
back to None so subprocess uses its default.
"""
if stream is None:
return None
try:
stream.fileno()
except (AttributeError, OSError, ValueError):
return None
return stream
def _display_host_for_bind(run_mod, host: str) -> str:
return run_mod._resolve_external_ip() if host in ("0.0.0.0", "::") else host
def _loopback_bind_host_for(host: str) -> str:
return "::1" if host == "::" else "127.0.0.1"
def _url_host(host: str) -> str:
return (
f"[{host}]" if ":" in host and not (host.startswith("[") and host.endswith("]")) else host
)
def _emit_run_cloudflare_notice(
run_mod, host: str, display_host: str, actual_port: int, secure: bool
) -> None:
from unsloth_cli._tool_policy import is_external_host
if not is_external_host(host):
return
run_mod._verify_global_reachability(display_host, actual_port)
run_mod._print_cloudflare_line(
secure = secure,
loopback_host = _loopback_bind_host_for(host),
)
def _studio_venv_python() -> Optional[Path]:
"""Return the studio venv Python binary, or None if not set up."""
if platform.system() == "Windows":
p = STUDIO_HOME / "unsloth_studio" / "Scripts" / "python.exe"
else:
p = STUDIO_HOME / "unsloth_studio" / "bin" / "python"
return p if p.is_file() else None
def _find_run_py() -> Optional[Path]:
"""Find studio/backend/run.py.
No CWD dependency — works from any directory.
Since studio/ is now a proper package (has __init__.py), it lives in
site-packages after pip install, right next to unsloth_cli/.
"""
# 1. Relative to __file__ (site-packages or editable repo root)
run_py = _PACKAGE_ROOT / "studio" / "backend" / "run.py"
if run_py.is_file():
return run_py
# 2. Unsloth venv's site-packages (Linux + Windows layouts)
for pattern in (
"lib/python*/site-packages/studio/backend/run.py",
"Lib/site-packages/studio/backend/run.py",
):
for match in (STUDIO_HOME / "unsloth_studio").glob(pattern):
return match
return None
def _install_state() -> dict:
"""verify_install() result for this install root.
STUDIO_HOME is an extra search root so a CLI installed outside the managed
venv still inspects the venv the desktop app launches.
"""
return _studio_deps.install_state(extra_roots = (STUDIO_HOME / "unsloth_studio",))
_RUN_MODULE = None
def _load_run_module():
"""Import studio.backend.run without relying on package resolution.
`studio update` can leave a partial ``site-packages/studio/backend/``
tree (plugin build artefacts only). That shadowed tree wins over an
editable install and breaks ``from studio.backend.run import ...``.
Loading by file path sidesteps the conflict.
"""
global _RUN_MODULE
if _RUN_MODULE is not None:
return _RUN_MODULE
run_py = _find_run_py()
if run_py is None:
raise ImportError("Could not find studio/backend/run.py. Re-run: unsloth studio setup")
loaded = sys.modules.get("studio.backend.run")
if loaded is not None:
# __file__ can be None for namespace packages from partial trees.
loaded_path = Path(getattr(loaded, "__file__", None) or "").resolve()
if loaded_path == run_py.resolve():
_RUN_MODULE = loaded
return _RUN_MODULE
spec = importlib.util.spec_from_file_location("studio.backend.run", run_py)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load studio backend from {run_py}")
module = importlib.util.module_from_spec(spec)
sys.modules["studio.backend.run"] = module
try:
spec.loader.exec_module(module)
except Exception:
sys.modules.pop("studio.backend.run", None)
raise
_RUN_MODULE = module
return _RUN_MODULE
def _find_setup_script() -> Optional[Path]:
"""Find studio/setup.sh or studio/setup.ps1.
No CWD dependency — works from any directory.
"""
name = "setup.ps1" if platform.system() == "Windows" else "setup.sh"
# 1. Relative to __file__ (site-packages or editable repo root)
s = _PACKAGE_ROOT / "studio" / name
if s.is_file():
return s
# 2. Unsloth venv's site-packages
for pattern in (
f"lib/python*/site-packages/studio/{name}",
f"Lib/site-packages/studio/{name}",
):
for match in (STUDIO_HOME / "unsloth_studio").glob(pattern):
return match
return None
# Mirror in studio/backend/run.py argparse + backend denylist test;
# bumping the cap in one place only desyncs.
_PARALLEL_MIN = 1
_PARALLEL_MAX = 64
_PARALLEL_DEFAULT_RUN = 4 # pre-PR hardcoded for `unsloth studio run`
# New Chat leaves the previous conversation generating and the admission queue caps decodes at
# the slot count, so at 1 every extra chat queues. _slots_that_fit_on_gpu() may cut it back.
_PARALLEL_DEFAULT_PLAIN = 4
def _resolve_secure(secure: bool, not_secure: bool) -> bool:
"""Reconcile the deprecated --not-secure alias with --secure/--no-secure.
Typer parses --secure and --not-secure as independent options, so the alias
cannot lean on Click's last-wins ordering the way --secure/--no-secure do.
Restore that ordering from argv: --not-secure only forces secure off when it
is the last of the secure flags on the command line, matching the backend's
BooleanOptionalAction.
"""
if not not_secure:
return secure
last_secure = max(
(i for i, a in enumerate(sys.argv) if a in ("--secure", "--no-secure")),
default = -1,
)
last_not_secure = max(
(i for i, a in enumerate(sys.argv) if a == "--not-secure"),
default = -1,
)
return secure if last_secure > last_not_secure else False
def _iter_editable_studio_source_roots(venv_dir: Path):
"""Yield repo roots from setuptools `__editable___*_finder.py` files in
*venv_dir*'s site-packages whose MAPPING includes a `studio` entry.
Returns the parent dir of the mapped `studio` package (i.e. the repo
root), so callers can append `/studio/...` to reach any subdir.
"""
import ast
import re
for sp_pattern in ("lib/python*/site-packages", "Lib/site-packages"):
for sp in venv_dir.glob(sp_pattern):
for finder in sp.glob("__editable___*_finder.py"):
try:
src = finder.read_text(encoding = "utf-8")
except (OSError, UnicodeDecodeError):
continue
# Tolerate single- or multi-line dict literals; [^}]* still
# rejects nested dicts, which the setuptools template never
# emits for editable installs.
m = re.search(r"^MAPPING\s*(?::[^=]*)?=\s*(\{[^}]*\})", src, re.M | re.S)
if not m:
continue
try:
mapping = ast.literal_eval(m.group(1))
except (SyntaxError, ValueError):
continue
# Defensive: literal_eval can return a set / list / None if the
# matched literal is not a dict (regex captures `{...}`).
if not isinstance(mapping, dict):
continue
studio_pkg = mapping.get("studio")
if studio_pkg:
yield Path(studio_pkg).parent
def _find_frontend_dist() -> Optional[Path]:
"""Locate a built `studio/frontend/dist` (containing index.html).
Probes (in order): package-local default, installer venv site-packages,
editable source roots referenced from the installer venv. Returns None
if nothing servable is found, so callers can decide to error or proceed
in `--api-only` mode.
Fixes the silent 404 when another `unsloth` on PATH shadows the
installer's binary and points `_PACKAGE_ROOT` at a site-packages copy
that never received a vite build.
"""
candidates: List[Path] = [_PACKAGE_ROOT / "studio" / "frontend" / "dist"]
venv_dir = STUDIO_HOME / "unsloth_studio"
for pattern in (
"lib/python*/site-packages/studio/frontend/dist",
"Lib/site-packages/studio/frontend/dist",
):
candidates.extend(venv_dir.glob(pattern))
for repo_root in _iter_editable_studio_source_roots(venv_dir):
candidates.append(repo_root / "studio" / "frontend" / "dist")
seen: set[Path] = set()
for c in candidates:
try:
resolved = c.resolve()
except OSError:
resolved = c
if resolved in seen:
continue
seen.add(resolved)
if (c / "index.html").is_file():
return c
return None
# ── helpers for `unsloth studio run` ────────────────────────────────
def _wait_for_server(port: int, timeout: int = 30) -> bool:
"""Poll ``GET /api/health`` until the server responds 200 or *timeout* expires."""
import urllib.request
import urllib.error
url = f"http://127.0.0.1:{port}/api/health"
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
with urllib.request.urlopen(url, timeout = 2) as resp:
if resp.status == 200:
return True
except (urllib.error.URLError, OSError, ConnectionError):
pass
time.sleep(0.5)
return False
def _create_api_key_inprocess(name: str) -> str:
"""Create an API key via direct storage call (no HTTP needed).
Bypasses the ``must_change_password`` gate that blocks HTTP
``POST /api/auth/api-keys`` on fresh installs. Safe because the
CLI already has filesystem access to ``~/.unsloth/studio``.
"""
storage = _load_backend_auth_storage()
raw_key, _row = storage.create_api_key(
username = storage.DEFAULT_ADMIN_USERNAME,
name = name,
)
return raw_key
def _load_backend_auth_storage():
run_py = _find_run_py()
backend_dir = run_py.parent if run_py is not None else _PACKAGE_ROOT / "studio" / "backend"
if backend_dir.is_dir() and str(backend_dir) not in sys.path:
sys.path.insert(0, str(backend_dir))
auth_dir = backend_dir / "auth"
storage_py = auth_dir / "storage.py"
loaded = sys.modules.get("auth.storage")
if loaded is not None:
# __file__ can be None for namespace packages from partial trees.
loaded_path = Path(getattr(loaded, "__file__", None) or "").resolve()
if loaded_path == storage_py.resolve():
return loaded
package = sys.modules.get("auth")
package_paths = [Path(path).resolve() for path in getattr(package, "__path__", [])]
if package is None or auth_dir.resolve() not in package_paths:
package = types.ModuleType("auth")
package.__path__ = [str(auth_dir)]
package.__package__ = "auth"
package.__file__ = str(auth_dir / "__init__.py")
sys.modules["auth"] = package
spec = importlib.util.spec_from_file_location("auth.storage", storage_py)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load backend auth storage from {storage_py}")
storage = importlib.util.module_from_spec(spec)
sys.modules["auth.storage"] = storage
spec.loader.exec_module(storage)
return storage
def _write_auth_secret(path: Path, secret: str) -> None:
path.parent.mkdir(parents = True, exist_ok = True)
fd, tmp_name = tempfile.mkstemp(prefix = f".{path.name}.", dir = path.parent)
tmp_path = Path(tmp_name)
try:
try:
os.chmod(tmp_path, 0o600)
except OSError:
pass
with os.fdopen(fd, "w", encoding = "utf-8") as f:
fd = -1
f.write(secret)
os.replace(tmp_path, path)
except Exception:
if fd >= 0:
os.close(fd)
tmp_path.unlink(missing_ok = True)
raise
try:
os.chmod(path, 0o600)
except OSError:
pass
def _connect_auth_db() -> sqlite3.Connection:
auth_dir = STUDIO_HOME / "auth"
auth_dir.mkdir(parents = True, exist_ok = True)
conn = sqlite3.connect(auth_dir / "auth.db")
# Mirror backend storage.get_connection: this path can create auth/ and
# auth.db (the pre-exposure gate writes here first), and sqlite3.connect
# makes the DB 0644 under a 022 umask. Keep both private.
for _path, _mode in ((auth_dir, 0o700), (auth_dir / "auth.db", 0o600)):
try:
os.chmod(_path, _mode)
except OSError:
pass
conn.execute(
"""
CREATE TABLE IF NOT EXISTS auth_user (
id INTEGER PRIMARY KEY,
username TEXT UNIQUE NOT NULL,
password_salt TEXT NOT NULL,
password_hash TEXT NOT NULL,
jwt_secret TEXT NOT NULL,
must_change_password INTEGER NOT NULL DEFAULT 0
);
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS refresh_tokens (
id INTEGER PRIMARY KEY,
token_hash TEXT NOT NULL,
username TEXT NOT NULL,
expires_at TEXT NOT NULL,
is_desktop INTEGER NOT NULL DEFAULT 0
);
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS api_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL,
key_prefix TEXT NOT NULL,
key_hash TEXT NOT NULL UNIQUE,
name TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
last_used_at TEXT,
expires_at TEXT,
is_active INTEGER NOT NULL DEFAULT 1
);
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS app_secrets (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
"""
)
auth_columns = {row[1] for row in conn.execute("PRAGMA table_info(auth_user)")}
if "must_change_password" not in auth_columns:
conn.execute(
"ALTER TABLE auth_user ADD COLUMN must_change_password INTEGER NOT NULL DEFAULT 0"
)
refresh_columns = {row[1] for row in conn.execute("PRAGMA table_info(refresh_tokens)")}
if "is_desktop" not in refresh_columns:
conn.execute("ALTER TABLE refresh_tokens ADD COLUMN is_desktop INTEGER NOT NULL DEFAULT 0")
conn.commit()
return conn
def _pbkdf2_hex(value: str, salt: bytes) -> str:
return hashlib.pbkdf2_hmac(
"sha256",
value.encode("utf-8"),
salt,
PBKDF2_ITERATIONS,
).hex()
def _hash_password(password: str) -> tuple[str, str]:
salt = secrets.token_hex(16)
pwd_hash = _pbkdf2_hex(password, salt.encode("utf-8"))
return salt, pwd_hash
def _get_or_create_api_key_pbkdf2_salt(conn: sqlite3.Connection) -> bytes:
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(API_KEY_PBKDF2_SALT_KEY,),
).fetchone()
if row is None:
salt_hex = secrets.token_hex(32)
conn.execute(
"INSERT OR IGNORE INTO app_secrets (key, value) VALUES (?, ?)",
(API_KEY_PBKDF2_SALT_KEY, salt_hex),
)
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(API_KEY_PBKDF2_SALT_KEY,),
).fetchone()
return bytes.fromhex(row[0])
def _ensure_cli_default_admin(conn: sqlite3.Connection) -> None:
row = conn.execute(
"SELECT 1 FROM auth_user WHERE username = ?",
(DEFAULT_ADMIN_USERNAME,),
).fetchone()
if row is not None:
return
bootstrap_password = secrets.token_urlsafe(32)
password_salt, password_hash = _hash_password(bootstrap_password)
conn.execute(
"""
INSERT INTO auth_user (
username,
password_salt,
password_hash,
jwt_secret,
must_change_password
)
VALUES (?, ?, ?, ?, ?)
""",
(
DEFAULT_ADMIN_USERNAME,
password_salt,
password_hash,
secrets.token_urlsafe(64),
1,
),
)
_write_auth_secret(
STUDIO_HOME / "auth" / BOOTSTRAP_PASSWORD_FILE,
bootstrap_password,
)
def _create_desktop_secret_in_cli() -> str:
raw_secret = DESKTOP_SECRET_PREFIX + secrets.token_urlsafe(48)
now = datetime.now(timezone.utc).isoformat()
conn = _connect_auth_db()
try:
_ensure_cli_default_admin(conn)
secret_hash = _pbkdf2_hex(raw_secret, _get_or_create_api_key_pbkdf2_salt(conn))
conn.execute(
"INSERT OR REPLACE INTO app_secrets (key, value) VALUES (?, ?)",
(DESKTOP_SECRET_HASH_KEY, secret_hash),
)
conn.execute(
"INSERT OR REPLACE INTO app_secrets (key, value) VALUES (?, ?)",
(DESKTOP_SECRET_CREATED_AT_KEY, now),
)
conn.commit()
return raw_secret
finally:
conn.close()
def _should_prompt_password_change(
*, cloudflare: Optional[bool], host: str, secure: bool, api_only: bool
) -> bool:
"""Whether this launch will expose Unsloth through the Cloudflare tunnel.
CLI mirror of run.py's _cloudflare_tunnel_should_start, minus the Colab
case (Colab launches never come through this CLI path). --secure implies
the tunnel; --cloudflare only tunnels non-api-only wildcard binds.
"""
if secure:
return True
if cloudflare is not True:
return False
return host in ("0.0.0.0", "::") and not api_only
def _prompt_streams_interactive() -> bool:
"""The prompt needs a real terminal for input and for the masked echo."""
try:
return sys.stdin.isatty() and sys.stderr.isatty()
except (AttributeError, ValueError):
return False
def _bootstrap_deadline_active() -> bool:
"""Whether the backend's bootstrap shutdown deadline will arm.
Mirror of studio/backend/auth/bootstrap_timeout.py bootstrap_timeout_seconds:
unset/blank/malformed UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT falls back to the 1h
default (a typo must not remove protection); 0 or negative disables it.
"""
raw = os.environ.get("UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT", "").strip()
if not raw:
return True
try:
return int(raw) > 0
except ValueError:
return True
def _cli_update_password(conn: sqlite3.Connection, username: str, new_password: str) -> None:
"""CLI mirror of backend update_password + change-password route effects.
One transaction: rehash, rotate the JWT secret, clear must_change_password,
revoke refresh tokens (PR #6651 finding), and drop the desktop secret. File
cleanup happens after commit; a failed unlink must not roll the change back.
"""
password_salt, password_hash = _hash_password(new_password)
with conn:
conn.execute(
"""
UPDATE auth_user
SET password_salt = ?, password_hash = ?, jwt_secret = ?, must_change_password = 0
WHERE username = ?
""",
(password_salt, password_hash, secrets.token_urlsafe(64), username),
)
conn.execute("DELETE FROM refresh_tokens WHERE username = ?", (username,))
conn.execute(
"DELETE FROM app_secrets WHERE key IN (?, ?)",
(DESKTOP_SECRET_HASH_KEY, DESKTOP_SECRET_CREATED_AT_KEY),
)
for stale in (BOOTSTRAP_PASSWORD_FILE, DESKTOP_SECRET_FILE):
stale_path = STUDIO_HOME / "auth" / stale
try:
stale_path.unlink(missing_ok = True)
except OSError as exc:
# The hash is already committed, so a failed unlink must NOT roll the
# change back. But a locked-yet-writable file (Windows AV, read-only
# auth dir) must be truncated: otherwise its stale plaintext survives
# and generate_bootstrap_password() would re-validate this revoked
# credential after a later reset-password deletes auth.db. Mirrors
# backend clear_bootstrap_password().
try:
stale_path.write_text("", encoding = "utf-8")
cleared = True
except OSError:
cleared = False
if cleared:
typer.echo(
f"Warning: could not remove stale {stale} file ({exc}); cleared its "
"contents so the old credential cannot be reused.",
err = True,
)
else:
typer.echo(
f"Warning: could not remove or clear stale {stale} file ({exc}); the "
"old credential is still on disk. Remove it manually to prevent reuse "
"after a reset.",
err = True,
)
def _apply_supplied_password_before_launch(supplied_password: "str | None") -> None:
"""Non-interactively set the INITIAL admin password (from --password /
UNSLOTH_STUDIO_PASSWORD / stdin) before the server binds, while the account
still has its auto-generated bootstrap password.
Only ever sets the FIRST password: an already-set one is a hard error (an
override would be an auth bypass on a public launch), and an invalid value
fails closed. Runs in the parent before any re-exec so the secret never
crosses to the child argv.
"""
if not supplied_password:
return
try:
conn = _connect_auth_db()
except (OSError, sqlite3.Error) as exc:
typer.echo(
f"Error: --password could not open the Unsloth auth database ({exc}); not starting.",
err = True,
)
raise typer.Exit(1)
try:
_ensure_cli_default_admin(conn)
conn.commit()
row = conn.execute(
"SELECT password_salt, password_hash, must_change_password "
"FROM auth_user WHERE username = ?",
(DEFAULT_ADMIN_USERNAME,),
).fetchone()
if not row:
typer.echo(
"Error: --password could not initialize the admin account; not starting.",
err = True,
)
raise typer.Exit(1)
if not row[2]:
typer.echo(
"Error: an Unsloth admin password is already set; --password only sets "
"the initial password. Run `unsloth studio reset-password` first "
"(or change it in the UI).",
err = True,
)
raise typer.Exit(1)
password_salt, password_hash = row[0], row[1]
def _is_current_password(candidate: str) -> bool:
return hmac.compare_digest(
_pbkdf2_hex(candidate, password_salt.encode("utf-8")), password_hash
)
problem = _password_prompt.validate_new_password(supplied_password, _is_current_password)
if problem is not None:
typer.echo(f"Error: {problem} Not starting.", err = True)
raise typer.Exit(1)
_cli_update_password(conn, DEFAULT_ADMIN_USERNAME, supplied_password)
typer.echo(f"Password updated for '{DEFAULT_ADMIN_USERNAME}'.", err = True)
except (OSError, sqlite3.Error) as exc:
# Any DB failure fails closed (typer.Exit is not caught here, so the
# deliberate Exit(1) branches above propagate unchanged).
typer.echo(
f"Error: --password could not update the Unsloth auth database ({exc}); not starting.",
err = True,
)
raise typer.Exit(1)
finally:
conn.close()
def _strip_seeded_bootstrap_password_or_exit(*, context: str) -> None:
"""Remove the seeded plaintext bootstrap password before a public re-exec.
Version-independent protection: a re-exec'd child of ANY version (including an
old studio-venv predating the pre-bind gate) then reads None instead of
injecting the default credential into the public page. must_change_password
stays set, so the login page still forces a change and the timer still arms.
Removal IS the protection, so if it fails (locked file, read-only auth dir)
fail closed rather than publish it.
"""
bootstrap_file = STUDIO_HOME / "auth" / BOOTSTRAP_PASSWORD_FILE
try:
bootstrap_file.unlink(missing_ok = True)
except OSError as exc:
typer.echo(
"Error: refusing to publish Unsloth on a public Cloudflare URL: "
f"could not remove the seeded bootstrap password file ({exc}), so an "
f"older Unsloth child could still serve the default credential ({context}). "
"Delete it manually or change the admin password (run `unsloth studio` "
"locally with a terminal attached, or `unsloth studio reset-password`), "
"then retry.",
err = True,
)
raise typer.Exit(1)
def _require_servable_frontend_or_exit(
*, frontend: Optional[Path], api_only: bool, cloudflare: Optional[bool], host: str, secure: bool
) -> Optional[Path]:
"""Fail closed BEFORE the pre-exposure gate if a public UI launch has no
login page to change the seeded password.
The gate strips the seeded .bootstrap_password on a headless public launch,
so if the child then cannot serve the login page the admin is locked out
(must_change_password=1, no file, no UI) until `unsloth studio reset-password`.
The login page is the ONLY in-band way to change the seeded password, so a
public non-api-only launch must have a servable dist before the strip.
Returns the dist to serve: a user-supplied --frontend (validated to contain
index.html) or the auto-resolved built dist. Returns `frontend` unchanged for
non-public or --api-only launches (no login page needed).
"""
if api_only or not _should_prompt_password_change(
cloudflare = cloudflare, host = host, secure = secure, api_only = api_only
):
return frontend
if frontend is not None:
# A user-supplied dist is not vetted by _find_frontend_dist, so verify it
# can serve the login page; else `--frontend /bad/path` bypasses the guard.
if (Path(frontend) / "index.html").is_file():
return frontend
typer.echo(
"Error: --frontend points at a directory with no index.html, so a "
"public Unsloth launch would have no login page to change the seeded "
"admin password. Point --frontend at a built dist, rebuild it (re-run "
"install.sh), or use --api-only.",
err = True,
)
raise typer.Exit(1)
# _find_frontend_dist only returns a path that already contains index.html.
resolved = _find_frontend_dist()
if resolved is not None:
return resolved
typer.echo(
"Error: the Unsloth frontend is not built, so a public launch would have "
"no login page to change the seeded admin password. Build it (re-run "
"install.sh), pass --frontend PATH to a built dist, or use --api-only.",
err = True,
)
raise typer.Exit(1)
def _validate_inproc_backend_before_strip(
*, cloudflare: Optional[bool], host: str, secure: bool, api_only: bool
) -> None:
"""In-venv (in-process) analogue of the re-exec launcher check.
In-venv there is no re-exec, so the backend is imported in-process only AFTER
the gate. On the headless public path the gate strips the seeded
.bootstrap_password, so a broken venv that fails at import would leave
must_change_password=1 with no password to log in. Import the backend up front
on that path and exit cleanly if broken, before anything is stripped.
Headless-only so an interactive prompt is not delayed behind the import.
"""
if not _should_prompt_password_change(
cloudflare = cloudflare, host = host, secure = secure, api_only = api_only
):
return
if _prompt_streams_interactive():
return
try:
_load_run_module()
except Exception as exc:
typer.echo(
f"Error: the Unsloth backend could not be loaded ({exc}); refusing to "
"expose Unsloth publicly before it is confirmed runnable. Re-run: "
"unsloth studio setup",
err = True,
)
raise typer.Exit(1)
def _tunnel_binary_confirmed_unavailable() -> bool:
"""True only if cloudflared is provably unavailable (found nowhere on PATH or
in the Unsloth cache AND the download failed), so the tunnel cannot start.
Used on the --secure path (loopback bind, so the tunnel is the ONLY public
exposure) to skip stripping the seeded recovery password before a public URL
that will never come up. Loads the stdlib-only cloudflare_tunnel helper by
file path so the check runs in the parent, before the strip.
Returns False on ANY uncertainty: a possible credential leak outweighs a
recoverable lockout, so the caller keeps the strip unless the tunnel is
provably dead.
"""
run_py = _find_run_py()
if run_py is None:
return False
backend_dir = run_py.parent
tunnel_py = backend_dir / "cloudflare_tunnel.py"
if not tunnel_py.is_file():
return False
# ensure_cloudflared() lazily imports utils.paths.storage_roots to resolve the
# Unsloth bin cache. The outer CLI hasn't added studio/backend to sys.path yet,
# so that import would fail and return None (a false "unavailable" that wrongly
# refuses --secure). Add the backend dir so the cache path resolves as in the child.
added_backend_path = False
try:
if str(backend_dir) not in sys.path:
sys.path.insert(0, str(backend_dir))
added_backend_path = True
spec = importlib.util.spec_from_file_location("studio.backend.cloudflare_tunnel", tunnel_py)
if spec is None or spec.loader is None:
return False
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module.ensure_cloudflared() is None
except Exception:
return False
finally:
if added_backend_path:
try:
sys.path.remove(str(backend_dir))
except ValueError:
pass
def _child_self_suppresses(*, in_studio_venv: bool, child_run_py: Optional[Path]) -> bool:
"""True when the child that will serve Unsloth is provably THIS install's
backend, whose pre-bind gate sets app.state.suppress_bootstrap_injection and
so never serves the seeded credential publicly -- even with .bootstrap_password
on disk. The parent-side strip is then unnecessary and can be skipped to avoid
a lockout if the tunnel never comes up, keeping the file for LOCAL recovery.
True iff we run in-process here, or the re-exec target is the outer install's
own run.py (identity match). False on ANY doubt -- a studio-venv console script
or a venv run.py that may predate the gate -- so the strip stays in force
wherever an old child is possible.
"""
if in_studio_venv:
return True
if child_run_py is None:
return False
try:
outer_run_py = (_PACKAGE_ROOT / "studio" / "backend" / "run.py").resolve()
return child_run_py.resolve() == outer_run_py
except OSError:
return False
def _enforce_password_change_before_exposure(
*,
cloudflare: Optional[bool],
host: str,
secure: bool,
api_only: bool,
child_self_suppresses: bool = False,
) -> None:
"""Force a terminal password change before the first public (tunnel) exposure.
When the launch will start the tunnel and the admin still has its
auto-generated bootstrap password, ask for a new one in the terminal (masked,
confirmed) before any server or tunnel exists. Committing here, in the parent,
keeps the password off argv/env and an older studio-venv child sees it
immediately. Without a terminal, warn and fall back to the bootstrap shutdown
timer (~1h, UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT).
"""
if not _should_prompt_password_change(
cloudflare = cloudflare, host = host, secure = secure, api_only = api_only
):
return
# Before public exposure we must PROVE the admin password is no longer the
# seeded default. If we cannot (auth DB won't open, or a fresh admin cannot be
# seeded + committed below), an old studio-venv child could regenerate a fresh
# bootstrap credential and serve it; stripping a file we can't vouch for cannot
# stop a regeneration. So those cases fail closed, as does a failure after the
# user typed a new password.
try:
conn = _connect_auth_db()
except (OSError, sqlite3.Error) as exc:
# Cannot open the auth DB, so cannot confirm a committed admin exists.
# Refuse rather than risk a child serving the default login; a transient
# lock clears on retry.
typer.echo(
"Error: refusing to publish Unsloth on a public Cloudflare URL: could "
f"not open the Unsloth auth database ({exc}) to confirm the admin "
"password was changed. Retry (a transient database lock clears), or "
"change the password first (run `unsloth studio` locally with a "
"terminal attached, or `unsloth studio reset-password`).",
err = True,
)
raise typer.Exit(1)
try:
try:
_ensure_cli_default_admin(conn)
# Persist a freshly seeded admin before we might re-exec: the INSERT is
# otherwise uncommitted and rolls back on conn.close(). If the seed or
# commit fails, no admin is committed, so a re-exec'd OLD child finds
# none, regenerates a fresh bootstrap password + file, and serves THAT
# -- stripping cannot stop a regeneration. Can't prove a committed
# admin, so fail closed.
conn.commit()
except (OSError, sqlite3.Error) as exc:
# Best-effort remove any half-written seed file (its row rolled back);
# the launch is refused regardless.
try:
(STUDIO_HOME / "auth" / BOOTSTRAP_PASSWORD_FILE).unlink(missing_ok = True)
except OSError:
pass
typer.echo(
"Error: refusing to publish Unsloth on a public Cloudflare URL: could "
f"not initialize the admin account ({exc}), so a re-exec'd Unsloth "
"child could regenerate and serve a default credential. Retry (a "
"transient database lock clears), or change the password first (run "
"`unsloth studio` locally with a terminal attached, or `unsloth "
"studio reset-password`).",
err = True,
)
raise typer.Exit(1)
try:
row = conn.execute(
"SELECT password_salt, password_hash, must_change_password "
"FROM auth_user WHERE username = ?",
(DEFAULT_ADMIN_USERNAME,),
).fetchone()
except (OSError, sqlite3.Error) as exc:
if child_self_suppresses:
# Could not read must_change back, but the child is this install's
# own backend and suppresses the injection, so nothing serves the
# seeded credential; proceed without stripping.
return
# The admin is committed above, so an old child finds it and won't
# regenerate; we just couldn't read must_change back. Strip the seeded
# file so nothing serves it, failing closed if the strip itself fails.
typer.echo(
f"Warning: could not read the Unsloth admin state back ({exc}); "
"removing the seeded bootstrap password before public exposure.",
err = True,
)
_strip_seeded_bootstrap_password_or_exit(context = "auth DB row unreadable")
return
if not row or not row[2]:
return
if not _prompt_streams_interactive():
# Only proceed headless if the bootstrap shutdown deadline will protect
# the launch: it never arms for api-only, and TIMEOUT=0 disables it.
if api_only or not _bootstrap_deadline_active():
typer.echo(
"Error: refusing to publish Unsloth on a public Cloudflare "
"URL: the default admin password was never changed, no "
"terminal is attached to change it here, and the bootstrap "
"shutdown deadline does not apply to this launch (api-only, "
"or UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0). Change the "
"password first (run `unsloth studio` locally and log in, "
"or re-run with a terminal attached), then retry.",
err = True,
)
raise typer.Exit(1)
if child_self_suppresses:
# The child is this install's own backend, whose pre-bind gate sets
# app.state.suppress_bootstrap_injection, so the seeded credential
# is never served publicly even with the file on disk. Skip the
# strip: unnecessary here, and it would lock the user out if the
# tunnel never comes up (e.g. a --secure loopback whose tunnel
# fails). Keep the file for LOCAL recovery; must_change stays set
# and the deadline arms.
typer.echo(
"Warning: Unsloth is being exposed publicly while the admin "
"account still uses its auto-generated bootstrap password. The "
"login page forces a change and the credential is never served "
"on the public page. Set a new password by running `unsloth "
"studio` locally with a terminal attached, or `unsloth studio "
"reset-password`; Unsloth shuts down after ~1h if the password "
"stays unchanged (UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT).",
err = True,
)
return
# The strip permanently removes the only plaintext recovery credential.
# On --secure the bind is loopback, so the tunnel is the ONLY public
# exposure: if cloudflared is provably unavailable no public URL can
# start, so stripping would just lock the user out. Refuse with the
# credential preserved. (A wildcard --cloudflare bind is public
# regardless of the tunnel, so it still strips below, as does any
# uncertainty.)
if secure and _tunnel_binary_confirmed_unavailable():
typer.echo(
"Error: refusing to expose Unsloth: the Cloudflare tunnel binary "
"(cloudflared) is unavailable and could not be downloaded, so no "
"public URL can start. The seeded bootstrap password is preserved "
"for recovery; fix connectivity and retry, or change the password "
"first (`unsloth studio` locally, or `unsloth studio "
"reset-password`).",
err = True,
)
raise typer.Exit(1)
# Mixed-version safety: an OLD studio-venv child (predating this gate)
# has no pre-bind suppression and would read the seeded credential back
# from disk and inject it into the public HTML until the deadline.
# Delete the file here, in the parent, so a fresh child of ANY version
# reads None. must_change_password stays set, so the login page still
# forces a change and the timer still arms; only the on-disk copy goes.
_strip_seeded_bootstrap_password_or_exit(context = "no terminal to change it")
typer.echo(
"Warning: Unsloth is being exposed publicly while the admin account "
"still uses its auto-generated bootstrap password. The seeded password "
"file has been removed so it is not served on the public page. Set a new "
"password by running `unsloth studio` locally with a terminal attached, "
"or `unsloth studio reset-password`; Unsloth shuts down after ~1h if the "
"password stays unchanged (UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT).",
err = True,
)
return
password_salt, password_hash = row[0], row[1]
def _is_current_password(candidate: str) -> bool:
return hmac.compare_digest(
_pbkdf2_hex(candidate, password_salt.encode("utf-8")), password_hash
)
typer.echo(
"Unsloth Studio will be exposed on the public internet, so set a "
"password now. Ctrl+C to abort.",
err = True,
)
try:
new_password = _password_prompt.prompt_new_password(_is_current_password)
except (KeyboardInterrupt, EOFError):
typer.echo(
"\nError: password change aborted; refusing to expose Unsloth "
"with the default admin password. Re-run and set a password, "
"or launch without --secure/--cloudflare.",
err = True,
)
raise typer.Exit(1)
_cli_update_password(conn, DEFAULT_ADMIN_USERNAME, new_password)
typer.echo(f"Password updated for '{DEFAULT_ADMIN_USERNAME}'.", err = True)
finally:
conn.close()
def _load_model_via_http(
port: int,
api_key: str,
model: str,
gguf_variant: Optional[str],
max_seq_length: int,
load_in_4bit: bool,
gpu_memory_mode: Literal["auto", "manual"] = "auto",
tensor_parallel: bool = False,
llama_extra_args: Optional[List[str]] = None,
timeout: int = 600,
) -> dict:
"""POST to ``/api/inference/load`` using the API key for auth."""
import json
import urllib.request
import urllib.error
payload: dict = {
"model_path": model,
"max_seq_length": max_seq_length,
"load_in_4bit": load_in_4bit,
}
if gguf_variant:
payload["gguf_variant"] = gguf_variant
if gpu_memory_mode == "manual":
payload["gpu_memory_mode"] = "manual"
payload["gpu_layers"] = -1
if tensor_parallel:
payload["tensor_parallel"] = True
if llama_extra_args:
payload["llama_extra_args"] = list(llama_extra_args)
data = json.dumps(payload).encode()
req = urllib.request.Request(
f"http://127.0.0.1:{port}/api/inference/load",
data = data,
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {api_key}",
},
method = "POST",
)
try:
with urllib.request.urlopen(req, timeout = timeout) as resp:
return json.loads(resp.read())
except urllib.error.HTTPError as exc:
body = exc.read().decode(errors = "replace")
raise RuntimeError(f"Model load failed (HTTP {exc.code}): {body}") from exc
def _format_context_length_line(load_result: dict) -> Optional[str]:
value = load_result.get("context_length")
if isinstance(value, bool):
return None
try:
value_int = int(value)
except (TypeError, ValueError):
return None
if value_int <= 0:
return None
return f" Context length: {value_int} tokens"
# ── unsloth studio (server) ──────────────────────────────────────────
@studio_app.callback(invoke_without_command = True)
def studio_default(
ctx: typer.Context,
port: int = typer.Option(8888, "--port", "-p"),
host: str = typer.Option("127.0.0.1", "--host", "-H"),
frontend: Optional[Path] = typer.Option(None, "--frontend", "-f"),
silent: bool = typer.Option(False, "--silent", "-q"),
api_only: bool = typer.Option(
False,
"--api-only",
help = "Run API server only, no frontend serving (for Tauri desktop app)",
),
parallel: int = typer.Option(
_PARALLEL_DEFAULT_PLAIN,
"--parallel",
"--n-parallel",
min = _PARALLEL_MIN,
max = _PARALLEL_MAX,
help = (
f"llama-server parallel decode slots ({_PARALLEL_MIN}..{_PARALLEL_MAX}). "
f"Default {_PARALLEL_DEFAULT_PLAIN}."
),
),
cloudflare: Optional[bool] = typer.Option(
None,
"--cloudflare/--no-cloudflare",
help = "Expose Unsloth on a PUBLIC internet URL via a free Cloudflare HTTPS "
"tunnel, for non-api-only wildcard binds (0.0.0.0 or ::). Off by default; "
"pass --cloudflare to enable it (--secure implies it). --no-cloudflare forces "
"it off but does not change a raw wildcard bind.",
),
secure: bool = typer.Option(
False,
"--secure/--no-secure",
help = "Expose ONLY a Cloudflare HTTPS link: bind localhost and fail closed "
"if the tunnel can't start. Without it, --no-secure also serves the raw "
"0.0.0.0 port, which is reachable from anywhere on the network.",
),
not_secure: bool = typer.Option(
False,
"--not-secure",
hidden = True,
help = "Deprecated alias for --no-secure.",
),
verbose: bool = typer.Option(
False,
"--verbose",
"-v",
help = "Log every API request, including the high-frequency polling that is "
"deduplicated by default.",
),
enable_tools: Optional[bool] = typer.Option(
None,
"--enable-tools/--disable-tools",
help = "Force server-side tools (web search, code execution) on or off for "
"every request. Default: on for every bind, with the per-chat UI toggle honored.",
),
disable_dns_pinning: bool = typer.Option(
False,
"--disable-dns-pinning",
help = "Allow hostname-based web fetches for enterprise proxies. WARNING: weakens "
"DNS-rebinding protection; hostname and redirect validation remain enabled.",
),
password: str = typer.Option(
"",
"--password",
help = "Set the INITIAL admin password non-interactively (headless setups), "
"only when none is set yet. Also reads the UNSLOTH_STUDIO_PASSWORD env var, or "
"`--password -` to read one line from stdin. A literal value is visible in the "
"process list and shell history. Rotate later with `unsloth studio reset-password`.",
),
):
"""Launch the Unsloth Studio server."""
# Back-compat: --not-secure is a deprecated alias for --no-secure.
secure = _resolve_secure(secure, not_secure)
# Runs before every subcommand (run/setup/update/...).
_ensure_studio_env_exported()
if ctx.invoked_subcommand is not None:
# Typer doesn't forward parent options to subcommands, so
# `unsloth studio --parallel N run ...` would silently drop N.
if parallel != _PARALLEL_DEFAULT_PLAIN:
typer.echo(
f"Error: --parallel on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put the flag after the "
f"subcommand: `unsloth studio {ctx.invoked_subcommand} "
f"--parallel {parallel} ...`",
err = True,
)
raise typer.Exit(2)
# Same for --cloudflare/--no-cloudflare: it would not reach the subcommand.
if cloudflare is not None:
_cf_flag = "--cloudflare" if cloudflare else "--no-cloudflare"
typer.echo(
f"Error: {_cf_flag} on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} {_cf_flag} ...`",
err = True,
)
raise typer.Exit(2)
# Same for --secure: it would not reach the subcommand.
if secure:
typer.echo(
f"Error: --secure on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} --secure ...`",
err = True,
)
raise typer.Exit(2)
# Same for --verbose: it would not reach the subcommand.
if verbose:
typer.echo(
f"Error: --verbose on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} --verbose ...`",
err = True,
)
raise typer.Exit(2)
# Same for --enable-tools/--disable-tools: it would not reach the subcommand.
if enable_tools is not None:
_tool_flag = "--enable-tools" if enable_tools else "--disable-tools"
typer.echo(
f"Error: {_tool_flag} on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} {_tool_flag} ...`",
err = True,
)
raise typer.Exit(2)
if disable_dns_pinning:
typer.echo(
"Error: --disable-dns-pinning on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio {ctx.invoked_subcommand}`, "
f"put it after the subcommand: `unsloth studio {ctx.invoked_subcommand} "
"--disable-dns-pinning ...`",
err = True,
)
raise typer.Exit(2)
# Same for --api-only: dropping it here would silently serve the UI.
if api_only:
typer.echo(
f"Error: --api-only on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} --api-only ...`",
err = True,
)
raise typer.Exit(2)
# Same for --password: it applies to the plain-server path only.
if password:
typer.echo(
f"Error: --password on `unsloth studio` applies to the "
f"plain-server path only. For `unsloth studio "
f"{ctx.invoked_subcommand}`, put it after the subcommand: "
f"`unsloth studio {ctx.invoked_subcommand} --password ...`",
err = True,
)
raise typer.Exit(2)
return
# --secure requires the tunnel; force a loopback bind.
if secure:
if cloudflare is False:
typer.echo(
"Error: --secure requires the Cloudflare tunnel; do not combine it "
"with --no-cloudflare.",
err = True,
)
raise typer.Exit(2)
if host not in ("127.0.0.1", "localhost", "::1"):
typer.echo(
"Note: --secure ignores -H (it binds loopback and serves only "
"through the Cloudflare tunnel). Drop --secure to bind "
f"{host} directly, or keep --secure for a tunnel-only public link.",
err = True,
)
host = "127.0.0.1"
# --verbose restores the per-request access logs that are suppressed by
# default (plain-server path; the `run` subcommand has its own --verbose).
if verbose:
_enable_verbose_access_logs()
if disable_dns_pinning:
os.environ["UNSLOTH_STUDIO_DISABLE_DNS_PINNING"] = "1"
else:
os.environ.setdefault("UNSLOTH_STUDIO_DISABLE_DNS_PINNING", "0")
# Use the studio venv if present and not already in it. Resolve the child
# launcher BEFORE the gate: a headless gate strips the seeded
# .bootstrap_password, so aborting afterward (venv/run.py missing) would leave
# must_change_password=1 with no password to log in.
studio_venv_dir = STUDIO_HOME / "unsloth_studio"
in_studio_venv = sys.prefix.startswith(str(studio_venv_dir))
studio_python = run_py = None
resolved_frontend = frontend
if not in_studio_venv:
studio_python = _studio_venv_python()
run_py = _find_run_py()
if not (studio_python and run_py):
typer.echo("Unsloth Studio not set up. Run install.sh first.")
raise typer.Exit(1)
# A public UI launch must have a servable login page BEFORE the gate can
# strip the seeded .bootstrap_password, or the child has no way to change
# it. Also returns the resolved dist so the child serves a real build
# regardless of where its __file__ lands (fixes the shadowed silent 404).
resolved_frontend = _require_servable_frontend_or_exit(
frontend = resolved_frontend,
api_only = api_only,
cloudflare = cloudflare,
host = host,
secure = secure,
)
# Non-public / api-only launches skip that validation but still forward an
# explicitly resolved dist for the same silent-404 reason.
if resolved_frontend is None and not api_only:
resolved_frontend = _find_frontend_dist()
else:
# Already in the studio venv: no re-exec, served in-process below. On the
# headless public path the gate strips the seeded .bootstrap_password, so
# validate BOTH FIRST -- else a bad dist or broken venv fails only after
# the strip (must_change_password=1, no password to log in). Frontend check
# first (cheap); the backend import is headless-only so an interactive
# prompt is not delayed behind it.
resolved_frontend = _require_servable_frontend_or_exit(
frontend = resolved_frontend,
api_only = api_only,
cloudflare = cloudflare,
host = host,
secure = secure,
)
_validate_inproc_backend_before_strip(
cloudflare = cloudflare, host = host, secure = secure, api_only = api_only
)
# A supplied --password / UNSLOTH_STUDIO_PASSWORD / stdin sets the initial
# admin password here in the parent, before the gate and any re-exec, so the
# secret never reaches the child argv; strip the env var so a re-exec'd child
# can't re-read it. The interactive gate below then no-ops.
_apply_supplied_password_before_launch(_password_prompt.resolve_supplied_password(password))
os.environ.pop(_password_prompt.SUPPLIED_PASSWORD_ENV, None)
# Public (tunnel) exposure with the seeded default password: force a terminal
# password change first, before any re-exec or server exists. The child is
# self-suppressing when we serve in-process or re-exec this install's own
# run.py (its pre-bind gate suppresses the injection), so the gate can skip
# the destructive strip.
_enforce_password_change_before_exposure(
cloudflare = cloudflare,
host = host,
secure = secure,
api_only = api_only,
child_self_suppresses = _child_self_suppresses(
in_studio_venv = in_studio_venv, child_run_py = run_py
),
)
if not in_studio_venv:
if studio_python and run_py:
if not silent:
typer.echo("Launching Unsloth Studio... Please wait...")
args = [
str(studio_python),
str(run_py),
"--host",
host,
"--port",
str(port),
"--parallel",
str(parallel),
]
# Forward the frontend dist resolved before the gate (skipped in
# --api-only, which serves no UI).
if resolved_frontend is not None:
args.extend(["--frontend", str(resolved_frontend)])
if silent:
args.append("--silent")
if api_only:
args.append("--api-only")
# Forward polarity explicitly: _find_run_py can fall back to an older
# run.py (--cloudflare defaulted on), so an unset default must not let a
# mixed install silently re-enable the tunnel. --secure implies it, so
# forward nothing then.
if cloudflare is True:
args.append("--cloudflare")
elif not secure:
args.append("--no-cloudflare")
args.append("--secure" if secure else "--no-secure")
# Forward an explicit tool policy; None -> run.py leaves it unset (tools on).
if enable_tools is True:
args.append("--enable-tools")
elif enable_tools is False:
args.append("--disable-tools")
# On Windows os.execvp keeps the parent alive, so Ctrl+C
# would orphan the child; use Popen+wait instead.
if sys.platform == "win32":
import subprocess as _sp
proc = _sp.Popen(args, **_windows_hidden_subprocess_kwargs())
try:
rc = proc.wait()
except KeyboardInterrupt:
# Child handles its own signal; let it finish.
rc = proc.wait()
if rc != 0:
typer.echo(
f"\nError: Unsloth server exited unexpectedly (code {rc}).",
err = True,
)
typer.echo(
"Check the error above. If a package is missing, "
"re-run: unsloth studio setup",
err = True,
)
raise typer.Exit(rc)
else:
os.execvp(str(studio_python), args)
else:
typer.echo("Unsloth Studio not set up. Run install.sh first.")
raise typer.Exit(1)
with _studio_deps.studio_backend_imports("unsloth studio"):
run_mod = _load_run_module()
run_server = run_mod.run_server
if not silent:
display_host = _display_host_for_bind(run_mod, host)
typer.echo(f"Starting Unsloth Studio on http://{_url_host(display_host)}:{port}")
run_kwargs = dict(
host = host,
port = port,
silent = silent,
api_only = api_only,
llama_parallel_slots = parallel,
cloudflare = cloudflare,
secure = secure,
enable_tools = enable_tools,
)
# Forward the frontend validated before the gate (in-venv path), so the
# in-process server serves exactly the dist we vouched for.
if resolved_frontend is not None:
run_kwargs["frontend_path"] = resolved_frontend
run_server(**run_kwargs)
try:
if run_mod._shutdown_event is not None:
# Event.wait() with no timeout blocks at C-level on Linux
# and swallows SIGINT; loop with a 1s timeout instead.
while not run_mod._shutdown_event.is_set():
run_mod._shutdown_event.wait(timeout = 1)
else:
while True:
time.sleep(1)
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
finally:
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
# ── unsloth studio run ───────────────────────────────────────────────
def _split_repo_variant(model_arg: str) -> tuple[str, Optional[str]]:
"""Split ``org/name:variant`` into ``(repo, variant)``; mirrors
llama.cpp's ``-hf <repo>:<quant>``. Local paths, Windows drives,
and ids without ``:`` pass through verbatim."""
s = model_arg.strip()
if not s:
return s, None
if s.startswith(("/", "./", "../", "~")) or s == ".":
return s, None
# Windows drive letter (e.g. "C:\path"): colon is a path separator.
if len(s) >= 2 and s[1] == ":" and s[0].isalpha():
return s, None
if ":" not in s:
return s, None
repo, _, variant = s.rpartition(":")
if not repo or not variant:
return s, None
# Quant labels never contain a slash; `foo:bar/baz` isn't repo:variant.
if "/" in variant:
return s, None
return repo, variant
def _expand_attached_np_short() -> None:
# Click clusters `-np8` as `-n -p 8` (-p = --port), dropping the parallel
# value. Split to `-np <N>` so typer's alias matches. Stops at `--`;
# accepts signed/junk forms so typer reports a clean error against `-np`.
# Kept in lockstep with the backend `_flag_name` recogniser.
i = 0
while i < len(sys.argv):
tok = sys.argv[i]
if tok == "--":
break
if len(tok) > 3 and tok.startswith("-np") and tok[3] != "=":
suffix = tok[3:]
first_numeric = suffix[0].isdigit() or (
len(suffix) > 1 and suffix[0] in {"-", "+"} and suffix[1].isdigit()
)
if first_numeric:
sys.argv[i : i + 1] = ["-np", suffix]
i += 2
continue
i += 1
def _consume_legacy_short_aliases(
args: List[str], aliases: tuple[str, ...], current: Optional[str], canonical: str
) -> tuple[Optional[str], List[str]]:
"""Pop exact-match legacy shorts (`-m`/`-hfr`/`-f`) from args;
leave clusters (`-mg`/`-fa`/...) for the llama-server tail. Inline
`-x=value` form also accepted."""
out: List[str] = []
value = current
i, n = 0, len(args)
while i < n:
tok = args[i]
if tok == "--": # end of options; tail is raw payload.
out.extend(args[i:])
break
name, sep, inline = tok.partition("=")
if name not in aliases:
out.append(tok)
i += 1
continue
if value is not None:
raise typer.BadParameter(f"{name} conflicts with {canonical} already provided")
if sep:
if inline == "": # `-m=` would become --model '' (Path('')='.').
raise typer.BadParameter(f"{name} requires a non-empty value")
value = inline
i += 1
elif i + 1 < n:
nxt = args[i + 1]
# `--long` is unambiguously a flag; single-dash `-x` may be a path.
if nxt.startswith("--") and nxt != "--":
raise typer.BadParameter(f"{name} expects a value but got the flag {nxt}")
value = nxt
i += 2
else:
raise typer.BadParameter(f"{name} requires a value")
return value, out
# Help panels so `unsloth run --help` groups options instead of one long list.
_RUN_PANEL_MODEL = "Model"
_RUN_PANEL_SERVER = "Server & network"
_RUN_PANEL_TOOLS = "Tool calls"
_RUN_PANEL_SAMPLING = "Sampling"
_RUN_PANEL_ADVANCED = "Advanced"
@studio_app.command(
context_settings = {
"allow_extra_args": True,
"ignore_unknown_options": True,
},
)
def run(
ctx: typer.Context,
model: Optional[str] = typer.Option(
None,
"--model",
"-hf",
"--hf-repo",
# `-m` / `-hfr` removed (Click would cluster `-mg`/`-md`/...).
# Exact-match `-m`/`-hfr` still work via the legacy shim below.
# `-hf` stays (multi-char shorts don't cluster).
rich_help_panel = _RUN_PANEL_MODEL,
help = (
"Model path or HF repo. Accepts llama.cpp-style "
"`org/repo:variant` syntax. `-hf` / `--hf-repo` match "
"llama-server's spelling."
),
),
gguf_variant: Optional[str] = typer.Option(
None,
"--gguf-variant",
rich_help_panel = _RUN_PANEL_MODEL,
help = "GGUF quant variant (e.g. UD-Q4_K_XL)",
),
verbose: bool = typer.Option(
False,
"--verbose",
"-v",
rich_help_panel = _RUN_PANEL_ADVANCED,
help = "Log every API request, including the high-frequency polling that is "
"deduplicated by default.",
),
max_seq_length: int = typer.Option(
0,
"--max-seq-length",
"--context-length",
rich_help_panel = _RUN_PANEL_MODEL,
help = "Runtime context length in tokens (0 = model default for GGUF; 2048 for hub models)",
),
gpu_memory_mode: Literal["auto", "manual"] = typer.Option(
"auto",
"--gpu-memory-mode",
rich_help_panel = _RUN_PANEL_MODEL,
help = (
"GPU memory strategy for GGUF models. Auto lets Unsloth select GPUs "
"and cap context to fit VRAM. Manual with default layers and context "
"delegates placement and sizing to llama.cpp --fit."
),
),
load_in_4bit: bool = typer.Option(
True, "--load-in-4bit/--no-load-in-4bit", rich_help_panel = _RUN_PANEL_MODEL
),
api_key_name: str = typer.Option(
"cli",
"--api-key-name",
rich_help_panel = _RUN_PANEL_ADVANCED,
help = "Label for the auto-generated API key",
),
port: int = typer.Option(8888, "--port", "-p", rich_help_panel = _RUN_PANEL_SERVER),
host: str = typer.Option("127.0.0.1", "--host", "-H", rich_help_panel = _RUN_PANEL_SERVER),
# `-f` removed (clustered `-fa`/`-fit*`); studio_default keeps it.
frontend: Optional[Path] = typer.Option(None, "--frontend", rich_help_panel = _RUN_PANEL_SERVER),
api_only: bool = typer.Option(
False,
"--api-only",
rich_help_panel = _RUN_PANEL_SERVER,
help = "Serve only the API (no web UI), for a headless model server. "
"Pairs with --secure to expose the API over the Cloudflare link alone.",
),
silent: bool = typer.Option(False, "--silent", "-q", rich_help_panel = _RUN_PANEL_ADVANCED),
enable_tools: Optional[bool] = typer.Option(
None,
"--enable-tools/--disable-tools",
rich_help_panel = _RUN_PANEL_TOOLS,
help = (
"Force server-side tools (web search, code execution) on or off for "
"every request. Default: on for every bind."
),
),
disable_dns_pinning: bool = typer.Option(
False,
"--disable-dns-pinning",
rich_help_panel = _RUN_PANEL_TOOLS,
help = "Allow hostname-based web fetches for enterprise proxies. WARNING: weakens "
"DNS-rebinding protection; hostname and redirect validation remain enabled.",
),
tool_call_healing: Optional[bool] = typer.Option(
None,
"--enable-tool-call-healing/--disable-tool-call-healing",
rich_help_panel = _RUN_PANEL_TOOLS,
help = (
"Promote text-form tool calls (small GGUFs often emit <tool_call>...) "
"back into structured calls on the client-tool passthrough. Default: on. "
"An explicit --disable-tool-call-healing is an absolute server kill-switch."
),
),
tool_call_nudging: Optional[bool] = typer.Option(
None,
"--enable-tool-call-nudging/--disable-tool-call-nudging",
rich_help_panel = _RUN_PANEL_TOOLS,
help = (
"On the non-streaming client-tool passthrough, retry once with a short "
"nudge when the model emitted a tool signal that healing could not repair. "
"Default: on. No effect on streaming requests or the server-side agentic loop."
),
),
temperature: Optional[float] = typer.Option(
None,
"--temperature",
min = 0.0,
max = 2.0,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = (
"Pin the sampling temperature for every request that omits it, overriding the "
"model's recommended value. Default: unset (use the per-model recommendation)."
),
),
top_p: Optional[float] = typer.Option(
None,
"--top-p",
min = 0.0,
max = 1.0,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = "Pin top-p (nucleus) sampling. Default: unset (per-model recommendation).",
),
top_k: Optional[int] = typer.Option(
None,
"--top-k",
min = -1,
max = 100,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = "Pin top-k sampling. Default: unset (per-model recommendation).",
),
min_p: Optional[float] = typer.Option(
None,
"--min-p",
min = 0.0,
max = 1.0,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = "Pin min-p sampling threshold. Default: unset (per-model recommendation).",
),
repetition_penalty: Optional[float] = typer.Option(
None,
"--repetition-penalty",
min = 1.0,
max = 2.0,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = "Pin the repetition penalty. Default: unset (per-model recommendation).",
),
presence_penalty: Optional[float] = typer.Option(
None,
"--presence-penalty",
min = 0.0,
max = 2.0,
rich_help_panel = _RUN_PANEL_SAMPLING,
help = "Pin the presence penalty. Default: unset (per-model recommendation).",
),
yes: bool = typer.Option(
False,
"--yes",
"-y",
rich_help_panel = _RUN_PANEL_ADVANCED,
help = "Accepted for backward compatibility; the tool policy no longer prompts.",
),
parallel: int = typer.Option(
_PARALLEL_DEFAULT_RUN,
"--parallel",
"--n-parallel",
"-np",
min = _PARALLEL_MIN,
max = _PARALLEL_MAX,
rich_help_panel = _RUN_PANEL_SERVER,
help = (
"llama-server parallel decode slots. N requests share one "
"loaded model; each slot gets ctx/N KV cache. Default "
f"{_PARALLEL_DEFAULT_RUN} (pre-PR hardcoded value)."
),
),
cloudflare: Optional[bool] = typer.Option(
None,
"--cloudflare/--no-cloudflare",
rich_help_panel = _RUN_PANEL_SERVER,
help = "Expose Unsloth on a PUBLIC internet URL via a free Cloudflare HTTPS "
"tunnel, for non-api-only wildcard binds (0.0.0.0 or ::). Off by default; "
"pass --cloudflare to enable it (--secure implies it). --no-cloudflare forces "
"it off but does not change a raw wildcard bind.",
),
secure: bool = typer.Option(
False,
"--secure/--no-secure",
rich_help_panel = _RUN_PANEL_SERVER,
help = "Expose ONLY a Cloudflare HTTPS link: bind localhost and fail closed "
"if the tunnel can't start. Without it, --no-secure also serves the raw "
"0.0.0.0 port, which is reachable from anywhere on the network.",
),
not_secure: bool = typer.Option(
False,
"--not-secure",
hidden = True,
help = "Deprecated alias for --no-secure.",
),
tensor_parallel: bool = typer.Option(
False,
"--tensor-parallel/--no-tensor-parallel",
rich_help_panel = _RUN_PANEL_MODEL,
help = (
"Split a GGUF across GPUs by tensor (--split-mode tensor) instead of "
"by layer. Multi-GPU only (no effect on one GPU); dense models gain "
"decode speed, MoE usually don't."
),
),
start_api_key_marker: bool = typer.Option(
False,
"--start-api-key-marker",
hidden = True,
help = "Emit an early API key marker for the unsloth start parent process.",
),
password: str = typer.Option(
"",
"--password",
rich_help_panel = _RUN_PANEL_ADVANCED,
help = "Set the INITIAL admin password non-interactively (headless setups), "
"only when none is set yet. Also reads the UNSLOTH_STUDIO_PASSWORD env var, or "
"`--password -` to read one line from stdin. A literal value is visible in the "
"process list and shell history. Rotate later with `unsloth studio reset-password`.",
),
):
"""Start Unsloth, load a model, print an API key -- one-liner server.
Unknown flags pass through to llama-server (GGUF only). Unsloth
rejects managed flags with HTTP 400: model identity, network
(--host/--port/--path/--api-prefix/--reuse-port), auth/TLS
(--api-key/--ssl-*), single-model UI (--ui/--models-*/--webui),
and parallel slots (use --parallel above). Full denylist in
studio/backend/core/inference/llama_server_args.py. Other knobs
(-c, -ngl, --jinja, --flash-attn, -t, ...) pass through and
last-wins-override Unsloth's auto-set value.
Example:
unsloth studio run --model unsloth/Qwen3-1.7B-GGUF --gguf-variant UD-Q4_K_XL
unsloth studio run --model unsloth/Qwen3-1.7B-GGUF --temperature 0.7 --seed 42 --parallel 8
unsloth studio run --model some-model --chat-template-file /path/to/tpl.jinja
unsloth studio run --model unsloth/Qwen3-27B-GGUF --gguf-variant Q8_0 --tensor-parallel
"""
# A newer outer CLI can re-exec into an older Studio venv; pass this signal via
# env so an older child ignores it instead of treating it as a llama-server arg.
inherited_start_api_key_marker = _consume_start_api_key_marker_env()
start_api_key_marker = start_api_key_marker or inherited_start_api_key_marker
# Back-compat: --not-secure is a deprecated alias for --no-secure.
secure = _resolve_secure(secure, not_secure)
extra_llama_args: List[str] = list(ctx.args) if ctx.args else []
# Tool-call healing/nudging are read from the env at backend import. Resolve here
# (before any re-exec/import) so the in-venv child inherits the decision. When the
# flag is omitted, respect a value the parent already set (e.g. `unsloth start`
# forwards its choice via the env) and otherwise apply the default: healing on,
# nudging on for a CLI-launched server.
_healing_disabled = (
os.environ.get("UNSLOTH_DISABLE_TOOL_CALL_HEALING") == "1"
if tool_call_healing is None
else not tool_call_healing
)
os.environ["UNSLOTH_DISABLE_TOOL_CALL_HEALING"] = "1" if _healing_disabled else "0"
if tool_call_nudging is not None:
os.environ["UNSLOTH_TOOL_CALL_NUDGE"] = "1" if tool_call_nudging else "0"
elif "UNSLOTH_TOOL_CALL_NUDGE" not in os.environ:
os.environ["UNSLOTH_TOOL_CALL_NUDGE"] = "1"
# Sampling overrides: the backend resolver reads UNSLOTH_SAMPLING_* to hard-pin a field
# (winning over both the client and the per-model recommendation). Only write a flag that
# was set explicitly so an omitted flag inherits any value the parent forwarded (e.g.
# `unsloth start`) and, when nothing is set, leaves the per-model recommendation in charge.
for _sampling_env, _sampling_value in (
("UNSLOTH_SAMPLING_TEMPERATURE", temperature),
("UNSLOTH_SAMPLING_TOP_P", top_p),
("UNSLOTH_SAMPLING_TOP_K", top_k),
("UNSLOTH_SAMPLING_MIN_P", min_p),
("UNSLOTH_SAMPLING_REPETITION_PENALTY", repetition_penalty),
("UNSLOTH_SAMPLING_PRESENCE_PENALTY", presence_penalty),
):
if _sampling_value is not None:
os.environ[_sampling_env] = str(_sampling_value)
# Set before any re-exec so the in-venv server inherits it via the env.
# `run --verbose` used to pass through to llama-server (its own -v); keep
# that by forwarding --log-verbose so we add Unsloth logs without dropping it.
if verbose:
_enable_verbose_access_logs()
if not any(a in ("--verbose", "-v", "--log-verbose") for a in extra_llama_args):
extra_llama_args.append("--log-verbose")
if disable_dns_pinning:
os.environ["UNSLOTH_STUDIO_DISABLE_DNS_PINNING"] = "1"
else:
os.environ.setdefault("UNSLOTH_STUDIO_DISABLE_DNS_PINNING", "0")
# Promote legacy exact `-m`/`-hfr`/`-f` back into typer params;
# clusters stay in extras.
model, extra_llama_args = _consume_legacy_short_aliases(
extra_llama_args,
("-m", "-hfr"),
model,
"--model",
)
legacy_frontend, extra_llama_args = _consume_legacy_short_aliases(
extra_llama_args,
("-f",),
str(frontend) if frontend is not None else None,
"--frontend",
)
if legacy_frontend is not None and frontend is None:
frontend = Path(legacy_frontend)
if model is None:
typer.echo(
"Error: Missing option '--model' / '-hf' / '--hf-repo' "
"(legacy aliases '-m' / '-hfr' are still accepted).",
err = True,
)
raise typer.Exit(2)
# 0. Parse llama.cpp `repo:variant` in --model; error if also paired
# with --gguf-variant and they disagree.
parsed_repo, embedded_variant = _split_repo_variant(model)
if embedded_variant:
if gguf_variant and gguf_variant != embedded_variant:
typer.echo(
f"Error: --model embeds variant '{embedded_variant}' but "
f"--gguf-variant '{gguf_variant}' was also provided.",
err = True,
)
raise typer.Exit(1)
model = parsed_repo
gguf_variant = gguf_variant or embedded_variant
# --secure requires the tunnel; force a loopback bind so the raw port is never public.
if secure:
if cloudflare is False:
typer.echo(
"Error: --secure requires the Cloudflare tunnel; do not combine it "
"with --no-cloudflare.",
err = True,
)
raise typer.Exit(2)
if host not in ("127.0.0.1", "localhost", "::1"):
typer.echo(
"Note: --secure ignores -H (it binds loopback and serves only "
"through the Cloudflare tunnel). Drop --secure to bind "
f"{host} directly, or keep --secure for a tunnel-only public link.",
err = True,
)
host = "127.0.0.1"
# Tool policy no longer depends on the bind: tools default on everywhere
# (--secure is a loopback tunnel; the operator owns a raw bind). Resolve here
# so the re-exec'd child inherits a concrete decision.
from unsloth_cli._tool_policy import is_external_host, resolve_tool_policy
enable_tools = resolve_tool_policy(
host = host,
flag = enable_tools,
yes = yes,
silent = silent,
)
# 1. Re-exec into the studio venv (same pattern as studio_default). Resolve
# the child launcher BEFORE the gate: a headless gate strips the seeded
# .bootstrap_password, so aborting afterward (venv/entry point missing) would
# leave must_change_password=1 with no password to log in.
studio_venv_dir = STUDIO_HOME / "unsloth_studio"
in_studio_venv = sys.prefix.startswith(str(studio_venv_dir))
studio_bin = None
resolved_frontend = frontend
if not in_studio_venv:
studio_python = _studio_venv_python()
if not studio_python:
typer.echo("Unsloth Studio not set up. Run install.sh first.")
raise typer.Exit(1)
# Re-exec via the studio venv's `unsloth` console-script.
studio_bin = studio_python.parent / "unsloth"
if not studio_bin.is_file():
typer.echo("Unsloth venv missing 'unsloth' entry point. Re-run: unsloth studio setup")
raise typer.Exit(1)
# `run` serves the same Unsloth UI (unless --api-only); a public launch must
# have a servable login page BEFORE the gate strips the seeded password, or
# the child has no way to change it. Validate here and forward the resolved
# dist so a shadowed child that can't self-resolve one still serves it.
resolved_frontend = _require_servable_frontend_or_exit(
frontend = frontend,
api_only = api_only,
cloudflare = cloudflare,
host = host,
secure = secure,
)
else:
# In-venv (in-process) run: validate the servable frontend and importable
# backend before the headless gate strips the seeded password. Frontend
# check first (cheap); backend import is headless-only so a prompt isn't
# delayed.
resolved_frontend = _require_servable_frontend_or_exit(
frontend = frontend,
api_only = api_only,
cloudflare = cloudflare,
host = host,
secure = secure,
)
_validate_inproc_backend_before_strip(
cloudflare = cloudflare, host = host, secure = secure, api_only = api_only
)
# A supplied --password / UNSLOTH_STUDIO_PASSWORD / stdin sets the initial
# admin password here in the parent, before the gate and any re-exec, so the
# secret never reaches the child argv; strip the env var so a re-exec'd child
# can't re-read it. The interactive gate below then no-ops.
_apply_supplied_password_before_launch(_password_prompt.resolve_supplied_password(password))
os.environ.pop(_password_prompt.SUPPLIED_PASSWORD_ENV, None)
# Public (tunnel) exposure with the seeded default password: force a terminal
# password change first, before any re-exec or server exists. The re-exec here
# runs the studio venv's `unsloth` console script (a possibly-OLD child), so it
# is NOT provably self-suppressing -- only the in-process case is, and the
# strip stays in force otherwise.
_enforce_password_change_before_exposure(
cloudflare = cloudflare,
host = host,
secure = secure,
api_only = api_only,
child_self_suppresses = _child_self_suppresses(
in_studio_venv = in_studio_venv, child_run_py = None
),
)
if not in_studio_venv:
args = [
str(studio_bin),
"studio",
"run",
"--model",
model,
"--max-seq-length",
str(max_seq_length),
"--api-key-name",
api_key_name,
"--port",
str(port),
"--host",
host,
]
if gpu_memory_mode != "auto":
args.extend(["--gpu-memory-mode", gpu_memory_mode])
if gguf_variant:
args.extend(["--gguf-variant", gguf_variant])
# Forward the explicit polarity; a future default flip on one
# layer must not silently invert behaviour for the other.
args.append("--load-in-4bit" if load_in_4bit else "--no-load-in-4bit")
# Forward the frontend resolved before the gate, not just a user-supplied
# one: the parent may have found a built dist the shadowed child cannot,
# and stripping without forwarding it would abort the child at frontend
# setup (lockout).
if resolved_frontend is not None:
args.extend(["--frontend", str(resolved_frontend)])
if api_only:
args.append("--api-only")
if silent:
args.append("--silent")
# Forward the resolved tool policy so the child doesn't re-resolve.
if enable_tools:
args.append("--enable-tools")
else:
args.append("--disable-tools")
# Forward --yes only if the user passed it; resolution no longer prompts.
if yes:
args.append("--yes")
# Typer claims --parallel outside ctx.args; without this the
# child reverts to its default and silently drops the value.
args.extend(["--parallel", str(parallel)])
# Always forward explicit polarity: a mixed-version studio venv whose old
# default was --cloudflare-on must not silently re-enable the tunnel.
# --secure implies it, so forward nothing then.
if cloudflare is True:
args.append("--cloudflare")
elif not secure:
args.append("--no-cloudflare")
args.append("--secure" if secure else "--no-secure")
args.append("--tensor-parallel" if tensor_parallel else "--no-tensor-parallel")
if verbose:
args.append("--verbose")
# llama-server pass-through extras → child ctx.args → load payload.
if extra_llama_args:
args.extend(extra_llama_args)
if start_api_key_marker:
os.environ[_START_API_KEY_MARKER_ENV] = "1"
try:
if sys.platform == "win32":
proc = subprocess.Popen(args)
try:
rc = proc.wait()
except KeyboardInterrupt:
rc = proc.wait()
raise typer.Exit(rc)
else:
os.execvp(str(studio_bin), args)
finally:
# execvp doesn't return on success; restore env after a Windows wait or a failed launch.
os.environ.pop(_START_API_KEY_MARKER_ENV, None)
# ── 2. Start server (always suppress built-in banner) ─────────────
with _studio_deps.studio_backend_imports("unsloth studio"):
run_mod = _load_run_module()
run_server = run_mod.run_server
# Match the route handlers' import path: run.py adds studio/backend/ to
# sys.path, so they import as `state.tool_policy`. Set this before
# run_server() starts uvicorn; once sockets are bound, routes can be hit.
from state.tool_policy import set_tool_policy
set_tool_policy(enable_tools)
run_kwargs = dict(
host = host,
port = port,
silent = True,
api_only = api_only,
llama_parallel_slots = parallel,
cloudflare = cloudflare,
secure = secure,
# Headless serving prints its own URL/API-key banner; the Tauri-only
# TAURI_PORT line would corrupt that machine-parseable output.
emit_tauri_port = False,
)
# Forward the frontend validated before the gate (in-venv path).
if resolved_frontend is not None:
run_kwargs["frontend_path"] = resolved_frontend
app = run_server(**run_kwargs)
actual_port = getattr(app.state, "server_port", port) or port
# Steps 3-5 can abort (health timeout, model-load error, or Ctrl+C during the
# slow load); tear the server and its children (llama-server, cloudflared) down
# on any abort so they never orphan.
from studio.backend.run import _graceful_shutdown, _server
try:
# 3. Wait for server health.
if not silent:
typer.echo("Starting Unsloth Studio...")
if not _wait_for_server(actual_port):
typer.echo("Error: server did not become healthy within 30 seconds.", err = True)
raise typer.Exit(1)
# 4. Create API key in-process.
api_key = _create_api_key_inprocess(api_key_name)
if start_api_key_marker:
# `unsloth start` reads this key from a private 0600 log to authenticate
# download-progress polling; the normal `unsloth run` output is unchanged.
typer.echo(f"UNSLOTH_START_API_KEY: {api_key}")
# 5. Load model via HTTP.
if not silent:
typer.echo(f"Loading model: {model}...")
try:
result = _load_model_via_http(
port = actual_port,
api_key = api_key,
model = model,
gguf_variant = gguf_variant,
max_seq_length = max_seq_length,
load_in_4bit = load_in_4bit,
gpu_memory_mode = gpu_memory_mode,
tensor_parallel = tensor_parallel,
llama_extra_args = extra_llama_args,
)
except RuntimeError as exc:
typer.echo(f"Error: {exc}", err = True)
raise typer.Exit(1)
except BaseException:
_graceful_shutdown(_server)
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
raise
loaded_model = result.get("model", model)
display_variant = f" ({gguf_variant})" if gguf_variant else ""
context_length_line = _format_context_length_line(result)
# 6. Print banner.
display_host = _display_host_for_bind(run_mod, host)
base_url = f"http://{_url_host(display_host)}:{actual_port}"
sdk_base_url = f"{base_url}/v1"
# run_server started the tunnel during the silent run above (wildcard or --secure).
_cf_url = getattr(app.state, "cloudflare_url", None)
# --secure: examples must use the public tunnel URL, not the loopback address.
if secure and _cf_url:
sdk_base_url = f"{_cf_url}/v1"
# Orange so the tool-policy notice stands out; printed under
# --silent / --yes too so the policy is never invisible.
_tool_notice_fg = (217, 119, 87)
_is_external = is_external_host(host)
if not enable_tools:
_tool_notice = "Server-side tools are DISABLED (--disable-tools)."
elif secure:
_tool_notice = (
"Server-side tools are ENABLED, reachable via the authenticated "
"Cloudflare HTTPS tunnel. Anyone with the API key can run code on "
"this machine. Do not share the API key. Pass --disable-tools to turn off."
)
elif _is_external:
_tool_notice = (
"Server-side tools are ENABLED and this port is network-reachable. "
"Anyone who can reach it with the API key can run code on this "
"machine. Do not share the API key. Pass --disable-tools to turn off."
)
else:
_tool_notice = (
"Server-side tools are ENABLED for loopback. Pass --disable-tools to turn off."
)
if not silent:
typer.echo("")
typer.echo("=" * 56)
if secure and _cf_url:
typer.echo(f" Unsloth Studio running (secure) at {_cf_url}")
typer.echo(f" On this machine only: {base_url}")
else:
typer.echo(f" Unsloth Studio running at {base_url}")
_emit_run_cloudflare_notice(run_mod, host, display_host, actual_port, secure)
typer.echo(f" Model loaded: {loaded_model}{display_variant}")
if context_length_line:
typer.echo(context_length_line)
typer.echo(f" API Key: {api_key}")
typer.echo("")
typer.echo(" OpenAI / Anthropic SDK base URL:")
typer.echo(f" {sdk_base_url}")
typer.echo("=" * 56)
typer.secho(_tool_notice, fg = _tool_notice_fg, bold = True)
typer.echo("")
typer.echo("OpenAI Chat Completions:")
typer.echo(f" curl {sdk_base_url}/chat/completions \\")
typer.echo(f' -H "Authorization: Bearer {api_key}" \\')
typer.echo(' -H "Content-Type: application/json" \\')
typer.echo(
""" -d '{"messages": [{"role": "user", "content": "Hello"}], "stream": true}'"""
)
typer.echo("")
typer.echo("Anthropic Messages:")
typer.echo(f" curl {sdk_base_url}/messages \\")
typer.echo(f' -H "Authorization: Bearer {api_key}" \\')
typer.echo(' -H "Content-Type: application/json" \\')
typer.echo(
""" -d '{"max_tokens": 256, "messages": [{"role": "user", "content": "Hello"}], "stream": true}'"""
)
typer.echo("")
typer.echo("OpenAI Responses:")
typer.echo(f" curl {sdk_base_url}/responses \\")
typer.echo(f' -H "Authorization: Bearer {api_key}" \\')
typer.echo(' -H "Content-Type: application/json" \\')
typer.echo(""" -d '{"input": "Hello", "stream": true}'""")
typer.echo("")
else:
# Silent still prints URL + API key + tool-status policy.
if secure and _cf_url:
typer.echo(f"URL: {_cf_url}")
typer.echo(f"Local: {base_url}")
else:
typer.echo(f"URL: {base_url}")
_emit_run_cloudflare_notice(run_mod, host, display_host, actual_port, secure)
if context_length_line:
typer.echo(context_length_line.strip())
typer.echo(f"API Key: {api_key}")
typer.secho(_tool_notice, fg = _tool_notice_fg, bold = True)
# 7. Wait for Ctrl+C.
try:
if run_mod._shutdown_event is not None:
while not run_mod._shutdown_event.is_set():
run_mod._shutdown_event.wait(timeout = 1)
else:
while True:
time.sleep(1)
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
finally:
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
# ── unsloth studio stop ───────────────────────────────────────────────
_PID_FILE = STUDIO_HOME / "studio.pid"
def _pid_alive(pid: int) -> bool:
"""Return True if a process with ``pid`` exists.
``os.kill(pid, 0)`` raises OSError (WinError 87) for every pid on Windows,
so use ``tasklist`` there and the signal-0 probe elsewhere.
"""
if sys.platform == "win32":
try:
out = subprocess.run(
["tasklist", "/FI", f"PID eq {int(pid)}", "/NH", "/FO", "CSV"],
capture_output = True,
text = True,
timeout = 10,
).stdout
except Exception:
# Can't determine -- assume alive; taskkill no-ops if already gone.
return True
return f'"{int(pid)}"' in out
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
@studio_app.command()
def stop():
"""Stop a running Unsloth Studio server.
Reads the PID from ~/.unsloth/studio/studio.pid and sends SIGTERM
(or TerminateProcess on Windows) to shut it down gracefully.
"""
import signal as _signal
if not _PID_FILE.is_file():
typer.echo("No running Unsloth server found (no PID file).")
raise typer.Exit(0)
pid_text = _PID_FILE.read_text(encoding = "utf-8").strip()
if not pid_text.isdigit():
typer.echo(f"Invalid PID file contents: {pid_text}")
_PID_FILE.unlink(missing_ok = True)
raise typer.Exit(1)
pid = int(pid_text)
# Check if still alive (os.kill(pid, 0) is invalid on Windows -- see _pid_alive).
if not _pid_alive(pid):
typer.echo(f"Unsloth server (PID {pid}) is not running. Cleaning up stale PID file.")
_PID_FILE.unlink(missing_ok = True)
raise typer.Exit(0)
# Send SIGTERM (graceful shutdown) or TerminateProcess on Windows
try:
if sys.platform == "win32":
# /T also stops llama-server children, which otherwise keep GPU and port.
subprocess.run(["taskkill", "/PID", str(pid), "/T", "/F"], check = True)
else:
os.kill(pid, _signal.SIGTERM)
typer.echo(f"Sent shutdown signal to Unsloth server (PID {pid}).")
except ProcessLookupError:
typer.echo(f"Unsloth server (PID {pid}) already exited.")
_PID_FILE.unlink(missing_ok = True)
raise typer.Exit(0)
except Exception as e:
typer.echo(f"Failed to stop Unsloth server (PID {pid}): {e}", err = True)
raise typer.Exit(1)
# Wait briefly for the process to exit and clean up.
for _ in range(10):
time.sleep(0.5)
if not _pid_alive(pid):
_PID_FILE.unlink(missing_ok = True)
typer.echo("Unsloth server stopped.")
raise typer.Exit(0)
typer.echo("Unsloth server is shutting down (may take a few seconds).")
# ── unsloth studio setup / update ─────────────────────────────────────
def _run_setup_script(*, verbose: bool = False) -> None:
"""Find and run the studio setup/update script."""
script = _find_setup_script()
if not script:
typer.echo("Error: Could not find setup script (setup.sh / setup.ps1).")
raise typer.Exit(1)
env = {**os.environ, "UNSLOTH_VERBOSE": "1"} if verbose else None
if platform.system() == "Windows":
powershell_args = ["powershell.exe"]
if _should_hide_windows_subprocesses():
powershell_args.extend(
["-NoLogo", "-NoProfile", "-NonInteractive", "-WindowStyle", "Hidden"]
)
# Use -Command + `*>&1` (not -File) so setup.ps1's Write-Host output
# (Information stream #6) merges into stdout. -File drops it when
# stdout is a pipe, e.g. `unsloth studio update --local 2>&1 | tee`.
# Single-quote escaping handles paths containing apostrophes.
script_pwsh_literal = str(script).replace("'", "''")
powershell_args.extend(
[
"-ExecutionPolicy",
"Bypass",
"-Command",
f"& '{script_pwsh_literal}' *>&1",
]
)
# Explicitly hand std handles to the child so CI tee sees setup.ps1's
# output. On Windows, subprocess.run defaults to close_fds=True
# (bInheritHandles=False); combined with CREATE_NO_WINDOW the child
# has no console and no inherited handles, so Write-Host writes to
# nothing. Passing stdout/stderr makes Python mark the std handles
# inheritable via PROC_THREAD_ATTRIBUTE_HANDLE_LIST. Empty update.log
# on windows-latest CI was the smoking gun (runs 25533694490/25534292239).
result = subprocess.run(
powershell_args,
env = env,
stdin = _stream_for_subprocess(sys.stdin),
stdout = _stream_for_subprocess(sys.stdout),
stderr = _stream_for_subprocess(sys.stderr),
**_windows_hidden_subprocess_kwargs(),
)
else:
result = subprocess.run(["bash", str(script)], env = env)
if result.returncode != 0:
raise typer.Exit(result.returncode)
_INSTALLER_URL_BASH = "https://unsloth.ai/install.sh"
_INSTALLER_URL_PWSH = "https://unsloth.ai/install.ps1"
def _refresh_desktop_shortcuts(*, verbose: bool = False) -> None:
"""Re-run installer with --shortcuts-only to refresh launchers post-update."""
env = {**os.environ}
if verbose:
env["UNSLOTH_VERBOSE"] = "1"
is_windows = platform.system() == "Windows"
installer_name = "install.ps1" if is_windows else "install.sh"
installer_url = _INSTALLER_URL_PWSH if is_windows else _INSTALLER_URL_BASH
# Prefer local checkout, fall back to package dir, then network fetch.
local_repo = (os.environ.get("STUDIO_LOCAL_REPO") or "").strip()
candidates: list[Path] = []
if local_repo:
candidates.append(Path(local_repo) / installer_name)
candidates.append(_PACKAGE_ROOT / installer_name)
args = ["--shortcuts-only"]
if verbose:
args.append("--verbose")
if is_windows:
ps_argv: list[str] = ["powershell.exe"]
if _should_hide_windows_subprocesses():
ps_argv.extend(["-NoLogo", "-NoProfile", "-NonInteractive", "-WindowStyle", "Hidden"])
for script in candidates:
try:
if script.is_file():
quoted = str(script).replace("'", "''")
argv = list(ps_argv)
argv.extend(
[
"-ExecutionPolicy",
"Bypass",
"-Command",
f"& '{quoted}' {' '.join(args)} *>&1",
]
)
result = subprocess.run(
argv,
env = env,
check = False,
**_windows_hidden_subprocess_kwargs(),
)
if result.returncode != 0:
typer.echo(f" refresh-launcher install.ps1 exited {result.returncode}")
return
except OSError:
continue
# PyPI installs lack install.ps1: fetch + pipe to powershell stdin.
try:
request = urllib.request.Request(
installer_url, headers = {"User-Agent": "unsloth-studio-update"}
)
with urllib.request.urlopen(request, timeout = 30) as response:
installer = response.read().decode("utf-8", errors = "replace")
except (urllib.error.URLError, TimeoutError, OSError) as exc:
typer.echo(f" refresh-launcher skipped: could not fetch {installer_url} ({exc})")
return
# install.ps1 auto-invokes `Install-UnslothStudio @args` at EOF; over
# stdin `$args` is empty so that triggers the full installer flow
# (deps, venv, prompts) before our shortcuts-only call. Strip it.
installer = re.sub(
r"(?m)^[ \t]*Install-UnslothStudio[ \t]+@args[ \t]*\r?\n?",
"",
installer,
)
# stdin-piped scripts have empty $args, so call Install-UnslothStudio explicitly.
marker_args = " ".join(args)
wrapper = installer + f"\nInstall-UnslothStudio {marker_args}\n"
# Write to a UTF-8 BOM tempfile and use -File rather than -Command -.
# `powershell.exe -Command -` reads stdin via [Console]::InputEncoding
# (CP1252/OEM on most Windows boxes), which mangles box-drawing chars
# in install.ps1. -File reads the BOM and decodes correctly. The
# prefix gives AV/EDR engines (and grep'ing users) a clear identity.
ps1_fd, ps1_path = tempfile.mkstemp(
prefix = "unsloth-studio-refresh-",
suffix = ".ps1",
)
try:
with os.fdopen(ps1_fd, "wb") as fh:
fh.write(b"\xef\xbb\xbf" + wrapper.encode("utf-8"))
argv = list(ps_argv)
argv.extend(["-ExecutionPolicy", "Bypass", "-File", ps1_path])
try:
result = subprocess.run(
argv,
env = env,
check = False,
**_windows_hidden_subprocess_kwargs(),
)
if result.returncode != 0:
typer.echo(
f" refresh-launcher fetched install.ps1 exited {result.returncode}"
)
except OSError as exc:
typer.echo(f" refresh-launcher skipped: powershell exec failed ({exc})")
finally:
try:
os.unlink(ps1_path)
except OSError:
pass
return
for script in candidates:
try:
if script.is_file():
result = subprocess.run(
["bash", str(script), *args],
env = env,
check = False,
)
if result.returncode != 0:
typer.echo(f" refresh-launcher install.sh exited {result.returncode}")
return
except OSError:
continue
# PyPI installs lack install.sh: fetch upstream.
try:
request = urllib.request.Request(
installer_url, headers = {"User-Agent": "unsloth-studio-update"}
)
with urllib.request.urlopen(request, timeout = 30) as response:
installer = response.read()
except (urllib.error.URLError, TimeoutError, OSError) as exc:
typer.echo(f" refresh-launcher skipped: could not fetch {installer_url} ({exc})")
return
try:
result = subprocess.run(
["bash", "-s", "--", *args],
input = installer,
env = env,
check = False,
)
if result.returncode != 0:
typer.echo(f" refresh-launcher fetched install.sh exited {result.returncode}")
except OSError as exc:
typer.echo(f" refresh-launcher skipped: bash exec failed ({exc})")
@studio_app.command(hidden = True)
def setup(
verbose: bool = typer.Option(
False,
"--verbose",
"-v",
help = "Full pip/build output during setup for troubleshooting.",
),
):
"""Run Unsloth setup (called by install.ps1 / install.sh)."""
_run_setup_script(verbose = verbose)
@studio_app.command()
def update(
local: bool = typer.Option(False, "--local", help = "Install from local repo instead of PyPI"),
package: str = typer.Option(
"unsloth", "--package", help = "Package name to install/update (for testing)"
),
verbose: bool = typer.Option(
False,
"--verbose",
"-v",
help = "Full pip/build output during update for troubleshooting.",
),
):
"""Update Unsloth Studio dependencies and rebuild."""
# Re-export UNSLOTH_STUDIO_HOME for env-mode installs so the refresh
# subprocess resolves the same install root the user originally chose.
_ensure_studio_env_exported()
# Ensure SKIP_STUDIO_BASE is not inherited from a parent install.ps1 session
os.environ.pop("SKIP_STUDIO_BASE", None)
os.environ["STUDIO_PACKAGE_NAME"] = package
if local:
os.environ["STUDIO_LOCAL_INSTALL"] = "1"
# Pass the repo root explicitly so install_python_stack.py doesn't
# have to guess from SCRIPT_DIR (which may be inside site-packages).
repo_root = Path(__file__).resolve().parents[2]
os.environ["STUDIO_LOCAL_REPO"] = str(repo_root)
else:
os.environ["STUDIO_LOCAL_INSTALL"] = "0"
os.environ.pop("STUDIO_LOCAL_REPO", None)
_release_self_exe_lock_windows()
try:
_run_setup_script(verbose = verbose)
except BaseException:
# Restore unsloth.exe from .deleteme if setup failed before pip
# produced a replacement; otherwise the user has no CLI for recovery.
_restore_self_exe_lock_windows()
raise
# On Windows clear the .deleteme orphan now that pip wrote a fresh
# unsloth.exe; on next update os.replace would overwrite it anyway,
# but leaving a stale binary around invites cross-version restore
# confusion from _restore_self_exe_lock_windows.
_cleanup_self_exe_lock_windows()
# Tauri desktop owns its own bundle entries; skip CLI launcher refresh
# so a Tauri-initiated update doesn't create duplicate shortcuts.
if os.environ.get("UNSLOTH_TAURI_UPDATE") == "1":
if verbose:
typer.echo(" refresh-launcher skipped (Tauri update)")
return
_refresh_desktop_shortcuts(verbose = verbose)
def _release_self_exe_lock_windows() -> None:
"""Rename running unsloth.exe so pip can replace it. setup.ps1 also retries."""
if platform.system() != "Windows":
return
try:
venv_scripts = Path(sys.executable).resolve().parent
except OSError:
return
exe = venv_scripts / "unsloth.exe"
if not exe.exists():
return
stale = exe.with_suffix(".exe.deleteme")
try:
# os.replace is atomic-overwrite on Windows; os.rename would raise
# FileExistsError if a prior aborted update left a .deleteme behind.
os.replace(exe, stale)
except OSError as e:
# Not fatal; setup.ps1 retries from a sibling process.
print(f"[update] could not rename {exe.name} -> {stale.name}: {e}")
def _restore_self_exe_lock_windows() -> None:
"""If setup failed before pip wrote a working unsloth.exe, restore .deleteme."""
if platform.system() != "Windows":
return
try:
venv_scripts = Path(sys.executable).resolve().parent
except OSError:
return
exe = venv_scripts / "unsloth.exe"
stale = exe.with_suffix(".exe.deleteme")
if not stale.exists():
return
# Treat a missing or zero-byte exe as "pip didn't produce a usable
# replacement"; otherwise leave the new binary alone.
if exe.exists():
try:
if exe.stat().st_size > 0:
return
except OSError:
return
try:
os.replace(stale, exe)
except OSError as e:
print(f"[update] could not restore {stale.name} -> {exe.name}: {e}")
def _cleanup_self_exe_lock_windows() -> None:
"""Remove the .deleteme orphan after a successful update on Windows."""
if platform.system() != "Windows":
return
try:
venv_scripts = Path(sys.executable).resolve().parent
except OSError:
return
stale = (venv_scripts / "unsloth.exe").with_suffix(".exe.deleteme")
try:
stale.unlink(missing_ok = True)
except OSError:
pass
# ── unsloth studio reset-password ────────────────────────────────────
@studio_app.command("desktop-capabilities", hidden = True)
def desktop_capabilities(
json_output: bool = typer.Option(
False,
"--json",
help = "Emit machine-readable JSON.",
),
):
state = _install_state()
payload = {
"desktop_protocol_version": 1,
# 2 adds studio_install_ok; the desktop treats < 2 as stale rather than
# guess at an absent field.
"desktop_manageability_version": 2,
"supports_provision_desktop_auth": True,
"supports_api_only": True,
"supports_desktop_backend_ownership": True,
# Did the install finish and are the backend's boot deps still there.
"studio_install_ok": bool(state["ok"]),
"studio_install_reason": state["reason"],
"version": "unknown",
}
try:
from importlib.metadata import version as package_version
payload["version"] = package_version("unsloth")
except Exception:
pass
if json_output:
typer.echo(json.dumps(payload, sort_keys = True))
return
for key, value in payload.items():
typer.echo(f"{key}: {value}")
@studio_app.command("verify-install")
def verify_install(
json_output: bool = typer.Option(
False,
"--json",
help = "Emit machine-readable JSON.",
),
):
"""Check that the Unsloth Studio dependency install completed.
Exits 0 when complete, 1 otherwise. setup.sh / setup.ps1 use the exit code
to decide whether the "already up to date" fast path may be taken.
"""
state = _install_state()
if json_output:
typer.echo(json.dumps(state, sort_keys = True))
raise typer.Exit(0 if state["ok"] else 1)
if state["ok"]:
typer.echo("Unsloth Studio install is complete.")
raise typer.Exit(0)
typer.echo(f"Unsloth Studio install is incomplete ({state['reason']}).")
if state["missing"]:
typer.echo(f" missing packages: {', '.join(state['missing'])}")
typer.echo(" repair with: unsloth studio update")
raise typer.Exit(1)
@studio_app.command("provision-desktop-auth", hidden = True)
def provision_desktop_auth():
"""Create/repair desktop auth state for the local machine."""
auth_dir = STUDIO_HOME / "auth"
secret = _create_desktop_secret_in_cli()
_write_auth_secret(auth_dir / DESKTOP_SECRET_FILE, secret)
typer.echo("Desktop auth ready.")
@studio_app.command("reset-password")
def reset_password():
"""Reset the Unsloth admin password.
Deletes the auth database so that a fresh admin account with a new
random password is created on the next server start. The Unsloth
server must be restarted after running this command.
"""
auth_dir = STUDIO_HOME / "auth"
db_file = auth_dir / "auth.db"
stale_files = [
auth_dir / BOOTSTRAP_PASSWORD_FILE,
auth_dir / DESKTOP_SECRET_FILE,
]
had_db = db_file.exists()
# Delete auth.db FIRST and prove it is gone before touching the seeded
# credential files. If it cannot be removed (a running Unsloth or Windows
# holds it open, or a read-only auth dir), abort with the credential files
# untouched: deleting them while an un-resettable DB (must_change_password=1)
# survives would lock a forgotten-password reset out of any recovery
# credential. Failing here leaves a consistent, still-recoverable state.
try:
db_file.unlink(missing_ok = True)
except OSError as exc:
typer.echo(
f"Error: could not delete the auth database ({exc}). Stop any running "
"Unsloth and retry; no credential files were changed.",
err = True,
)
raise typer.Exit(1)
# The DB is gone, so the next start re-seeds. Invalidate the seeded plaintext
# credential files so that re-seed generates a FRESH password instead of
# reusing a stale one: unlink only ignores FileNotFoundError, so a
# locked/undeletable file (Windows AV, read-only dir) would otherwise survive
# and generate_bootstrap_password() would read it back and re-validate the
# credential this reset revoked. Truncate on unlink failure; if a file can be
# neither removed nor truncated, fail closed -- the DB is already gone, so a
# surviving plaintext would be reused, and the user must remove it manually.
for path in stale_files:
try:
path.unlink(missing_ok = True)
except OSError:
try:
path.write_text("", encoding = "utf-8")
except OSError as exc:
typer.echo(
f"Error: could not remove or clear {path.name} ({exc}); delete "
"it manually before restarting Unsloth or the old password may "
"be reused.",
err = True,
)
raise typer.Exit(1)
if not had_db:
typer.echo("No auth database found -- nothing to reset.")
raise typer.Exit(0)
typer.echo("Auth database deleted. Restart Unsloth Studio to get a new password.")