unsloth/studio/backend/auth/storage.py
Michael Han a00fe86c13
Studio: read model text as utf-8 so umlauts survive on Windows (#7467)
* Studio: read model text as utf-8 so umlauts survive on Windows

Chat rejects or mangles non-ASCII on Windows: "ä ö ü" in a prompt, a chat
template, or a model path comes back as mojibake, or the load dies with
UnicodeDecodeError.

open() and Path.read_text() fall back to locale.getencoding() when no encoding
is passed. On Windows that is the ANSI codepage (cp1252, cp932, cp1251, ... by
system locale), never UTF-8. Hugging Face writes these files as raw UTF-8, so
every read of one decodes with the wrong codec:

- tokenizer_config.json, which holds the chat template. Templates routinely
  carry -> arrows, smart quotes and CJK, so this is the common path into chat
- config.json and adapter_config.json
- modules.json, Ollama manifests, and the .py sources the remote-code scanner
  reads before a model is allowed to load

The llama-server and embedding-server stdout readers have the same problem via
subprocess(text = True); they now decode utf-8 with errors = "replace" so a
stray byte cannot kill a log reader.

Encoding arguments only, no logic changes.

tests/test_chat_text_encoding.py covers a config.json and a chat template
holding umlauts, arrows and CJK, plus the remote-code scanner reading a source
file with umlauts. Those pass anywhere the locale is already UTF-8, so a fourth
test re-runs the readers under -X warn_default_encoding and fails on any
platform if an encoding argument goes missing again.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: name utf-8 explicitly on the remaining text I/O, with an AST guard (#7465)

* Studio: name utf-8 explicitly on the remaining text I/O

Follow-up to the model-text reads in #7467, covering the rest of the backend:
system probes (nvidia-smi, amd-smi, powershell, git, node), package installers,
/proc and /sys readers, and internal marker files (pid, install id, bootstrap
password, Colab credentials).

Same reason as #7467. open(), Path.read_text()/write_text() and
subprocess(text = True) fall back to locale.getencoding(), which on Windows is
the ANSI codepage rather than UTF-8. These paths are mostly ASCII today, so this
is hardening, not a live bug. Encoding arguments only, no logic changes.

Adds tests/test_text_io_encoding.py: an AST guard walking every backend source
and asserting text I/O names its encoding, so the class of bug cannot creep back
in one call at a time. 275 files.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Catch aliased subprocess and positional Path.open, migrate legacy JSONL

The guard only matched a receiver literally named subprocess, so worker.py's
`import subprocess as _sp` hid three text = True installs that decode pip
output with the ANSI codepage. It also skipped any .open() with more than one
positional argument, though Path.open takes buffering/encoding/errors/newline
positionally.

Resuming a scrape written by an older release is the other half: those JSONL
lines are in the locale codepage, so the UTF-8 preload raised, the dedup keys
were silently forgotten and duplicates were appended to a now mixed-encoding
file. Decode with the locale codepage as fallback and rewrite as UTF-8 before
the append handle opens, since Windows cannot replace a file it holds open.

* Stream the JSONL preload and keep a torn line from relabelling the shard

Reading the whole shard to migrate it was wrong twice over. These files reach
gigabytes on a large scrape, so the preload now streams line by line and the
rewrite streams through a temp file.

Worse, one interrupted append used to condemn the file: the whole-file UTF-8
decode failed, every byte was retried as cp1252, and the rewrite persisted
mojibake over records that were fine. A line now counts as legacy only if the
locale codepage both decodes it and yields valid JSON, which a torn UTF-8 line
does not. Damaged lines are skipped and copied through byte for byte.

When the rewrite cannot be written at all, the append handle opens with the
legacy encoding rather than mixing UTF-8 into the file.

install_wheel takes run = subprocess.run as a parameter, so the guard cannot
see it. Both wheel installs there now name their encoding.

* Decide the shard's encoding from the file, not one line at a time

Some byte strings parse both ways. cp1251 `Р°` is D0 B0, which is also valid
UTF-8 for `а`, so a UTF-8-first parse quietly showed the wrong text instead of
migrating it.

A line now yields both readings, and the file decides. Any line that parses
under the codepage but not as UTF-8 is unambiguous evidence, and ambiguous lines
then follow that verdict, which is enough for any real shard: ordinary Cyrillic
or Japanese prose is invalid UTF-8 several times per line. Keys for ambiguous
lines are re-derived from the legacy reading during the rewrite.

A shard is undecidable only if every line is ambiguous, and nothing can tell
those apart.

latin-1 is also tried after the locale codepage, so a scrape carried from
Windows to a UTF-8 machine still has a reading rather than none. Requiring valid
JSON, not just a decode, keeps that from claiming torn lines.

* Weigh the whole shard, and never lose a record on the fallback path

One structurally valid JSON line carrying a stray 0x96 parses as cp1252, so a
single-line verdict let it relabel a healthy shard and mojibake every good
record in it. Each line with non-ASCII bytes now votes: parsing only under the
codepage is evidence for legacy, parsing as UTF-8 is evidence against, since
codepage text rarely forms valid multibyte UTF-8. Ties leave the file alone.

When the migration cannot be written the append handle uses the legacy codepage,
and errors = "replace" quietly turned characters it cannot hold into question
marks while write() still reported success. That path now escapes to \uXXXX
instead, which is ASCII, so every codepage holds it and json.loads returns the
exact characters. Nothing needs replacing, so errors = "strict" is safe.

stream_installer runs sys.executable, so its output is now decoded as UTF-8 by
utf8_child_env rather than read as the ANSI codepage.

* Only rewrite a shard we can attribute, and append ASCII when we cannot

latin-1 was doing too much work. It reads any byte, so it gave a moved shard a
reading, but it is the right text only for cp1252: cp1251 Привет came back as
Ïðèâåò and the rewrite made that permanent. The codepage is now trusted only
when it is the locale's, and an untrusted reading is never written back.

That leaves three cases where the file holds bytes UTF-8 cannot read and we are
not converting it: no codepage to attribute it to, ambiguous lines outvoting the
unambiguous ones, and a preload that could not read the file at all. All three
used to append UTF-8 into it. They now append pure ASCII, which every
ASCII-compatible codepage stores identically, so the file keeps decoding exactly
as it did and no record is lost.

Keys from the two readings are also kept apart. A damaged line in a healthy
shard was marked seen through its codepage reading, so the retry that would have
replaced the unreadable record was refused as a duplicate.

* Let the flash-attn install stub take the kwargs the installer now passes

_run_kwargs gained encoding and errors, so the one stub in this file that
spelled its signature out rejected the call. The other four here already take
**kwargs; this one now matches.

* Do not let a stuck temp file mask the migration failure

unlink() on the failure path could raise in its own right, on a stale
.utf8.tmp directory or a temp another process holds. That escaped the
constructor instead of returning False, so the caller never reached the ASCII
append fallback that keeps the shard single-encoding.

The pip fallback in install_wheel also spawns a Python child, so it gets
utf8_child_env like the probe above it already had. The uv and nvidia-smi
children are native binaries, where PYTHONIOENCODING would do nothing.

* Stop converting legacy shards; the encoding that wrote them is unknowable

trusted only ever meant that the bytes parse under this machine's codepage,
which for a single-byte codepage is nearly always true. A cp1251 shard opened on
a cp1252 Windows box decodes cleanly and would have been rewritten with Привет
as Ïðèâåò. That is the fourth way this rewrite could corrupt a shard, and the
common cause is that a file's encoding cannot be recovered from its bytes.

So the rewrite is gone. The shard is left exactly as found, and appends are pure
ASCII whenever it holds bytes UTF-8 cannot read, which is what actually
delivered the no-mixed-encoding guarantee the rewrite was added for. Dedup keys
still come from whichever reading parses, since ids are ASCII either way.

This also removes the temp file, so there is no longer any file mode or ACL to
carry across.

* Scan the sandbox shim; it is shipped code, not a build artifact

sandbox_site is on the sandboxed child's PYTHONPATH for every Python run
(tools.py:332, 2660), so excluding it let two unannotated text calls through in
code we ship. Both read and write the remap sidecar, which holds file paths.

The exclusion list is meant for build output only, so the directory comes off
it and the two calls name their encoding.

* Force the worker's pip children to UTF-8, and read DBCS keys with a DBCS codec

The three installer calls run sys.executable -m pip with an inherited
environment, so the parent decoded UTF-8 while the child emitted the ANSI
codepage. They now go through utf8_child_env like the other Python children.

Two tests asserted no env kwarg was passed as a stand-in for no HIP flag being
injected. They now assert the flag itself, which is the guarantee they were
written for and does not depend on how the env is delivered.

Separately, latin-1 cannot stand in for a double-byte codepage while recovering
dedup keys: cp932 表 is 95 5C, and the trail byte reads as a JSON backslash, so
the record failed to parse and its id was forgotten, appending a duplicate on
resume. cp932, cp936, cp949 and cp950 are tried too. The reading is still only
ever used for keys, which are ASCII and identical whichever codec parses.

* Require more than one legacy line before trusting its dedup keys

A shard whose valid records are all ASCII casts no UTF-8 votes, so a single
damaged line won the vote by itself, its key was remembered, and the retry that
would have replaced the unreadable record was refused.

One such line is genuinely undecidable: a legacy record with one accented
character and an ASCII record with one stray byte are the same shape. Reading it
as damage costs a duplicate; reading it as legacy loses the record for good.
Only one of those is recoverable, so it is now read as damage.

A real legacy shard has a legacy line for every record carrying an umlaut, so
its dedup is unaffected.

* Append ASCII whenever the shard already holds non-ASCII bytes

The gate asked whether any line was undecodable as UTF-8, which misses a shard
where every legacy line happens to be valid UTF-8 too. A cp1251 shard of Р°
records is bytes D0 B0 throughout, so appending 世界 as UTF-8 left a file where
cp1251 reads the old records correctly and the new one as mojibake, and UTF-8
does the reverse. No single decoding recovered the whole scrape.

The gate is now simply whether the shard holds any non-ASCII byte at all, which
covers both cases and is easier to reason about: if what is already there reads
differently under different encodings, do not add more bytes that do.

Appending ASCII costs only \uXXXX escapes, which json.loads turns back into the
exact characters, and it leaves the new record correct under either reading.

* Skip the two Linux-gated flash-attn tests off Linux

_should_try_runtime_flash_attn_install ends in sys.platform.startswith(
"linux"), and the threshold test one line above already asserts exactly that,
so the two tests that drive _ensure_flash_attn_for_long_context past the gate
cannot pass anywhere else: the call returns before it reports a status. They
were written on Linux and only surface once the suite actually runs on Windows
or macOS, where both fail on an empty status list. This PR is about making the
backend behave on Windows, so its own suite should be runnable there.

* Fail closed when a KFD topology node does not decode

This PR pins that read to utf-8, which turns an undecodable byte into
UnicodeDecodeError. That is a ValueError, not an OSError, so it slips past the
handler one line below and escapes a helper whose docstring promises to fail
closed on any unreadable node. The caller would then lose the whole HIP-order
map on a machine that has AMD GPUs, and the reason the helper fails closed is
that dropping a node shifts every later ordinal and lets a similar-capacity GPU
pass the total-size guard while showing another card's usage.

Widening the handler is the same one-line change main already made in #7487, so
the two agree and the eventual merge is clean.

* Tighten the comments added in this branch

* Treat an undecodable marker and undecodable metadata as malformed, not fatal

Two more places where pinning the decode changed the failure mode. A
UnicodeDecodeError is a ValueError, so neither `except OSError` nor
`except (JSONDecodeError, OSError)` catches it, and both sites had a documented
fallback that stopped being reached.

An undecodable .transport marker used to read as an unknown value, and the
caller then safely purged and restarted the partial download. It now aborts
prepare_cache_for_transport instead, so the transfer fails rather than retrying.

Undecodable .meta.json used to fall back to the file's own name, the same way
invalid JSON does. It now aborts URI construction for the entire unstructured
seed, so one corrupt byte in original_filename takes out the whole dataset.

Both handlers are widened, matching the KFD fix earlier on this branch.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Widen two more decode guards, and pin the kernel installer's pipe

Same shape as the ones already fixed here: the read was pinned to UTF-8 while
the handler around it still only catches OSError, and UnicodeDecodeError is a
ValueError.

hf_cache_snapshot_dir answers whether a model is already on disk, and the
offline embedding checks turn a raise into a 500. A torn refs/main used to
decode into a nonsense commit and miss the snapshot dir; it now skips that cache
root and keeps looking. _remove_pid_file runs first in _graceful_shutdown, so a
corrupt studio.pid raising there abandoned the inference, export, training and
tunnel children the rest of that function exists to kill.

ssm_runtime's source-build path builds its subprocess kwargs in a dict and
splats them through _run_with_heartbeat, so neither the encoding guard nor the
earlier sweep saw the text = True in it: pip's output was still decoded with the
Windows ANSI codepage, where a non-ASCII path or a compiler diagnostic mojibakes
or raises over an install that was going fine. It now pins the same
utf-8/replace pair install_wheel uses, and the HIP branch extends that env
rather than replacing it. The guard learned the dict-literal shape and reddens
on the old code (ssm_runtime.py:253).

* Tighten the comments around the UTF-8 text I/O pins

Collapse the multi-line rationales added with the encoding pins down to a
line or two each, drop what the code already says, and use one wording for
the repeated child-env note.

* Do not let an unreadable bootstrap password stop startup, and narrow the kwargs guard

ensure_default_admin calls _load_bootstrap_password for every existing admin and
the lifespan calls that with no handler, so pinning the decode turned a damaged
or pre-pin .bootstrap_password file into a backend that will not start. We write
that file ourselves in UTF-8, so a byte that will not decode belongs to a file
whose plaintext is worthless anyway; it now reads as no bootstrap password, the
same answer as an absent file. A readable one still loads.

The new kwargs check also judged every dict literal in the tree, so an unrelated
payload carrying "text": True would have been reported as subprocess
configuration with a misleading message, and a dict that fills in its encoding on
a later line would have been reported too. It now only judges a dict that
actually reaches a call, either splatted through a name or written at the call
site, and treats a later kw["encoding"] assignment as satisfying it. The
ssm_runtime shape it was written for is still caught, and a test pins both
directions.

* Stop reading a UTF-8 record a second time

_read_line always parsed the line under the codepage as well, even when it had
already read as UTF-8. Both callers take the UTF-8 reading when there is one and
never look at the other, so on a healthy shard the second parse is pure waste,
and this file reads all of one on every resume of a scrape it expects to reach
gigabytes. Measured on 200,000 records, 76 MB: 1.96s before, 0.81s after, so the
double reading was costing 2.8x.

The early return is limited to a record, since the key lookup deliberately falls
through to the codepage reading when UTF-8 yields something that is not one. A
line UTF-8 cannot read still tries the codepage, latin-1 and the double-byte
encodings as before, which is what the second reading is for.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin the scanned source fixture's line endings

test_remote_code_scan_reads_non_ascii_sources compared a file's contents against
the string it wrote, but wrote it in text mode, so Windows translated the line
ends on the way out and the read back differed by a carriage return. That is the
writer's doing, not the encoding the test is about, and it was the one failure on
the Windows runner that belonged to this branch. The fixture now writes with
newline = "" so the bytes on disk are the string on every platform.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the newer comments to their point

Shorten the widened-guard and state store notes added since the last pass,
and collapse the line-ending note on the scanned source fixture.

* Read the scraper checkpoint as UTF-8 only, never as a codepage

A checkpoint holds nothing but base64 cursors and booleans, so one written by
an older locale-encoded release is byte-identical to a UTF-8 one and already
reads back. The codepage fallback can therefore only ever contribute non-ASCII:
if a single-byte reading of the file were all ASCII, the UTF-8 read would have
succeeded first.

So the only file it changes the answer for is a damaged one, and there it turns
a safe reset into a resume on a mojibaked cursor. GitHub answers that with
INVALID_CURSOR_ARGUMENTS at HTTP 200, gh_client returns the partial document,
and the scraper reads zero nodes and an empty pageInfo, which marks the stream
done. Every later resume then skips it entirely.

Reading UTF-8 only restores the earlier behaviour of dropping a checkpoint that
will not decode, which re-scrapes from the first page while the writers dedup
the replay. The shard scan below keeps its codepage reading; those records do
carry non-ASCII.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate the remaining tilelang install tests to Linux

_tilelang_platform_supported() returns False off Linux, so _ensure_tilelang_backend
returns before the install and the subprocess mock these six assert on is never
called. They fail on macOS runners for that reason alone. The rest of the file
already carries this marker; these were missed.

* Gate the Windows-incompatible worker and ROCm tests

Two different gates, because the production code has two. The causal-conv1d and
flash-linear-attention installers bail out on sys.platform == 'win32' alone and
run everywhere else including macOS, so those cases get not_on_windows; marking
them linux_only would skip tests that legitimately pass off Linux. The DRM and
KFD readers return early unless platform.system() is Linux, and their fixtures
build a fake sysfs tree needing PCI addresses like 0000:00:02.0 as directory
names, which Windows cannot represent, so those get linux_only.

The two visible-utilization cases failed for a different reason: on Windows
get_visible_gpu_utilization takes the AMD adapter branch ahead of the torch
fallback under test, and probing it imports torch, which the runner lacks.
Stubbing that branch empty leaves every other platform unchanged.

* Treat unparseable JSON nesting as a parse failure, and guard os.fdopen

json.loads answers nesting it cannot descend with RecursionError, a
RuntimeError, so _parse let it escape where the catch-all it replaced
discarded the record. Both callers run _parse outside any further handler,
so one damaged checkpoint or shard line aborted the scraper at startup.

The encoding guard also missed os.fdopen, which is open() on a descriptor
and takes the same locale default in text mode. It flags exactly the two
text-mode calls that were left unencoded; the swap lock file's reader was
already pinned to UTF-8 while its writer still used the codepage.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Write the non-ASCII source fixture without a 3.10-only argument

Path.write_text() only grew newline in 3.10, and pyproject declares
requires-python >=3.9, so this raised TypeError there. open() takes the same
argument on every supported version and pins the bytes on disk the same way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten encoding comments

* Follow subprocess calls through callable aliases in the encoding guard

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-28 21:27:27 -07:00

944 lines
31 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""SQLite storage for auth data (user credentials + JWT secret)."""
import hashlib
import hmac
import ipaddress
import os
import secrets
import sqlite3
import threading
from datetime import datetime, timezone
from typing import Optional, Tuple
from utils.paths import auth_db_path, ensure_dir
DB_PATH = auth_db_path()
DEFAULT_ADMIN_USERNAME = "unsloth"
# Single source for the password policy; models/auth.py ChangePasswordRequest
# and the terminal prompt both enforce it. Keep the unsloth_cli mirror in sync.
MIN_PASSWORD_LENGTH = 8
# Plaintext bootstrap password file beside auth.db, deleted on first password
# change so the credential never lingers on disk.
_BOOTSTRAP_PW_PATH = DB_PATH.parent / ".bootstrap_password"
# In-process cache to avoid re-reading the file on every HTML serve.
_bootstrap_password: Optional[str] = None
def generate_bootstrap_password() -> str:
"""Generate a 4-word diceware passphrase and persist it to disk.
Persisted (the DB stores only the hash) so it survives restarts; later
calls return the persisted value.
"""
global _bootstrap_password
# Cached in this process?
if _bootstrap_password is not None:
return _bootstrap_password
# Persisted from a previous run?
if _BOOTSTRAP_PW_PATH.is_file():
_bootstrap_password = _BOOTSTRAP_PW_PATH.read_text(encoding = "utf-8").strip()
if _bootstrap_password:
return _bootstrap_password
# First startup: generate a fresh passphrase.
import diceware
_bootstrap_password = diceware.get_passphrase(
options = diceware.handle_options(args = ["-n", "4", "-d", "", "-c"])
)
# Persist so the same passphrase survives restarts until password change.
ensure_dir(_BOOTSTRAP_PW_PATH.parent)
_BOOTSTRAP_PW_PATH.write_text(_bootstrap_password, encoding = "utf-8")
try:
os.chmod(_BOOTSTRAP_PW_PATH, 0o600)
except OSError:
pass
return _bootstrap_password
def get_bootstrap_password() -> Optional[str]:
"""Return the cached bootstrap password, or None if not yet generated."""
return _bootstrap_password
def _load_bootstrap_password() -> Optional[str]:
"""Load an existing bootstrap password without creating one."""
global _bootstrap_password
_bootstrap_password = None
if _BOOTSTRAP_PW_PATH.is_file():
# No caller handles a raise, so an unreadable file has to mean "no bootstrap
# password", not a dead backend. We write UTF-8, so bytes that will not
# decode are damage whose plaintext is worthless anyway.
try:
bootstrap_password = _BOOTSTRAP_PW_PATH.read_text(encoding = "utf-8").strip()
except (OSError, UnicodeDecodeError):
return _bootstrap_password
if bootstrap_password:
_bootstrap_password = bootstrap_password
return _bootstrap_password
def clear_bootstrap_password() -> None:
"""Delete the persisted bootstrap password file (after a password change).
Best-effort: the new hash is already committed, so a locked/undeletable file
(Windows AV, read-only auth dir) must not fail the change.
"""
global _bootstrap_password
_bootstrap_password = None
if _BOOTSTRAP_PW_PATH.is_file():
try:
_BOOTSTRAP_PW_PATH.unlink(missing_ok = True)
except OSError as e:
# Removal failed (Windows AV, read-only auth dir). The hash is already
# committed, so don't fail the change -- but truncate the file so its
# stale plaintext can't be re-seeded by generate_bootstrap_password()
# if a later reset-password deletes auth.db and re-validates it.
try:
_BOOTSTRAP_PW_PATH.write_text("", encoding = "utf-8")
cleared = True
except OSError:
cleared = False
import sys
if cleared:
message = (
f"Warning: could not delete {_BOOTSTRAP_PW_PATH.name} ({e}); "
"cleared its contents so the old bootstrap password cannot be reused."
)
else:
# Neither removed nor truncated: stale plaintext is still on disk
# and would be reused if auth.db is reset. Don't claim otherwise.
message = (
f"Warning: could not delete or clear {_BOOTSTRAP_PW_PATH.name} ({e}); "
"its old bootstrap password is still on disk. Remove it manually to "
"prevent reuse after a reset."
)
print(message, file = sys.stderr, flush = True)
def _hash_token(token: str) -> str:
"""SHA-256 hash helper for refresh token storage.
Plain SHA-256 is intentional: refresh tokens are 384-bit random strings, so
a slow KDF adds no security while costing per-refresh latency. API keys use
the separate ``_pbkdf2_api_key`` helper, only to satisfy CodeQL's
``py/weak-sensitive-data-hashing`` query, not for crypto reasons.
"""
return hashlib.sha256(token.encode("utf-8")).hexdigest()
def get_connection() -> sqlite3.Connection:
"""Get a connection to the auth database, creating tables if needed."""
ensure_dir(DB_PATH.parent)
conn = sqlite3.connect(DB_PATH)
# Keep the auth dir + DB private (they hold the JWT/identity secrets and
# password hashes); sqlite3.connect would otherwise create the DB 0644 under
# a 022 umask, letting another OS user read the identity secret and forge proofs.
for _path, _mode in ((DB_PATH.parent, 0o700), (DB_PATH, 0o600)):
try:
os.chmod(_path, _mode)
except OSError:
pass
conn.row_factory = sqlite3.Row
# WAL lets token reads run concurrently with refresh-token writes;
# busy_timeout bounds lock waits. Matches the other Unsloth SQLite stores.
# Set busy_timeout first: switching journal_mode needs a lock, so if a
# refresh-token write already holds one, journal_mode=WAL raises SQLITE_BUSY;
# with busy_timeout already in effect it waits instead of failing and leaving
# this connection on SQLite's default zero lock wait.
try:
conn.execute("PRAGMA busy_timeout=5000")
conn.execute("PRAGMA journal_mode=WAL")
except sqlite3.Error:
pass
conn.execute(
"""
CREATE TABLE IF NOT EXISTS auth_user (
id INTEGER PRIMARY KEY,
username TEXT UNIQUE NOT NULL,
password_salt TEXT NOT NULL,
password_hash TEXT NOT NULL,
jwt_secret TEXT NOT NULL,
must_change_password INTEGER NOT NULL DEFAULT 0
);
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS refresh_tokens (
id INTEGER PRIMARY KEY,
token_hash TEXT NOT NULL,
username TEXT NOT NULL,
expires_at TEXT NOT NULL,
is_desktop INTEGER NOT NULL DEFAULT 0
);
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS api_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL,
key_prefix TEXT NOT NULL,
key_hash TEXT NOT NULL UNIQUE,
name TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
last_used_at TEXT,
expires_at TEXT,
is_active INTEGER NOT NULL DEFAULT 1,
is_internal INTEGER NOT NULL DEFAULT 0
);
"""
)
api_key_columns = {row["name"] for row in conn.execute("PRAGMA table_info(api_keys)")}
if "is_internal" not in api_key_columns:
conn.execute("ALTER TABLE api_keys ADD COLUMN is_internal INTEGER NOT NULL DEFAULT 0")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS app_secrets (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
"""
)
columns = {row["name"] for row in conn.execute("PRAGMA table_info(auth_user)")}
if "must_change_password" not in columns:
conn.execute(
"ALTER TABLE auth_user ADD COLUMN must_change_password INTEGER NOT NULL DEFAULT 0"
)
refresh_columns = {row["name"] for row in conn.execute("PRAGMA table_info(refresh_tokens)")}
if "is_desktop" not in refresh_columns:
conn.execute("ALTER TABLE refresh_tokens ADD COLUMN is_desktop INTEGER NOT NULL DEFAULT 0")
conn.commit()
return conn
# ── API-key PBKDF2 salt ────────────────────────────────────────────────
#
# Module-level cache for the persistent API-key PBKDF2 salt, populated lazily
# via ``_get_or_create_api_key_pbkdf2_salt``. No lock needed: (a) ``INSERT OR
# IGNORE`` is atomic at the SQLite layer and (b) concurrent populations
# converge on the same value, so the worst case is a harmless duplicate read
# on startup.
_api_key_pbkdf2_salt_cache: Optional[bytes] = None
def _get_or_create_api_key_pbkdf2_salt() -> bytes:
"""Return the persistent API-key PBKDF2 salt, generating it once if missing.
Hex-encoded 32-byte random value in ``app_secrets``. Regenerated only when
the row is missing (fresh install, or operator deleted it).
"""
global _api_key_pbkdf2_salt_cache
if _api_key_pbkdf2_salt_cache is not None:
return _api_key_pbkdf2_salt_cache
conn = get_connection()
try:
cur = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
("api_key_pbkdf2_salt",),
)
row = cur.fetchone()
if row is None:
new_value = secrets.token_hex(32) # 32 bytes -> 64 hex chars
conn.execute(
"INSERT OR IGNORE INTO app_secrets (key, value) VALUES (?, ?)",
("api_key_pbkdf2_salt", new_value),
)
conn.commit()
cur = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
("api_key_pbkdf2_salt",),
)
row = cur.fetchone()
salt = bytes.fromhex(row["value"])
finally:
conn.close()
_api_key_pbkdf2_salt_cache = salt
return salt
# Secret answering the /api/auth/identity challenge (HMAC(secret, nonce)). Lives
# in this same-user DB so a port squatter or remote/fake server can't forge a
# proof. Separate from the per-user JWT secret.
_IDENTITY_SECRET_DB_KEY = "studio_identity_secret"
_identity_secret_cache: Optional[bytes] = None
def get_or_create_identity_secret() -> bytes:
"""Return the identity secret (hex 32-byte row in app_secrets), creating it once."""
global _identity_secret_cache
if _identity_secret_cache is not None:
return _identity_secret_cache
conn = get_connection()
try:
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(_IDENTITY_SECRET_DB_KEY,),
).fetchone()
if row is None:
conn.execute(
"INSERT OR IGNORE INTO app_secrets (key, value) VALUES (?, ?)",
(_IDENTITY_SECRET_DB_KEY, secrets.token_hex(32)),
)
conn.commit()
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(_IDENTITY_SECRET_DB_KEY,),
).fetchone()
secret = bytes.fromhex(row["value"])
finally:
conn.close()
_identity_secret_cache = secret
return secret
def compute_identity_proof(nonce: bytes, host: str, port: int) -> str:
"""HMAC-SHA256 proof that the caller holds this install's identity secret,
bound to the loopback address and port the connection landed on. A proof
relayed from an Unsloth on a different address/port (a squatter proxying to the
real one, e.g. localhost resolving to ::1 while Unsloth is on 127.0.0.1) was
computed for that other endpoint and won't match the one the client dialed."""
try:
host = ipaddress.ip_address(host).compressed # normalise 127.0.0.1 / ::1 forms
except ValueError:
host = (host or "").lower()
msg = b"|".join([nonce, host.encode(), str(int(port)).encode()])
return hmac.new(get_or_create_identity_secret(), msg, hashlib.sha256).hexdigest()
# Capability secret for public ``/p`` preview share links. HMAC(secret, ref)
# turns the deterministic preview ref into an unguessable bearer capability, so a
# guessed run/checkpoint name can't reach inference. Dedicated (not the per-user
# JWT secret) so rotating it revokes every shared link without touching logins.
_PREVIEW_LINK_SECRET_DB_KEY = "preview_link_secret"
_preview_link_secret_cache: Optional[bytes] = None
def get_or_create_preview_link_secret() -> bytes:
"""Return the preview-link signing secret (hex 32-byte row in app_secrets), creating it once."""
global _preview_link_secret_cache
if _preview_link_secret_cache is not None:
return _preview_link_secret_cache
conn = get_connection()
try:
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(_PREVIEW_LINK_SECRET_DB_KEY,),
).fetchone()
if row is None:
conn.execute(
"INSERT OR IGNORE INTO app_secrets (key, value) VALUES (?, ?)",
(_PREVIEW_LINK_SECRET_DB_KEY, secrets.token_hex(32)),
)
conn.commit()
row = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(_PREVIEW_LINK_SECRET_DB_KEY,),
).fetchone()
secret = bytes.fromhex(row["value"])
finally:
conn.close()
_preview_link_secret_cache = secret
return secret
def rotate_preview_link_secret() -> bytes:
"""Rotate the preview-link secret, immediately revoking every outstanding ``/p`` share link."""
global _preview_link_secret_cache
new_secret_hex = secrets.token_hex(32)
conn = get_connection()
try:
conn.execute(
"INSERT OR REPLACE INTO app_secrets (key, value) VALUES (?, ?)",
(_PREVIEW_LINK_SECRET_DB_KEY, new_secret_hex),
)
conn.commit()
finally:
conn.close()
secret = bytes.fromhex(new_secret_hex)
_preview_link_secret_cache = secret
return secret
_API_KEY_PBKDF2_ITERATIONS = 100_000
DESKTOP_SECRET_PREFIX = "desktop-"
_DESKTOP_SECRET_HASH_KEY = "desktop_secret_hash"
_DESKTOP_SECRET_CREATED_AT_KEY = "desktop_secret_created_at"
def _pbkdf2_api_key(raw_key: str) -> str:
"""PBKDF2-HMAC-SHA256 an API key with a persistent server-side salt.
For API-key storage ONLY, not refresh tokens. The slow KDF is only to
appease CodeQL's ``py/weak-sensitive-data-hashing`` query, not a crypto
requirement (API keys are random 128-bit tokens). The salt lives in
``app_secrets`` so dumping ``api_keys`` alone can't derive hashes.
"""
salt = _get_or_create_api_key_pbkdf2_salt()
dk = hashlib.pbkdf2_hmac(
"sha256",
raw_key.encode("utf-8"),
salt,
_API_KEY_PBKDF2_ITERATIONS,
)
return dk.hex()
def _pbkdf2_desktop_secret(raw_secret: str) -> str:
return _pbkdf2_api_key(raw_secret)
# Memoize the deterministic raw-key -> PBKDF2-hash derivation so the 100k-round
# KDF runs once per key instead of on every authenticated request. Keyed by a
# salted HMAC of the key (not the key itself); revocation/expiry are still
# enforced by the SQLite read on every call, so a cache hit only skips the KDF.
# Only keys present in the DB are cached, so unknown-key spam can't grow it.
_api_key_hash_cache: dict[str, str] = {}
_API_KEY_HASH_CACHE_MAX = 4096
_api_key_hash_cache_lock = threading.Lock()
def _api_key_cache_id(raw_key: str) -> str:
"""Cache id for a raw key: salted HMAC-SHA256 (not the key itself)."""
return hmac.new(
_get_or_create_api_key_pbkdf2_salt(), raw_key.encode("utf-8"), hashlib.sha256
).hexdigest()
def _reset_api_key_hash_cache() -> None:
"""Drop memoized derivations (tests / salt change)."""
with _api_key_hash_cache_lock:
_api_key_hash_cache.clear()
def is_initialized() -> bool:
"""Check if auth is ready for login (at least one user exists in DB)."""
conn = get_connection()
cur = conn.execute("SELECT COUNT(*) AS c FROM auth_user")
row = cur.fetchone()
conn.close()
return bool(row["c"])
def create_initial_user(
username: str,
password: str,
jwt_secret: str,
*,
must_change_password: bool = False,
) -> None:
"""
Create the initial admin user in the database.
Raises sqlite3.IntegrityError if username already exists.
"""
from .hashing import hash_password
salt, pwd_hash = hash_password(password)
conn = get_connection()
try:
conn.execute(
"""
INSERT INTO auth_user (
username,
password_salt,
password_hash,
jwt_secret,
must_change_password
)
VALUES (?, ?, ?, ?, ?)
""",
(username, salt, pwd_hash, jwt_secret, int(must_change_password)),
)
conn.commit()
finally:
conn.close()
def delete_user(username: str) -> None:
"""
Delete a user from the database.
Used for rollback when user creation fails partway through bootstrap.
"""
conn = get_connection()
try:
conn.execute("DELETE FROM auth_user WHERE username = ?", (username,))
conn.commit()
finally:
conn.close()
def get_user_and_secret(username: str) -> Optional[Tuple[str, str, str, bool]]:
"""
Get user's password salt, hash, and JWT secret.
Returns (password_salt, password_hash, jwt_secret, must_change_password)
or None if user not found.
"""
conn = get_connection()
try:
cur = conn.execute(
"""
SELECT password_salt, password_hash, jwt_secret, must_change_password
FROM auth_user
WHERE username = ?
""",
(username,),
)
row = cur.fetchone()
if not row:
return None
return (
row["password_salt"],
row["password_hash"],
row["jwt_secret"],
bool(row["must_change_password"]),
)
finally:
conn.close()
def get_jwt_secret(username: str) -> Optional[str]:
"""Return the current JWT signing secret for a user."""
conn = get_connection()
try:
cur = conn.execute(
"SELECT jwt_secret FROM auth_user WHERE username = ?",
(username,),
)
row = cur.fetchone()
return row["jwt_secret"] if row else None
finally:
conn.close()
def requires_password_change(username: str) -> bool:
"""Return whether the user must change the seeded default password."""
conn = get_connection()
try:
cur = conn.execute(
"SELECT must_change_password FROM auth_user WHERE username = ?",
(username,),
)
row = cur.fetchone()
return bool(row and row["must_change_password"])
finally:
conn.close()
def load_jwt_secret() -> str:
"""
Load the JWT secret from the database.
Raises RuntimeError if no auth user has been created yet.
"""
conn = get_connection()
try:
cur = conn.execute("SELECT jwt_secret FROM auth_user LIMIT 1")
row = cur.fetchone()
if not row:
raise RuntimeError(
"Auth is not initialized. Wait for the seeded admin bootstrap to complete."
)
return row["jwt_secret"]
finally:
conn.close()
def ensure_default_admin() -> bool:
"""Seed the default admin account on first startup.
Uses a randomly generated diceware passphrase as the bootstrap password.
Returns True when the default admin was created in this call.
"""
if get_user_and_secret(DEFAULT_ADMIN_USERNAME) is not None:
_load_bootstrap_password()
return False
bootstrap_pw = generate_bootstrap_password()
try:
create_initial_user(
username = DEFAULT_ADMIN_USERNAME,
password = bootstrap_pw,
jwt_secret = secrets.token_urlsafe(64),
must_change_password = True,
)
return True
except sqlite3.IntegrityError:
return False
def update_password(
username: str,
new_password: str,
*,
revoke_refresh_tokens: bool = False,
) -> bool:
"""Update password, clear first-login requirement, rotate JWT secret.
``revoke_refresh_tokens`` deletes the user's refresh tokens in the SAME
transaction: a separate delete could fail after the password commit and
leave a pre-change token still able to mint access tokens.
"""
from .hashing import hash_password
salt, pwd_hash = hash_password(new_password)
jwt_secret = secrets.token_urlsafe(64)
conn = get_connection()
try:
cursor = conn.execute(
"""
UPDATE auth_user
SET password_salt = ?, password_hash = ?, jwt_secret = ?, must_change_password = 0
WHERE username = ?
""",
(salt, pwd_hash, jwt_secret, username),
)
if revoke_refresh_tokens and cursor.rowcount > 0:
conn.execute("DELETE FROM refresh_tokens WHERE username = ?", (username,))
conn.commit()
if cursor.rowcount > 0:
clear_bootstrap_password()
clear_desktop_secret()
return cursor.rowcount > 0
finally:
conn.close()
def save_refresh_token(
token: str,
username: str,
expires_at: str,
*,
is_desktop: bool = False,
) -> None:
"""
Store a hashed refresh token with its associated username and expiry.
"""
token_hash = _hash_token(token)
conn = get_connection()
try:
conn.execute(
"""
INSERT INTO refresh_tokens (token_hash, username, expires_at, is_desktop)
VALUES (?, ?, ?, ?)
""",
(token_hash, username, expires_at, int(is_desktop)),
)
conn.commit()
finally:
conn.close()
def consume_refresh_token(token: str) -> Optional[Tuple[str, bool]]:
"""Atomically validate-and-delete a refresh token for single-use rotation.
DELETE RETURNING fuses validate and delete into one statement so two
concurrent refresh requests cannot both consume the same token.
"""
token_hash = _hash_token(token)
now = datetime.now(timezone.utc).isoformat()
conn = get_connection()
try:
conn.execute(
"DELETE FROM refresh_tokens WHERE expires_at < ?",
(now,),
)
cur = conn.execute(
"""
DELETE FROM refresh_tokens
WHERE token_hash = ? AND expires_at >= ?
RETURNING username, is_desktop
""",
(token_hash, now),
)
row = cur.fetchone()
conn.commit()
if row is None:
return None
return row["username"], bool(row["is_desktop"])
finally:
conn.close()
def verify_refresh_token(token: str) -> Optional[Tuple[str, bool]]:
"""
Verify a refresh token and return the username plus desktop marker.
Returns the username and desktop marker if valid and not expired, None otherwise.
The token is NOT consumed — it stays valid until it expires.
"""
token_hash = _hash_token(token)
conn = get_connection()
try:
# Opportunistically clean up expired tokens
conn.execute(
"DELETE FROM refresh_tokens WHERE expires_at < ?",
(datetime.now(timezone.utc).isoformat(),),
)
conn.commit()
cur = conn.execute(
"""
SELECT id, username, expires_at, is_desktop FROM refresh_tokens
WHERE token_hash = ?
""",
(token_hash,),
)
row = cur.fetchone()
if row is None:
return None
# Check expiry
expires_at = datetime.fromisoformat(row["expires_at"])
if datetime.now(timezone.utc) > expires_at:
conn.execute("DELETE FROM refresh_tokens WHERE id = ?", (row["id"],))
conn.commit()
return None
return row["username"], bool(row["is_desktop"])
finally:
conn.close()
def revoke_user_refresh_tokens(username: str) -> None:
"""Revoke all refresh tokens for a user (e.g. on logout)."""
conn = get_connection()
try:
conn.execute("DELETE FROM refresh_tokens WHERE username = ?", (username,))
conn.commit()
finally:
conn.close()
def create_desktop_secret() -> str:
"""Create/rotate the local desktop credential and return it once."""
ensure_default_admin()
raw_secret = DESKTOP_SECRET_PREFIX + secrets.token_urlsafe(48)
secret_hash = _pbkdf2_desktop_secret(raw_secret)
now = datetime.now(timezone.utc).isoformat()
conn = get_connection()
try:
conn.execute(
"INSERT OR REPLACE INTO app_secrets (key, value) VALUES (?, ?)",
(_DESKTOP_SECRET_HASH_KEY, secret_hash),
)
conn.execute(
"INSERT OR REPLACE INTO app_secrets (key, value) VALUES (?, ?)",
(_DESKTOP_SECRET_CREATED_AT_KEY, now),
)
conn.commit()
return raw_secret
finally:
conn.close()
def validate_desktop_secret(raw_secret: str) -> Optional[str]:
"""Return the real admin username when the desktop secret matches."""
if not raw_secret.startswith(DESKTOP_SECRET_PREFIX):
return None
if get_user_and_secret(DEFAULT_ADMIN_USERNAME) is None:
return None
secret_hash = _pbkdf2_desktop_secret(raw_secret)
conn = get_connection()
try:
cur = conn.execute(
"SELECT value FROM app_secrets WHERE key = ?",
(_DESKTOP_SECRET_HASH_KEY,),
)
row = cur.fetchone()
if row is None:
return None
if not secrets.compare_digest(row["value"], secret_hash):
return None
return DEFAULT_ADMIN_USERNAME
finally:
conn.close()
def clear_desktop_secret() -> None:
"""Remove backend-side desktop auth state."""
conn = get_connection()
try:
conn.execute(
"DELETE FROM app_secrets WHERE key IN (?, ?)",
(_DESKTOP_SECRET_HASH_KEY, _DESKTOP_SECRET_CREATED_AT_KEY),
)
conn.commit()
finally:
conn.close()
# ---------------------------------------------------------------------------
# API key management
# ---------------------------------------------------------------------------
API_KEY_PREFIX = "sk-unsloth-"
def create_api_key(
username: str,
name: str,
expires_at: Optional[str] = None,
internal: bool = False,
) -> Tuple[str, dict]:
"""Create a new API key for *username*.
Returns ``(raw_key, row_dict)`` where *raw_key* is shown to the user
exactly once. The database only stores the PBKDF2 hash.
Pass ``internal=True`` for keys minted by workflows (e.g. data-recipe
runs) that should not appear in user-facing key listings.
"""
raw_key = API_KEY_PREFIX + secrets.token_hex(16)
key_hash = _pbkdf2_api_key(raw_key)
key_prefix = raw_key[len(API_KEY_PREFIX) : len(API_KEY_PREFIX) + 8]
now = datetime.now(timezone.utc).isoformat()
conn = get_connection()
try:
conn.execute(
"""
INSERT INTO api_keys (username, key_prefix, key_hash, name, created_at, expires_at, is_internal)
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
username,
key_prefix,
key_hash,
name,
now,
expires_at,
1 if internal else 0,
),
)
conn.commit()
cur = conn.execute("SELECT * FROM api_keys WHERE key_hash = ?", (key_hash,))
row = cur.fetchone()
return raw_key, dict(row)
finally:
conn.close()
def list_api_keys(username: str, include_internal: bool = False) -> list:
"""Return API keys for *username*. Internal workflow keys are hidden
by default so they do not clutter user-facing UIs."""
conn = get_connection()
try:
if include_internal:
cur = conn.execute(
"""
SELECT id, username, key_prefix, name, created_at, last_used_at,
expires_at, is_active, is_internal
FROM api_keys
WHERE username = ?
ORDER BY created_at DESC
""",
(username,),
)
else:
cur = conn.execute(
"""
SELECT id, username, key_prefix, name, created_at, last_used_at,
expires_at, is_active, is_internal
FROM api_keys
WHERE username = ? AND is_internal = 0
ORDER BY created_at DESC
""",
(username,),
)
return [dict(row) for row in cur.fetchall()]
finally:
conn.close()
def revoke_api_key(username: str, key_id: int) -> bool:
"""Soft-delete an API key. Returns True if a matching row was found."""
conn = get_connection()
try:
cursor = conn.execute(
"UPDATE api_keys SET is_active = 0 WHERE id = ? AND username = ?",
(key_id, username),
)
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def revoke_internal_api_key(key_id: int) -> bool:
"""Revoke an internal workflow-minted key without requiring a username.
Used by the recipe runner to retire its sk-unsloth-* key once the job
terminates, shrinking the window a leaked key could be abused.
"""
conn = get_connection()
try:
cursor = conn.execute(
"UPDATE api_keys SET is_active = 0 WHERE id = ? AND is_internal = 1",
(key_id,),
)
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def validate_api_key(raw_key: str) -> Optional[str]:
"""Validate *raw_key* and return the owning username, or ``None``.
Also updates ``last_used_at`` on success.
"""
cache_id = _api_key_cache_id(raw_key)
cached_hash = _api_key_hash_cache.get(cache_id)
key_hash = cached_hash if cached_hash is not None else _pbkdf2_api_key(raw_key)
conn = get_connection()
try:
cur = conn.execute(
"SELECT id, username, is_active, expires_at FROM api_keys WHERE key_hash = ?",
(key_hash,),
)
row = cur.fetchone()
if row is None:
return None
# Real key: memoize so later requests skip the KDF. Bounded; clear on overflow.
if cached_hash is None:
with _api_key_hash_cache_lock:
if len(_api_key_hash_cache) >= _API_KEY_HASH_CACHE_MAX:
_api_key_hash_cache.clear()
_api_key_hash_cache[cache_id] = key_hash
if not row["is_active"]:
return None
if row["expires_at"] is not None:
expires = datetime.fromisoformat(row["expires_at"])
if datetime.now(timezone.utc) > expires:
return None
conn.execute(
"UPDATE api_keys SET last_used_at = ? WHERE id = ?",
(datetime.now(timezone.utc).isoformat(), row["id"]),
)
conn.commit()
return row["username"]
finally:
conn.close()