* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
168 lines
5.4 KiB
Python
168 lines
5.4 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Pydantic schemas for Export API."""
|
|
|
|
from pathlib import Path, PureWindowsPath
|
|
|
|
from pydantic import BaseModel, Field, field_validator
|
|
from typing import List, Optional, Literal, Dict, Any
|
|
|
|
|
|
def _validate_save_directory(value: str) -> str:
|
|
"""Validate save_directory — allows absolute paths (user may want a different drive)."""
|
|
if value is None:
|
|
raise ValueError("save_directory is required")
|
|
raw = str(value).strip()
|
|
if not raw:
|
|
raise ValueError("save_directory must not be empty")
|
|
if "\x00" in raw:
|
|
raise ValueError("save_directory may not contain null bytes")
|
|
if any(ch in raw for ch in ("\r", "\n")):
|
|
raise ValueError("save_directory may not contain control characters")
|
|
path = Path(raw).expanduser()
|
|
path_parts = (*path.parts, *PureWindowsPath(raw).parts, *raw.replace("\\", "/").split("/"))
|
|
if any(len(part) > 255 for part in path_parts if part not in ("", ".", "/", "\\")):
|
|
raise ValueError("save_directory path components must be <= 255 characters")
|
|
if (
|
|
".." in path.parts
|
|
or ".." in PureWindowsPath(raw).parts
|
|
or ".." in raw.replace("\\", "/").split("/")
|
|
):
|
|
raise ValueError("save_directory may not contain '..' segments")
|
|
return raw
|
|
|
|
|
|
class LoadCheckpointRequest(BaseModel):
|
|
"""Request for loading a checkpoint into the export backend."""
|
|
|
|
checkpoint_path: str = Field(..., description = "Path to the checkpoint directory")
|
|
max_seq_length: int = Field(
|
|
2048,
|
|
ge = 128,
|
|
le = 32768,
|
|
description = "Maximum sequence length for loading the model",
|
|
)
|
|
load_in_4bit: bool = Field(
|
|
True,
|
|
description = "Whether to load the model in 4-bit quantization",
|
|
)
|
|
trust_remote_code: bool = Field(
|
|
False,
|
|
description = "Allow loading models with custom code. Only enable for checkpoints/base models you trust.",
|
|
)
|
|
|
|
|
|
class ExportStatusResponse(BaseModel):
|
|
"""Current export backend status."""
|
|
|
|
current_checkpoint: Optional[str] = Field(
|
|
None,
|
|
description = "Path to the currently loaded checkpoint, if any",
|
|
)
|
|
is_vision: bool = Field(
|
|
False,
|
|
description = "True if the loaded checkpoint is a vision model",
|
|
)
|
|
is_peft: bool = Field(
|
|
False,
|
|
description = "True if the loaded checkpoint is a PEFT (LoRA) model",
|
|
)
|
|
|
|
|
|
class ExportOperationResponse(BaseModel):
|
|
"""Generic response for export operations."""
|
|
|
|
success: bool = Field(..., description = "True if the operation succeeded")
|
|
message: str = Field(..., description = "Human-readable status or error message")
|
|
details: Optional[Dict[str, Any]] = Field(
|
|
default = None,
|
|
description = "Optional extra details about the operation",
|
|
)
|
|
|
|
|
|
class ExportCommonOptions(BaseModel):
|
|
"""Common options for export operations that save locally and/or push to Hub."""
|
|
|
|
save_directory: str = Field(
|
|
...,
|
|
description = "Local directory where the exported artifacts will be written",
|
|
)
|
|
|
|
@field_validator("save_directory", mode = "before")
|
|
@classmethod
|
|
def _check_save_directory(cls, v):
|
|
return _validate_save_directory(v)
|
|
|
|
push_to_hub: bool = Field(
|
|
False,
|
|
description = "If True, also push the exported model to the Hugging Face Hub",
|
|
)
|
|
repo_id: Optional[str] = Field(
|
|
None,
|
|
description = "Hugging Face Hub repository ID (username/model-name)",
|
|
)
|
|
hf_token: Optional[str] = Field(
|
|
None,
|
|
description = "Hugging Face access token used for Hub operations",
|
|
)
|
|
private: bool = Field(
|
|
False,
|
|
description = "If True, create a private repository on the Hub (where applicable)",
|
|
)
|
|
base_model_id: Optional[str] = Field(
|
|
None,
|
|
description = "HuggingFace model ID of the base model (for model card metadata)",
|
|
)
|
|
|
|
|
|
class ExportMergedModelRequest(ExportCommonOptions):
|
|
"""Request for exporting a merged PEFT model."""
|
|
|
|
format_type: Literal["16-bit (FP16)", "4-bit (FP4)"] = Field(
|
|
"16-bit (FP16)",
|
|
description = "Export precision / format for the merged model",
|
|
)
|
|
|
|
|
|
class ExportBaseModelRequest(ExportCommonOptions):
|
|
"""Request for exporting a non-PEFT (base) model."""
|
|
|
|
# Uses fields from ExportCommonOptions only
|
|
|
|
|
|
class ExportGGUFRequest(BaseModel):
|
|
"""Request for exporting the current model to GGUF format."""
|
|
|
|
save_directory: str = Field(
|
|
...,
|
|
description = "Directory where GGUF files will be saved",
|
|
)
|
|
|
|
@field_validator("save_directory", mode = "before")
|
|
@classmethod
|
|
def _check_save_directory(cls, v):
|
|
return _validate_save_directory(v)
|
|
|
|
quantization_method: str = Field(
|
|
"Q4_K_M",
|
|
description = 'GGUF quantization method (e.g. "Q4_K_M")',
|
|
)
|
|
push_to_hub: bool = Field(
|
|
False,
|
|
description = "If True, also push GGUF artifacts to the Hugging Face Hub",
|
|
)
|
|
repo_id: Optional[str] = Field(
|
|
None,
|
|
description = "Hugging Face Hub repository ID for GGUF upload",
|
|
)
|
|
hf_token: Optional[str] = Field(
|
|
None,
|
|
description = "Hugging Face token for GGUF upload",
|
|
)
|
|
|
|
|
|
class ExportLoRAAdapterRequest(ExportCommonOptions):
|
|
"""Request for exporting only the LoRA adapter (not merged)."""
|
|
|
|
# Uses fields from ExportCommonOptions only
|