554c289538* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
99 lines
2.5 KiB
Python
99 lines
2.5 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Path utilities for model and dataset handling."""
|
|
|
|
from .path_utils import (
|
|
normalize_path,
|
|
is_local_path,
|
|
is_model_cached,
|
|
get_cache_path,
|
|
resolve_cached_repo_id_case,
|
|
get_cache_case_resolution_stats,
|
|
reset_cache_case_resolution_state,
|
|
)
|
|
from .storage_roots import (
|
|
studio_root,
|
|
assets_root,
|
|
datasets_root,
|
|
dataset_uploads_root,
|
|
recipe_datasets_root,
|
|
outputs_root,
|
|
exports_root,
|
|
auth_root,
|
|
auth_db_path,
|
|
studio_db_path,
|
|
rag_root,
|
|
rag_db_path,
|
|
rag_uploads_root,
|
|
documents_root,
|
|
project_workspaces_root,
|
|
tmp_root,
|
|
seed_uploads_root,
|
|
unstructured_seed_cache_root,
|
|
unstructured_uploads_root,
|
|
oxc_validator_tmp_root,
|
|
tensorboard_root,
|
|
legacy_hf_cache_dir,
|
|
hf_default_cache_dir,
|
|
lmstudio_model_dirs,
|
|
well_known_model_dirs,
|
|
ensure_dir,
|
|
ensure_studio_directories,
|
|
resolve_under_root,
|
|
resolve_output_dir,
|
|
resolve_export_dir,
|
|
resolve_export_write_dir,
|
|
resolve_tensorboard_dir,
|
|
resolve_dataset_path,
|
|
)
|
|
|
|
# Re-export shim: mark project-path helpers as used so the import-hoist
|
|
# safety net does not flag them as unused.
|
|
_REEXPORTED = (documents_root, project_workspaces_root, resolve_export_write_dir)
|
|
|
|
__all__ = [
|
|
"normalize_path",
|
|
"is_local_path",
|
|
"is_model_cached",
|
|
"get_cache_path",
|
|
"resolve_cached_repo_id_case",
|
|
"get_cache_case_resolution_stats",
|
|
"reset_cache_case_resolution_state",
|
|
"studio_root",
|
|
"assets_root",
|
|
"datasets_root",
|
|
"dataset_uploads_root",
|
|
"recipe_datasets_root",
|
|
"outputs_root",
|
|
"exports_root",
|
|
"auth_root",
|
|
"auth_db_path",
|
|
"studio_db_path",
|
|
"rag_root",
|
|
"rag_db_path",
|
|
"rag_uploads_root",
|
|
"documents_root",
|
|
"project_workspaces_root",
|
|
"tmp_root",
|
|
"seed_uploads_root",
|
|
"unstructured_seed_cache_root",
|
|
"unstructured_uploads_root",
|
|
"oxc_validator_tmp_root",
|
|
"tensorboard_root",
|
|
"legacy_hf_cache_dir",
|
|
"hf_default_cache_dir",
|
|
"lmstudio_model_dirs",
|
|
"well_known_model_dirs",
|
|
"ensure_dir",
|
|
"ensure_studio_directories",
|
|
"resolve_under_root",
|
|
"resolve_output_dir",
|
|
"resolve_export_dir",
|
|
"resolve_export_write_dir",
|
|
"resolve_tensorboard_dir",
|
|
"resolve_dataset_path",
|
|
]
|
|
|
|
# Bind the re-exports so the import-hoist verifier counts them as used.
|
|
_ = (rag_root, rag_db_path, rag_uploads_root)
|