Remove the no-op rmtree guard around the GGUF save (#7479)

* Remove the no-op rmtree guard around the GGUF save

patch_unsloth_gguf_save saves shutil.rmtree and restores it, but never
replaces it, so the context manager does nothing. Its comment claims it
prevents deletion of the directory save_pretrained just created.

It reads as a copy of the patch_unsloth_save sibling above it, minus the one
line that does the work. Completing it is not the right fix though: the GGUF
call forces push_to_hub=False and the merge cleanup that would remove the save
directory is gated on push_to_hub=True, so nothing on this path calls rmtree.
That was verified on a real LoRA-backed q8_0 export with every rmtree call
logged, in the discussion on #7149.

Drop the dead context manager rather than leave code that looks like a guard
and is not. Behaviour is unchanged; the following step comments are renumbered
to stay contiguous.

* Note why no rmtree guard is needed at the GGUF call site

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
This commit is contained in:
Vineeth Sai Varikuntla 2026-07-27 03:26:46 -07:00 committed by GitHub
commit 274f5ff569
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -186,32 +186,23 @@ def _save_pretrained_gguf(
if tokenizer is None:
tokenizer = self.tokenizer
# 4. Patch environment so Unsloth treats this embedding model correctly
@contextlib.contextmanager
def patch_unsloth_gguf_save():
# Prevent deletion of the directory self.save_pretrained just created
original_rmtree = shutil.rmtree
try:
yield
finally:
shutil.rmtree = original_rmtree
# 4. Call Unsloth's GGUF saver on the inner model targeting the transformer subdirectory
# No rmtree guard here: the merge cleanup that deletes save_directory is gated on
# push_to_hub, which is forced False below.
result = unsloth_save_pretrained_gguf(
inner_model,
save_directory = transformer_dir,
tokenizer = tokenizer,
quantization_method = quantization_method,
first_conversion = first_conversion,
push_to_hub = False, # Force local first to move files
token = token,
max_shard_size = max_shard_size,
temporary_location = temporary_location,
maximum_memory_usage = maximum_memory_usage,
)
# 5. Call Unsloth's GGUF saver on the inner model targeting the transformer subdirectory
with patch_unsloth_gguf_save():
result = unsloth_save_pretrained_gguf(
inner_model,
save_directory = transformer_dir,
tokenizer = tokenizer,
quantization_method = quantization_method,
first_conversion = first_conversion,
push_to_hub = False, # Force local first to move files
token = token,
max_shard_size = max_shard_size,
temporary_location = temporary_location,
maximum_memory_usage = maximum_memory_usage,
)
# 6. Move GGUF files from the subdirectory (0_Transformer) to the root save_directory
# 5. Move GGUF files from the subdirectory (0_Transformer) to the root save_directory
gguf_files = result.get("gguf_files", [])
new_gguf_locations = []
@ -241,7 +232,7 @@ def _save_pretrained_gguf(
result["gguf_files"] = new_gguf_locations
# 7. Add branding
# 6. Add branding
try:
FastSentenceTransformer._add_unsloth_branding(save_directory)
@ -256,7 +247,7 @@ def _save_pretrained_gguf(
except:
pass
# 8. Handle Push to Hub if requested
# 7. Handle Push to Hub if requested
if push_to_hub:
if token is None:
token = get_token()