1212a0023d
Export checkpoint loading always used unsloth's from_pretrained default of device_map="sequential", which stacks the whole model on GPU0. On a multi-GPU host this OOMs GPU0 while the other GPUs sit empty, so a GGUF export that would comfortably fit across the machine fails with CUDA out of memory (#7053). Add _multi_gpu_device_map_kwargs(): when the CUDA/ROCm host exposes more than one visible GPU and get_device_map resolves to "balanced" (the same policy the inference loader already uses), pass device_map="balanced" to every from_pretrained in load_checkpoint. In every other case -- single GPU, CPU, MLX, or any probe failure -- it returns {} so the loader default is untouched. Fixes #7053 |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_node_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| node_prebuilt_pins.json | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||