From 6b3eb504b2a3f0ab2e9ec97436ef025284fadd18 Mon Sep 17 00:00:00 2001 From: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Date: Thu, 26 Mar 2026 13:55:46 +0400 Subject: [PATCH] Fix Colab setup skipping llama.cpp installation (#4618) * Fix Colab setup skipping llama.cpp installation The early exit 0 in the Colab no-venv path prevented setup.sh from ever reaching the llama.cpp install section. Remove the early exit and instead guard only the venv-dependent Python deps section, so execution continues through to the llama.cpp prebuilt/source install. * Simplify _SKIP_PYTHON_DEPS initialization * Add --local flag to setup.sh in Colab notebook --- studio/Unsloth_Studio_Colab.ipynb | 304 +++++++++++++++--------------- studio/setup.sh | 24 +-- 2 files changed, 158 insertions(+), 170 deletions(-) diff --git a/studio/Unsloth_Studio_Colab.ipynb b/studio/Unsloth_Studio_Colab.ipynb index 7191dfde41..1479b1a05b 100644 --- a/studio/Unsloth_Studio_Colab.ipynb +++ b/studio/Unsloth_Studio_Colab.ipynb @@ -1,157 +1,153 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "view-in-github", - "colab_type": "text" - }, - "source": [ - "\"Open" - ] - }, - { - "cell_type": "markdown", - "id": "6b87de59", - "metadata": { - "id": "6b87de59" - }, - "source": [ - "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n", - "
\n", - "\n", - "\n", - " Join Discord if you need help + ⭐ Star us on Github ⭐\n", - "
\n", - "\n", - "To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n", - "\n", - "### Unsloth Studio\n", - "\n", - "Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). Currently, installation may take 30+ mins so use a newer GPU.\n", - "\n", - "\n", - "We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n", - "\n", - "[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Studio Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)" - ] - }, - { - "cell_type": "markdown", - "id": "e4206349", - "metadata": { - "id": "e4206349" - }, - "source": [ - "

" - ] - }, - { - "cell_type": "markdown", - "id": "27da2957", - "metadata": { - "id": "27da2957" - }, - "source": [ - "### Setup: Clone repo and run setup" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "27e68f91", - "metadata": { - "id": "27e68f91" - }, - "outputs": [], - "source": [ - "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n", - "%cd /content/unsloth\n", - "!chmod +x studio/setup.sh && ./studio/setup.sh" - ] - }, - { - "cell_type": "markdown", - "id": "3e1771a9", - "metadata": { - "id": "3e1771a9" - }, - "source": [ - "### Start Unsloth Studio" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "277e431e", - "metadata": { - "id": "277e431e" - }, - "outputs": [], - "source": [ - "import sys, time\n", - "sys.path.insert(0, \"/content/unsloth/studio/backend\")\n", - "from colab import start\n", - "start()" - ] - }, - { - "cell_type": "code", - "source": [ - "from google.colab import output\n", - "output.serve_kernel_port_as_iframe(8888, height = 1200, width = \"100%\")\n", - "for _ in range(10000): time.sleep(300), print(\"=\", end = \"\")" - ], - "metadata": { - "id": "wb9UELh--XzX" - }, - "id": "wb9UELh--XzX", - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "id": "f2b0c6a1", - "metadata": { - "id": "f2b0c6a1" - }, - "source": [ - "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n", - "\n", - "Some other resources:\n", - "1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n", - "2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n", - "3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n", - "4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n", - "5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n", - "\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - " Join Discord if you need help + ⭐️ Star us on Github ⭐️\n", - "\n", - " This notebook is licensed AGPL-3.0\n", - "
" - ] - } - ], - "metadata": { - "accelerator": "GPU", - "colab": { - "gpuType": "T4", - "provenance": [], - "include_colab_link": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "name": "python" - } + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "view-in-github", + "colab_type": "text" + }, + "source": [ + "\"Open" + ] }, - "nbformat": 4, - "nbformat_minor": 5 + { + "cell_type": "markdown", + "id": "6b87de59", + "metadata": { + "id": "6b87de59" + }, + "source": [ + "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n", + "
\n", + "\n", + "\n", + " Join Discord if you need help + ⭐ Star us on Github ⭐\n", + "
\n", + "\n", + "To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n", + "\n", + "### Unsloth Studio\n", + "\n", + "Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). Currently, installation may take 30+ mins so use a newer GPU.\n", + "\n", + "\n", + "We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n", + "\n", + "[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Studio Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)" + ] + }, + { + "cell_type": "markdown", + "id": "e4206349", + "metadata": { + "id": "e4206349" + }, + "source": [ + "

" + ] + }, + { + "cell_type": "markdown", + "id": "27da2957", + "metadata": { + "id": "27da2957" + }, + "source": [ + "### Setup: Clone repo and run setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "27e68f91", + "metadata": { + "id": "27e68f91" + }, + "outputs": [], + "source": "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n%cd /content/unsloth\n!chmod +x studio/setup.sh && ./studio/setup.sh --local" + }, + { + "cell_type": "markdown", + "id": "3e1771a9", + "metadata": { + "id": "3e1771a9" + }, + "source": [ + "### Start Unsloth Studio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "277e431e", + "metadata": { + "id": "277e431e" + }, + "outputs": [], + "source": [ + "import sys, time\n", + "sys.path.insert(0, \"/content/unsloth/studio/backend\")\n", + "from colab import start\n", + "start()" + ] + }, + { + "cell_type": "code", + "source": [ + "from google.colab import output\n", + "output.serve_kernel_port_as_iframe(8888, height = 1200, width = \"100%\")\n", + "for _ in range(10000): time.sleep(300), print(\"=\", end = \"\")" + ], + "metadata": { + "id": "wb9UELh--XzX" + }, + "id": "wb9UELh--XzX", + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "id": "f2b0c6a1", + "metadata": { + "id": "f2b0c6a1" + }, + "source": [ + "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n", + "\n", + "Some other resources:\n", + "1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n", + "2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n", + "3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n", + "4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n", + "5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n", + "\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + " Join Discord if you need help + ⭐️ Star us on Github ⭐️\n", + "\n", + " This notebook is licensed AGPL-3.0\n", + "
" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [], + "include_colab_link": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 } \ No newline at end of file diff --git a/studio/setup.sh b/studio/setup.sh index f103dd95a4..4e46a12fe0 100755 --- a/studio/setup.sh +++ b/studio/setup.sh @@ -337,30 +337,22 @@ fast_install() { cd "$SCRIPT_DIR" -# On Colab without a venv, skip all venv-dependent sections (Python deps -# update, llama.cpp build) -- the backend deps were already installed above. +# On Colab without a venv, skip venv-dependent Python deps sections but +# continue to llama.cpp install so GGUF inference is available. if [ "$_COLAB_NO_VENV" = true ]; then echo "✅ Studio backend dependencies installed into system Python" - - echo "" - echo "╔══════════════════════════════════════╗" - echo "║ Setup Complete! ║" - echo "╠══════════════════════════════════════╣" - echo "║ Unsloth Studio is ready to start ║" - echo "║ in your Colab notebook! ║" - echo "║ ║" - echo "║ from colab import start ║" - echo "║ start() ║" - echo "╚══════════════════════════════════════╝" - exit 0 fi # ── Check if Python deps need updating ── # Compare installed package version against PyPI latest. # Skip all Python dependency work if versions match (fast update path). -_PKG_NAME="${STUDIO_PACKAGE_NAME:-unsloth}" +# On Colab (no venv), skip the entire venv-dependent Python deps section. _SKIP_PYTHON_DEPS=false -if [ "${SKIP_STUDIO_BASE:-0}" != "1" ] && [ "${STUDIO_LOCAL_INSTALL:-0}" != "1" ]; then +if [ "$_COLAB_NO_VENV" = true ]; then + _SKIP_PYTHON_DEPS=true +fi +_PKG_NAME="${STUDIO_PACKAGE_NAME:-unsloth}" +if [ "$_SKIP_PYTHON_DEPS" != true ] && [ "${SKIP_STUDIO_BASE:-0}" != "1" ] && [ "${STUDIO_LOCAL_INSTALL:-0}" != "1" ]; then # Only check when NOT called from install.sh (which just installed the package) INSTALLED_VER=$("$VENV_DIR/bin/python" -c " from importlib.metadata import version