Merge remote-tracking branch 'upstream/main' into fix-oom-labels-vulkan-6414
This commit is contained in:
LeoBorcherding
commit
4e7c0c2a1b
162 changed files with 2884 additions and 418 deletions
|
|
@ -1,134 +1,145 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/unslothai/unsloth/blob/main/studio/Unsloth_Studio_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
]
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/unslothai/unsloth/blob/main/studio/Unsloth_Studio_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6b87de59"
|
||||
},
|
||||
"source": [
|
||||
"To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
|
||||
"<div class=\"align-center\">\n",
|
||||
"<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
|
||||
"<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
|
||||
"<a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n",
|
||||
"\n",
|
||||
"### Unsloth Studio\n",
|
||||
"\n",
|
||||
"Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). NEW! Installation should now only take 2 mins!\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n",
|
||||
"\n",
|
||||
"[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Unsloth Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)"
|
||||
],
|
||||
"id": "6b87de59"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e4206349"
|
||||
},
|
||||
"source": [
|
||||
"<p align=\"left\"><img src=\"https://github.com/unslothai/unsloth/raw/main/studio/frontend/public/studio%20github%20landscape%20colab%20display.png\" width=\"600\"></p>"
|
||||
],
|
||||
"id": "e4206349"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27da2957"
|
||||
},
|
||||
"source": [
|
||||
"### Setup: Clone repo and run setup"
|
||||
],
|
||||
"id": "27da2957"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "27e68f91"
|
||||
},
|
||||
"source": "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n%cd /content/unsloth\n!chmod +x studio/setup.sh && ./studio/setup.sh --local",
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "27e68f91"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3e1771a9"
|
||||
},
|
||||
"source": [
|
||||
"### Start Unsloth Studio"
|
||||
],
|
||||
"id": "3e1771a9"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "277e431e"
|
||||
},
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"sys.path.insert(0, \"/content/unsloth/studio/backend\")\n",
|
||||
"from colab import start\n",
|
||||
"\n",
|
||||
"# On Colab, start() auto-opens a Cloudflare link and prints admin login credentials.\n",
|
||||
"# Use the Cloudflare link above the ready card to open Studio (in-cell iframes often stay blank).\n",
|
||||
"start()\n",
|
||||
"\n",
|
||||
"# To skip the Cloudflare tunnel and try the in-notebook proxy iframe only:\n",
|
||||
"# start(cloudflare=False)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": [],
|
||||
"id": "277e431e"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f2b0c6a1"
|
||||
},
|
||||
"source": [
|
||||
"And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
|
||||
"\n",
|
||||
"Some other resources:\n",
|
||||
"1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n",
|
||||
"2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n",
|
||||
"3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n",
|
||||
"4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n",
|
||||
"5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n",
|
||||
"\n",
|
||||
"<div class=\"align-center\">\n",
|
||||
" <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
|
||||
" <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
|
||||
" <a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
|
||||
"\n",
|
||||
" Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
|
||||
"\n",
|
||||
" <b>This notebook is licensed <a href=\"https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0\">AGPL-3.0</a></b>\n",
|
||||
"</div>"
|
||||
],
|
||||
"id": "f2b0c6a1"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "T4",
|
||||
"provenance": [],
|
||||
"include_colab_link": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6b87de59",
|
||||
"metadata": {
|
||||
"id": "6b87de59"
|
||||
},
|
||||
"source": [
|
||||
"To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
|
||||
"<div class=\"align-center\">\n",
|
||||
"<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
|
||||
"<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
|
||||
"<a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n",
|
||||
"\n",
|
||||
"### Unsloth Studio\n",
|
||||
"\n",
|
||||
"Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). NEW! Installation should now only take 2 mins!\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n",
|
||||
"\n",
|
||||
"[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Unsloth Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e4206349",
|
||||
"metadata": {
|
||||
"id": "e4206349"
|
||||
},
|
||||
"source": [
|
||||
"<p align=\"left\"><img src=\"https://github.com/unslothai/unsloth/raw/main/studio/frontend/public/studio%20github%20landscape%20colab%20display.png\" width=\"600\"></p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "27da2957",
|
||||
"metadata": {
|
||||
"id": "27da2957"
|
||||
},
|
||||
"source": [
|
||||
"### Setup: Clone repo and run setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "27e68f91",
|
||||
"metadata": {
|
||||
"id": "27e68f91"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n%cd /content/unsloth\n!chmod +x studio/setup.sh && ./studio/setup.sh --local"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3e1771a9",
|
||||
"metadata": {
|
||||
"id": "3e1771a9"
|
||||
},
|
||||
"source": [
|
||||
"### Start Unsloth Studio"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "277e431e",
|
||||
"metadata": {
|
||||
"id": "277e431e"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": "import sys\nsys.path.insert(0, \"/content/unsloth/studio/backend\")\nfrom colab import start\n\n# Default: in-tab iframe only. start() blocks to keep the kernel alive.\nstart()\n\n# For a shareable Cloudflare link, replace start() above with:\n# start(cloudflare=True)"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f2b0c6a1",
|
||||
"metadata": {
|
||||
"id": "f2b0c6a1"
|
||||
},
|
||||
"source": [
|
||||
"And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
|
||||
"\n",
|
||||
"Some other resources:\n",
|
||||
"1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n",
|
||||
"2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n",
|
||||
"3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n",
|
||||
"4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n",
|
||||
"5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n",
|
||||
"\n",
|
||||
"<div class=\"align-center\">\n",
|
||||
" <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
|
||||
" <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
|
||||
" <a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
|
||||
"\n",
|
||||
" Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
|
||||
"\n",
|
||||
" <b>This notebook is licensed <a href=\"https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0\">AGPL-3.0</a></b>\n",
|
||||
"</div>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "T4",
|
||||
"provenance": [],
|
||||
"include_colab_link": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
vision_all_linear: false
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
vision_all_linear: false
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
- "query"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
- "value"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "Wqkv"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -37,6 +37,7 @@ lora:
|
|||
- "shared_mlp.output_linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -37,6 +37,7 @@ lora:
|
|||
- "shared_mlp.output_linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
- "v_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "v_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -37,6 +37,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "out_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -37,6 +37,7 @@ lora:
|
|||
- "out_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ lora:
|
|||
- "v_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -36,6 +36,7 @@ lora:
|
|||
- "gate_up_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -36,6 +36,7 @@ lora:
|
|||
- "gate_up_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@ lora:
|
|||
- "down_proj"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
|
||||
logging:
|
||||
enable_wandb: false
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ lora:
|
|||
- "all-linear"
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ lora:
|
|||
vision_all_linear: true
|
||||
use_rslora: false
|
||||
use_loftq: false
|
||||
use_dora: false
|
||||
finetune_vision_layers: true
|
||||
finetune_language_layers: true
|
||||
finetune_attention_modules: true
|
||||
|
|
|
|||
|
|
@ -1,9 +1,7 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""
|
||||
Colab helpers for Unsloth Studio. Uses Colab's built-in proxy.
|
||||
"""
|
||||
"""Colab helpers for Unsloth Studio. Uses Colab's built-in proxy."""
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
|
@ -22,11 +20,9 @@ logger = get_logger(__name__)
|
|||
|
||||
|
||||
def get_colab_url(port: int = 8888) -> str:
|
||||
"""
|
||||
Get the Colab proxy URL for a port.
|
||||
"""Get the Colab proxy URL for a port.
|
||||
|
||||
Retries up to 3 times, validating the result is a real HTTPS Colab URL.
|
||||
Falls back to http://localhost:{port} only when all attempts fail.
|
||||
Retries 3x validating a real HTTPS Colab URL; falls back to localhost on failure.
|
||||
"""
|
||||
import time as _time
|
||||
|
||||
|
|
@ -55,28 +51,244 @@ def get_colab_url(port: int = 8888) -> str:
|
|||
return fallback
|
||||
|
||||
|
||||
def show_link(port: int = 8888, *, _url: "str | None" = None):
|
||||
"""Display a styled clickable link to the UI.
|
||||
|
||||
*_url* is an optional pre-fetched proxy URL; pass it to avoid a second eval_js round-trip.
|
||||
"""
|
||||
from IPython.display import display, HTML
|
||||
|
||||
url = _url if _url is not None else get_colab_url(port)
|
||||
|
||||
# Truncated display URL; try/except so an odd URL shape still renders the link.
|
||||
def _short_colab_url(url: str, port: int) -> str:
|
||||
"""Truncated display form of a Colab proxy URL; falls back to the full URL."""
|
||||
try:
|
||||
port_prefix = f"{port}-"
|
||||
idx = url.index(port_prefix)
|
||||
next_dash = url.index("-", idx + len(port_prefix))
|
||||
short_url = url[: next_dash + 1] + "..."
|
||||
return url[: next_dash + 1] + "..."
|
||||
except (ValueError, IndexError):
|
||||
short_url = url
|
||||
return url
|
||||
|
||||
# Plain-text line so the URL shows even if HTML display fails.
|
||||
logger.info(f"🌐 Unsloth Studio URL: {url}")
|
||||
|
||||
html = f"""
|
||||
def _is_colab_proxy_url(url: str, port: int) -> bool:
|
||||
"""True when *url* looks like a real Colab kernel proxy, not a localhost fallback."""
|
||||
return bool(url and isinstance(url, str) and url.startswith("https://") and str(port) in url)
|
||||
|
||||
|
||||
def _is_colab_runtime() -> bool:
|
||||
"""True on a hosted Colab notebook kernel.
|
||||
|
||||
Reuses the backend's main Colab detector (``/content`` + Colab env / ``google.colab``)
|
||||
instead of a single env var, which is not always present on hosted runtimes.
|
||||
"""
|
||||
try:
|
||||
from main import _IS_COLAB
|
||||
return bool(_IS_COLAB)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _colab_login_credentials_path() -> Path:
|
||||
from auth.storage import DB_PATH
|
||||
return DB_PATH.parent / ".colab_notebook_login"
|
||||
|
||||
|
||||
def _store_colab_login_credentials(username: str, password: str) -> None:
|
||||
"""Persist Colab admin credentials for notebook re-runs after interrupt."""
|
||||
path = _colab_login_credentials_path()
|
||||
try:
|
||||
path.parent.mkdir(parents = True, exist_ok = True)
|
||||
path.write_text(f"{username}\n{password}\n")
|
||||
try:
|
||||
import os
|
||||
os.chmod(path, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
except OSError as e:
|
||||
logger.info(f"Could not persist Colab login credentials ({e}).")
|
||||
|
||||
|
||||
def _load_colab_login_credentials() -> "tuple[str, str] | None":
|
||||
"""Return stored Colab admin credentials from a previous ``start()`` run, if any."""
|
||||
path = _colab_login_credentials_path()
|
||||
try:
|
||||
if not path.is_file():
|
||||
return None
|
||||
lines = path.read_text().splitlines()
|
||||
if len(lines) >= 2 and lines[0] and lines[1]:
|
||||
return lines[0], lines[1]
|
||||
except OSError as e:
|
||||
logger.info(f"Could not load Colab login credentials ({e}).")
|
||||
return None
|
||||
|
||||
|
||||
def _clear_colab_login_credentials() -> None:
|
||||
"""Drop the cached Colab credentials once they no longer authenticate."""
|
||||
path = _colab_login_credentials_path()
|
||||
try:
|
||||
path.unlink(missing_ok = True)
|
||||
except OSError as e:
|
||||
logger.info(f"Could not clear Colab login credentials ({e}).")
|
||||
|
||||
|
||||
def _colab_credentials_still_valid(username: str, password: str) -> bool:
|
||||
"""True when *password* still matches the stored admin hash.
|
||||
|
||||
Guards against redisplaying a cached first-run password after the user has
|
||||
changed the admin password through the app, which would print credentials
|
||||
that no longer authenticate to the current Cloudflare tunnel.
|
||||
"""
|
||||
try:
|
||||
from auth.storage import get_user_and_secret
|
||||
from auth.hashing import verify_password
|
||||
except Exception as e:
|
||||
logger.info(f"Could not load auth to validate cached Colab credentials ({e}).")
|
||||
return False
|
||||
try:
|
||||
row = get_user_and_secret(username)
|
||||
if not row:
|
||||
return False
|
||||
salt, pwd_hash = row[0], row[1]
|
||||
return bool(verify_password(password, salt, pwd_hash))
|
||||
except Exception as e:
|
||||
logger.info(f"Could not validate cached Colab credentials ({e}).")
|
||||
return False
|
||||
|
||||
|
||||
def _colab_wants_cloudflare(cloudflare: "bool | None") -> bool:
|
||||
"""Resolve whether to open a Cloudflare tunnel.
|
||||
|
||||
``None`` auto-enables on real Colab (the in-cell proxy embed is often blank);
|
||||
pass ``False`` to opt out.
|
||||
"""
|
||||
if cloudflare is not None:
|
||||
return cloudflare
|
||||
return _is_colab_runtime()
|
||||
|
||||
|
||||
def _finalize_colab_admin_password() -> "tuple[str, str] | None":
|
||||
"""Clear the bootstrap-password gate on Colab so Cloudflare tunnels can start.
|
||||
|
||||
Returns ``(username, password)`` for display in the notebook. On first run the
|
||||
random admin password is finalized; on later runs (e.g. after interrupt) the
|
||||
stored credentials are re-displayed so the Cloudflare link stays usable.
|
||||
Anyone who can read this cell already controls the runtime.
|
||||
"""
|
||||
if not _is_colab_runtime():
|
||||
return None
|
||||
try:
|
||||
from auth.storage import (
|
||||
DEFAULT_ADMIN_USERNAME,
|
||||
ensure_default_admin,
|
||||
generate_bootstrap_password,
|
||||
get_bootstrap_password,
|
||||
requires_password_change,
|
||||
update_password,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Could not load auth for Colab setup ({e}); Cloudflare link may be blocked."
|
||||
)
|
||||
return None
|
||||
|
||||
try:
|
||||
ensure_default_admin()
|
||||
username = DEFAULT_ADMIN_USERNAME
|
||||
if not requires_password_change(username):
|
||||
creds = _load_colab_login_credentials()
|
||||
if creds is not None and _colab_credentials_still_valid(username, creds[1]):
|
||||
return creds
|
||||
# The admin password was changed through the app after the first run,
|
||||
# so the cached copy is stale; drop it instead of printing dead credentials.
|
||||
_clear_colab_login_credentials()
|
||||
return None
|
||||
password = get_bootstrap_password() or generate_bootstrap_password()
|
||||
if not update_password(username, password):
|
||||
logger.warning(
|
||||
"Could not finalize Colab admin password; Cloudflare link may be blocked."
|
||||
)
|
||||
return None
|
||||
_store_colab_login_credentials(username, password)
|
||||
return username, password
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Could not finalize Colab admin password ({e}); Cloudflare link may be blocked."
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _colab_login_html(username: str, password: str) -> str:
|
||||
"""Notebook card with Colab admin credentials (shown once after auto-finalize)."""
|
||||
return f"""
|
||||
<div style="display: inline-block; padding: 20px; background: #ffffff; border: 2px solid #000000;
|
||||
border-radius: 12px; margin: 10px 0; font-family: system-ui, -apple-system, sans-serif;">
|
||||
<h2 style="color: #000000; margin: 0 0 12px 0; font-size: 22px; font-weight: 800;">
|
||||
Unsloth Studio Login (Colab)
|
||||
</h2>
|
||||
<p style="color: #333333; margin: 0 0 12px 0; font-size: 14px; font-weight: bold;">
|
||||
Log in to Studio with the Cloudflare link above using these credentials. This cell
|
||||
is visible only in your notebook session.
|
||||
</p>
|
||||
<p style="color: #333333; margin: 0; font-size: 14px; font-family: monospace; font-weight: bold;">
|
||||
Username: <code>{username}</code><br>
|
||||
Password: <code>{password}</code>
|
||||
</p>
|
||||
</div>
|
||||
"""
|
||||
|
||||
|
||||
def _show_colab_login_credentials(username: str, password: str) -> None:
|
||||
"""Display Colab admin credentials in the notebook output."""
|
||||
from IPython.display import HTML, display
|
||||
|
||||
logger.info(f"🔐 Unsloth Studio login — user: {username}")
|
||||
display(HTML(_colab_login_html(username, password)))
|
||||
|
||||
|
||||
def _ready_card_html(
|
||||
url: str,
|
||||
port: int,
|
||||
*,
|
||||
has_cloudflare_link: bool = False,
|
||||
cloudflare_requested: bool = False,
|
||||
) -> str:
|
||||
"""Branded ready card for the in-notebook Studio view.
|
||||
|
||||
Colab ``*.prod.colab.dev`` proxy URLs are session-scoped and 404 when opened as a
|
||||
top-level tab or on another device, so never ``window.open`` them. On real Colab the
|
||||
Cloudflare link is the supported entry point because in-cell proxy embeds often stay blank.
|
||||
"""
|
||||
short_url = _short_colab_url(url, port)
|
||||
if _is_colab_runtime() or _is_colab_proxy_url(url, port):
|
||||
if has_cloudflare_link:
|
||||
embed_note = (
|
||||
"Open Studio with the Cloudflare link above. In-cell proxy previews on "
|
||||
"current Colab often stay blank, so the tunnel link is the supported path."
|
||||
)
|
||||
elif cloudflare_requested:
|
||||
embed_note = (
|
||||
"Could not open a Cloudflare tunnel, so Studio may be unreachable on Colab. "
|
||||
"Check the logs above and re-run this cell. Pass "
|
||||
'<code style="background:#f3f3f3;padding:2px 6px;border-radius:4px;">'
|
||||
"cloudflare=True</code> after fixing any tunnel errors."
|
||||
)
|
||||
else:
|
||||
embed_note = (
|
||||
"Colab proxy links cannot be opened in a new tab (they 404 outside this "
|
||||
'notebook). Re-run with <code style="background:#f3f3f3;padding:2px 6px;'
|
||||
'border-radius:4px;">start(cloudflare=True)</code> for a working link.'
|
||||
)
|
||||
return f"""
|
||||
<div style="display: inline-block; padding: 20px; background: #ffffff; border: 2px solid #000000;
|
||||
border-radius: 12px; margin: 10px 0; font-family: system-ui, -apple-system, sans-serif;">
|
||||
<h2 style="color: #000000; margin: 0 0 12px 0; font-size: 26px; font-weight: 800;
|
||||
display: flex; align-items: center; gap: 12px;">
|
||||
<img src="https://github.com/unslothai/unsloth/raw/main/studio/frontend/public/unsloth-gem.png"
|
||||
height="48" style="display:block;">
|
||||
Unsloth Studio is Ready!
|
||||
</h2>
|
||||
<p style="color: #333333; margin: 0 0 8px 0; font-size: 15px; font-weight: bold;">
|
||||
{embed_note}
|
||||
</p>
|
||||
<p style="color: #666666; margin: 16px 0 0 0; font-size: 13px; font-family: monospace; font-weight: bold;">
|
||||
{short_url}
|
||||
</p>
|
||||
</div>
|
||||
"""
|
||||
|
||||
return f"""
|
||||
<div style="display: inline-block; padding: 20px; background: #ffffff; border: 2px solid #000000;
|
||||
border-radius: 12px; margin: 10px 0; font-family: system-ui, -apple-system, sans-serif;">
|
||||
<h2 style="color: #000000; margin: 0 0 12px 0; font-size: 26px; font-weight: 800;
|
||||
|
|
@ -100,15 +312,52 @@ def show_link(port: int = 8888, *, _url: "str | None" = None):
|
|||
</p>
|
||||
</div>
|
||||
"""
|
||||
display(HTML(html))
|
||||
|
||||
|
||||
def show_link(
|
||||
port: int = 8888,
|
||||
*,
|
||||
_url: "str | None" = None,
|
||||
has_cloudflare_link: bool = False,
|
||||
cloudflare_requested: bool = False,
|
||||
):
|
||||
"""Display a styled ready card for the UI.
|
||||
|
||||
Colab proxy URLs are informational only (no new-tab open; they 404 outside the cell);
|
||||
non-proxy URLs keep a clickable open button. *_url* is an optional pre-fetched proxy
|
||||
URL to avoid a second eval_js round-trip.
|
||||
"""
|
||||
from IPython.display import display, HTML
|
||||
|
||||
url = _url if _url is not None else get_colab_url(port)
|
||||
logger.info(f"🌐 Unsloth Studio URL: {url}")
|
||||
display(
|
||||
HTML(
|
||||
_ready_card_html(
|
||||
url,
|
||||
port,
|
||||
has_cloudflare_link = has_cloudflare_link,
|
||||
cloudflare_requested = cloudflare_requested,
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _warn_colab_cloudflare_missing(*, use_cloudflare: bool, cloudflare_url: "str | None") -> None:
|
||||
"""Log a prominent warning when Colab expected a tunnel but none was opened."""
|
||||
if not use_cloudflare or cloudflare_url or not _is_colab_runtime():
|
||||
return
|
||||
logger.warning(
|
||||
"Colab Cloudflare tunnel unavailable — Studio is unlikely to be reachable in this "
|
||||
"notebook. Check the logs above for tunnel or auth errors, then re-run start()."
|
||||
)
|
||||
|
||||
|
||||
def _bootstrap_password_pending() -> bool:
|
||||
"""True while the default admin still owes a bootstrap-password change.
|
||||
|
||||
While pending, main.py injects that password into same-origin GETs, and a public
|
||||
tunnel GET (no Origin) reads as same-origin, so sharing the link would leak admin
|
||||
access. Fails safe to pending if the state cannot be read.
|
||||
While pending, a public tunnel GET (no Origin) reads as same-origin and gets the
|
||||
injected password, so sharing the link would leak admin access. Fails safe to pending.
|
||||
"""
|
||||
try:
|
||||
from auth.storage import requires_password_change, DEFAULT_ADMIN_USERNAME
|
||||
|
|
@ -121,9 +370,8 @@ def _bootstrap_password_pending() -> bool:
|
|||
def start_cloudflare_tunnel(port: int) -> "str | None":
|
||||
"""Open a shareable Cloudflare quick tunnel to localhost:*port*, or None.
|
||||
|
||||
run_server suppresses the tunnel on Colab by design, so we start it directly.
|
||||
Refused while the bootstrap password is pending; any failure collapses to None
|
||||
and the Colab proxy still works.
|
||||
run_server suppresses the tunnel on Colab, so we start it directly. Refused while the
|
||||
bootstrap password is pending; any failure collapses to None (Colab proxy still works).
|
||||
"""
|
||||
if _bootstrap_password_pending():
|
||||
logger.warning(
|
||||
|
|
@ -152,9 +400,9 @@ def start_cloudflare_tunnel(port: int) -> "str | None":
|
|||
def _publish_cloudflare_url(cloudflare_url: "str | None") -> None:
|
||||
"""Publish a directly-started tunnel URL onto app.state so /api/health advertises it.
|
||||
|
||||
run_server only sets this when it opens the tunnel itself, which it skips on Colab,
|
||||
so we set it here. Otherwise the frontend's API examples fall back to an
|
||||
unreachable server_url. Best-effort.
|
||||
run_server sets this only when it opens the tunnel itself (skipped on Colab), so we
|
||||
set it here; otherwise the frontend's API examples fall back to an unreachable
|
||||
server_url. Best-effort.
|
||||
"""
|
||||
if not cloudflare_url:
|
||||
return
|
||||
|
|
@ -183,8 +431,7 @@ def _stop_cloudflare_tunnel() -> None:
|
|||
def _is_studio_healthy(port: int, timeout: float = 2.0) -> bool:
|
||||
"""True only if Unsloth Studio (not some other app) answers /api/health on *port*.
|
||||
|
||||
The service-marker check stops the reuse path reusing or tunneling a foreign
|
||||
process that merely serves /api/health.
|
||||
The service-marker check stops the reuse path reusing or tunneling a foreign process.
|
||||
"""
|
||||
import json, urllib.request
|
||||
try:
|
||||
|
|
@ -222,31 +469,45 @@ def _shareable_link_html(cloudflare_url: str) -> str:
|
|||
"""
|
||||
|
||||
|
||||
def _show_and_embed(port: int, *, cloudflare_url: "str | None" = None):
|
||||
"""Render the Unsloth header + iframe for *port*, with a shareable-link card above
|
||||
when *cloudflare_url* is set. Falls back to serve_kernel_port_as_iframe."""
|
||||
url = get_colab_url(port)
|
||||
logger.info(f"🌐 Unsloth Studio URL: {url}")
|
||||
if cloudflare_url:
|
||||
logger.info(f"🔗 Shareable Cloudflare link: {cloudflare_url}")
|
||||
# Height for serve_kernel_port_as_iframe (~82vh on a 1080p screen, clamped).
|
||||
_COLAB_IFRAME_HEIGHT = 900
|
||||
|
||||
|
||||
def _embed_kernel_port_iframe(port: int) -> bool:
|
||||
"""Embed Studio via Colab's native kernel-port iframe helper.
|
||||
|
||||
Only trusted on a real Colab runtime: colabtools can import ``google.colab`` and
|
||||
queue browser-side JS without appending an iframe, so callers outside Colab must use
|
||||
the HTML iframe path instead.
|
||||
"""
|
||||
if not _is_colab_runtime():
|
||||
return False
|
||||
try:
|
||||
from google.colab import output as colab_output
|
||||
except ImportError:
|
||||
return False
|
||||
try:
|
||||
colab_output.serve_kernel_port_as_iframe(
|
||||
port,
|
||||
height = _COLAB_IFRAME_HEIGHT,
|
||||
width = "100%",
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.info(f"serve_kernel_port_as_iframe failed ({e}); trying HTML iframe.")
|
||||
return False
|
||||
|
||||
|
||||
def _embed_html_iframe(url: str, port: int) -> bool:
|
||||
"""Fallback embed: raw HTML iframe when the Colab helper is unavailable."""
|
||||
try:
|
||||
from IPython.display import HTML, display
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
iframe_id = f"unsloth-studio-{port}"
|
||||
|
||||
# Truncated header URL — best-effort, falls back to full URL.
|
||||
try:
|
||||
port_prefix = f"{port}-"
|
||||
idx = url.index(port_prefix)
|
||||
next_dash = url.index("-", idx + len(port_prefix))
|
||||
short_url = url[: next_dash + 1] + "..."
|
||||
except (ValueError, IndexError):
|
||||
short_url = url
|
||||
|
||||
if cloudflare_url:
|
||||
display(HTML(_shareable_link_html(cloudflare_url)))
|
||||
|
||||
short_url = _short_colab_url(url, port)
|
||||
iframe_id = f"unsloth-studio-{port}"
|
||||
try:
|
||||
display(
|
||||
HTML(f"""
|
||||
<div style="font-family:system-ui,-apple-system,sans-serif;margin:8px 0;
|
||||
|
|
@ -266,41 +527,101 @@ def _show_and_embed(port: int, *, cloudflare_url: "str | None" = None):
|
|||
</div>
|
||||
""")
|
||||
)
|
||||
except Exception:
|
||||
# Fallback: Colab's built-in helper.
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.info(f"HTML iframe embed failed ({e}).")
|
||||
return False
|
||||
|
||||
|
||||
def _show_and_embed(
|
||||
port: int,
|
||||
*,
|
||||
cloudflare_url: "str | None" = None,
|
||||
colab_login: "tuple[str, str] | None" = None,
|
||||
cloudflare_requested: bool = False,
|
||||
):
|
||||
"""Render the Unsloth ready card + iframe for *port*.
|
||||
|
||||
Prefer Colab's ``serve_kernel_port_as_iframe`` on real Colab; raw HTML iframe is the
|
||||
fallback. Cloudflare cards stay clickable.
|
||||
"""
|
||||
url = get_colab_url(port)
|
||||
logger.info(f"🌐 Unsloth Studio URL: {url}")
|
||||
if cloudflare_url:
|
||||
logger.info(f"🔗 Shareable Cloudflare link: {cloudflare_url}")
|
||||
|
||||
_warn_colab_cloudflare_missing(
|
||||
use_cloudflare = cloudflare_requested,
|
||||
cloudflare_url = cloudflare_url,
|
||||
)
|
||||
|
||||
if cloudflare_url:
|
||||
try:
|
||||
from google.colab import output as colab_output
|
||||
colab_output.serve_kernel_port_as_iframe(port, height = 900, width = "100%")
|
||||
except ImportError:
|
||||
pass
|
||||
from IPython.display import HTML, display
|
||||
display(HTML(_shareable_link_html(cloudflare_url)))
|
||||
except Exception as e:
|
||||
logger.info(f"Could not render Cloudflare link card ({e}).")
|
||||
|
||||
if colab_login:
|
||||
try:
|
||||
_show_colab_login_credentials(*colab_login)
|
||||
except Exception as e:
|
||||
logger.info(f"Could not render Colab login card ({e}).")
|
||||
|
||||
try:
|
||||
show_link(
|
||||
port,
|
||||
_url = url,
|
||||
has_cloudflare_link = bool(cloudflare_url),
|
||||
cloudflare_requested = cloudflare_requested,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.info(f"Could not render Unsloth link card ({e}).")
|
||||
|
||||
# On Colab with a working tunnel, skip the in-cell proxy embed (often blank).
|
||||
if _is_colab_runtime() and cloudflare_url:
|
||||
return
|
||||
|
||||
# Real Colab: kernel helper needs only the port (works when eval_js failed).
|
||||
if _is_colab_runtime():
|
||||
if _embed_kernel_port_iframe(port):
|
||||
return
|
||||
_embed_html_iframe(url, port)
|
||||
|
||||
|
||||
def start(port: int = 8888, *, cloudflare: bool = False):
|
||||
def start(port: int = 8888, *, cloudflare: "bool | None" = None):
|
||||
"""Start Unsloth Studio in Colab and display the URL.
|
||||
|
||||
Args:
|
||||
port: Port to bind/serve on.
|
||||
cloudflare: Opt in to a shareable Cloudflare HTTPS link reachable from any
|
||||
device (default OFF). It exposes Unsloth's login page beyond Colab, so it
|
||||
stays an explicit opt-in; the default shows only the in-tab proxy iframe.
|
||||
cloudflare: Shareable Cloudflare HTTPS link. ``None`` (default) auto-enables on
|
||||
real Colab because the in-cell proxy embed is often blank; pass ``False`` to
|
||||
skip the tunnel or ``True`` to force it on other runtimes.
|
||||
|
||||
Usage:
|
||||
start() # Colab-proxy iframe only (default)
|
||||
start(cloudflare=True) # also open a shareable Cloudflare link
|
||||
start() # Cloudflare link on Colab (auto); proxy iframe elsewhere
|
||||
start(cloudflare=False) # Colab proxy iframe only (often blank on current Colab)
|
||||
start(cloudflare=True) # force Cloudflare link on any runtime
|
||||
"""
|
||||
import time
|
||||
|
||||
logger.info("🦥 Starting Unsloth Studio...")
|
||||
use_cloudflare = _colab_wants_cloudflare(cloudflare)
|
||||
|
||||
# Fast path: Unsloth already running (cell re-run). Re-launching would collide on
|
||||
# the port, so just re-show the link and iframe.
|
||||
# Fast path: already running (cell re-run); re-show link/iframe instead of rebinding the port.
|
||||
if _is_studio_healthy(port):
|
||||
logger.info(f" Unsloth is already running on port {port} — reusing existing server.")
|
||||
# try/finally: tear the tunnel down even if interrupted mid-start/render.
|
||||
try:
|
||||
cf_url = start_cloudflare_tunnel(port) if cloudflare else None
|
||||
colab_login = _finalize_colab_admin_password() if use_cloudflare else None
|
||||
cf_url = start_cloudflare_tunnel(port) if use_cloudflare else None
|
||||
_publish_cloudflare_url(cf_url)
|
||||
_show_and_embed(port, cloudflare_url = cf_url)
|
||||
_show_and_embed(
|
||||
port,
|
||||
cloudflare_url = cf_url,
|
||||
colab_login = colab_login,
|
||||
cloudflare_requested = use_cloudflare,
|
||||
)
|
||||
for _ in range(10000):
|
||||
time.sleep(300)
|
||||
print("=", end = "", flush = True)
|
||||
|
|
@ -313,7 +634,6 @@ def start(port: int = 8888, *, cloudflare: bool = False):
|
|||
logger.info(" Loading backend...")
|
||||
from run import run_server
|
||||
|
||||
# Auto-detect frontend path
|
||||
repo_root = Path(__file__).parent.parent
|
||||
frontend_path = repo_root / "frontend" / "dist"
|
||||
|
||||
|
|
@ -323,8 +643,7 @@ def start(port: int = 8888, *, cloudflare: bool = False):
|
|||
|
||||
logger.info(" Starting server...")
|
||||
try:
|
||||
# cloudflare=False: this helper owns the tunnel (Colab's own
|
||||
# start(cloudflare=...) drives it), so pin it off explicitly.
|
||||
# cloudflare=False: this helper owns the tunnel (via start(cloudflare=...)), so pin it off.
|
||||
app = run_server(
|
||||
host = "0.0.0.0",
|
||||
port = port,
|
||||
|
|
@ -339,14 +658,12 @@ def start(port: int = 8888, *, cloudflare: bool = False):
|
|||
logger.error(f"❌ Unsloth Studio failed to start: {exc}")
|
||||
return
|
||||
|
||||
# run_server auto-increments the port if in use; read back the bound port so the
|
||||
# proxy URL and iframe point at the right place.
|
||||
# run_server may auto-increment the port; read back the bound port for the proxy URL/iframe.
|
||||
actual_port: int = getattr(getattr(app, "state", None), "server_port", None) or port
|
||||
|
||||
logger.info(f" Server started on port {actual_port}!")
|
||||
|
||||
# Poll health endpoint before showing the link — avoids the race where ready_event
|
||||
# fires but the process hasn't finished binding.
|
||||
# Poll health before showing the link: avoids the race where ready_event fires pre-bind.
|
||||
import urllib.request
|
||||
|
||||
server_ready = False
|
||||
|
|
@ -365,12 +682,17 @@ def start(port: int = 8888, *, cloudflare: bool = False):
|
|||
)
|
||||
return
|
||||
|
||||
# Open the tunnel now the server is healthy, publish its URL for /api/health, and
|
||||
# tear it down on interrupt (try/finally) rather than orphan the process.
|
||||
# Server healthy: finalize Colab auth, open the tunnel, publish URL, tear down on interrupt.
|
||||
try:
|
||||
cf_url = start_cloudflare_tunnel(actual_port) if cloudflare else None
|
||||
colab_login = _finalize_colab_admin_password() if use_cloudflare else None
|
||||
cf_url = start_cloudflare_tunnel(actual_port) if use_cloudflare else None
|
||||
_publish_cloudflare_url(cf_url)
|
||||
_show_and_embed(actual_port, cloudflare_url = cf_url)
|
||||
_show_and_embed(
|
||||
actual_port,
|
||||
cloudflare_url = cf_url,
|
||||
colab_login = colab_login,
|
||||
cloudflare_requested = use_cloudflare,
|
||||
)
|
||||
|
||||
# Keep kernel alive so the daemon server thread runs.
|
||||
for _ in range(10000):
|
||||
|
|
|
|||
|
|
@ -1048,6 +1048,21 @@ class ExportBackend:
|
|||
"Use the safetensors adapter instead.",
|
||||
None,
|
||||
)
|
||||
# llama.cpp's convert_lora_to_gguf.py has no concept of DoRA's
|
||||
# lora_magnitude_vector tensors: it only reads the standard
|
||||
# lora_A/lora_B delta, so exporting a DoRA adapter would silently
|
||||
# drop the magnitude rescaling and produce a GGUF LoRA file that
|
||||
# loads fine but no longer matches the trained model.
|
||||
_peft_config = getattr(self.current_model, "peft_config", {}).get("default")
|
||||
if getattr(_peft_config, "use_dora", False):
|
||||
return (
|
||||
False,
|
||||
"GGUF LoRA export is not supported for DoRA adapters: the GGUF LoRA "
|
||||
"format has no way to represent DoRA's magnitude vectors, so the "
|
||||
"exported file would silently lose the DoRA behavior. Use the "
|
||||
"safetensors adapter instead, or merge to a full GGUF model.",
|
||||
None,
|
||||
)
|
||||
outtype = str(gguf_outtype).lower()
|
||||
if outtype not in _GGUF_LORA_OUTTYPES:
|
||||
return (
|
||||
|
|
|
|||
|
|
@ -307,6 +307,26 @@ def _native_linux_system_rocm_lib_dirs(binary_dir: str = "") -> "list[str]":
|
|||
_DEFAULT_MAX_TOKENS_FLOOR = 32768
|
||||
_DEFAULT_FIRST_TOKEN_TIMEOUT_S = 1200.0 # 20 min
|
||||
|
||||
|
||||
def _finalize_reasoning_only_cumulative(
|
||||
cumulative: str, reasoning_text: str, finish_reason: Optional[str], promote_reasoning_only: bool
|
||||
) -> str:
|
||||
"""Close a live thinking block and promote it only after a clean stop.
|
||||
|
||||
Local inference streams cumulative snapshots. Replacing ``<think>...`` with
|
||||
bare reasoning at EOF makes the final snapshot shorter, so suffix-based
|
||||
route consumers drop the intended fallback. Keep the snapshot append-only.
|
||||
A length-truncated thought is not a final answer, so close it without
|
||||
promotion and let the client surface the ``length`` terminal state. Raw
|
||||
consumers that do not split reasoning from visible content can disable the
|
||||
fallback to avoid returning the same reasoning twice.
|
||||
"""
|
||||
visible_fallback = (
|
||||
reasoning_text if promote_reasoning_only and finish_reason != "length" else ""
|
||||
)
|
||||
return cumulative + "</think>" + visible_fallback
|
||||
|
||||
|
||||
# Only large streamed tool payloads get an early provisional card; render_html
|
||||
# is exempt because it needs immediate artifact feedback.
|
||||
_PROVISIONAL_ARGS_MIN_CHARS = 256
|
||||
|
|
@ -2781,6 +2801,7 @@ class LlamaCppBackend:
|
|||
"found": False,
|
||||
"mtp_token": None,
|
||||
"supports_mtp": False,
|
||||
"mtp_probe_inconclusive": True,
|
||||
"ngram_mod_flavor": None,
|
||||
"supports_ngram_mod": False,
|
||||
"spec_draft_n_max_flag": None,
|
||||
|
|
@ -2813,6 +2834,9 @@ class LlamaCppBackend:
|
|||
supports_no_cache_prompt = False
|
||||
supports_metrics = False
|
||||
supports_slot_save = False
|
||||
saw_spec_type = False
|
||||
probe_ok = False
|
||||
help_text = ""
|
||||
try:
|
||||
probe_env = cls._llama_server_env_for_binary(bin_path)
|
||||
result = subprocess.run(
|
||||
|
|
@ -2824,6 +2848,7 @@ class LlamaCppBackend:
|
|||
check = False,
|
||||
env = probe_env,
|
||||
)
|
||||
probe_ok = result.returncode == 0
|
||||
help_text = (result.stdout or "") + "\n" + (result.stderr or "")
|
||||
# Split into per-flag blocks (each --flag line + its indented
|
||||
# continuation), so the "argument has been removed" description
|
||||
|
|
@ -2868,17 +2893,19 @@ class LlamaCppBackend:
|
|||
return False
|
||||
return "argument has been removed" not in desc
|
||||
|
||||
# MTP token from the --spec-type line.
|
||||
spec_line = ""
|
||||
for line in help_text.splitlines():
|
||||
if "--spec-type" in line:
|
||||
spec_line = line
|
||||
break
|
||||
# PR #22673 used draft-mtp; later renamed to mtp.
|
||||
if "draft-mtp" in spec_line:
|
||||
mtp_token = "draft-mtp"
|
||||
elif re.search(r"[|,\[]mtp[|,\]]", spec_line):
|
||||
mtp_token = "mtp"
|
||||
# MTP token from the full --spec-type help block (decl + indented
|
||||
# continuation). First-line-only probing missed builds putting the
|
||||
# enum on the next line (#7302). Prefer draft-mtp (PR #22673) over mtp.
|
||||
spec_help = blocks.get("--spec-type") or ""
|
||||
if not spec_help:
|
||||
# Fallback: join --spec-type lines, avoiding incidental "mtp" in --help.
|
||||
spec_help = "\n".join(
|
||||
line for line in help_text.splitlines() if "--spec-type" in line
|
||||
)
|
||||
mtp_token = cls._mtp_token_from_spec_help(spec_help)
|
||||
# Only a resolved --spec-type block confirms missing MTP; empty/crash
|
||||
# leaves saw_spec_type False so supports_mtp fails open.
|
||||
saw_spec_type = bool(spec_help.strip()) and "--spec-type" in spec_help
|
||||
|
||||
# ngram-mod flag flavor. Post-rename builds advertise both new
|
||||
# args (real) and legacy ones (stubs); pre-rename builds only
|
||||
|
|
@ -2914,11 +2941,29 @@ class LlamaCppBackend:
|
|||
supports_slot_save = _is_real("--slot-save-path")
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
logger.debug(f"llama-server --help probe failed: {exc}")
|
||||
saw_spec_type = False
|
||||
probe_ok = False
|
||||
help_text = ""
|
||||
|
||||
help_nonempty = bool(help_text.strip())
|
||||
# Confirmed only when a successful --help lists a --spec-type block with
|
||||
# mtp/draft-mtp; nonempty --help without it is a definitive pre-spec
|
||||
# binary; failed/empty probes stay inconclusive (#7302).
|
||||
if saw_spec_type and probe_ok:
|
||||
supports_mtp = mtp_token is not None
|
||||
mtp_probe_inconclusive = False
|
||||
elif help_nonempty and probe_ok:
|
||||
supports_mtp = False
|
||||
mtp_probe_inconclusive = False
|
||||
else:
|
||||
supports_mtp = False
|
||||
mtp_probe_inconclusive = True
|
||||
|
||||
info = {
|
||||
"found": True,
|
||||
"mtp_token": mtp_token,
|
||||
"supports_mtp": mtp_token is not None,
|
||||
"supports_mtp": supports_mtp,
|
||||
"mtp_probe_inconclusive": mtp_probe_inconclusive,
|
||||
"ngram_mod_flavor": ngram_mod_flavor,
|
||||
"supports_ngram_mod": ngram_mod_flavor is not None,
|
||||
"spec_draft_n_max_flag": spec_draft_n_max_flag,
|
||||
|
|
@ -2934,6 +2979,21 @@ class LlamaCppBackend:
|
|||
cls._capability_cache[cache_key] = info
|
||||
return info
|
||||
|
||||
@staticmethod
|
||||
def _mtp_token_from_spec_help(spec_help: str) -> Optional[str]:
|
||||
"""Extract ``draft-mtp`` / ``mtp`` from a ``--spec-type`` help snippet.
|
||||
|
||||
Prefers ``draft-mtp`` (llama.cpp PR #22673) over the later bare ``mtp``
|
||||
rename. Returns ``None`` when neither token appears as an enum value.
|
||||
"""
|
||||
text = spec_help or ""
|
||||
if "draft-mtp" in text:
|
||||
return "draft-mtp"
|
||||
# Bare `mtp` enum token (`|mtp|`, `,mtp,`, ...), not a substring.
|
||||
if re.search(r"(?<![A-Za-z0-9_-])mtp(?![A-Za-z0-9_-])", text):
|
||||
return "mtp"
|
||||
return None
|
||||
|
||||
# ── GPU allocation ────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -6078,6 +6138,24 @@ class LlamaCppBackend:
|
|||
and ("unknown" in text or "unsupported" in text or "not supported" in text)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _mmproj_retry_failure_message(*, projector_confirmed: bool, detail: str) -> str:
|
||||
"""User-facing error when the text-only --mmproj strip retry also fails.
|
||||
|
||||
Confirmed projector-format mismatches keep the historical wording.
|
||||
Bare signal crashes (common on some ROCm/driver paths) must not be
|
||||
reported as "Vision projector incompatible" — that misled #7302.
|
||||
"""
|
||||
if projector_confirmed:
|
||||
return (
|
||||
"Vision projector incompatible with this llama.cpp "
|
||||
"build, and the text-only retry also failed: " + detail
|
||||
)
|
||||
return (
|
||||
"Vision model failed to start (llama-server crashed with "
|
||||
"--mmproj), and the text-only retry also failed: " + detail
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _output_has_nonprojector_diagnostic(output: str) -> bool:
|
||||
"""True when the output already names a concrete non-projector cause (out
|
||||
|
|
@ -8453,23 +8531,29 @@ class LlamaCppBackend:
|
|||
self._kill_process()
|
||||
# The #6415 split-axis abort is latched earlier (first spawn).
|
||||
# Skip if a cancel/unload is pending (mirrors the MTP guard).
|
||||
_projector_msg = self._is_projector_incompatibility(out)
|
||||
_signal_mmproj_guess = self._is_signal_crash(
|
||||
_crash_rc
|
||||
) and not self._output_has_nonprojector_diagnostic(out)
|
||||
if (
|
||||
launched_with_mmproj
|
||||
and not self._cancel_event.is_set()
|
||||
and (
|
||||
self._is_projector_incompatibility(out)
|
||||
or (
|
||||
self._is_signal_crash(_crash_rc)
|
||||
and not self._output_has_nonprojector_diagnostic(out)
|
||||
)
|
||||
)
|
||||
and (_projector_msg or _signal_mmproj_guess)
|
||||
):
|
||||
logger.warning(
|
||||
"llama-server could not load this model's vision "
|
||||
"projector (--mmproj). The installed llama.cpp build is "
|
||||
"likely too old for it. Loading text-only for this "
|
||||
"session; run 'unsloth studio update' to enable vision."
|
||||
)
|
||||
if _projector_msg:
|
||||
logger.warning(
|
||||
"llama-server could not load this model's vision "
|
||||
"projector (--mmproj). The installed llama.cpp build is "
|
||||
"likely too old for it. Loading text-only for this "
|
||||
"session; run 'unsloth studio update' to enable vision."
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"llama-server crashed while loading this model's vision "
|
||||
"projector (--mmproj). Retrying text-only for this "
|
||||
"session; if this persists, run 'unsloth studio update' "
|
||||
"or check GPU/driver logs."
|
||||
)
|
||||
cmd = self._strip_mmproj_args(_last_spawn_cmd)
|
||||
# This retry bypasses _spawn_and_wait, so refresh the
|
||||
# launched-argv snapshot itself -- the zero-offload
|
||||
|
|
@ -8497,14 +8581,16 @@ class LlamaCppBackend:
|
|||
"(e.g. ROCR_VISIBLE_DEVICES=0 exposes only the first "
|
||||
"GPU) before launching Unsloth Studio."
|
||||
)
|
||||
_retry_detail = self._classify_llama_start_failure(
|
||||
"\n".join(self._stdout_lines[-50:]),
|
||||
gguf_path,
|
||||
self._model_identifier,
|
||||
_retry_rc,
|
||||
)
|
||||
raise RuntimeError(
|
||||
"Vision projector incompatible with this llama.cpp "
|
||||
"build, and the text-only retry also failed: "
|
||||
+ self._classify_llama_start_failure(
|
||||
"\n".join(self._stdout_lines[-50:]),
|
||||
gguf_path,
|
||||
self._model_identifier,
|
||||
_retry_rc,
|
||||
self._mmproj_retry_failure_message(
|
||||
projector_confirmed = _projector_msg,
|
||||
detail = _retry_detail,
|
||||
)
|
||||
)
|
||||
else:
|
||||
|
|
@ -8734,18 +8820,29 @@ class LlamaCppBackend:
|
|||
caps = self.probe_server_capabilities(binary)
|
||||
mtp_token = caps.get("mtp_token") if caps else None
|
||||
if not mtp_token:
|
||||
logger.warning(
|
||||
"Requested MTP speculative decoding but "
|
||||
"llama-server lacks --spec-type mtp/draft-mtp; "
|
||||
"run `unsloth studio update`. Loading without "
|
||||
"speculative decoding."
|
||||
)
|
||||
inconclusive = bool(caps.get("mtp_probe_inconclusive")) if caps else True
|
||||
if inconclusive:
|
||||
logger.info(
|
||||
"Requested MTP speculative decoding but llama-server MTP "
|
||||
"capability probe was inconclusive; loading without "
|
||||
"speculative decoding."
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"Requested MTP speculative decoding but "
|
||||
"llama-server lacks --spec-type mtp/draft-mtp; "
|
||||
"run `unsloth studio update`. Loading without "
|
||||
"speculative decoding."
|
||||
)
|
||||
# Override an inherited LLAMA_ARG_SPEC_TYPE=draft-mtp (CLI wins
|
||||
# over env) so the child matches the binary-capability gate and
|
||||
# the no-MTP budget, like the sibling no-head/non-MTP fallbacks.
|
||||
flags.append("--spec-default")
|
||||
self._speculative_type = "default"
|
||||
self._spec_fallback_reason = "binary_no_mtp"
|
||||
if inconclusive:
|
||||
self._spec_fallback_reason = None
|
||||
else:
|
||||
self._spec_fallback_reason = "binary_no_mtp"
|
||||
return False
|
||||
draft_n_max = _resolved_draft_n_max()
|
||||
n_max_flag = caps.get("spec_draft_n_max_flag") or "--spec-draft-n-max"
|
||||
|
|
@ -10544,6 +10641,7 @@ class LlamaCppBackend:
|
|||
reasoning_effort: Optional[str] = None,
|
||||
preserve_thinking: Optional[bool] = None,
|
||||
seed: Optional[int] = None,
|
||||
promote_reasoning_only: bool = True,
|
||||
_allow_respawn_retry: bool = True,
|
||||
) -> Generator[Union[str, dict], None, None]:
|
||||
"""
|
||||
|
|
@ -10626,7 +10724,12 @@ class LlamaCppBackend:
|
|||
# model put its whole reply in reasoning
|
||||
# (e.g. Qwen3 always-think). Show it as
|
||||
# the main response, not a thinking block.
|
||||
cumulative = reasoning_text
|
||||
cumulative = _finalize_reasoning_only_cumulative(
|
||||
cumulative,
|
||||
reasoning_text,
|
||||
_metadata_finish_reason,
|
||||
promote_reasoning_only,
|
||||
)
|
||||
yield cumulative
|
||||
_stream_done = True
|
||||
break # exit inner while
|
||||
|
|
@ -10723,6 +10826,7 @@ class LlamaCppBackend:
|
|||
reasoning_effort = reasoning_effort,
|
||||
preserve_thinking = preserve_thinking,
|
||||
seed = seed,
|
||||
promote_reasoning_only = promote_reasoning_only,
|
||||
_allow_respawn_retry = False,
|
||||
)
|
||||
return
|
||||
|
|
@ -10764,6 +10868,7 @@ class LlamaCppBackend:
|
|||
confirm_tool_calls: bool = False,
|
||||
bypass_permissions: bool = False,
|
||||
permission_mode: Optional[str] = None,
|
||||
promote_reasoning_only: bool = True,
|
||||
) -> Generator[dict, None, None]:
|
||||
"""
|
||||
Agentic loop: let the model call tools, execute them, and continue.
|
||||
|
|
@ -11106,7 +11211,12 @@ class LlamaCppBackend:
|
|||
),
|
||||
}
|
||||
else:
|
||||
cumulative_display = reasoning_accum
|
||||
cumulative_display = _finalize_reasoning_only_cumulative(
|
||||
cumulative_display,
|
||||
reasoning_accum,
|
||||
_iter_finish_reason,
|
||||
promote_reasoning_only,
|
||||
)
|
||||
if not _suppress_visible_output:
|
||||
yield {
|
||||
"type": "content",
|
||||
|
|
@ -11570,7 +11680,12 @@ class LlamaCppBackend:
|
|||
if _reasoning_started_at is not None and not _reasoning_summary_emitted:
|
||||
_reasoning_summary_emitted = True
|
||||
yield _reasoning_summary_event(_reasoning_started_at)
|
||||
cumulative_display = reasoning_accum
|
||||
cumulative_display = _finalize_reasoning_only_cumulative(
|
||||
cumulative_display,
|
||||
reasoning_accum,
|
||||
_iter_finish_reason,
|
||||
promote_reasoning_only,
|
||||
)
|
||||
if not _suppress_visible_output:
|
||||
yield {
|
||||
"type": "content",
|
||||
|
|
@ -12134,7 +12249,12 @@ class LlamaCppBackend:
|
|||
"text": _strip_tool_markup(cumulative, final = True),
|
||||
}
|
||||
else:
|
||||
cumulative = reasoning_text
|
||||
cumulative = _finalize_reasoning_only_cumulative(
|
||||
cumulative,
|
||||
reasoning_text,
|
||||
_metadata_finish_reason,
|
||||
promote_reasoning_only,
|
||||
)
|
||||
yield {"type": "content", "text": cumulative}
|
||||
_stream_done = True
|
||||
break # exit inner while
|
||||
|
|
|
|||
|
|
@ -891,6 +891,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing: str = "unsloth",
|
||||
use_rslora: bool = False,
|
||||
use_loftq: bool = False,
|
||||
use_dora: bool = False,
|
||||
modules_to_save: list = None,
|
||||
) -> bool:
|
||||
"""
|
||||
|
|
@ -993,6 +994,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing = use_gradient_checkpointing,
|
||||
random_state = 3407,
|
||||
use_rslora = use_rslora,
|
||||
use_dora = use_dora,
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1} if use_loftq else None,
|
||||
)
|
||||
# Audio VLM models support VLM-style layer selection
|
||||
|
|
@ -1023,6 +1025,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing = use_gradient_checkpointing,
|
||||
random_state = 3407,
|
||||
use_rslora = use_rslora,
|
||||
use_dora = use_dora,
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1} if use_loftq else None,
|
||||
task_type = None,
|
||||
)
|
||||
|
|
@ -1042,6 +1045,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing = use_gradient_checkpointing,
|
||||
random_state = 3407,
|
||||
use_rslora = use_rslora,
|
||||
use_dora = use_dora,
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1} if use_loftq else None,
|
||||
)
|
||||
|
||||
|
|
@ -1067,6 +1071,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing = use_gradient_checkpointing,
|
||||
random_state = 3407,
|
||||
use_rslora = use_rslora,
|
||||
use_dora = use_dora,
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1} if use_loftq else None,
|
||||
modules_to_save = modules_to_save,
|
||||
)
|
||||
|
|
@ -1087,6 +1092,7 @@ class UnslothTrainer:
|
|||
use_gradient_checkpointing = use_gradient_checkpointing,
|
||||
random_state = 3407,
|
||||
use_rslora = use_rslora,
|
||||
use_dora = use_dora,
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1} if use_loftq else None,
|
||||
modules_to_save = modules_to_save,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -196,6 +196,7 @@ def _build_training_worker_config(values: dict[str, Any]) -> dict[str, Any]:
|
|||
"gradient_checkpointing": values.get("gradient_checkpointing", "unsloth"),
|
||||
"use_rslora": values.get("use_rslora", False),
|
||||
"use_loftq": values.get("use_loftq", False),
|
||||
"use_dora": values.get("use_dora", False),
|
||||
"train_on_completions": values.get("train_on_completions", False),
|
||||
"finetune_vision_layers": values.get("finetune_vision_layers", True),
|
||||
"finetune_language_layers": values.get("finetune_language_layers", True),
|
||||
|
|
@ -455,6 +456,7 @@ class _MLXTrainerAdapter:
|
|||
use_gradient_checkpointing: Union[str, bool] = "unsloth",
|
||||
use_rslora: bool = False,
|
||||
use_loftq: bool = False,
|
||||
use_dora: bool = False,
|
||||
) -> bool:
|
||||
self._peft_config = {
|
||||
"use_lora": bool(use_lora),
|
||||
|
|
@ -465,6 +467,7 @@ class _MLXTrainerAdapter:
|
|||
"gradient_checkpointing": use_gradient_checkpointing,
|
||||
"use_rslora": bool(use_rslora),
|
||||
"use_loftq": bool(use_loftq),
|
||||
"use_dora": bool(use_dora),
|
||||
"finetune_vision_layers": bool(finetune_vision_layers),
|
||||
"finetune_language_layers": bool(finetune_language_layers),
|
||||
"finetune_attention_modules": bool(finetune_attention_modules),
|
||||
|
|
@ -572,6 +575,7 @@ class _MLXTrainerAdapter:
|
|||
"gradient_checkpointing": "unsloth",
|
||||
"use_rslora": False,
|
||||
"use_loftq": False,
|
||||
"use_dora": False,
|
||||
"finetune_vision_layers": True,
|
||||
"finetune_language_layers": True,
|
||||
"finetune_attention_modules": True,
|
||||
|
|
|
|||
|
|
@ -1547,6 +1547,10 @@ def _run_mlx_training(event_queue, stop_queue, config):
|
|||
message = "LoftQ is not supported for MLX training yet."
|
||||
_send("error", error = message)
|
||||
raise NotImplementedError(message)
|
||||
if config.get("use_dora"):
|
||||
message = "DoRA is not supported for MLX training yet."
|
||||
_send("error", error = message)
|
||||
raise NotImplementedError(message)
|
||||
if config.get("is_embedding"):
|
||||
message = "Embedding model training is not supported for MLX training yet."
|
||||
_send("error", error = message)
|
||||
|
|
@ -3186,6 +3190,7 @@ def run_training_process(*, event_queue: Any, stop_queue: Any, config: dict) ->
|
|||
use_gradient_checkpointing = config.get("gradient_checkpointing", "unsloth"),
|
||||
use_rslora = config.get("use_rslora", False),
|
||||
use_loftq = config.get("use_loftq", False),
|
||||
use_dora = config.get("use_dora", False),
|
||||
)
|
||||
elif use_lora:
|
||||
_send_status(event_queue, "Configuring LoRA adapters...")
|
||||
|
|
@ -3202,6 +3207,7 @@ def run_training_process(*, event_queue: Any, stop_queue: Any, config: dict) ->
|
|||
use_gradient_checkpointing = config.get("gradient_checkpointing", "unsloth"),
|
||||
use_rslora = config.get("use_rslora", False),
|
||||
use_loftq = config.get("use_loftq", False),
|
||||
use_dora = config.get("use_dora", False),
|
||||
)
|
||||
else:
|
||||
_send_status(event_queue, "Preparing model for full finetuning...")
|
||||
|
|
@ -3630,6 +3636,7 @@ def _run_embedding_training(event_queue: Any, stop_queue: Any, config: dict) ->
|
|||
use_gradient_checkpointing = gradient_checkpointing,
|
||||
random_state = config.get("random_seed", 3407),
|
||||
use_rslora = config.get("use_rslora", False),
|
||||
use_dora = config.get("use_dora", False),
|
||||
loftq_config = {"loftq_bits": 4, "loftq_iter": 1}
|
||||
if config.get("use_loftq")
|
||||
else None,
|
||||
|
|
|
|||
|
|
@ -445,7 +445,11 @@ def _run_llama_cpp_startup_probes(app: FastAPI) -> None:
|
|||
import structlog as _structlog
|
||||
|
||||
_log = _structlog.get_logger(__name__)
|
||||
if _caps.get("found") and not _caps.get("supports_mtp"):
|
||||
if (
|
||||
_caps.get("found")
|
||||
and not _caps.get("supports_mtp")
|
||||
and not _caps.get("mtp_probe_inconclusive")
|
||||
):
|
||||
_msg = (
|
||||
"llama.cpp prebuilt lacks MTP support "
|
||||
"(--spec-type mtp/draft-mtp). Run `unsloth studio update`. "
|
||||
|
|
|
|||
|
|
@ -70,7 +70,10 @@ class LoadRequest(BaseModel):
|
|||
|
||||
cache_type_kv: Optional[str] = Field(
|
||||
None,
|
||||
description = "KV cache data type for both K and V (e.g. 'f16', 'bf16', 'q8_0', 'q4_1', 'q5_1')",
|
||||
description = (
|
||||
"KV cache data type for both K and V "
|
||||
"(e.g. 'f16', 'bf16', 'q8_0', 'q4_0', 'q4_1', 'q5_0', 'q5_1', 'iq4_nl', 'f32')"
|
||||
),
|
||||
)
|
||||
gpu_ids: Optional[List[int]] = Field(
|
||||
None,
|
||||
|
|
@ -442,7 +445,10 @@ class LoadResponse(BaseModel):
|
|||
)
|
||||
cache_type_kv: Optional[str] = Field(
|
||||
None,
|
||||
description = "KV cache data type for K and V (e.g. 'f16', 'bf16', 'q8_0')",
|
||||
description = (
|
||||
"KV cache data type for K and V "
|
||||
"(e.g. 'f16', 'bf16', 'q8_0', 'q4_0', 'q4_1', 'q5_0', 'q5_1', 'iq4_nl', 'f32')"
|
||||
),
|
||||
)
|
||||
chat_template: Optional[str] = Field(
|
||||
None,
|
||||
|
|
@ -602,7 +608,11 @@ class InferenceStatusResponse(BaseModel):
|
|||
)
|
||||
cache_type_kv: Optional[str] = Field(
|
||||
None,
|
||||
description = "KV cache quantization dtype (e.g. 'q8_0'), or None for default",
|
||||
description = (
|
||||
"KV cache quantization dtype "
|
||||
"(e.g. 'f16', 'bf16', 'q8_0', 'q4_0', 'q4_1', 'q5_0', 'q5_1', 'iq4_nl', 'f32'), "
|
||||
"or None for default"
|
||||
),
|
||||
)
|
||||
chat_template: Optional[str] = Field(
|
||||
None, description = "Model's default chat template (Jinja2 source), if any"
|
||||
|
|
|
|||
|
|
@ -470,6 +470,7 @@ class TrainingStartRequest(BaseModel):
|
|||
gradient_checkpointing: str = Field("", description = "Gradient checkpointing setting")
|
||||
use_rslora: bool = Field(False, description = "Use RSLoRA")
|
||||
use_loftq: bool = Field(False, description = "Use LoftQ")
|
||||
use_dora: bool = Field(False, description = "Use DoRA")
|
||||
train_on_completions: bool = Field(False, description = "Train on completions only")
|
||||
|
||||
# Vision-specific LoRA parameters
|
||||
|
|
@ -545,6 +546,37 @@ class TrainingStartRequest(BaseModel):
|
|||
raise ValueError("Either num_epochs or max_steps must be > 0; both cannot be 0.")
|
||||
return self
|
||||
|
||||
@model_validator(mode = "after")
|
||||
def _validate_lora_variant_flags(self) -> "TrainingStartRequest":
|
||||
# The frontend only ever sends one of these and never under Full
|
||||
# Finetuning, but a direct API/YAML/CLI caller can bypass that. Nothing
|
||||
# downstream breaks (full finetune ignores them, MLX rejects use_dora/
|
||||
# use_loftq outright), but reject early here for a clear error instead
|
||||
# of a silently-ignored flag.
|
||||
active = [
|
||||
name
|
||||
for name, enabled in (
|
||||
("use_rslora", self.use_rslora),
|
||||
("use_loftq", self.use_loftq),
|
||||
("use_dora", self.use_dora),
|
||||
)
|
||||
if enabled
|
||||
]
|
||||
if len(active) > 1:
|
||||
raise ValueError(
|
||||
f"Only one LoRA variant may be enabled at a time; got {active}. "
|
||||
"use_rslora, use_loftq, and use_dora are mutually exclusive."
|
||||
)
|
||||
# getattr, not self.training_type: model_construct() (used by tests that
|
||||
# validate a single field in isolation) leaves required fields unset, and
|
||||
# this is a mode="after" validator so it still runs on that partial instance.
|
||||
if getattr(self, "training_type", None) == "Full Finetuning" and active:
|
||||
raise ValueError(
|
||||
f"{active[0]} requires an adapter method (LoRA/QLoRA or "
|
||||
"Continued Pretraining); it has no effect under Full Finetuning."
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
class TrainingJobResponse(BaseModel):
|
||||
"""Immediate response when training is initiated"""
|
||||
|
|
|
|||
|
|
@ -160,11 +160,26 @@ class ChatInferenceSettings(BaseModel):
|
|||
fastMode: Optional[bool] = None
|
||||
|
||||
|
||||
class ChatPresetLoadConfig(BaseModel):
|
||||
model_config = ConfigDict(extra = "forbid")
|
||||
|
||||
customContextLength: Optional[int] = Field(default = None, gt = 0)
|
||||
maxSeqLength: Optional[float] = None
|
||||
kvCacheDtype: Optional[str] = None
|
||||
speculativeType: Optional[str] = None
|
||||
specDraftNMax: Optional[int] = Field(default = None, ge = 1, le = 16)
|
||||
tensorParallel: Optional[bool] = None
|
||||
gpuMemoryMode: Optional[Literal["manual"]] = None
|
||||
gpuLayers: Optional[int] = None
|
||||
nCpuMoe: Optional[int] = Field(default = None, ge = 0)
|
||||
|
||||
|
||||
class ChatPreset(BaseModel):
|
||||
model_config = ConfigDict(extra = "forbid")
|
||||
|
||||
name: str
|
||||
params: ChatInferenceSettings
|
||||
loadConfig: Optional[ChatPresetLoadConfig] = None
|
||||
|
||||
|
||||
class ChatSettingsPayload(BaseModel):
|
||||
|
|
|
|||
|
|
@ -1794,7 +1794,16 @@ router = APIRouter()
|
|||
studio_router = APIRouter()
|
||||
|
||||
|
||||
_ARTIFACT_PREVIEW_FRAME_ANCESTORS = "'self' tauri://localhost http://tauri.localhost"
|
||||
# Packaged desktop runs at tauri://localhost (macOS/Linux) or http://tauri.localhost
|
||||
# (Windows WebView2); the web build is same-origin ('self'). The `tauri dev` shell,
|
||||
# however, serves the frontend from the Vite dev origin (http://localhost:5173),
|
||||
# so the packaged allowlist alone leaves the preview blocked in dev with an
|
||||
# "ancestor violates frame-ancestors" error. This shell exposes no server resource
|
||||
# (it only renders postMessage'd HTML in a no-same-origin sandbox), so also allowing
|
||||
# any localhost/127.0.0.1 dev origin to frame it is safe and unblocks the dev shell.
|
||||
_ARTIFACT_PREVIEW_FRAME_ANCESTORS = (
|
||||
"'self' tauri://localhost http://tauri.localhost http://localhost:* http://127.0.0.1:*"
|
||||
)
|
||||
_ARTIFACT_PREVIEW_FRAME_STRICT_CSP = (
|
||||
"default-src 'none'; "
|
||||
"script-src 'unsafe-inline'; "
|
||||
|
|
@ -5843,10 +5852,15 @@ async def get_status(current_subject: str = Depends(get_current_subject)):
|
|||
try:
|
||||
_bin = type(llama_backend)._find_llama_server_binary()
|
||||
_caps = type(llama_backend).probe_server_capabilities(_bin)
|
||||
_supports_mtp = bool(_caps.get("supports_mtp", False))
|
||||
# Fail open on inconclusive probes: False means a definitive
|
||||
# "binary lacks MTP" to API consumers.
|
||||
_supports_mtp = bool(
|
||||
_caps.get("supports_mtp", False)
|
||||
or (_caps.get("found", False) and _caps.get("mtp_probe_inconclusive", False))
|
||||
)
|
||||
except Exception:
|
||||
_bin = None
|
||||
_supports_mtp = True # fail open
|
||||
_supports_mtp = False # no usable binary: MTP genuinely unavailable
|
||||
try:
|
||||
from utils.llama_cpp_freshness import check_prebuilt_freshness
|
||||
_freshness = check_prebuilt_freshness(_bin)
|
||||
|
|
@ -13358,6 +13372,7 @@ async def anthropic_messages(
|
|||
disable_parallel_tool_use = _disable_parallel,
|
||||
bypass_permissions = bool(payload.bypass_permissions),
|
||||
permission_mode = getattr(payload, "permission_mode", None),
|
||||
promote_reasoning_only = False,
|
||||
)
|
||||
|
||||
if payload.stream:
|
||||
|
|
@ -13397,6 +13412,7 @@ async def anthropic_messages(
|
|||
max_tokens = payload.max_tokens,
|
||||
stop = stop,
|
||||
cancel_event = cancel_event,
|
||||
promote_reasoning_only = False,
|
||||
)
|
||||
|
||||
if payload.stream:
|
||||
|
|
|
|||
|
|
@ -2692,7 +2692,10 @@ async def get_kv_cache_estimate(
|
|||
repo_id: str = Query(..., description = "HF repo ID or local path"),
|
||||
quant: str = Query(..., description = "Quantization label (e.g. Q4_K_M)"),
|
||||
n_ctx: int = Query(..., ge = 1, description = "Context length to size the KV cache for"),
|
||||
cache_type_kv: Optional[str] = Query(None, description = "KV cache dtype (e.g. q8_0)"),
|
||||
cache_type_kv: Optional[str] = Query(
|
||||
None,
|
||||
description = "KV cache dtype (e.g. q8_0, q4_0, q5_0, iq4_nl, f32)",
|
||||
),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""Estimate KV cache + weight bytes for a downloaded GGUF at n_ctx.
|
||||
|
|
|
|||
|
|
@ -332,6 +332,7 @@ async def start_training(
|
|||
else "unsloth",
|
||||
"use_rslora": request.use_rslora,
|
||||
"use_loftq": request.use_loftq,
|
||||
"use_dora": request.use_dora,
|
||||
"train_on_completions": request.train_on_completions,
|
||||
"finetune_vision_layers": request.finetune_vision_layers,
|
||||
"finetune_language_layers": request.finetune_language_layers,
|
||||
|
|
|
|||
|
|
@ -68,16 +68,15 @@ def _emitter_client_text(events: list[str]) -> str:
|
|||
|
||||
|
||||
def test_anthropic_emitter_closes_reasoning_only_think_block():
|
||||
# A reasoning-only reply streams <think>X live then shrinks to bare X at EOF.
|
||||
# This emitter diffs cumulative snapshots and drops the shrink, so without a
|
||||
# closing pass the client text would end on an unclosed <think>. finish()
|
||||
# must balance it.
|
||||
# Anthropic asks the GGUF generator not to promote reasoning into a duplicate
|
||||
# visible fallback, so its final cumulative snapshot only balances the block.
|
||||
emitter = AnthropicStreamEmitter()
|
||||
events = emitter.start("msg_1", "m")
|
||||
events += emitter.feed({"type": "content", "text": "<think>The capital"})
|
||||
events += emitter.feed({"type": "content", "text": "<think>The capital of France is Paris."})
|
||||
# The generator's final bare-text shrink (dropped by the cumulative diff).
|
||||
events += emitter.feed({"type": "content", "text": "The capital of France is Paris."})
|
||||
events += emitter.feed(
|
||||
{"type": "content", "text": "<think>The capital of France is Paris.</think>"}
|
||||
)
|
||||
events += emitter.finish()
|
||||
|
||||
assert _emitter_client_text(events) == "<think>The capital of France is Paris.</think>"
|
||||
|
|
@ -1563,6 +1562,44 @@ class TestAnthropicMessagesToolRouting:
|
|||
assert entry["context_length"] == 2048
|
||||
assert monitor.active_count() == 0
|
||||
|
||||
@pytest.mark.parametrize("stream", [False, True])
|
||||
@pytest.mark.parametrize("with_tools", [False, True])
|
||||
def test_reasoning_only_output_is_not_duplicated(self, monkeypatch, stream, with_tools):
|
||||
reasoning = "The capital of France is Paris."
|
||||
|
||||
def _gen_plain(**kwargs):
|
||||
assert kwargs["promote_reasoning_only"] is False
|
||||
yield f"<think>{reasoning}"
|
||||
yield f"<think>{reasoning}</think>"
|
||||
|
||||
def _gen_tools(**kwargs):
|
||||
assert kwargs["promote_reasoning_only"] is False
|
||||
yield {"type": "content", "text": f"<think>{reasoning}"}
|
||||
yield {"type": "content", "text": f"<think>{reasoning}</think>"}
|
||||
|
||||
_mock_backend(
|
||||
monkeypatch,
|
||||
generate_chat_completion = _gen_plain,
|
||||
generate_chat_completion_with_tools = _gen_tools,
|
||||
)
|
||||
payload_fields = {"stream": stream}
|
||||
if with_tools:
|
||||
payload_fields.update(
|
||||
{
|
||||
"enable_tools": True,
|
||||
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
||||
}
|
||||
)
|
||||
payload = _basic_payload(**payload_fields)
|
||||
|
||||
response = _drive(anthropic_messages(payload, request = self._Request(), current_subject = "t"))
|
||||
if stream:
|
||||
body = self._sse_blob(self._consume_response(response))
|
||||
assert body.count(reasoning) == 1
|
||||
else:
|
||||
body = json.loads(response.body)
|
||||
assert body["content"][0]["text"] == f"<think>{reasoning}</think>"
|
||||
|
||||
def test_tool_use_non_streaming_records_api_monitor_reply(self, monkeypatch):
|
||||
import routes.inference as inf_mod
|
||||
|
||||
|
|
|
|||
|
|
@ -91,6 +91,28 @@ def test_chat_settings_payload_accepts_fast_mode_presets():
|
|||
assert dumped["customPresets"][0]["params"]["fastMode"] is True
|
||||
|
||||
|
||||
def test_chat_settings_payload_accepts_preset_load_config():
|
||||
payload = chat_history.ChatSettingsPayload.model_validate(
|
||||
{
|
||||
"customPresets": [
|
||||
{
|
||||
"name": "GGUF preset",
|
||||
"params": {"temperature": 0.7, "maxTokens": 512},
|
||||
"loadConfig": {
|
||||
"customContextLength": 256,
|
||||
"kvCacheDtype": "q8_0",
|
||||
"tensorParallel": False,
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
dumped = payload.model_dump(exclude_unset = True)
|
||||
assert dumped["customPresets"][0]["loadConfig"]["customContextLength"] == 256
|
||||
assert dumped["customPresets"][0]["loadConfig"]["kvCacheDtype"] == "q8_0"
|
||||
|
||||
|
||||
def test_chat_settings_payload_accepts_nudge_tool_calls():
|
||||
# extra="forbid" 400s PUT /api/chat/settings on unknown keys, so the
|
||||
# frontend's persisted nudgeToolCalls needs a payload field (like
|
||||
|
|
|
|||
479
studio/backend/tests/test_colab_embed.py
Normal file
479
studio/backend/tests/test_colab_embed.py
Normal file
|
|
@ -0,0 +1,479 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Regression coverage for Colab iframe embedding (#7344)."""
|
||||
|
||||
import sys
|
||||
import types
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import colab
|
||||
|
||||
|
||||
def _mock_google_colab_modules(colab_mod):
|
||||
"""Mock ``google`` and ``google.colab`` for environments without Google packages."""
|
||||
google_mod = types.ModuleType("google")
|
||||
google_mod.colab = colab_mod
|
||||
return {"google": google_mod, "google.colab": colab_mod}
|
||||
|
||||
|
||||
def test_short_colab_url_truncates_proxy_host():
|
||||
url = "https://8888-gpu-a100-s-kkb-usc1f0-9hzedjcxrlu8-f.us-central1-0.prod.colab.dev/"
|
||||
assert colab._short_colab_url(url, 8888) == "https://8888-gpu-..."
|
||||
|
||||
|
||||
def test_short_colab_url_falls_back_on_unexpected_shape():
|
||||
assert colab._short_colab_url("https://example.com", 8888) == "https://example.com"
|
||||
|
||||
|
||||
def test_is_colab_proxy_url_requires_https_proxy():
|
||||
assert colab._is_colab_proxy_url("https://8888-test.prod.colab.dev/", 8888) is True
|
||||
assert colab._is_colab_proxy_url("http://localhost:8888", 8888) is False
|
||||
assert colab._is_colab_proxy_url("http://127.0.0.1:8888", 8888) is False
|
||||
|
||||
|
||||
def test_ready_card_html_does_not_open_colab_proxy_in_new_tab():
|
||||
"""Colab proxy hosts 404 as top-level tabs (#7349 reporter); never window.open them."""
|
||||
html = colab._ready_card_html("https://8888-test.prod.colab.dev/", 8888)
|
||||
assert "window.open" not in html
|
||||
assert 'href="https://8888-test.prod.colab.dev/"' not in html
|
||||
assert "start(cloudflare=True)" in html
|
||||
|
||||
|
||||
def test_ready_card_html_points_to_cloudflare_when_link_ready(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
html = colab._ready_card_html(
|
||||
"https://8888-test.prod.colab.dev/",
|
||||
8888,
|
||||
has_cloudflare_link = True,
|
||||
)
|
||||
assert "Cloudflare link above" in html
|
||||
|
||||
|
||||
def test_ready_card_html_warns_when_cloudflare_tunnel_missing(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
html = colab._ready_card_html(
|
||||
"https://8888-test.prod.colab.dev/",
|
||||
8888,
|
||||
cloudflare_requested = True,
|
||||
)
|
||||
assert "Could not open a Cloudflare tunnel" in html
|
||||
|
||||
|
||||
def test_warn_colab_cloudflare_missing_logs_on_colab_without_tunnel(monkeypatch):
|
||||
warnings: list[str] = []
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(colab.logger, "warning", lambda msg, **kwargs: warnings.append(msg))
|
||||
colab._warn_colab_cloudflare_missing(use_cloudflare = True, cloudflare_url = None)
|
||||
assert warnings
|
||||
assert "Cloudflare tunnel unavailable" in warnings[0]
|
||||
|
||||
|
||||
def test_warn_colab_cloudflare_missing_skips_when_tunnel_ready(monkeypatch, caplog):
|
||||
import logging
|
||||
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
with caplog.at_level(logging.WARNING):
|
||||
colab._warn_colab_cloudflare_missing(
|
||||
use_cloudflare = True,
|
||||
cloudflare_url = "https://share.trycloudflare.com",
|
||||
)
|
||||
assert "Cloudflare tunnel unavailable" not in caplog.text
|
||||
|
||||
|
||||
def test_is_colab_runtime_uses_backend_colab_detector(monkeypatch):
|
||||
fake_main = types.ModuleType("main")
|
||||
fake_main._IS_COLAB = True
|
||||
monkeypatch.setitem(sys.modules, "main", fake_main)
|
||||
assert colab._is_colab_runtime() is True
|
||||
fake_main._IS_COLAB = False
|
||||
assert colab._is_colab_runtime() is False
|
||||
|
||||
|
||||
def test_ready_card_html_uses_cloudflare_hint_on_colab_runtime_localhost(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
html = colab._ready_card_html("http://localhost:8888", 8888)
|
||||
assert "window.open" not in html
|
||||
assert "start(cloudflare=True)" in html
|
||||
|
||||
|
||||
def test_ready_card_html_keeps_open_button_for_localhost_outside_colab(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
html = colab._ready_card_html("http://localhost:8888", 8888)
|
||||
assert "window.open" in html
|
||||
assert 'href="http://localhost:8888"' in html
|
||||
assert "Open Unsloth Studio" in html
|
||||
|
||||
|
||||
def test_embed_kernel_port_iframe_uses_colab_helper(monkeypatch):
|
||||
colab_output = MagicMock()
|
||||
google_colab = SimpleNamespace(output = colab_output)
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
with patch.dict("sys.modules", _mock_google_colab_modules(google_colab)):
|
||||
assert colab._embed_kernel_port_iframe(8888) is True
|
||||
colab_output.serve_kernel_port_as_iframe.assert_called_once_with(
|
||||
8888,
|
||||
height = colab._COLAB_IFRAME_HEIGHT,
|
||||
width = "100%",
|
||||
)
|
||||
|
||||
|
||||
def test_embed_kernel_port_iframe_returns_false_without_colab():
|
||||
with patch.dict("sys.modules", _mock_google_colab_modules(None)):
|
||||
assert colab._embed_kernel_port_iframe(8888) is False
|
||||
|
||||
|
||||
def test_embed_kernel_port_iframe_skips_colabtools_without_runtime(monkeypatch):
|
||||
"""colabtools can queue JS without appending an iframe; only trust the helper on Colab."""
|
||||
colab_output = MagicMock()
|
||||
google_colab = SimpleNamespace(output = colab_output)
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
with patch.dict("sys.modules", _mock_google_colab_modules(google_colab)):
|
||||
assert colab._embed_kernel_port_iframe(8888) is False
|
||||
colab_output.serve_kernel_port_as_iframe.assert_not_called()
|
||||
|
||||
|
||||
def test_show_and_embed_prefers_kernel_port_iframe(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"https://{port}-test.prod.colab.dev/")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port,
|
||||
*,
|
||||
_url = None,
|
||||
has_cloudflare_link = False,
|
||||
cloudflare_requested = False: calls.append("show_link"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_kernel_port_iframe",
|
||||
lambda port: calls.append("kernel_iframe") or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append("html_iframe") or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888)
|
||||
|
||||
assert calls == ["show_link", "kernel_iframe"]
|
||||
|
||||
|
||||
def test_show_and_embed_falls_back_to_html_iframe(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"https://{port}-test.prod.colab.dev/")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port, *, _url = None, has_cloudflare_link = False: None,
|
||||
)
|
||||
monkeypatch.setattr(colab, "_embed_kernel_port_iframe", lambda port: False)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append((url, port)) or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888)
|
||||
|
||||
assert calls == [("https://8888-test.prod.colab.dev/", 8888)]
|
||||
|
||||
|
||||
def test_colab_wants_cloudflare_auto_enables_on_runtime(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
assert colab._colab_wants_cloudflare(None) is True
|
||||
assert colab._colab_wants_cloudflare(True) is True
|
||||
assert colab._colab_wants_cloudflare(False) is False
|
||||
|
||||
|
||||
def test_colab_wants_cloudflare_defaults_off_outside_runtime(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
assert colab._colab_wants_cloudflare(None) is False
|
||||
assert colab._colab_wants_cloudflare(True) is True
|
||||
|
||||
|
||||
def test_finalize_colab_admin_password_skips_outside_runtime(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
assert colab._finalize_colab_admin_password() is None
|
||||
|
||||
|
||||
def test_finalize_colab_admin_password_clears_bootstrap_gate(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(colab, "_load_colab_login_credentials", lambda: None)
|
||||
stored: list[tuple[str, str]] = []
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_store_colab_login_credentials",
|
||||
lambda username, password: stored.append((username, password)),
|
||||
)
|
||||
|
||||
storage = SimpleNamespace(
|
||||
DEFAULT_ADMIN_USERNAME = "unsloth",
|
||||
ensure_default_admin = MagicMock(),
|
||||
get_bootstrap_password = MagicMock(return_value = "alpha-beta-gamma"),
|
||||
generate_bootstrap_password = MagicMock(return_value = "alpha-beta-gamma"),
|
||||
requires_password_change = MagicMock(return_value = True),
|
||||
update_password = MagicMock(return_value = True),
|
||||
)
|
||||
auth_pkg = types.ModuleType("auth")
|
||||
auth_pkg.storage = storage
|
||||
with patch.dict("sys.modules", {"auth": auth_pkg, "auth.storage": storage}):
|
||||
result = colab._finalize_colab_admin_password()
|
||||
|
||||
assert result == ("unsloth", "alpha-beta-gamma")
|
||||
storage.ensure_default_admin.assert_called_once()
|
||||
storage.update_password.assert_called_once_with("unsloth", "alpha-beta-gamma")
|
||||
assert stored == [("unsloth", "alpha-beta-gamma")]
|
||||
|
||||
|
||||
def test_start_skips_finalize_when_cloudflare_disabled(monkeypatch):
|
||||
import time
|
||||
|
||||
finalize_calls: list[str] = []
|
||||
monkeypatch.setattr(colab, "_is_studio_healthy", lambda port: True)
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_finalize_colab_admin_password",
|
||||
lambda: finalize_calls.append("finalize") or ("unsloth", "secret"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab, "start_cloudflare_tunnel", lambda port: "https://share.trycloudflare.com"
|
||||
)
|
||||
monkeypatch.setattr(colab, "_publish_cloudflare_url", lambda url: None)
|
||||
monkeypatch.setattr(colab, "_show_and_embed", lambda port, **kwargs: None)
|
||||
monkeypatch.setattr(colab, "_stop_cloudflare_tunnel", lambda: None)
|
||||
monkeypatch.setattr(time, "sleep", lambda _: (_ for _ in ()).throw(KeyboardInterrupt))
|
||||
|
||||
colab.start(cloudflare = False)
|
||||
|
||||
assert finalize_calls == []
|
||||
|
||||
|
||||
def test_finalize_colab_admin_password_redisplay_on_rerun(monkeypatch):
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_load_colab_login_credentials",
|
||||
lambda: ("unsloth", "saved-pass"),
|
||||
)
|
||||
monkeypatch.setattr(colab, "_colab_credentials_still_valid", lambda username, password: True)
|
||||
|
||||
storage = SimpleNamespace(
|
||||
DEFAULT_ADMIN_USERNAME = "unsloth",
|
||||
ensure_default_admin = MagicMock(),
|
||||
get_bootstrap_password = MagicMock(),
|
||||
generate_bootstrap_password = MagicMock(),
|
||||
requires_password_change = MagicMock(return_value = False),
|
||||
update_password = MagicMock(),
|
||||
)
|
||||
auth_pkg = types.ModuleType("auth")
|
||||
auth_pkg.storage = storage
|
||||
with patch.dict("sys.modules", {"auth": auth_pkg, "auth.storage": storage}):
|
||||
result = colab._finalize_colab_admin_password()
|
||||
|
||||
assert result == ("unsloth", "saved-pass")
|
||||
storage.update_password.assert_not_called()
|
||||
|
||||
|
||||
def test_finalize_colab_admin_password_drops_stale_cached_credentials(monkeypatch):
|
||||
"""After an in-app password change the cached first-run password no longer
|
||||
authenticates, so it must not be redisplayed (#7349 Codex review)."""
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_load_colab_login_credentials",
|
||||
lambda: ("unsloth", "stale-pass"),
|
||||
)
|
||||
monkeypatch.setattr(colab, "_colab_credentials_still_valid", lambda username, password: False)
|
||||
cleared: list[bool] = []
|
||||
monkeypatch.setattr(colab, "_clear_colab_login_credentials", lambda: cleared.append(True))
|
||||
|
||||
storage = SimpleNamespace(
|
||||
DEFAULT_ADMIN_USERNAME = "unsloth",
|
||||
ensure_default_admin = MagicMock(),
|
||||
get_bootstrap_password = MagicMock(),
|
||||
generate_bootstrap_password = MagicMock(),
|
||||
requires_password_change = MagicMock(return_value = False),
|
||||
update_password = MagicMock(),
|
||||
)
|
||||
auth_pkg = types.ModuleType("auth")
|
||||
auth_pkg.storage = storage
|
||||
with patch.dict("sys.modules", {"auth": auth_pkg, "auth.storage": storage}):
|
||||
result = colab._finalize_colab_admin_password()
|
||||
|
||||
assert result is None
|
||||
assert cleared == [True]
|
||||
storage.update_password.assert_not_called()
|
||||
|
||||
|
||||
def test_colab_credentials_still_valid_matches_stored_hash(monkeypatch):
|
||||
from auth.hashing import hash_password
|
||||
|
||||
salt, pwd_hash = hash_password("right-pass")
|
||||
storage = SimpleNamespace(
|
||||
get_user_and_secret = MagicMock(return_value = (salt, pwd_hash, "jwt", False)),
|
||||
)
|
||||
with patch.dict("sys.modules", {"auth.storage": storage}):
|
||||
assert colab._colab_credentials_still_valid("unsloth", "right-pass") is True
|
||||
assert colab._colab_credentials_still_valid("unsloth", "wrong-pass") is False
|
||||
|
||||
|
||||
def test_colab_credentials_still_valid_false_when_user_missing(monkeypatch):
|
||||
storage = SimpleNamespace(get_user_and_secret = MagicMock(return_value = None))
|
||||
with patch.dict("sys.modules", {"auth.storage": storage}):
|
||||
assert colab._colab_credentials_still_valid("unsloth", "any") is False
|
||||
|
||||
|
||||
def test_colab_login_html_includes_credentials():
|
||||
html = colab._colab_login_html("unsloth", "alpha-beta-gamma-delta")
|
||||
assert "unsloth" in html
|
||||
assert "alpha-beta-gamma-delta" in html
|
||||
|
||||
|
||||
def test_show_and_embed_renders_cloudflare_before_colab_login(monkeypatch):
|
||||
displayed: list[str] = []
|
||||
ipython_display = SimpleNamespace(
|
||||
HTML = lambda html: SimpleNamespace(html = html),
|
||||
display = lambda html: displayed.append(html.html),
|
||||
)
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: "https://8888-test.prod.colab.dev/")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port, *, _url = None, has_cloudflare_link = False, cloudflare_requested = False: None,
|
||||
)
|
||||
monkeypatch.setattr(colab, "_embed_kernel_port_iframe", lambda port: True)
|
||||
with patch.dict("sys.modules", {"IPython.display": ipython_display}):
|
||||
colab._show_and_embed(
|
||||
8888,
|
||||
cloudflare_url = "https://share.trycloudflare.com",
|
||||
colab_login = ("unsloth", "secret-pass"),
|
||||
)
|
||||
|
||||
assert len(displayed) == 2
|
||||
assert "share.trycloudflare.com" in displayed[0]
|
||||
assert "secret-pass" in displayed[1]
|
||||
|
||||
|
||||
def test_show_and_embed_skips_iframe_on_colab_when_cloudflare_ready(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"https://{port}-test.prod.colab.dev/")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port, *, _url = None, has_cloudflare_link = False, cloudflare_requested = False: None,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_kernel_port_iframe",
|
||||
lambda port: calls.append("kernel_iframe") or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append("html_iframe") or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888, cloudflare_url = "https://share.trycloudflare.com")
|
||||
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_show_and_embed_uses_kernel_helper_on_colab_runtime_despite_localhost(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"http://localhost:{port}")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port,
|
||||
*,
|
||||
_url = None,
|
||||
has_cloudflare_link = False,
|
||||
cloudflare_requested = False: calls.append("show_link"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_kernel_port_iframe",
|
||||
lambda port: calls.append("kernel_iframe") or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append("html_iframe") or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888)
|
||||
|
||||
assert calls == ["show_link", "kernel_iframe"]
|
||||
|
||||
|
||||
def test_show_and_embed_skips_kernel_helper_for_localhost_outside_colab(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"http://localhost:{port}")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: False)
|
||||
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port,
|
||||
*,
|
||||
_url = None,
|
||||
has_cloudflare_link = False,
|
||||
cloudflare_requested = False: calls.append("show_link"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_kernel_port_iframe",
|
||||
lambda port: calls.append("kernel_iframe") or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append("html_iframe") or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888)
|
||||
|
||||
assert calls == ["show_link", "html_iframe"]
|
||||
|
||||
|
||||
def test_show_and_embed_still_embeds_when_show_link_fails(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
monkeypatch.setattr(colab, "get_colab_url", lambda port: f"https://{port}-test.prod.colab.dev/")
|
||||
monkeypatch.setattr(colab, "_is_colab_runtime", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"show_link",
|
||||
lambda port, *, _url = None: (_ for _ in ()).throw(RuntimeError("no display")),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_kernel_port_iframe",
|
||||
lambda port: calls.append("kernel_iframe") or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
colab,
|
||||
"_embed_html_iframe",
|
||||
lambda url, port: calls.append("html_iframe") or True,
|
||||
)
|
||||
|
||||
colab._show_and_embed(8888)
|
||||
|
||||
assert calls == ["kernel_iframe"]
|
||||
|
|
@ -165,6 +165,23 @@ def test_skips_local_path():
|
|||
assert "local" in d.reason
|
||||
|
||||
|
||||
def test_scans_inactive_hf_cache_snapshot_path(tmp_path):
|
||||
# An inactive HF cache loads by snapshot path; the gate must recover the repo id +
|
||||
# commit from models--org--repo/snapshots/<rev> and scan that exact commit, not exempt
|
||||
# it and not fall back to the default branch (an older commit may hold a dropped pickle).
|
||||
snapshot = tmp_path / "models--evil--repo" / "snapshots" / "deadbeef"
|
||||
snapshot.mkdir(parents = True)
|
||||
status = {
|
||||
"scansDone": True,
|
||||
"filesWithIssues": [{"path": "pytorch_model.bin", "level": "unsafe"}],
|
||||
}
|
||||
with _patch_status(status) as model_info:
|
||||
d = evaluate_file_security(str(snapshot))
|
||||
assert d.blocked is True
|
||||
assert model_info.call_args.args[0] == "evil/repo"
|
||||
assert model_info.call_args.kwargs["revision"] == "deadbeef"
|
||||
|
||||
|
||||
def test_remote_gguf_named_repo_is_still_scanned():
|
||||
# Only LOCAL paths skip the Hub scan, so a remote .gguf repo is still scanned and a
|
||||
# poisoned pickle smuggled into it is blocked.
|
||||
|
|
|
|||
|
|
@ -789,9 +789,12 @@ class TestLoadHubDownloadExclusion:
|
|||
|
||||
# The gguf_load_in_flight marker must be entered before the hub-download
|
||||
# guard and the unload so a concurrent load can't race the download
|
||||
# manager. The llama_extra_args inheritance that used to sit between the
|
||||
# marker and the guard now runs in _guard_chat_load_against_training, ahead
|
||||
# of the GGUF branch, so it is no longer a landmark inside this slice.
|
||||
# manager. The llama_extra_args inheritance moved out of the branch into
|
||||
# _resolve_inherited_extra_args, which must still run BEFORE it: the
|
||||
# inherited value (e.g. a carried --no-mmproj) shapes the guard's
|
||||
# require_mmproj. Anchor on the call form so the assertion pins the
|
||||
# endpoint's call site, not the function definition.
|
||||
assert source.index("= _resolve_inherited_extra_args(") < source.index("if config.is_gguf:")
|
||||
assert (
|
||||
gguf_branch.index("enter_context(gguf_load_in_flight")
|
||||
< gguf_branch.index("_hub_download_blocks_gguf_load")
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue