Commit graph

555 commits

Author SHA1 Message Date
Roland Tannous
60997a75eb Install transformers into both site-packages and overlay to fix sub-package resolution during version switch 2026-02-22 19:44:12 +00:00
Roland Tannous
7cde520176 Move transformers overlay to local .venv_overlay/, add huggingface-hub to overlay install 2026-02-22 19:34:46 +00:00
Roland Tannous
15cb9b0f37 Use sys.path overlay to switch transformers versions in-process instead of modifying site-packages 2026-02-22 19:19:12 +00:00
Roland Tannous
0050e78aa3 Fix in-memory transformers version detection and aggressive module purge for 5.1.0/4.57.1 switching 2026-02-22 19:08:18 +00:00
Roland Tannous
1c2653fcc2 aggressive reload_transformers 2026-02-22 18:52:06 +00:00
Roland Tannous
4d06258e93 Auto-switch transformers version (5.1.0/4.57.1) for Ministral-3, GLM-4.7-Flash, Qwen3-30B-A3B models with LoRA adapter resolution 2026-02-22 18:29:40 +00:00
Roland Tannous
132cdb0547 fix: correct vision LoRA defaults for VLMs and remove vision fields from text-only model configs 2026-02-22 15:09:10 +00:00
Roland Tannous
202b7cdfa7 fix: pass full Processor as processing_class for VLM SFTTrainer 2026-02-22 14:11:12 +00:00
Shine1i
0786728323 refactor: extract reusable helpers and streamline seed inspection flow 2026-02-22 03:41:59 +01:00
Shine1i
869ac64e18 feat: enhance dataset seed handling with inspection and UI improvements 2026-02-22 03:39:34 +01:00
Shine1i
3b29d088c0 merge: nightly into feature/canvas-lab 2026-02-22 02:31:32 +01:00
Roland Tannous
c051e3d532 fix: load proper vision processor from base model when FastVisionModel returns raw tokenizer, add tokenize=False to vision chat template 2026-02-21 04:40:29 +00:00
Manan17
f6ebeb1d42 Mapping proper tokenizer for VLMs 2026-02-21 01:57:05 +00:00
Manan17
3fa9e773c2 fixed the vlm's text only errors 2026-02-20 22:23:26 +00:00
Roland Tannous
ed476534f7 Merge pull request #186 from unslothai/fix/colab-setup-fixes
Fix/colab setup fixes
2026-02-20 23:08:25 +04:00
Roland Tannous
08ff8de31d add huggingface-hub==0.36.0 due to colab error 2026-02-20 18:26:20 +00:00
Roland Tannous
48e232b38c add huggingface-hub==0.36.0 due to colab error 2026-02-20 18:24:48 +00:00
Roland Tannous
34131da9a4 moved transformers4.57.1 to no-extra-deps 2026-02-20 18:08:58 +00:00
Shine1i
360b4daf6d feat: enhance execution log tracking, progress updates, and data visualization
- Added `log_lines` field to track and display runtime logs for executions.
- Enhanced progress tracking with terminal-like log outputs and live log scrolling.
- Introduced detailed "model usage" and "dropped columns" analysis in `ExecutionsView`.
- Optimized UI components for displaying dataset metrics, including input/output token averages.
2026-02-20 13:51:19 +01:00
Shine1i
391b633cae feat: enhance progress tracking for execution jobs
- Added logic to calculate and manage column-level progress for job executions.
- Introduced `progress_columns_total` and `_column_done` fields for more granular progress updates.
- Improved overall progress computation by considering total columns and individual progress per column.
2026-02-20 12:52:45 +01:00
Shine1i
13e153e448 feat: refactor and extend recipe execution logic
- Extracted shared execution utilities into `execution-helpers.ts` for reusability across features.
- Replaced deprecated `/preview` endpoint and its logic with unified job execution handling.
- Consolidated job execution flows ("Preview" and "Full Run") into shared `runJobExecution` logic.
- Enhanced execution progress tracking with support for column-level progress reporting.
- Added support for handling execution job events and improved error reporting from the backend.
- Updated backend to better manage dataset access errors and provide more informative error messages.
- Cleaned up redundant code in `use-recipe-studio-actions` and streamlined execution APIs.
2026-02-20 12:47:51 +01:00
Shine1i
f3296b1953 feat: add dataset pagination support for recipe executions
- Introduced backend changes to handle dataset pagination with limit, offset, and total row support.
- Updated frontend execution view with dataset pagination controls, including "Next" and "Prev" buttons.
- Extended recipe execution logic to manage dataset pagination details like page number, page size, and total records.
2026-02-20 12:12:02 +01:00
Shine1i
1259b75d15 feat: add support for full recipe executions with detailed progress and analysis
- Introduced "Full Run" support in execution logic, including progress tracking, cancellation, and job status updates.
- Extended backend to manage full execution jobs, handle dataset previews, and return detailed analysis and artifacts.
- Updated frontend components to support full runs, with execution sorting, live updates, and detailed execution views.
- Enhanced `ExecutionsView` with progress indicators, status filtering, and dataset preview capabilities.
- Added IndexedDB schema migration to track additional execution metadata.
2026-02-20 12:05:42 +01:00
Shine1i
d378e48c2a feat: introduce execution tracking and analysis for recipe preview
- Added `ExecutionsView` with execution history tracking, live updates, and detailed data analysis.
- Implemented IndexedDB support via Dexie to persist execution records locally.
- Enhanced backend preview logic to return execution analysis and artifacts.
- Updated studio header with view toggling between "Editor" and "Executions."
2026-02-20 11:34:25 +01:00
Wasim Yousef Said
6dd0e11439 Merge branch 'nightly' into feature/canvas-lab 2026-02-20 01:23:44 -08:00
Manan17
798bfb8f6f Setting it to total cpu_count // 4 2026-02-20 06:32:01 +00:00
Manan17
fdeccec259 Fixing compare feature 2026-02-19 20:15:44 +00:00
Roland Tannous
18c41c2b08 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Roland Tannous
e2b7b4b54c change train_on_completions to true 2026-02-19 06:02:16 +00:00
Manan17
56869c63bd Passing use_auth = True and also having different checks which is missed by the is_vision function 2026-02-19 02:55:46 +00:00
Roland Tannous
adc0c78dbc reduce dataset_num_proc to 1/4 of cpu_count 2026-02-18 20:53:32 +00:00
Roland Tannous
6aceaec323 Merge pull request #155 from unslothai/fix/sft-tokenizer-unwrap-for-vlm-text
fix: Unwrap ProcessorMixin to raw tokenizer for text-only SFTTrainer on VLM-architecture models
2026-02-18 23:19:57 +04:00
Roland Tannous
e33920974b fix: unwrap ProcessorMixin to raw tokenizer for text-only SFTTrainer on VLM-architecture models 2026-02-18 19:13:20 +00:00
Roland Tannous
a6e2fa5b3a Merge remote-tracking branch 'origin/nightly' into fix/dataset-mapping-vlm-text-datasets 2026-02-18 18:11:45 +04:00
Roland Tannous
940328ce1e Merge remote-tracking branch 'origin/nightly' into feature/colab-notebook 2026-02-18 17:41:07 +04:00
Roland Tannous
d57b2742ab renamed UNSLOTH_FLEX_ATTENTION to UNSLOTH_ENABLE_FLEX_ATTENTION 2026-02-18 09:21:53 +00:00
Roland Tannous
5a02ed4f0f Disable flex attention on Blackwell+ GPUs (sm_120+) at startup 2026-02-18 08:58:25 +00:00
Roland Tannous
d69431fa57 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Manan17
76cd1dc24c fixing the hangup of training after multiple back to back training processes 2026-02-18 08:18:13 +00:00
Manan17
c37bf686a6 Dividing the total cpu_count // 3 2026-02-18 07:59:57 +00:00
Roland Tannous
14edb08cf5 Merge pull request #148 from unslothai/fix/linear
linear fix
2026-02-18 11:10:16 +04:00
Manan17
58116e7e7a fix the linear path on backend 2026-02-18 07:08:32 +00:00
Roland Tannous
d2f7eaf085 Merge pull request #145 from unslothai/fix/check-format-sample
fix: stream HF datasets in check-format endpoint to avoid full d…
2026-02-18 10:49:17 +04:00
Roland Tannous
d7853efd21 debug statements 2026-02-18 00:37:09 +00:00
Roland Tannous
0d8b67b706 fix: normalize target_modules [all-linear] list to string for Unsloth/PEFT compatibility 2026-02-18 00:32:21 +00:00
Roland Tannous
d2332622d1 fix: defensively rename VLM chat column to match model's forward() signature 2026-02-17 23:49:13 +00:00
Roland Tannous
3d0d1c7020 fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Roland Tannous
5864dece26 fix: fix: stream HF datasets in check-format endpoint to avoid full downloads; add info logging to model config endpoints 2026-02-17 22:53:29 +00:00
Shine1i
dc0cec772d feat: enhance training stop and reset flow with detailed checks 2026-02-17 23:32:22 +01:00
Leo Borcherding
3e3c315b01 Merge nightly into feature/colab-notebook - resolved setup.sh conflicts 2026-02-17 15:51:14 -06:00