0c803242ef* feat(studio): add Continued Pretraining (CPT) support Implements CPT as a first-class training method in Unsloth Studio, resolving feature request #4565. Changes: - frontend/src/types/training.ts: add 'cpt' to TrainingMethod union - frontend/src/lib/vram.ts: add 'cpt' to VramTrainingMethod (fp16 footprint) - frontend/src/features/export/constants.ts: add CPT to METHOD_LABELS - frontend/src/features/training/api/mappers.ts: map 'cpt' -> 'Continued Pretraining', force packing=true and train_on_completions=false for CPT payloads - frontend/src/features/studio/sections/model-section.tsx: add 'Continued Pretraining' option (purple dot) to Method selector; update tooltip - frontend/src/features/onboarding/.../model-selection-step.tsx: add CPT to onboarding wizard method dropdown - backend/models/training.py: update training_type field description - backend/core/training/worker.py: detect is_cpt flag, force packing=True, train_on_completions=False, pass is_cpt to _train_worker - backend/core/training/trainer.py: _train_worker reads is_cpt kwarg, forces packing on, skips train_on_responses_only for raw-text pretraining CPT behaviour: - Full model weights (no LoRA adapters), same as Full Finetuning - Sequence packing always enabled for GPU efficiency - Trains on every token (no chat-format masking) - VRAM estimated at fp16 (2.0 bytes/param) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mappers.ts * Add CPT raw dataset support and UI fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add missing training methods module * Handle invalid raw-text rows and expose raw in onboarding --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com> Co-authored-by: Etherll <mrmrmidessam@gmail.com> |
||
|---|---|---|
| .. | ||
| datasets | ||
| hardware | ||
| inference | ||
| models | ||
| paths | ||
| .gitkeep | ||
| __init__.py | ||
| cache_cleanup.py | ||
| downsample.py | ||
| native_path_leases.py | ||
| subprocess_compat.py | ||
| transformers_version.py | ||
| utils.py | ||
| wheel_utils.py | ||