diff --git a/README.md b/README.md index 9ad41546cf..af02f3f667 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ For Windows install instructions, see [here](https://docs.unsloth.ai/get-started - 📣 NEW! **[Qwen3](https://docs.unsloth.ai/basics/qwen3-how-to-run-and-fine-tune)** is now supported! Qwen3-30B-A3B fits on 17.5GB VRAM. - 📣 NEW! Introducing **[Dynamic 2.0](https://docs.unsloth.ai/basics/unsloth-dynamic-2.0-ggufs)** quants that set new benchmarks on 5-shot MMLU & KL Divergence. - 📣 **[Llama 4](https://unsloth.ai/blog/llama4)**, Meta's latest models including Scout & Maverick are now supported. -- 📣 NEW! [**EVERYTHING** is now supported](https://unsloth.ai/blog/gemma3#everything) incuding: FFT, ALL models (Mixtral, MOE, Cohere, Mamba) and all training algorithms (KTO, DoRA) etc. MultiGPU support coming very soon. +- 📣 NEW! [**EVERYTHING** is now supported](https://unsloth.ai/blog/gemma3#everything) including: FFT, ALL models (Mixtral, MOE, Cohere, Mamba) and all training algorithms (KTO, DoRA) etc. MultiGPU support coming very soon. To enable full-finetuning, set ```full_finetuning = True``` and for 8-bit finetuning, set ```load_in_8bit = True``` - 📣 **Gemma 3** by Google: [Read Blog](https://unsloth.ai/blog/gemma3). We [uploaded GGUFs, 4-bit models](https://huggingface.co/collections/unsloth/gemma-3-67d12b7e8816ec6efa7e4e5b). - 📣 Introducing Long-context [Reasoning (GRPO)](https://unsloth.ai/blog/grpo) in Unsloth. Train your own reasoning model with just 5GB VRAM. Transform Llama, Phi, Mistral etc. into reasoning LLMs! @@ -118,7 +118,7 @@ See [here](https://github.com/unslothai/unsloth/edit/main/README.md#advanced-pip Follow the instructions to install [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive). 6. **Install PyTorch:** - You will need the correct version of PyTorch that is compatibile with your CUDA drivers, so make sure to select them carefully. + You will need the correct version of PyTorch that is compatible with your CUDA drivers, so make sure to select them carefully. [Install PyTorch](https://pytorch.org/get-started/locally/). 7. **Install Unsloth:** @@ -143,7 +143,7 @@ trainer = SFTTrainer( For **advanced installation instructions** or if you see weird errors during installations: 1. Install `torch` and `triton`. Go to https://pytorch.org to install it. For example `pip install torch torchvision torchaudio triton` -2. Confirm if CUDA is installated correctly. Try `nvcc`. If that fails, you need to install `cudatoolkit` or CUDA drivers. +2. Confirm if CUDA is installed correctly. Try `nvcc`. If that fails, you need to install `cudatoolkit` or CUDA drivers. 3. Install `xformers` manually. You can try installing `vllm` and seeing if `vllm` succeeds. Check if `xformers` succeeded with `python -m xformers.info` Go to https://github.com/facebookresearch/xformers. Another option is to install `flash-attn` for Ampere GPUs. 4. Double check that your versions of Python, CUDA, CUDNN, `torch`, `triton`, and `xformers` are compatible with one another. The [PyTorch Compatibility Matrix](https://github.com/pytorch/pytorch/blob/main/RELEASE.md#release-compatibility-matrix) may be useful. 5. Finally, install `bitsandbytes` and check it with `python -m bitsandbytes` @@ -244,7 +244,7 @@ from unsloth import FastLanguageModel, FastModel import torch from trl import SFTTrainer, SFTConfig from datasets import load_dataset -max_seq_length = 2048 # Supports RoPE Scaling interally, so choose any! +max_seq_length = 2048 # Supports RoPE Scaling internally, so choose any! # Get LAION dataset url = "https://huggingface.co/datasets/laion/OIG/resolve/main/unified_chip2.jsonl" dataset = load_dataset("json", data_files = {"train" : url}, split = "train") diff --git a/tests/qlora/README.md b/tests/qlora/README.md index e535c38760..c05dcf446d 100644 --- a/tests/qlora/README.md +++ b/tests/qlora/README.md @@ -39,7 +39,7 @@ For the unsloth test, the model's behavior is as expected: - after merging, the model's response contains the answer For the huggingface test, the model's behavior is as expected: -- before training, the model's response does not contains the answer +- before training, the model's response does not contain the answer - after training, the model's response contains the answer - after using peft's `merge_and_unload`, the model's response does not contain the answer - after using my custom merge function, the model's response contains the answer diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.py index 882de28cba..6065af1fc1 100644 --- a/unsloth/models/_utils.py +++ b/unsloth/models/_utils.py @@ -147,7 +147,7 @@ class HideLoggingMessage(logging.Filter): def filter(self, x): return not (self.text in x.getMessage()) pass -# The speedups for torchdynamo mostly come wih GPU Ampere or higher and which is not detected here. +# The speedups for torchdynamo mostly come with GPU Ampere or higher and which is not detected here. from transformers.training_args import logger as transformers_training_args_logger transformers_training_args_logger.addFilter(HideLoggingMessage("The speedups")) # torch.distributed process group is initialized, but parallel_mode != ParallelMode.DISTRIBUTED. diff --git a/unsloth/save.py b/unsloth/save.py index 578f2b49a1..5652e15098 100644 --- a/unsloth/save.py +++ b/unsloth/save.py @@ -1717,7 +1717,7 @@ def push_to_ollama( tag=tag ) - print("Succesfully pushed to ollama") + print("Successfully pushed to ollama")