From ddf118a8fc4aa4acda6c94ca99d17674cb58b7dd Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Thu, 21 Nov 2024 05:01:44 -0800 Subject: [PATCH 1/6] Vision support (#1315) * Fix pad token * Update llama.py * Typo * ignored labels * Revert "ignored labels" This reverts commit 4b25138ac7275bcec147577d97efcd9e3be7e178. * More patching * Update _utils.py * Update _utils.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Feat/all tmp (#1219) * Update save.py Check whether path is in /tmp dir for Kaggle environment * Update save.py Move temporary_location to /tmp in Kaggle * Enhance Kaggle environment support in save and tokenizer utilities --------- Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com> Co-authored-by: Erland366 * Bug fixes * Update pyproject.toml * Update _utils.py * Update __init__.py * Update __init__.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Tied weights * Revert "Tied weights" This reverts commit 820cd4efef5d173f2200ef184b3d8856ba09d75b. * Tied weights * Utils * CE Loss patching * Update __init__.py * Update __init__.py * Patching * Update cross_entropy_loss.py * CE Loss * Update _utils.py * Update _utils.py * CE Loss * Update _utils.py * Update _utils.py * Layernorm * Update _utils.py * Update _utils.py * Post patch * Update _utils.py * Update llama.py * Update _utils.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * typing * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * int64 * Update _utils.py * Update cross_entropy_loss.py * constexpr * constexpr * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update _utils.py * Update _utils.py * Update _utils.py * CE * Update cross_entropy_loss.py * Update _utils.py * Update llama.py * Update _utils.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update utils.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * typing * Update rope_embedding.py * types * Disable compiling * Update _utils.py * Update _utils.py * Forward hook * Update _utils.py * Update llama.py * Update _utils.py * Update llama.py * Update llama.py * Update _utils.py * Update pyproject.toml * Update _utils.py * Update llama.py * CE Loss * Update cross_entropy_loss.py * Update _utils.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254) * Fix: cast logits to float32 in cross_entropy_forward to prevent errors * Update cross_entropy_loss.py --------- Co-authored-by: Daniel Han * Throw error when inferencing longer than max_popsition_embeddings (#1236) * Throw error when inferencing longer than max_popsition_embeddings without rope scaling * Update llama.py --------- Co-authored-by: Daniel Han * CLI now handles user input strings for dtype correctly (#1235) Co-authored-by: root * Update flex_attention.py * Update _utils.py * Update _utils.py * Update flex_attention.py * Update flex_attention.py * Update loader.py * Update loader.py * Update flex_attention.py * Update flex_attention.py * Update flex_attention.py * Update flex_attention.py * Update _utils.py * Update cross_entropy_loss.py * Update _utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * triton_cast * Update utils.py * Qwen 2.5 Coder * Fix/export mistral (#1281) * Enhance install_python_non_blocking to handle protobuf installation and process management * Revert "Enhance install_python_non_blocking to handle protobuf installation and process management" This reverts commit a3b796a05841fb8d93c652c845591e12cf81ea93. * Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 * Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266" This reverts commit f00fbf5eac7ad4f5d48c70b98d770255d1a9ef58. * Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 * Update __init__.py --------- Co-authored-by: Daniel Han * DOC Update - Update README.md with os.environ in example (#1269) * Update README.md with os.environ in example Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup. As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required. Small change but a bit time saver for those who straight away copies the tutorials * Update README.md --------- Co-authored-by: Daniel Han * fix/get_chat_template (#1246) * Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to * Remove type hinting * Update chat_templates.py --------- Co-authored-by: Daniel Han * fix/sft-trainer (#1276) * Add patch for SFTTrainer to maintain backward compatibility with TRL changes * Update trainer.py * Update trainer.py * Refactor trainer patch to maintain backward compatibility with TRL changes * Update trainer.py * Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch --------- Co-authored-by: Daniel Han * Update __init__.py * Update trainer.py * Update trainer.py * Update trainer.py * Update tokenizer_utils.py * Update llama.py * Fix #853 * fix/sfttrainer-compatibility (#1293) * Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance * Update trainer.py * Update trainer.py --------- Co-authored-by: Daniel Han * Update rms_layernorm.py * Update rms_layernorm.py * Gemma * Update rms_layernorm.py * Update gemma2.py * Cut Cross Entropy * Update llama.py * Cut Cross Entropy * Update llama.py * Update llama.py * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update mapper.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * patch_fast_lora * vision * Update fast_lora.py * Update _utils.py * Update _utils.py * Vision * Update trainer.py * Update save.py * FastBaseVisionModel * Update loader_utils.py * Update vision.py * Update loader.py * Update vision.py * Update loader.py * Update vision.py * Update _utils.py * tokenizer_name * Update loader.py * Update vision.py * Update save.py * Update save.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update vision.py * Update _utils.py --------- Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com> Co-authored-by: Erland366 Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com> Co-authored-by: Datta Nimmaturi Co-authored-by: Edwin Fennell Co-authored-by: root Co-authored-by: Uday Girish Maradana --- unsloth/__init__.py | 2 +- unsloth/kernels/__init__.py | 1 + unsloth/kernels/fast_lora.py | 78 +++++ unsloth/kernels/rms_layernorm.py | 10 +- unsloth/models/__init__.py | 2 +- unsloth/models/_utils.py | 211 +++++++++--- unsloth/models/gemma2.py | 11 +- unsloth/models/llama.py | 50 ++- unsloth/models/loader.py | 340 ++++++++++++++------ unsloth/models/loader_utils.py | 120 +++++++ unsloth/models/mapper.py | 48 ++- unsloth/models/vision.py | 535 ++++++++++++------------------- unsloth/save.py | 171 +++++++++- unsloth/trainer.py | 31 +- 14 files changed, 1087 insertions(+), 523 deletions(-) create mode 100644 unsloth/models/loader_utils.py diff --git a/unsloth/__init__.py b/unsloth/__init__.py index 745b210208..980425e1f1 100644 --- a/unsloth/__init__.py +++ b/unsloth/__init__.py @@ -55,7 +55,7 @@ else: pass # Reduce VRAM usage by reducing fragmentation -os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True" +os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True,roundup_power2_divisions:[64:128,256:64,>:32]" # Hugging Face Hub faster downloads if "HF_HUB_ENABLE_HF_TRANSFER" not in os.environ: diff --git a/unsloth/kernels/__init__.py b/unsloth/kernels/__init__.py index 82e7641693..ef5fa5da70 100644 --- a/unsloth/kernels/__init__.py +++ b/unsloth/kernels/__init__.py @@ -42,6 +42,7 @@ from .fast_lora import ( apply_lora_mlp_geglu_approx, apply_lora_qkv, apply_lora_o, + fast_lora_forward, ) from .utils import fast_dequantize, fast_gemv, QUANT_STATE, fast_linear_forward, matmul_lora diff --git a/unsloth/kernels/fast_lora.py b/unsloth/kernels/fast_lora.py index 2177b43b9e..c2b7929a29 100644 --- a/unsloth/kernels/fast_lora.py +++ b/unsloth/kernels/fast_lora.py @@ -410,3 +410,81 @@ def apply_lora_o(self, X): O = LoRA_W.apply(X, OW, OW_quant, OA, OB, OS) return O pass + + +IDENTITY_DROPOUT = torch.nn.Identity +@torch._disable_dynamo +def fast_lora_forward(self, x: torch.Tensor, *args, **kwargs) -> torch.Tensor: + raise NotImplementedError( + "Unsloth: Currently not supported yet - reshaping done incorrectly" + ) + self._check_forward_args(x, *args, **kwargs) + adapter_names = kwargs.pop("adapter_names", None) + + if self.disable_adapters: + if self.merged: + self.unmerge() + result = self.base_layer(x, *args, **kwargs) + elif adapter_names is not None: + result = self._mixed_batch_forward(x, *args, adapter_names=adapter_names, **kwargs) + elif self.merged: + result = self.base_layer(x, *args, **kwargs) + else: + # Fastpath + if len(self.active_adapters) == 1: + active_adapter = self.active_adapters[0] + if active_adapter not in self.lora_A.keys(): return self.base_layer(x, *args, **kwargs) + + dropout = self.lora_dropout[active_adapter] + if isinstance(dropout, IDENTITY_DROPOUT) and not self.use_dora[active_adapter]: + lora_A = self.lora_A[active_adapter].weight + lora_B = self.lora_B[active_adapter].weight + scaling = self.scaling[active_adapter] + W = self.base_layer.weight + return LoRA_W.apply(x, W, QUANT_STATE(W), lora_A, lora_B, scaling) + pass + pass + + result = self.base_layer(x, *args, **kwargs) + # As per Tim Dettmers, for 4bit, we need to defensively clone here. + # The reason is that in some cases, an error can occur that backprop + # does not work on a manipulated view. This issue may be solved with + # newer PyTorch versions but this would need extensive testing to be + # sure. + result = result.clone() + + for active_adapter in self.active_adapters: + if active_adapter not in self.lora_A.keys(): + continue + lora_A = self.lora_A[active_adapter] + lora_B = self.lora_B[active_adapter] + dropout = self.lora_dropout[active_adapter] + scaling = self.scaling[active_adapter] + + requires_conversion = not torch.is_autocast_enabled() + if requires_conversion: + expected_dtype = result.dtype + x = x.to(lora_A.weight.dtype) + + if not self.use_dora[active_adapter]: + result = result + lora_B(lora_A(dropout(x))) * scaling + else: + if isinstance(dropout, torch.nn.Identity) or not self.training: + base_result = result + else: + x = dropout(x) + base_result = None + + result = result + self.lora_magnitude_vector[active_adapter]( + x, + lora_A=lora_A, + lora_B=lora_B, + scaling=scaling, + base_layer=self.get_base_layer(), + base_result=base_result, + ) + if requires_conversion: + result = result.to(expected_dtype) + + return result +pass diff --git a/unsloth/kernels/rms_layernorm.py b/unsloth/kernels/rms_layernorm.py index 4b22f8c3e5..b74d636c63 100644 --- a/unsloth/kernels/rms_layernorm.py +++ b/unsloth/kernels/rms_layernorm.py @@ -57,6 +57,7 @@ pass @triton.jit def _rms_layernorm_backward( dY, dY_row_stride, + dX, dX_row_stride, X, X_row_stride, W, W_row_stride, r, r_row_stride, @@ -78,6 +79,9 @@ def _rms_layernorm_backward( X += row_idx * X_row_stride r += row_idx * r_row_stride + if GEMMA: dX += row_idx * dY_row_stride + else: dX = dY + dY_row = tl.load(dY + col_offsets, mask = mask, other = 0).to(tl.float32) X_row = tl.load(X + col_offsets, mask = mask, other = 0).to(tl.float32) W_row = tl.load(W + col_offsets, mask = mask, other = 0).to(tl.float32) @@ -91,7 +95,7 @@ def _rms_layernorm_backward( rowsum_dY_normed = tl.sum(dY_W * normed, axis = 0) output = inv_var/n_cols * (n_cols*dY_W - normed*rowsum_dY_normed) - tl.store(dY + col_offsets, output, mask = mask) + tl.store(dX + col_offsets, output, mask = mask) pass @@ -172,9 +176,11 @@ class Fast_RMS_Layernorm(torch.autograd.Function): n_cols : int n_rows, n_cols = dY.shape # dW = X + dX = torch.empty_like(dY, device = "cuda:0") if ctx.GEMMA else dY _rms_layernorm_backward[(n_rows,)]( dY, dY.stride(0), + dX, dX.stride(0), X, X .stride(0), W, W .stride(0), r, r .stride(0), @@ -184,7 +190,7 @@ class Fast_RMS_Layernorm(torch.autograd.Function): BLOCK_SIZE = ctx.BLOCK_SIZE, num_warps = ctx.num_warps, ) - dX = dY.view(*shape) + dX = dX.view(*shape) return dX, None, None, None pass pass diff --git a/unsloth/models/__init__.py b/unsloth/models/__init__.py index e67a9e5fad..3230cdc207 100644 --- a/unsloth/models/__init__.py +++ b/unsloth/models/__init__.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -from .loader import FastLanguageModel +from .loader import FastLanguageModel, FastVisionModel from .llama import FastLlamaModel from .mistral import FastMistralModel from .qwen2 import FastQwen2Model diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.py index daa81d97ac..ee85ba3c36 100644 --- a/unsloth/models/_utils.py +++ b/unsloth/models/_utils.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "2024.11.7" +__version__ = "2024.11.8" __all__ = [ "prepare_model_for_kbit_training", @@ -52,6 +52,17 @@ __all__ = [ "unpatch_unsloth_gradient_checkpointing", "patch_gradient_checkpointing", "unpatch_gradient_checkpointing", + + "HAS_CUT_CROSS_ENTROPY", + "fused_linear_cross_entropy", + "patch_unsloth_smart_gradient_checkpointing", + "unpatch_unsloth_smart_gradient_checkpointing", + "create_gradient_checkpointing_buffer", + + "patch_compiled_autograd", + "process_vision_info", + "unsloth_compile_transformers", + "patch_fast_lora", ] import torch @@ -70,6 +81,7 @@ from unsloth_zoo.patching_utils import ( patch_layernorm, patch_torch_compile, patch_model_and_tokenizer, + patch_compiled_autograd, ) from unsloth_zoo.gradient_checkpointing import ( Unsloth_Offloaded_Gradient_Checkpointer, @@ -81,6 +93,21 @@ from unsloth_zoo.gradient_checkpointing import ( unsloth_gradient_checkpoint, patch_gradient_checkpointing, unpatch_gradient_checkpointing, + + patch_unsloth_smart_gradient_checkpointing, + unpatch_unsloth_smart_gradient_checkpointing, + create_gradient_checkpointing_buffer, +) +from unsloth_zoo.loss_utils import ( + HAS_CUT_CROSS_ENTROPY, + fused_linear_cross_entropy, +) +from unsloth_zoo.vision_utils import ( + process_vision_info, +) +from unsloth_zoo.compiler import ( + get_transformers_model_type, + unsloth_compile_transformers as _unsloth_compile_transformers, ) # ============================================= @@ -120,6 +147,22 @@ try: except: pass +# The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function. +try: + from accelerate.utils.modeling import logger as accelerate_utils_modeling_logger + accelerate_utils_modeling_logger.addFilter(HideLoggingMessage("The model weights are not tied")) + del accelerate_utils_modeling_logger +except: + pass + +# Setting `pad_token_id` to `eos_token_id` +try: + from transformers.generation.utils import logger as transformers_generation_utils_logger + transformers_generation_utils_logger.addFilter(HideLoggingMessage("Setting `pad_token_id` to `eos_token_id`")) + del transformers_generation_utils_logger +except: + pass + # ============================================= # ============================================= @@ -282,54 +325,60 @@ from transformers.models.llama.modeling_llama import logger # ============================================= # Get Xformers -from xformers import __version__ as xformers_version -# Temporarily disable 0.0.27 and higher - inference issues -if False: #Version(xformers_version) >= Version("0.0.27"): - raise ImportError( - "Unsloth: If you are in Colab, we updated the top cell install instructions - please change it to below "\ - "then press Disconnect Runtime and then Restart it.\n"\ - "\n"\ - "%%capture\n" - "# Installs Unsloth, Xformers (Flash Attention) and all other packages!\n" - '!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"\n' - '!pip install --no-deps "xformers<=0.0.27" trl peft accelerate bitsandbytes\n'\ - '\n'\ - f"Otherwise in local machines, your xformers version of {xformers_version} is too new.\n"\ - 'Please downgrade xformers via `pip install --force-reinstall "xformers<=0.0.27"' - ) -pass - -if Version(torch_version) < Version("2.2.0") and Version(xformers_version) >= Version("0.0.24"): - raise ImportError( - f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ - f"Please install xformers < 0.0.24 for torch = {torch_version}." - ) -elif Version(torch_version) < Version("2.3.0") and Version(xformers_version) >= Version("0.0.26"): - raise ImportError( - f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ - f"Please install xformers < 0.0.26 for torch = {torch_version}." - ) -elif Version(torch_version) < Version("2.4.0") and Version(xformers_version) > Version("0.0.27"): - raise ImportError( - f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ - f"Please install xformers <= 0.0.27 for torch = {torch_version}." - ) -pass - -from xformers._cpp_lib import _register_extensions try: - _register_extensions() # Check if C++ modules are loaded correctly -except Exception as error: - raise ImportError( - "Unsloth: Xformers was not installed correctly.\n"\ - "Please install xformers separately first.\n"\ - "Then confirm if it's correctly installed by running:\n"\ - "python -m xformers.info\n\n" - "Longer error message:\n" + str(error) - ) + from xformers import __version__ as xformers_version + # Temporarily disable 0.0.27 and higher - inference issues + if False: #Version(xformers_version) >= Version("0.0.27"): + raise ImportError( + "Unsloth: If you are in Colab, we updated the top cell install instructions - please change it to below "\ + "then press Disconnect Runtime and then Restart it.\n"\ + "\n"\ + "%%capture\n" + "# Installs Unsloth, Xformers (Flash Attention) and all other packages!\n" + '!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"\n' + '!pip install --no-deps "xformers<=0.0.27" trl peft accelerate bitsandbytes\n'\ + '\n'\ + f"Otherwise in local machines, your xformers version of {xformers_version} is too new.\n"\ + 'Please downgrade xformers via `pip install --force-reinstall "xformers<=0.0.27"' + ) + pass + + if Version(torch_version) < Version("2.2.0") and Version(xformers_version) >= Version("0.0.24"): + raise ImportError( + f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ + f"Please install xformers < 0.0.24 for torch = {torch_version}." + ) + elif Version(torch_version) < Version("2.3.0") and Version(xformers_version) >= Version("0.0.26"): + raise ImportError( + f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ + f"Please install xformers < 0.0.26 for torch = {torch_version}." + ) + elif Version(torch_version) < Version("2.4.0") and Version(xformers_version) > Version("0.0.27"): + raise ImportError( + f"Unsloth: You have torch = {torch_version} but xformers = {xformers_version}.\n"\ + f"Please install xformers <= 0.0.27 for torch = {torch_version}." + ) + pass + + from xformers._cpp_lib import _register_extensions + try: + _register_extensions() # Check if C++ modules are loaded correctly + except Exception as error: + raise ImportError( + "Unsloth: Xformers was not installed correctly.\n"\ + "Please install xformers separately first.\n"\ + "Then confirm if it's correctly installed by running:\n"\ + "python -m xformers.info\n\n" + "Longer error message:\n" + str(error) + ) + pass + import xformers.ops.fmha as xformers + xformers_attention = xformers.memory_efficient_attention +except: + xformers = None + xformers_attention = None + xformers_version = None pass -import xformers.ops.fmha as xformers -xformers_attention = xformers.memory_efficient_attention # Check TRL version from trl import __version__ as trl_version @@ -658,7 +707,7 @@ BitsAndBytesConfig__init__ = BitsAndBytesConfig__init__.replace( ) def _prepare_backend( - self, cpu: bool = False, sagemaker_dp = False, backend: str = None, + self, cpu = False, sagemaker_dp = False, backend: str = None, ) -> tuple[str, DistributedType]: return None, DistributedType.NO pass @@ -1047,3 +1096,69 @@ def patch_tokenizer(model, tokenizer): model.config.update({"unsloth_version" : __version__}) return model, tokenizer pass + + +def patch_fast_lora(): + import peft.tuners.lora.bnb + peft.tuners.lora.bnb.Linear4bit.forward = fast_lora_forward +pass + + +def unsloth_compile_transformers( + model_name, + token = None, + revision = None, + trust_remote_code = False, + sdpa_dynamic_mask = True, + sdpa_bool_masks = True, + sdpa_gqa_replace = True, + sdpa_dynamic_compile = True, + compile_attention = True, + disable_causal_masks = True, + compile_torch_modules = True, + compile_custom_modules = True, + compile_function_calls = True, + fuse_lm_head = True, + gradient_checkpointing = True, + manual_replacements = True, + epilogue_fusion = True, + max_autotune = False, + shape_padding = True, + cudagraphs = False, + debug = False, + import_from_cache = False, + disable = False, +): + if disable: return + model_types = get_transformers_model_type( + model_name = model_name, + token = token, + revision = revision, + trust_remote_code = trust_remote_code, + ) + for model_type in model_types: + _unsloth_compile_transformers( + model_type, + sdpa_dynamic_mask = sdpa_dynamic_mask, + sdpa_bool_masks = sdpa_bool_masks, + sdpa_gqa_replace = sdpa_gqa_replace, + sdpa_dynamic_compile = sdpa_dynamic_compile, + compile_attention = compile_attention, + disable_causal_masks = disable_causal_masks, + compile_torch_modules = compile_torch_modules, + compile_custom_modules = compile_custom_modules, + compile_function_calls = compile_function_calls, + fuse_lm_head = fuse_lm_head, + gradient_checkpointing = gradient_checkpointing, + manual_replacements = manual_replacements, + epilogue_fusion = epilogue_fusion, + max_autotune = max_autotune, + shape_padding = shape_padding, + cudagraphs = cudagraphs, + debug = debug, + import_from_cache = import_from_cache, + disable = disable, + ) + pass + return model_types +pass diff --git a/unsloth/models/gemma2.py b/unsloth/models/gemma2.py index 4eb9d64313..62ecb9690f 100644 --- a/unsloth/models/gemma2.py +++ b/unsloth/models/gemma2.py @@ -60,8 +60,7 @@ if HAS_FLASH_ATTENTION_SOFTCAPPING: from flash_attn import flash_attn_func # [TODO] We must randomnly use torch.compile? -# I checked the gradients and formulas and I'm sure it's correct. -# I'm stumped :( +# Gemma 2 uses double RMS Layernorms, so the backward passes should not overwrite the gradients! @torch.compile(fullgraph = False, dynamic = True, options = torch_compile_options) def fast_rms_layernorm_gemma2_compiled(layernorm, X, gemma = True): old_dtype = X.dtype @@ -207,7 +206,7 @@ def Gemma2DecoderLayer_fast_forward( hidden_states += residual else: residual = hidden_states - hidden_states = fast_rms_layernorm_gemma2_compiled(self.input_layernorm, hidden_states, gemma = True) + hidden_states = fast_rms_layernorm(self.input_layernorm, hidden_states, gemma = True) hidden_states, self_attn_weights, present_key_value = self.self_attn( hidden_states=hidden_states, causal_mask=causal_mask, @@ -218,14 +217,14 @@ def Gemma2DecoderLayer_fast_forward( use_cache=use_cache, padding_mask=padding_mask, ) - hidden_states = fast_rms_layernorm_gemma2_compiled(self.post_attention_layernorm, hidden_states, gemma = True) + hidden_states = fast_rms_layernorm(self.post_attention_layernorm, hidden_states, gemma = True) hidden_states = residual + hidden_states # Fully Connected residual = hidden_states - hidden_states = fast_rms_layernorm_gemma2_compiled(self. pre_feedforward_layernorm, hidden_states, gemma = True) + hidden_states = fast_rms_layernorm(self. pre_feedforward_layernorm, hidden_states, gemma = True) hidden_states = self.mlp(hidden_states) - hidden_states = fast_rms_layernorm_gemma2_compiled(self.post_feedforward_layernorm, hidden_states, gemma = True) + hidden_states = fast_rms_layernorm(self.post_feedforward_layernorm, hidden_states, gemma = True) hidden_states = residual + hidden_states pass diff --git a/unsloth/models/llama.py b/unsloth/models/llama.py index 47a57024a2..0256fc1830 100644 --- a/unsloth/models/llama.py +++ b/unsloth/models/llama.py @@ -719,25 +719,33 @@ def LlamaModel_fast_forward( pass # Gemma2 has alternating SWA and global attn + use_static_mask = True + dynamic_SWA_mask = None + dynamic_GA_mask = None if IS_GEMMA2: if HAS_FLASH_ATTENTION_SOFTCAPPING and attention_mask is None: self.SWA_mask = True self.GA_mask = False elif attention_mask is not None: - self.SWA_mask = _prepare_4d_causal_attention_mask_for_sdpa( + + # Fixes https://github.com/unslothai/unsloth/issues/853 + # Unsloth needs a 2D mask, not a [2, 1, n, n] mask! + dynamic_SWA_mask = _prepare_4d_causal_attention_mask_for_sdpa( attention_mask, (batch_size, seq_length), inputs_embeds, past_key_values_length, sliding_window = self.config.sliding_window, - ) - self.GA_mask = _prepare_4d_causal_attention_mask_for_sdpa( + )[0][0] + dynamic_GA_mask = _prepare_4d_causal_attention_mask_for_sdpa( attention_mask, (batch_size, seq_length), inputs_embeds, past_key_values_length, sliding_window = None, - ) + )[0][0] + use_static_mask = False + elif not hasattr(self, "SWA_mask"): if HAS_FLEX_ATTENTION: # Use Flex Attention instead! @@ -772,7 +780,12 @@ def LlamaModel_fast_forward( past_key_value = past_key_values[idx] if past_key_values is not None else None mask = causal_mask - if IS_GEMMA2: mask = self.SWA_mask if (idx % 2 == 0) else self.GA_mask + if IS_GEMMA2: + if (idx % 2 == 0): + mask = self.SWA_mask if use_static_mask else dynamic_SWA_mask + else: + mask = self. GA_mask if use_static_mask else dynamic_GA_mask + pass if offloaded_gradient_checkpointing: hidden_states = Unsloth_Offloaded_Gradient_Checkpointer.apply( @@ -955,14 +968,39 @@ def CausalLM_fast_forward(fast_forward_inference): ) pass hidden_states = outputs[0] + bsz, q_len, hd = hidden_states.shape lm_head = self.lm_head.weight + logit_softcapping = getattr(self.config, "final_logit_softcapping", 0) + logit_scaling = getattr(self.config, "logit_scale", 0) + if bsz == 1 and q_len == 1: logits = torch.mv(lm_head, hidden_states.ravel().to(lm_head.dtype)) logits = logits.unsqueeze(0).unsqueeze(0) elif num_logits_to_keep != 0: logits = self.lm_head(hidden_states[:, -num_logits_to_keep:, :].to(lm_head.dtype)) else: + if HAS_CUT_CROSS_ENTROPY and labels is not None: + n_items = kwargs.get("num_items_in_batch", None) or kwargs.get("n_items", None) + loss = fused_linear_cross_entropy( + hidden_states = hidden_states, + lm_weight = lm_head, + labels = labels, + num_items_in_batch = n_items, + logit_softcapping = logit_softcapping, + ) + if not return_dict: + output = (logits,) + outputs[1:] + return (loss,) + output if loss is not None else output + + return CausalLMOutputWithPast( + loss=loss, + logits=None, + past_key_values=outputs.past_key_values, + hidden_states=outputs.hidden_states, + attentions=outputs.attentions, + ) + pass logits = self.lm_head(hidden_states.to(lm_head.dtype)) pass @@ -974,8 +1012,6 @@ def CausalLM_fast_forward(fast_forward_inference): pass loss = None - logit_softcapping = getattr(self.config, "final_logit_softcapping", 0) - logit_scaling = getattr(self.config, "logit_scale", 0) if labels is not None: shift_logits = logits if not hasattr(self, "extra_ignored_labels"): diff --git a/unsloth/models/loader.py b/unsloth/models/loader.py index 7a6322d248..232fe6acff 100644 --- a/unsloth/models/loader.py +++ b/unsloth/models/loader.py @@ -20,8 +20,8 @@ from .cohere import FastCohereModel from transformers import AutoConfig from transformers import __version__ as transformers_version from peft import PeftConfig, PeftModel -from .mapper import INT_TO_FLOAT_MAPPER, FLOAT_TO_INT_MAPPER, MAP_TO_UNSLOTH_16bit -import os +from .loader_utils import get_model_name +import os, contextlib, sys try: from huggingface_hub.utils import get_token except: @@ -63,105 +63,6 @@ def _get_dtype(dtype): pass -def __get_model_name( - model_name, - load_in_4bit = True, - INT_TO_FLOAT_MAPPER = None, - FLOAT_TO_INT_MAPPER = None, - MAP_TO_UNSLOTH_16bit = None, -): - model_name = str(model_name) - lower_model_name = model_name.lower() - - if not SUPPORTS_FOURBIT and lower_model_name in INT_TO_FLOAT_MAPPER: - - model_name = INT_TO_FLOAT_MAPPER[lower_model_name] - logger.warning_once( - f"Unsloth: Your transformers version of {transformers_version} does not support native "\ - f"4bit loading.\nThe minimum required version is 4.37.\n"\ - f'Try `pip install --upgrade "transformers>=4.37"`\n'\ - f"to obtain the latest transformers build, then restart this session.\n"\ - f"For now, we shall load `{model_name}` instead (still 4bit, just slower downloading)." - ) - return model_name - - elif not load_in_4bit and lower_model_name in INT_TO_FLOAT_MAPPER: - - new_model_name = INT_TO_FLOAT_MAPPER[lower_model_name] - # logger.warning_once( - # f"Unsloth: You passed in `{model_name}` which is a 4bit model, yet you set\n"\ - # f"`load_in_4bit = False`. We shall load `{new_model_name}` instead." - # ) - return new_model_name - - elif not load_in_4bit and lower_model_name in MAP_TO_UNSLOTH_16bit: - - new_model_name = MAP_TO_UNSLOTH_16bit[lower_model_name] - return new_model_name - - elif load_in_4bit and SUPPORTS_FOURBIT and lower_model_name in FLOAT_TO_INT_MAPPER: - - new_model_name = FLOAT_TO_INT_MAPPER[lower_model_name] - # logger.warning_once( - # f"Unsloth: You passed in `{model_name}` and `load_in_4bit = True`.\n"\ - # f"We shall load `{new_model_name}` for 4x faster loading." - # ) - return new_model_name - pass - - return None -pass - - -def _get_new_mapper(): - try: - import requests - new_mapper = "https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/models/mapper.py" - with requests.get(new_mapper, timeout = 3) as new_mapper: new_mapper = new_mapper.text - new_mapper = new_mapper[new_mapper.find("__INT_TO_FLOAT_MAPPER"):] - new_mapper = new_mapper\ - .replace("INT_TO_FLOAT_MAPPER", "NEW_INT_TO_FLOAT_MAPPER")\ - .replace("FLOAT_TO_INT_MAPPER", "NEW_FLOAT_TO_INT_MAPPER")\ - .replace("MAP_TO_UNSLOTH_16bit", "NEW_MAP_TO_UNSLOTH_16bit") - - exec(new_mapper, globals()) - return NEW_INT_TO_FLOAT_MAPPER, NEW_FLOAT_TO_INT_MAPPER, NEW_MAP_TO_UNSLOTH_16bit - except: - return {}, {}, {} - pass -pass - - -def get_model_name(model_name, load_in_4bit = True): - new_model_name = __get_model_name( - model_name = model_name, - load_in_4bit = load_in_4bit, - INT_TO_FLOAT_MAPPER = INT_TO_FLOAT_MAPPER, - FLOAT_TO_INT_MAPPER = FLOAT_TO_INT_MAPPER, - MAP_TO_UNSLOTH_16bit = MAP_TO_UNSLOTH_16bit, - ) - if new_model_name is None and model_name.count("/") == 1 and model_name[0].isalnum(): - # Try checking if a new Unsloth version allows it! - NEW_INT_TO_FLOAT_MAPPER, NEW_FLOAT_TO_INT_MAPPER, NEW_MAP_TO_UNSLOTH_16bit = _get_new_mapper() - upgraded_model_name = __get_model_name( - model_name = model_name, - load_in_4bit = load_in_4bit, - INT_TO_FLOAT_MAPPER = NEW_INT_TO_FLOAT_MAPPER, - FLOAT_TO_INT_MAPPER = NEW_FLOAT_TO_INT_MAPPER, - MAP_TO_UNSLOTH_16bit = NEW_MAP_TO_UNSLOTH_16bit, - ) - if upgraded_model_name is not None: - raise NotImplementedError( - f"Unsloth: {model_name} is not supported in your current Unsloth version! Please update Unsloth via:\n\n"\ - 'pip uninstall unsloth -y\n'\ - 'pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"' - ) - pass - pass - return new_model_name if new_model_name is not None else model_name -pass - - class FastLanguageModel(FastLlamaModel): @staticmethod def from_pretrained( @@ -333,7 +234,8 @@ class FastLanguageModel(FastLlamaModel): else: raise NotImplementedError( f"Unsloth: {model_name} not supported yet!\n"\ - "Make an issue to https://github.com/unslothai/unsloth!", + "Maybe you're doing vision finetuning? Please use FastVisionModel instead!\n"\ + "Otherwise, make an issue to https://github.com/unslothai/unsloth!", ) pass @@ -411,4 +313,236 @@ class FastLanguageModel(FastLlamaModel): pass return model, tokenizer pass -pass \ No newline at end of file +pass + + +from ._utils import ( + patch_compiling_bitsandbytes, + patch_model_and_tokenizer, + prepare_model_for_kbit_training, + patch_unsloth_smart_gradient_checkpointing, + patch_compiled_autograd, + process_vision_info, + unsloth_compile_transformers, +) +from ..kernels import ( + patch_loss_functions, + post_patch_loss_function, +) +from .vision import FastBaseVisionModel + + +class FastVisionModel(FastBaseVisionModel): + @staticmethod + def from_pretrained( + model_name = "unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit", + max_seq_length = None, # [TODO] No effect + dtype = None, + load_in_4bit = True, + token = None, + device_map = "sequential", + rope_scaling = None, # [TODO] No effect + fix_tokenizer = True, # [TODO] No effect + trust_remote_code = False, + use_gradient_checkpointing = "unsloth", + resize_model_vocab = None, # [TODO] No effect + revision = None, + *args, **kwargs, + ): + if token is None: token = get_token() + + patch_compiled_autograd() + patch_compiling_bitsandbytes() + if use_gradient_checkpointing == "unsloth": + patch_unsloth_smart_gradient_checkpointing() + + old_model_name = model_name + model_name = get_model_name(model_name, load_in_4bit) + + with contextlib.redirect_stdout(open(os.devnull, "w")): + patch_loss_functions(torch_compile = False) + model_types = unsloth_compile_transformers( + model_name = model_name, + sdpa_dynamic_mask = True, + sdpa_bool_masks = True, + sdpa_gqa_replace = True, + sdpa_dynamic_compile = True, + compile_attention = True, + disable_causal_masks = True, + compile_torch_modules = True, + compile_custom_modules = True, + compile_function_calls = True, + fuse_lm_head = True, + gradient_checkpointing = True, + manual_replacements = True, + epilogue_fusion = True, + max_autotune = False, + shape_padding = True, + cudagraphs = False, + debug = False, + import_from_cache = False, + disable = False, + ) + pass + + # First check if it's a normal model via AutoConfig + from huggingface_hub.utils import disable_progress_bars, enable_progress_bars, are_progress_bars_disabled + was_disabled = are_progress_bars_disabled() + disable_progress_bars() + + autoconfig_error = None + peft_error = None + try: + model_config = AutoConfig.from_pretrained( + model_name, + token = token, + revision = revision, + trust_remote_code = trust_remote_code, + ) + is_model = True + except Exception as error: + autoconfig_error = str(error) + is_model = False + try: + peft_config = PeftConfig.from_pretrained( + model_name, + token = token, + revision = revision, + trust_remote_code = trust_remote_code, + ) + is_peft = True + except Exception as error: + peft_error = str(error) + is_peft = False + pass + + # Both config.json and adapter_config.json should not exist! + + # Old transformers versions check + both_exist = (is_model and is_peft) and not SUPPORTS_LLAMA32 + + # New transformers need to check manually. + if SUPPORTS_LLAMA32: + # Check if folder exists locally + if os.path.isdir(model_name): + exist_adapter_config = os.path.exists(os.path.join(model_name, "adapter_config.json")) + exist_config = os.path.exists(os.path.join(model_name, "config.json")) + both_exist = exist_adapter_config and exist_config + else: + files = HfFileSystem(token = token).glob(os.path.join(model_name, "*.json")) + files = (os.path.split(x)[-1] for x in files) + if sum(x == "adapter_config.json" or x == "config.json" for x in files) >= 2: + both_exist = True + pass + pass + pass + + # Error out if both LoRA and normal model config exists. + if both_exist: + raise RuntimeError( + "Unsloth: Your repo has a LoRA adapter and a base model.\n"\ + "You have 2 files `config.json` and `adapter_config.json`.\n"\ + "We must only allow one config file.\n"\ + "Please separate the LoRA and base models to 2 repos." + ) + + elif not is_model and not is_peft: + error = autoconfig_error or peft_error + # Old transformers version + if "rope_scaling" in error.lower() and not SUPPORTS_LLAMA31: + raise ImportError( + f"Unsloth: Your transformers version of {transformers_version} does not support new RoPE scaling methods.\n"\ + f"This includes Llama 3.1. The minimum required version is 4.43.2\n"\ + f'Try `pip install --upgrade "transformers>=4.43.2"`\n'\ + f"to obtain the latest transformers build, then restart this session."\ + ) + raise RuntimeError(autoconfig_error or peft_error) + pass + + # Get base model for PEFT: + if is_peft: + # Check base model again for PEFT + model_name = get_model_name(peft_config.base_model_name_or_path, load_in_4bit) + model_config = AutoConfig.from_pretrained( + model_name, + token = token, + revision = revision, + trust_remote_code = trust_remote_code, + ) + pass + + if not was_disabled: enable_progress_bars() + + # Check if this is local model since the tokenizer gets overwritten + if os.path.exists(os.path.join(old_model_name, "tokenizer_config.json")) and \ + os.path.exists(os.path.join(old_model_name, "tokenizer.json")) and \ + os.path.exists(os.path.join(old_model_name, "special_tokens_map.json")): + + tokenizer_name = old_model_name + else: + tokenizer_name = None + pass + + model, tokenizer = FastBaseVisionModel.from_pretrained( + model_name = model_name, + max_seq_length = max_seq_length, + dtype = _get_dtype(dtype), + load_in_4bit = load_in_4bit, + token = token, + device_map = device_map, + trust_remote_code = trust_remote_code, + revision = revision if not is_peft else None, + model_types = model_types, + tokenizer_name = tokenizer_name, + *args, **kwargs, + ) + + if resize_model_vocab is not None: + model.resize_token_embeddings(resize_model_vocab) + pass + + # In case the model supports tagging, add the unsloth tag. + if hasattr(model, "add_model_tags"): + model.add_model_tags(["unsloth",]) + pass + if hasattr(tokenizer, "add_model_tags"): + tokenizer.add_model_tags(["unsloth",]) + pass + + if load_in_4bit: + # Fix up bitsandbytes config + quantization_config = \ + { + # Sometimes torch_dtype is not a string!! + "bnb_4bit_compute_dtype" : model.config.to_dict()["torch_dtype"], + "bnb_4bit_quant_type" : "nf4", + "bnb_4bit_use_double_quant" : True, + "llm_int8_enable_fp32_cpu_offload" : False, + "llm_int8_has_fp16_weight" : False, + "llm_int8_skip_modules" : None, + "llm_int8_threshold" : 6.0, + "load_in_4bit" : True, + "load_in_8bit" : False, + "quant_method" : "bitsandbytes", + } + model.config.update({"quantization_config" : quantization_config}) + pass + + if is_peft: + # From https://github.com/huggingface/peft/issues/184 + # Now add PEFT adapters + model.enable_input_require_grads() + model = PeftModel.from_pretrained( + model, + old_model_name, + token = token, + revision = revision, + is_trainable = True, + trust_remote_code = trust_remote_code, + ) + # Patch it as well! + model = FastBaseVisionModel.patch_peft_model(model, use_gradient_checkpointing) + pass + return model, tokenizer + pass +pass diff --git a/unsloth/models/loader_utils.py b/unsloth/models/loader_utils.py new file mode 100644 index 0000000000..b778b7e95b --- /dev/null +++ b/unsloth/models/loader_utils.py @@ -0,0 +1,120 @@ +# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .mapper import INT_TO_FLOAT_MAPPER, FLOAT_TO_INT_MAPPER, MAP_TO_UNSLOTH_16bit +# https://github.com/huggingface/transformers/pull/26037 allows 4 bit loading! +from packaging.version import Version +from transformers import __version__ as transformers_version +transformers_version = Version(transformers_version) +SUPPORTS_FOURBIT = transformers_version >= Version("4.37") + + +def __get_model_name( + model_name, + load_in_4bit = True, + INT_TO_FLOAT_MAPPER = None, + FLOAT_TO_INT_MAPPER = None, + MAP_TO_UNSLOTH_16bit = None, +): + model_name = str(model_name) + lower_model_name = model_name.lower() + + if not SUPPORTS_FOURBIT and lower_model_name in INT_TO_FLOAT_MAPPER: + + model_name = INT_TO_FLOAT_MAPPER[lower_model_name] + print( + f"Unsloth: Your transformers version of {transformers_version} does not support native "\ + f"4bit loading.\nThe minimum required version is 4.37.\n"\ + f'Try `pip install --upgrade "transformers>=4.37"`\n'\ + f"to obtain the latest transformers build, then restart this session.\n"\ + f"For now, we shall load `{model_name}` instead (still 4bit, just slower downloading)." + ) + return model_name + + elif not load_in_4bit and lower_model_name in INT_TO_FLOAT_MAPPER: + + new_model_name = INT_TO_FLOAT_MAPPER[lower_model_name] + # logger.warning_once( + # f"Unsloth: You passed in `{model_name}` which is a 4bit model, yet you set\n"\ + # f"`load_in_4bit = False`. We shall load `{new_model_name}` instead." + # ) + return new_model_name + + elif not load_in_4bit and lower_model_name in MAP_TO_UNSLOTH_16bit: + + new_model_name = MAP_TO_UNSLOTH_16bit[lower_model_name] + return new_model_name + + elif load_in_4bit and SUPPORTS_FOURBIT and lower_model_name in FLOAT_TO_INT_MAPPER: + + new_model_name = FLOAT_TO_INT_MAPPER[lower_model_name] + # logger.warning_once( + # f"Unsloth: You passed in `{model_name}` and `load_in_4bit = True`.\n"\ + # f"We shall load `{new_model_name}` for 4x faster loading." + # ) + return new_model_name + pass + + return None +pass + + +def _get_new_mapper(): + try: + import requests + new_mapper = "https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/models/mapper.py" + with requests.get(new_mapper, timeout = 3) as new_mapper: new_mapper = new_mapper.text + new_mapper = new_mapper[new_mapper.find("__INT_TO_FLOAT_MAPPER"):] + new_mapper = new_mapper\ + .replace("INT_TO_FLOAT_MAPPER", "NEW_INT_TO_FLOAT_MAPPER")\ + .replace("FLOAT_TO_INT_MAPPER", "NEW_FLOAT_TO_INT_MAPPER")\ + .replace("MAP_TO_UNSLOTH_16bit", "NEW_MAP_TO_UNSLOTH_16bit") + + exec(new_mapper, globals()) + return NEW_INT_TO_FLOAT_MAPPER, NEW_FLOAT_TO_INT_MAPPER, NEW_MAP_TO_UNSLOTH_16bit + except: + return {}, {}, {} + pass +pass + + +def get_model_name(model_name, load_in_4bit = True): + new_model_name = __get_model_name( + model_name = model_name, + load_in_4bit = load_in_4bit, + INT_TO_FLOAT_MAPPER = INT_TO_FLOAT_MAPPER, + FLOAT_TO_INT_MAPPER = FLOAT_TO_INT_MAPPER, + MAP_TO_UNSLOTH_16bit = MAP_TO_UNSLOTH_16bit, + ) + if new_model_name is None and model_name.count("/") == 1 and model_name[0].isalnum(): + # Try checking if a new Unsloth version allows it! + NEW_INT_TO_FLOAT_MAPPER, NEW_FLOAT_TO_INT_MAPPER, NEW_MAP_TO_UNSLOTH_16bit = _get_new_mapper() + upgraded_model_name = __get_model_name( + model_name = model_name, + load_in_4bit = load_in_4bit, + INT_TO_FLOAT_MAPPER = NEW_INT_TO_FLOAT_MAPPER, + FLOAT_TO_INT_MAPPER = NEW_FLOAT_TO_INT_MAPPER, + MAP_TO_UNSLOTH_16bit = NEW_MAP_TO_UNSLOTH_16bit, + ) + if upgraded_model_name is not None: + raise NotImplementedError( + f"Unsloth: {model_name} is not supported in your current Unsloth version! Please update Unsloth via:\n\n"\ + 'pip uninstall unsloth unsloth_zoo -y\n'\ + 'pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"\n'\ + 'pip install --upgrade --no-cache-dir "git+https://github.com/unslothai/unsloth-zoo.git"\n'\ + ) + pass + pass + return new_model_name if new_model_name is not None else model_name +pass diff --git a/unsloth/models/mapper.py b/unsloth/models/mapper.py index d4f1278e1d..fc1dc8cdb0 100644 --- a/unsloth/models/mapper.py +++ b/unsloth/models/mapper.py @@ -409,12 +409,12 @@ __INT_TO_FLOAT_MAPPER = \ "Qwen/Qwen2.5-Coder-32B", ), "unsloth/Qwen2.5-Coder-0.5B-Instruct-bnb-4bit" : ( - "unsloth/Qwen2.5-Coder-Instruct-0.5B", - "Qwen/Qwen2.5-Coder-Instruct-0.5B", + "unsloth/Qwen2.5-Coder-0.5B-Instruct", + "Qwen/Qwen2.5-Coder-0.5B-Instruct", ), "unsloth/Qwen2.5-Coder-1.5B-Instruct-bnb-4bit" : ( - "unsloth/Qwen2.5-Coder-Instruct-1.5B", - "Qwen/Qwen2.5-Coder-Instruct-1.5B", + "unsloth/Qwen2.5-Coder-1.5B-Instruct", + "Qwen/Qwen2.5-Coder-1.5B-Instruct", ), "unsloth/Qwen2.5-Coder-3B-Instruct-bnb-4bit" : ( "unsloth/Qwen2.5-Coder-3B-Instruct", @@ -452,6 +452,46 @@ __INT_TO_FLOAT_MAPPER = \ "unsloth/Llama-3.1-Nemotron-70B-Instruct", "nvidia/Llama-3.1-Nemotron-70B-Instruct-HF", ), + "unsloth/Qwen2-VL-2B-Instruct-bnb-4bit" : ( + "unsloth/Qwen2-VL-2B-Instruct", + "Qwen/Qwen2-VL-2B-Instruct", + ), + "unsloth/Qwen2-VL-7B-Instruct-bnb-4bit" : ( + "unsloth/Qwen2-VL-7B-Instruct", + "Qwen/Qwen2-VL-7B-Instruct", + ), + "unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit" : ( + "unsloth/Llama-3.2-11B-Vision-Instruct", + "meta-llama/Llama-3.2-11B-Vision-Instruct", + ), + "unsloth/Llama-3.2-90B-Vision-Instruct-bnb-4bit" : ( + "unsloth/Llama-3.2-90B-Vision-Instruct", + "meta-llama/Llama-3.2-90B-Vision-Instruct", + ), + "unsloth/Llama-3.2-11B-Vision-bnb-4bit" : ( + "unsloth/Llama-3.2-11B-Vision", + "meta-llama/Llama-3.2-11B-Vision", + ), + "unsloth/Llama-3.2-90B-Vision-bnb-4bit" : ( + "unsloth/Llama-3.2-90B-Vision", + "meta-llama/Llama-3.2-90B-Vision", + ), + "unsloth/Pixtral-12B-2409-bnb-4bit" : ( + "unsloth/Pixtral-12B-2409", + "mistralai/Pixtral-12B-2409", + ), + "unsloth/Pixtral-12B-2409-Base-bnb-4bit" : ( + "unsloth/Pixtral-12B-Base-2409", + "mistralai/Pixtral-12B-Base-2409", + ), + "unsloth/llava-1.5-7b-hf-bnb-4bit" : ( + "unsloth/llava-1.5-7b-hf", + "llava-hf/llava-1.5-7b-hf", + ), + "unsloth/llava-v1.6-mistral-7b-hf-bnb-4bit" : ( + "unsloth/llava-v1.6-mistral-7b-hf", + "llava-hf/llava-v1.6-mistral-7b-hf", + ), } INT_TO_FLOAT_MAPPER = {} diff --git a/unsloth/models/vision.py b/unsloth/models/vision.py index 0b8c08a371..d083144651 100644 --- a/unsloth/models/vision.py +++ b/unsloth/models/vision.py @@ -1,58 +1,86 @@ +# Unsloth Zoo - Utilities for Unsloth # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved. # -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. # -# http://www.apache.org/licenses/LICENSE-2.0 +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. +# You should have received a copy of the GNU Lesser General Public License +# along with this program. If not, see . +import torch +from transformers import ( + BitsAndBytesConfig, + AutoModelForVision2Seq, + AutoProcessor, +) from .llama import * -from ..kernels import patch_layernorm, unpatch_layernorm -from ..kernels import patch_rms_layernorm, unpatch_rms_layernorm -from ..kernels import patch_llama_for_causal_lm, unpatch_llama_for_causal_lm -from ._utils import patch_gradient_checkpointing +from ..kernels import ( + post_patch_loss_function, +) +from ._utils import __version__ +from peft import LoraConfig, TaskType, get_peft_model +from transformers import set_seed as transformers_set_seed +from unsloth_zoo.peft_utils import ( + get_peft_regex, + merge_and_overwrite_lora, +) -from transformers import AutoProcessor -try: - from transformers import MllamaForConditionalGeneration -except: - raise ImportError( - "Unsloth: Please update your transformers version to 4.46.0 for Llama 3.2 support!" - ) +__all__ = [ + "FastBaseVisionModel", +] + +def _wrap_fast_inference(generate, device_type, dtype, model): + # Wraps inference with bfloat16 / float16 + @torch.inference_mode + def _fast_generate(*args, **kwargs): + # For num_logits_to_keep + kwargs["num_logits_to_keep"] = 1 + + # Remove token_type_ids + kwargs.pop("token_type_ids", None) + + # Check pad_token + model_eos_token_id = getattr(model.config, "eos_token_id", None) + if model_eos_token_id is not None and hasattr(model_eos_token_id, "__iter__"): + model_eos_token_id = model_eos_token_id[0] + + kwargs["pad_token_id"] = kwargs.pop("pad_token_id", model_eos_token_id) + + try: + kwargs["pixel_values"] = kwargs["pixel_values"].to(model.dtype) + except: + pass + + # Autocasted + with torch.autocast(device_type = device_type, dtype = dtype): + output = generate(*args, **kwargs) + pass + return output + pass + return _fast_generate pass -class FastVisionModel: - - def pre_patch(self): - patch_gradient_checkpointing() - patch_layernorm() - patch_rms_layernorm() - patch_llama_for_causal_lm() - pass - - def post_unpatch(self): - unpatch_layernorm() - unpatch_rms_layernorm() - unpatch_llama_for_causal_lm() - pass +class FastBaseVisionModel: @staticmethod def from_pretrained( - model_name = "llava-hf/llava-1.5-7b-hf", + model_name = "unsloth/llama-3-8b-bnb-4bit", max_seq_length = None, dtype = None, load_in_4bit = True, token = None, device_map = "sequential", - rope_scaling = None, trust_remote_code = False, + model_types = None, + tokenizer_name = None, **kwargs, ): if trust_remote_code: @@ -67,7 +95,7 @@ class FastVisionModel: max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3) statistics = \ - f"==((====))== Unsloth {__version__}: Fast {model_patcher.__name__[4:-5]} patching. Transformers = {transformers_version}.\n"\ + f"==((====))== Unsloth {__version__}: Fast {model_types[0].title()} vision patching. Transformers = {transformers_version}.\n"\ f" \\\ /| GPU: {gpu_stats.name}. Max memory: {max_memory} GB. Platform = {platform_system}.\n"\ f"O^O/ \_/ \\ Pytorch: {torch.__version__}. CUDA = {gpu_stats.major}.{gpu_stats.minor}. CUDA Toolkit = {torch.version.cuda}.\n"\ f"\ / Bfloat16 = {str(SUPPORTS_BFLOAT16).upper()}. FA [Xformers = {xformers_version}. FA2 = {HAS_FLASH_ATTENTION}]\n"\ @@ -81,6 +109,7 @@ class FastVisionModel: pass # Return old flag os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = old_hf_transfer + os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1" get_statistics() # For debugging - we use a download counter to see if environments are not breaking @@ -105,160 +134,36 @@ class FastVisionModel: ) pass + kwargs.pop("attn_implementation", None); # No need since we auto call it + # Cannot be None, since HF now checks for the config if load_in_4bit: kwargs["quantization_config"] = bnb_config - self.pre_patch() - model = MllamaForConditionalGeneration.from_pretrained( + model = AutoModelForVision2Seq.from_pretrained( model_name, device_map = device_map, torch_dtype = dtype, - # quantization_config = bnb_config, + # quantization_config = bnb_config, token = token, - max_position_embeddings = max_position_embeddings, trust_remote_code = trust_remote_code, - attn_implementation = "sdpa", + # attn_implementation = "sdpa", [TODO] Pixtral for eg fails **kwargs, ) - self.post_unpatch() - # Return old flag os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = old_hf_transfer # We currently only support NVIDIA GPUs - AMD / Intel is a work in progress! post_check = check_nvidia() # Counteract saved tokenizers + tokenizer_name = model_name if tokenizer_name is None else tokenizer_name tokenizer = AutoProcessor.from_pretrained( - model_name, - ) - model = FastVisionModel.post_patch(model) - - # Patch Trainer - from transformers.trainer import Trainer - try: - if Trainer._inner_training_loop.__name__ != "_fast_inner_training_loop": - inner_training_loop = inspect.getsource(Trainer._inner_training_loop) - Trainer._original_training_loop = inner_training_loop - else: - inner_training_loop = Trainer._original_training_loop - except: - raise RuntimeError('Unsloth currently does not support multi GPU setups - but we are working on it!') - pass - - if ((post_check - pre_check) >= 1).sum() > 1: - raise RuntimeError('Unsloth currently does not support multi GPU setups - but we are working on it!') - - import transformers.trainer - items_in_trainer = dir(transformers.trainer) - good_items = [] - for item in items_in_trainer: - # TODO: Support Deepspeed - if item.startswith(("deepspeed", "xm", "met", "smp")): continue - if item in inner_training_loop: good_items.append(item) - pass - exec("from transformers.trainer import (" + ", ".join(x for x in good_items) + ")", globals()) - - start = re.search('logger\.info\([\"\'].+?Running training', inner_training_loop).span(0)[0] - end = inner_training_loop.find("\n\n", start) - original_debug = inner_training_loop[start:end] - spaces = re.search('\n([\s\t]{1,})', original_debug).group(0)[1:] - front_spaces = re.match('([\s\t]{1,})', inner_training_loop).group(0) - - debug_info = """debug_info = \\ - f"==((====))== Unsloth - 2x faster free finetuning | Num GPUs = {args.world_size}\\n"\\ - f" \\\\\\ /| Num examples = {num_examples:,} | Num Epochs = {num_train_epochs:,}\\n"\\ - f"O^O/ \\_/ \\ Batch size per device = {self._train_batch_size:,} | Gradient Accumulation steps = {args.gradient_accumulation_steps}\\n"\\ - f"\\ / Total batch size = {total_train_batch_size:,} | Total steps = {max_steps:,}\\n"\\ - f' "-____-" Number of trainable parameters = {get_model_param_count(model, trainable_only=True):,}' - logger.warning(debug_info) - import subprocess, re, gc, numpy as np - a = np.array([0,]) - try: - a = subprocess.check_output('nvidia-smi --query-gpu=memory.used --format=csv', shell = True) - a = re.findall(rb'([\\d]{1,})[\\s]{1,}M', a) - a = np.array([int(x.decode('utf-8'))/1024 for x in a]) - except: - if not torch.cuda.is_available(): - raise RuntimeError('Unsloth: We do not support AMD / Intel machines yet - it is a work in progress!') - if ((a - PRE_CHECK) >= 1).sum() > 1: - raise RuntimeError('Unsloth currently does not support multi GPU setups - but we are working on it!') - for _ in range(3): - gc.collect() - torch.cuda.empty_cache()""" - - debug_info = debug_info.split('\n') - debug_info = "\n".join([debug_info[0]] + [spaces + x[8:] for x in debug_info[1:]]) - inner_training_loop = inner_training_loop.replace(original_debug, debug_info) - - debug_info = """n_total_devices = total_train_batch_size // \\ - args.gradient_accumulation_steps // self._train_batch_size - if n_total_devices > 1: - logger.warning_once('Unsloth currently does not support multi GPU setups - but we are working on it!') - debug_info =""" - debug_info = debug_info.split('\n') - debug_info = "\n".join([debug_info[0]] + [spaces + x[8:] for x in debug_info[1:]]) - inner_training_loop = inner_training_loop.replace("debug_info =", debug_info, 1) - - front_spaces = re.match(r"[\t\s]{1,}", inner_training_loop).group(0) - inner_training_loop = re.sub(r"^" + front_spaces, "", inner_training_loop, flags = re.MULTILINE) - inner_training_loop = inner_training_loop.replace( - "train_dataloader = tpu_spmd_dataloader(train_dataloader)", - "raise RuntimeError('Unsloth: TPUs are not yet supported!')" - ) - inner_training_loop = inner_training_loop.replace( - "self.accelerator.free_memory()", - "self.accelerator.free_memory()\n" + \ - front_spaces + "if self.is_deepspeed_enabled:"\ - "raise RuntimeError('Unsloth: Deepspeed is not yet supported!')\n", 1, + tokenizer_name, + padding_side = "right", + token = token, ) - check_batches = """train_dataloader = self.get_train_dataloader() - ga = args.gradient_accumulation_steps - bsz = self._train_batch_size - total_batches = bsz * ga * args.world_size - n_total_devices = total_batches // ga // bsz - if n_total_devices > 1: - logger.warning_once('Unsloth currently does not support multi GPU setups - but we are working on it!') - divisor = n_total_devices / 1 - bsz = self._train_batch_size = max(int(bsz / divisor), 1) - if total_batches // ga // bsz > 1: - divisor = n_total_devices / 1 - ga = args.gradient_accumulation_steps = max(int(ga / divisor), 1)""" - check_batches = check_batches.split('\n') - check_batches = "\n".join([check_batches[0]] + [front_spaces + x[8:] for x in check_batches[1:]]) - inner_training_loop = inner_training_loop.replace( - "train_dataloader = self.get_train_dataloader()", - check_batches, 1, - ) - inner_training_loop = inner_training_loop.replace( - "_inner_training_loop", - "_fast_inner_training_loop", 1, - ) - exec(inner_training_loop, globals()) - - Trainer._inner_training_loop = _fast_inner_training_loop - inner_training_loop = inner_training_loop.replace( - "is_torch_tpu_available()", - "False", - ) - if "n_total_devices >" not in inner_training_loop: - raise RuntimeError('Unsloth currently does not support multi GPU setups - but we are working on it!') - pass - inner_training_loop = inner_training_loop.replace( - "is_sagemaker_mp_enabled()", - "False", - ) - exec(inner_training_loop, globals()) - Trainer._inner_training_loop = _fast_inner_training_loop - - # Save max_seq_length - model.max_seq_length = max_position_embeddings - internal_model = model - while hasattr(internal_model, "model"): - internal_model.max_seq_length = max_position_embeddings - internal_model = internal_model.model - pass - internal_model.max_seq_length = max_position_embeddings + model, tokenizer = patch_tokenizer(model, tokenizer) + model = post_patch_loss_function(model) # Fix up config for transformers uploading PEFT # Not necessary anymore since we require transformers>=4.37! @@ -271,115 +176,76 @@ class FastVisionModel: pass # Log Unsloth version for future fastpaths for inference - model.config.update({"unsloth_version" : __version__}) + if hasattr(model, "config"): + model.config.update({"unsloth_version" : __version__}) + pass + patch_saving_functions(model, vision = True) + patch_saving_functions(tokenizer, vision = True) - # Add save modules - patch_saving_functions(model) - Trainer._inner_training_loop = _fast_inner_training_loop - - # Also fix torch_dtype + # Save tokenizer for inference purposes + tokenizer.padding_side = "left" # Force inference internal_model = model while hasattr(internal_model, "model"): - if hasattr(internal_model, "config"): - if internal_model.config.torch_dtype == "float32": - internal_model.config.torch_dtype = torch.float32 - elif internal_model.config.torch_dtype == "bfloat16": - internal_model.config.torch_dtype = torch.bfloat16 - elif internal_model.config.torch_dtype == "float16": - internal_model.config.torch_dtype = torch.float16 - pass - pass + internal_model._saved_temp_tokenizer = tokenizer internal_model = internal_model.model pass - if hasattr(internal_model, "config"): - if internal_model.config.torch_dtype == "float32": - internal_model.config.torch_dtype = torch.float32 - elif internal_model.config.torch_dtype == "bfloat16": - internal_model.config.torch_dtype = torch.bfloat16 - elif internal_model.config.torch_dtype == "float16": - internal_model.config.torch_dtype = torch.float16 - pass - pass + internal_model._saved_temp_tokenizer = tokenizer return model, tokenizer pass - @staticmethod - def post_patch(model): - # Patch model - layers = model.model.layers - lm_head = model.get_output_embeddings().weight - - # Also patch all dtypes - BnB seems to not allocate the correct type? - # BnB default dtype seems to be float16! - correct_dtype = lm_head.weight.dtype - - for name, module in model.named_modules(): - if isinstance(module, (Bnb_Linear4bit, Peft_Linear4bit)): - weight = module.weight - quant_state = weight.quant_state - - if type(quant_state) is list: - # BnB seems to have float16 as default! - module.weight.quant_state[2] = correct_dtype # Cast to correct dtype - else: - # https://github.com/TimDettmers/bitsandbytes/pull/763/files - quant_state.dtype = correct_dtype - pass - pass - pass - - # Clear deleted GPU items - for _ in range(3): - gc.collect() - torch.cuda.empty_cache() - return model - pass - - @staticmethod def get_peft_model( model, - r = 16, - target_modules = "all-linear", - lora_alpha = 16, - lora_dropout = 0, - bias = "none", - layers_to_transform = None, - layers_pattern = None, + r = 16, + target_modules = None, + lora_alpha = 16, + lora_dropout = 0, + bias = "none", + finetune_vision_layers = True, + finetune_language_layers = True, + finetune_attention_modules = True, + finetune_mlp_modules = True, + layers_to_transform = None, + layers_pattern = None, use_gradient_checkpointing = True, - random_state = 3407, - max_seq_length = 2048, # not used anymore - use_rslora = False, - modules_to_save = None, - init_lora_weights = True, - loftq_config = {}, - temporary_location = "_unsloth_temporary_saved_buffers", + random_state = 3407, + max_seq_length = 2048, # not used anymore + use_rslora = False, + modules_to_save = None, + init_lora_weights = True, + loftq_config = {}, + temporary_location = "_unsloth_temporary_saved_buffers", **kwargs, ): transformers_set_seed(random_state) - # Get LoRA - arguments = dict( - r = r, - lora_alpha = lora_alpha, - target_modules = target_modules, - lora_dropout = lora_dropout, - bias = bias, - layers_to_transform = layers_to_transform, - init_lora_weights = init_lora_weights, - # loftq_config = loftq_config, - # use_rslora = use_rslora, - modules_to_save = modules_to_save, - **kwargs, - ) + if type(r) is not int: + raise TypeError(f"Unsloth: Rank of {str(r)} must be an integer.") + if r <= 0: + raise TypeError(f"Unsloth: Rank of {str(r)} must be larger than 0.") - lora_config = LoraConfig(**arguments) + if isinstance(model, PeftModelForCausalLM): + raise RuntimeError("Unsloth: You already added LoRA adapters to your model!") - model = _get_peft_model(model, lora_config) - - model = FastVisionModel.patch_peft_model(model, use_gradient_checkpointing) + if target_modules == "all-linear": + finetune_vision_layers = True + finetune_language_layers = True + finetune_attention_modules = True + finetune_mlp_modules = True + pass + if target_modules is None: + target_modules = get_peft_regex( + model, + finetune_vision_layers = finetune_vision_layers, + finetune_language_layers = finetune_language_layers, + finetune_attention_modules = finetune_attention_modules, + finetune_mlp_modules = finetune_mlp_modules, + ) + else: + assert(type(target_modules) in (list, tuple,)) + pass # Clear deleted GPU items for _ in range(3): @@ -387,6 +253,29 @@ class FastVisionModel: torch.cuda.empty_cache() pass + lora_config = LoraConfig( + r = r, + lora_alpha = lora_alpha, + target_modules = target_modules, + lora_dropout = lora_dropout, + bias = bias, + task_type = TaskType.CAUSAL_LM, + ) + model = prepare_model_for_kbit_training( + model, + use_gradient_checkpointing = use_gradient_checkpointing, + ) + model = get_peft_model(model, lora_config) + + model = FastBaseVisionModel.patch_peft_model(model, use_gradient_checkpointing) + + # Clear deleted GPU items + for _ in range(3): + gc.collect() + torch.cuda.empty_cache() + pass + patch_saving_functions(model, vision = True) + return model pass @@ -396,6 +285,11 @@ class FastVisionModel: model, use_gradient_checkpointing = True, ): + if not isinstance(model, PeftModelForCausalLM): + raise TypeError( + "Unsloth: Your model needs to call `.get_peft_model` first!" + ) + pass model = prepare_model_for_kbit_training( model, @@ -403,20 +297,6 @@ class FastVisionModel: use_reentrant = True, ) - # Fix up config for transformers uploading PEFT - for active_adapter in model.peft_config.keys(): - # Not necessary since we requires transformers >= 4.37 - if False: - name = model.peft_config[active_adapter].base_model_name_or_path - if name.startswith("unsloth/") and name.endswith("-bnb-4bit"): - name = name[:len(name) - len("-bnb-4bit")] - model.peft_config[active_adapter].base_model_name_or_path = name - pass - # Add revision to enable future fast inference paths - # [TODO] Bugs out!see https://github.com/unslothai/unsloth/issues/492 - # model.peft_config[active_adapter].revision = f"unsloth" - pass - from transformers.trainer import Trainer if Trainer._inner_training_loop.__name__ != "_fast_inner_training_loop": raise RuntimeError( @@ -426,24 +306,7 @@ class FastVisionModel: 'Thank you for your understanding and we appreciate it immensely!' ) pass - - logger.warning_once( - f"Unsloth {__version__} patched {len(model.model.model.layers)} layers with "\ - f"{n_qkv} QKV layers, {n_o} O layers and {n_mlp} MLP layers.", - ) - patch_saving_functions(model) - - # Patch cross entropy loss labels - # Fixes https://github.com/unslothai/unsloth/issues/10 - max_seq_length = model.max_seq_length - extra_ignored_labels = torch.full((max_seq_length, 1), -100, device = "cuda:0") - model.model.extra_ignored_labels = extra_ignored_labels - internal_model = model - while hasattr(internal_model, "model"): - internal_model.max_seq_length = max_seq_length - internal_model = internal_model.model - pass - internal_model.max_seq_length = max_seq_length + patch_saving_functions(model, vision = True) # Patch tokenizer to pad to the right internal_model = model @@ -468,37 +331,40 @@ class FastVisionModel: @staticmethod def for_inference(model): - # if model.config.model_type == "qwen2": - # FastLlamaModel.for_training(model) - # return - # pass + model.gradient_checkpointing = False + model.training = False - internal_model = model - internal_model.gradient_checkpointing = False - internal_model.training = False - - while hasattr(internal_model, "model"): - internal_model = internal_model.model - internal_model.gradient_checkpointing = False - internal_model.training = False - pass - if hasattr(internal_model, "training"): - internal_model.training = False + for name, module in model.named_modules(): + if hasattr(module, "gradient_checkpointing"): + module.gradient_checkpointing = False + if hasattr(module, "training"): + module.training = False pass - # Also check if lm_head / embeddings are trained - internal_model = model - while not hasattr(internal_model, "lm_head"): - internal_model = internal_model.model - pass - lm_head = internal_model.lm_head.weight - device_type = lm_head.device.type dtype = model.config.torch_dtype - if type(dtype) is str: if dtype == "float16": dtype = torch.float16 elif dtype == "bfloat16": dtype = torch.bfloat16 pass + device_type = model.device.type + + # Wrap model.generate + if model.generate.__name__ != "_fast_generate": + model._unwrapped_old_generate = model.generate + model.generate = _wrap_fast_inference(model.generate, device_type, dtype, model) + pass + + # Patch tokenizer to pad to the left + internal_model = model + while hasattr(internal_model, "model"): + if hasattr(internal_model, "_saved_temp_tokenizer"): + internal_model._saved_temp_tokenizer.padding_side = "left" + pass + internal_model = internal_model.model + pass + if hasattr(internal_model, "_saved_temp_tokenizer"): + internal_model._saved_temp_tokenizer.padding_side = "left" + pass # Also disable training for embeddings for NEFTune if hasattr(model, "get_input_embeddings"): @@ -516,23 +382,32 @@ class FastVisionModel: @staticmethod def for_training(model, use_gradient_checkpointing = True): + model.gradient_checkpointing = use_gradient_checkpointing + model.training = True + + for name, module in model.named_modules(): + if hasattr(module, "gradient_checkpointing"): + module.gradient_checkpointing = use_gradient_checkpointing + if hasattr(module, "training"): + module.training = True + pass + + # Also revert model.generate + if hasattr(model, "_unwrapped_old_generate"): + model.generate = model._unwrapped_old_generate + del model._unwrapped_old_generate + pass + + # Patch tokenizer to pad to the right internal_model = model - internal_model.gradient_checkpointing = use_gradient_checkpointing - internal_model.training = True - - # Delete all fast inference loras - for param in model.parameters(): - if hasattr(param, "_fast_lora"): - del param._fast_lora - pass - while hasattr(internal_model, "model"): + if hasattr(internal_model, "_saved_temp_tokenizer"): + internal_model._saved_temp_tokenizer.padding_side = "right" + pass internal_model = internal_model.model - internal_model.gradient_checkpointing = use_gradient_checkpointing - internal_model.training = True pass - if hasattr(internal_model, "training"): - internal_model.training = True + if hasattr(internal_model, "_saved_temp_tokenizer"): + internal_model._saved_temp_tokenizer.padding_side = "right" pass # Also re-enable training for embeddings for NEFTune @@ -548,3 +423,5 @@ class FastVisionModel: return model pass pass + + diff --git a/unsloth/save.py b/unsloth/save.py index b4c6b499cf..b503b2b47a 100644 --- a/unsloth/save.py +++ b/unsloth/save.py @@ -2041,8 +2041,153 @@ def unsloth_convert_lora_to_ggml_and_save_locally( print("Unsloth: Done.") print(f"Unsloth: Conversion completed! Output file: {output_file}") print("\nThis GGML making function was made by Maheswar. Ping him @Maheswar on the Unsloth Discord or on HuggingFace (@mahiatlinux) if you like this!") +pass -def patch_saving_functions(model): + +from unsloth_zoo.peft_utils import merge_and_overwrite_lora +from .models.loader_utils import get_model_name + +@torch.inference_mode +def unsloth_generic_save( + model, + tokenizer, + save_directory : Union[str, os.PathLike] = "unsloth_finetuned_merge", + save_method : str = "lora", # ["lora", "merged_16bit", "merged_4bit"] + push_to_hub : bool = False, + token : Optional[Union[str, bool]] = None, + is_main_process : bool = True, + state_dict : Optional[dict] = None, + save_function : Callable = torch.save, + max_shard_size : Union[int, str] = "5GB", + safe_serialization : bool = True, + variant : Optional[str] = None, + save_peft_format : bool = True, + + # Push to hub + use_temp_dir : Optional[bool] = None, + commit_message : Optional[str] = "Trained with Unsloth", + private : Optional[bool] = None, + create_pr : bool = False, + revision : str = None, + commit_description : str = "Upload model trained with Unsloth 2x faster", + tags : List[str] = None, + + # Our functions + temporary_location : str = "_unsloth_temporary_saved_buffers", + maximum_memory_usage : float = 0.9, +): + if token is None and push_to_hub: token = get_token() + + merge_and_overwrite_lora( + get_model_name, + create_huggingface_repo, + model, + save_location = save_directory, + push_to_hub = push_to_hub, + token = token, + upload_location = save_directory if push_to_hub else None, + low_disk_space_usage = True, + private = private, + ) + return +pass + + +def unsloth_generic_save_pretrained_merged( + self, + save_directory : Union[str, os.PathLike], + tokenizer = None, + save_method : str = "merged_16bit", # ["lora", "merged_16bit", "merged_4bit"] + push_to_hub : bool = False, + token : Optional[Union[str, bool]] = None, + is_main_process : bool = True, + state_dict : Optional[dict] = None, + save_function : Callable = torch.save, + max_shard_size : Union[int, str] = "5GB", + safe_serialization : bool = True, + variant : Optional[str] = None, + save_peft_format : bool = True, + tags : List[str] = None, + temporary_location : str = "_unsloth_temporary_saved_buffers", + maximum_memory_usage : float = 0.75, +): + """ + Same as .push_to_hub(...) except 4bit weights are auto + converted to float16 with as few overhead as possible. + + Choose for `save_method` to be either: + 1. `16bit`: Merge LoRA into float16 weights. Useful for GGUF / llama.cpp. + 2. `4bit`: Merge LoRA into int4 weights. Useful for DPO / HF inference. + 3. `lora`: Save LoRA adapters with no merging. Useful for HF inference. + """ + if tokenizer is None: + logger.warning_once( + "Unsloth: You're not saving a tokenizer as well?\n"\ + "You can do it separately via `tokenizer.save_pretrained(...)`" + ) + pass + + arguments = dict(locals()) + arguments["model"] = self + del arguments["self"] + unsloth_generic_save(**arguments) + for _ in range(3): + gc.collect() +pass + + +def unsloth_generic_push_to_hub_merged( + self, + repo_id : str, + tokenizer = None, + save_method : str = "merged_16bit", # ["lora", "merged_16bit", "merged_4bit"] + use_temp_dir : Optional[bool] = None, + commit_message : Optional[str] = "Trained with Unsloth", + private : Optional[bool] = None, + token : Union[bool, str, None] = None, + max_shard_size : Union[int, str, None] = "5GB", + create_pr : bool = False, + safe_serialization : bool = True, + revision : str = None, + commit_description : str = "Upload model trained with Unsloth 2x faster", + tags : Optional[List[str]] = None, + temporary_location : str = "_unsloth_temporary_saved_buffers", + maximum_memory_usage : float = 0.75, +): + """ + Same as .push_to_hub(...) except 4bit weights are auto + converted to float16 with as few overhead as possible. + + Choose for `save_method` to be either: + 1. `16bit`: Merge LoRA into float16 weights. Useful for GGUF / llama.cpp. + 2. `4bit`: Merge LoRA into int4 weights. Useful for DPO / HF inference. + 3. `lora`: Save LoRA adapters with no merging. Useful for HF inference. + """ + if tokenizer is None: + logger.warning_once( + "Unsloth: You're not saving a tokenizer as well?\n"\ + "You can do it separately via `tokenizer.push_to_hub(...)`" + ) + pass + + arguments = dict(locals()) + arguments["model"] = self + arguments["save_directory"] = repo_id + arguments["push_to_hub"] = True + del arguments["self"] + del arguments["repo_id"] + unsloth_generic_save(**arguments) + for _ in range(3): + gc.collect() +pass + + +def not_implemented_save(*args, **kwargs): + raise NotImplementedError("Unsloth: Sorry GGUF is currently not supported for vision models!") +pass + + +def patch_saving_functions(model, vision = False): import inspect import types from typing import Callable, Optional, Union, List @@ -2131,14 +2276,22 @@ def patch_saving_functions(model): pass # Add saving methods to top level model - if hasattr(model, "config"): - # Counteract tokenizers - model.push_to_hub_merged = types.MethodType(unsloth_push_to_hub_merged, model) - model.save_pretrained_merged = types.MethodType(unsloth_save_pretrained_merged, model) - model.push_to_hub_gguf = types.MethodType(unsloth_push_to_hub_gguf, model) - model.save_pretrained_gguf = types.MethodType(unsloth_save_pretrained_gguf, model) - model.push_to_hub_ggml = types.MethodType(unsloth_convert_lora_to_ggml_and_push_to_hub, model) - model.save_pretrained_ggml = types.MethodType(unsloth_convert_lora_to_ggml_and_save_locally, model) + if not vision: + if hasattr(model, "config"): + # Counteract tokenizers + model.push_to_hub_merged = types.MethodType(unsloth_push_to_hub_merged, model) + model.save_pretrained_merged = types.MethodType(unsloth_save_pretrained_merged, model) + model.push_to_hub_gguf = types.MethodType(unsloth_push_to_hub_gguf, model) + model.save_pretrained_gguf = types.MethodType(unsloth_save_pretrained_gguf, model) + model.push_to_hub_ggml = types.MethodType(unsloth_convert_lora_to_ggml_and_push_to_hub, model) + model.save_pretrained_ggml = types.MethodType(unsloth_convert_lora_to_ggml_and_save_locally, model) + pass + else: + # Vision only 1 option + model.push_to_hub_merged = types.MethodType(unsloth_generic_push_to_hub_merged, model) + model.save_pretrained_merged = types.MethodType(unsloth_generic_save_pretrained_merged, model) + model.push_to_hub_gguf = types.MethodType(not_implemented_save, model) + model.save_pretrained_gguf = types.MethodType(not_implemented_save, model) pass return model pass diff --git a/unsloth/trainer.py b/unsloth/trainer.py index 00956ed41b..012be4b0cb 100644 --- a/unsloth/trainer.py +++ b/unsloth/trainer.py @@ -20,13 +20,13 @@ from functools import wraps import trl import inspect from trl import SFTTrainer -try: - from trl import SFTConfig as TrainingArguments -except: - from transformers import TrainingArguments -pass from . import is_bfloat16_supported -from unsloth_zoo.training_utils import unsloth_train as _unsloth_train +from unsloth_zoo.training_utils import ( + unsloth_train as _unsloth_train, +) +from unsloth_zoo.vision_utils import ( + UnslothVisionDataCollator, +) from packaging.version import Version import dataclasses @@ -35,6 +35,7 @@ __all__ = [ "UnslothTrainer", "unsloth_train", "_patch_trl_trainer", + "UnslothVisionDataCollator", ] # Unsloth gradient accumulation fix: @@ -60,7 +61,11 @@ else: pass pass - +try: + from trl import SFTConfig as TrainingArguments +except: + from transformers import TrainingArguments +pass @dataclass class UnslothTrainingArguments(TrainingArguments): embedding_learning_rate : Optional[float] = field( @@ -134,7 +139,7 @@ pass # From `trl>=0.13.0`, they changed how to pass several params to the trainer # We need to patch to make the transition smooth -def create_backwards_compatible_trainer(trainer_class, config_class): +def _backwards_compatible_trainer(trainer_class, config_class): original_init = trainer_class.__init__ @wraps(original_init) @@ -167,6 +172,7 @@ def create_backwards_compatible_trainer(trainer_class, config_class): } # Get parameters that exist in Config but not in TrainingArguments + from transformers import TrainingArguments moved_params = \ set(inspect.signature(config_class) .parameters.keys()) - \ set(inspect.signature(TrainingArguments).parameters.keys()) @@ -207,14 +213,13 @@ def _patch_trl_trainer(): import trl.trainer trl_classes = dir(trl.trainer) - - non_convertable_trainer = set(["PPOv2", "AlignProp"]) - trl_trainers = set(x[:-len("Trainer")] for x in trl_classes if x.endswith("Trainer")) - non_convertable_trainer - trl_configs = set(x[:-len("Config")] for x in trl_classes if x.endswith("Config")) - non_convertable_trainer + trl_trainers = set(x[:-len("Trainer")] for x in trl_classes if x.endswith("Trainer")) + trl_configs = set(x[:-len("Config")] for x in trl_classes if x.endswith("Config")) trl_classes = list(trl_trainers & trl_configs) for x in trl_classes: - exec(f"trl.{x}Trainer.__init__ = create_backwards_compatible_trainer(trl.{x}Trainer, trl.{x}Config)", globals()) + try: exec(f"trl.{x}Trainer.__init__ = _backwards_compatible_trainer(trl.{x}Trainer, trl.{x}Config)", globals()) + except: continue pass trl.__UNSLOTH_BACKWARDS_COMPATIBLE__ = True From 967f9fb23d8b8e612989d5b8be9b113fa66af7f5 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Thu, 21 Nov 2024 06:07:06 -0800 Subject: [PATCH 2/6] Update vision.py --- unsloth/models/vision.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/unsloth/models/vision.py b/unsloth/models/vision.py index d083144651..69fb3fd986 100644 --- a/unsloth/models/vision.py +++ b/unsloth/models/vision.py @@ -41,7 +41,7 @@ def _wrap_fast_inference(generate, device_type, dtype, model): @torch.inference_mode def _fast_generate(*args, **kwargs): # For num_logits_to_keep - kwargs["num_logits_to_keep"] = 1 + # kwargs["num_logits_to_keep"] = 1 # Remove token_type_ids kwargs.pop("token_type_ids", None) From 7296f5eed7888996b0381648aa5dae5dcd2e05d9 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Thu, 21 Nov 2024 06:45:40 -0800 Subject: [PATCH 3/6] Update _utils.py --- unsloth/models/_utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.py index ee85ba3c36..69f36f0d46 100644 --- a/unsloth/models/_utils.py +++ b/unsloth/models/_utils.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "2024.11.8" +__version__ = "2024.11.9" __all__ = [ "prepare_model_for_kbit_training", From 6d34ab821b10d0b867d7caff83127716eb90764b Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Thu, 21 Nov 2024 11:24:12 -0800 Subject: [PATCH 4/6] Vision (#1318) * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Update README.md * Update README.md * Update README.md * Update README.md --------- Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com> --- README.md | 36 +++++++++++-------------- docs github button.png | Bin 0 -> 11805 bytes images/documentation github button.png | Bin 0 -> 11807 bytes images/documentation lighter.png | Bin 0 -> 11777 bytes 4 files changed, 15 insertions(+), 21 deletions(-) create mode 100644 docs github button.png create mode 100644 images/documentation github button.png create mode 100644 images/documentation lighter.png diff --git a/README.md b/README.md index 4d68d996f0..07d4e0abb6 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,9 @@ - + -### Finetune Llama 3.2, Mistral, Phi-3.5 & Gemma 2-5x faster with 80% less memory! +### Finetune Llama 3.2, Mistral, Phi-3.5, Qwen 2.5 & Gemma 2-5x faster with 80% less memory! ![](https://i.ibb.co/sJ7RhGG/image-41.png) @@ -23,43 +23,37 @@ All notebooks are **beginner friendly**! Add your dataset, click "Run All", and | Unsloth supports | Free Notebooks | Performance | Memory use | |-----------|---------|--------|----------| | **Llama 3.2 (3B)** | [▶️ Start for free](https://colab.research.google.com/drive/1T5-zKWM_5OD21QHwXHiV9ixTRR7k3iB9?usp=sharing) | 2x faster | 60% less | +| **Llama 3.2 Vision (11B)** | [▶️ Start for free](https://colab.research.google.com/drive/1j0N4XTY1zXXy7mPAhOC1_gMYZ2F2EBlk?usp=sharing) | 2x faster | 40% less | | **Llama 3.1 (8B)** | [▶️ Start for free](https://colab.research.google.com/drive/1Ys44kVvmeZtnICzWz0xgpRnrIOjZAuxp?usp=sharing) | 2x faster | 60% less | | **Phi-3.5 (mini)** | [▶️ Start for free](https://colab.research.google.com/drive/1lN6hPQveB_mHSnTOYifygFcrO8C1bxq4?usp=sharing) | 2x faster | 50% less | | **Gemma 2 (9B)** | [▶️ Start for free](https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing) | 2x faster | 63% less | -| **Mistral Small (22B)** | [▶️ Start for free](https://colab.research.google.com/drive/1oCEHcED15DzL8xXGU1VTx5ZfOJM8WY01?usp=sharing) | 2x faster | 60% less | -| **Ollama** | [▶️ Start for free](https://colab.research.google.com/drive/1WZDi7APtQ9VsvOrQSSC5DDtxq159j8iZ?usp=sharing) | 1.9x faster | 43% less | +| **Qwen 2.5 (7B)** | [▶️ Start for free](https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing) | 2x faster | 63% less | | **Mistral v0.3 (7B)** | [▶️ Start for free](https://colab.research.google.com/drive/1_yNCks4BTD5zOnjozppphh5GzMFaMKq_?usp=sharing) | 2.2x faster | 73% less | +| **Ollama** | [▶️ Start for free](https://colab.research.google.com/drive/1WZDi7APtQ9VsvOrQSSC5DDtxq159j8iZ?usp=sharing) | 1.9x faster | 43% less | | **ORPO** | [▶️ Start for free](https://colab.research.google.com/drive/11t4njE3c4Lxl-07OD8lJSMKkfyJml3Tn?usp=sharing) | 1.9x faster | 43% less | | **DPO Zephyr** | [▶️ Start for free](https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing) | 1.9x faster | 43% less | -- **Kaggle Notebooks** for [Llama 3.1 (8B)](https://www.kaggle.com/danielhanchen/kaggle-llama-3-1-8b-unsloth-notebook), [Gemma 2 (9B)](https://www.kaggle.com/code/danielhanchen/kaggle-gemma-7b-unsloth-notebook/), [Mistral (7B)](https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook) -- Run [Llama 3.2 1B 3B notebook](https://colab.research.google.com/drive/1hoHFpf7ROqk_oZHzxQdfPW9yvTxnvItq?usp=sharing) and [Llama 3.2 conversational notebook](https://colab.research.google.com/drive/1T5-zKWM_5OD21QHwXHiV9ixTRR7k3iB9?usp=sharing) -- Run [Llama 3.1 conversational notebook](https://colab.research.google.com/drive/15OyFkGoCImV9dSsewU1wa2JuKB4-mDE_?usp=sharing) and [Mistral v0.3 ChatML](https://colab.research.google.com/drive/15F1xyn8497_dUbxZP4zWmPZ3PJx1Oymv?usp=sharing) +- See [all our notebooks](https://docs.unsloth.ai/get-started/unsloth-notebooks) and [all our models](https://docs.unsloth.ai/get-started/all-our-models) +- **Kaggle Notebooks** for [Llama 3.2 Kaggle notebook](https://www.kaggle.com/danielhanchen/kaggle-llama-3-2-1b-3b-unsloth-notebook), [Llama 3.1 (8B)](https://www.kaggle.com/danielhanchen/kaggle-llama-3-1-8b-unsloth-notebook), [Gemma 2 (9B)](https://www.kaggle.com/code/danielhanchen/kaggle-gemma-7b-unsloth-notebook/), [Mistral (7B)](https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook) +- Run notebooks for [Llama 3.2 conversational](https://colab.research.google.com/drive/1T5-zKWM_5OD21QHwXHiV9ixTRR7k3iB9?usp=sharing), [Llama 3.1 conversational](https://colab.research.google.com/drive/15OyFkGoCImV9dSsewU1wa2JuKB4-mDE_?usp=sharing) and [Mistral v0.3 ChatML](https://colab.research.google.com/drive/15F1xyn8497_dUbxZP4zWmPZ3PJx1Oymv?usp=sharing) - This [text completion notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing) is for continued pretraining / raw text - This [continued pretraining notebook](https://colab.research.google.com/drive/1tEd1FrOXWMnCU9UIvdYhs61tkxdMuKZu?usp=sharing) is for learning another language -- Click [here](https://github.com/unslothai/unsloth/wiki) for detailed documentation for Unsloth. +- Click [here](https://docs.unsloth.ai/) for detailed documentation for Unsloth. ## 🦥 Unsloth.ai News +- 📣 NEW! [Vision models](https://unsloth.ai/blog/vision) now supported! [Llama 3.2 Vision (11B)](https://colab.research.google.com/drive/1j0N4XTY1zXXy7mPAhOC1_gMYZ2F2EBlk?usp=sharing), [Qwen 2.5 VL (7B)](https://colab.research.google.com/drive/1whHb54GNZMrNxIsi2wm2EY_-Pvo2QyKh?usp=sharing) and [Pixtral (12B) 2409](https://colab.research.google.com/drive/1K9ZrdwvZRE96qGkCq_e88FgV3MLnymQq?usp=sharing) +- 📣 NEW! Qwen-2.5 including [Coder](https://colab.research.google.com/drive/18sN803sU23XuJV9Q8On2xgqHSer6-UZF?usp=sharing) models are now supported with bugfixes. 14b fits in a Colab GPU! [Qwen 2.5 conversational notebook] - 📣 NEW! We found and helped fix a [gradient accumulation bug](https://unsloth.ai/blog/gradient)! Please update Unsloth and transformers. -- 📣 NEW! [Llama 3.2 Conversational notebook](https://colab.research.google.com/drive/1T5-zKWM_5OD21QHwXHiV9ixTRR7k3iB9?usp=sharing) includes training only on completions / outputs (increase accuracy), ShareGPT standardization and more! -- 📣 NEW! [Llama 3.2 Kaggle notebook](https://www.kaggle.com/danielhanchen/kaggle-llama-3-2-1b-3b-unsloth-notebook) and [Llama 3.2 Kaggle conversational notebook](https://www.kaggle.com/code/danielhanchen/kaggle-llama-3-2-1b-3b-conversational-unsloth/notebook) -- 📣 NEW! [Qwen 2.5 7b notebook](https://colab.research.google.com/drive/1Kose-ucXO1IBaZq5BvbwWieuubP7hxvQ?usp=sharing) finetuning is supported! Qwen 2.5 comes in multiple sizes - check our [4bit uploads](https://huggingface.co/unsloth) for 4x faster downloads!. 14b fits in a Colab GPU! [Qwen 2.5 conversational notebook](https://colab.research.google.com/drive/1qN1CEalC70EO1wGKhNxs1go1W9So61R5?usp=sharing) - 📣 NEW! [Mistral Small 22b notebook](https://colab.research.google.com/drive/1oCEHcED15DzL8xXGU1VTx5ZfOJM8WY01?usp=sharing) finetuning fits in under 16GB of VRAM! -- 📣 NEW! [Phi-3.5 (mini)](https://colab.research.google.com/drive/1lN6hPQveB_mHSnTOYifygFcrO8C1bxq4?usp=sharing) now supported -- 📣 NEW! [Gemma-2-2b](https://colab.research.google.com/drive/1weTpKOjBZxZJ5PQ-Ql8i6ptAY2x-FWVA?usp=sharing) now supported! Try out [Chat interface](https://colab.research.google.com/drive/1i-8ESvtLRGNkkUQQr_-z_rcSAIo9c3lM?usp=sharing)! -- 📣 NEW! [Llama 3.1 8b, 70b](https://colab.research.google.com/drive/1Ys44kVvmeZtnICzWz0xgpRnrIOjZAuxp?usp=sharing) & [Mistral Nemo-12b](https://colab.research.google.com/drive/17d3U-CAIwzmbDRqbZ9NnpHxCkmXB6LZ0?usp=sharing) both Base and Instruct are now supported
Click for more news +- 📣 Try out [Chat interface](https://colab.research.google.com/drive/1i-8ESvtLRGNkkUQQr_-z_rcSAIo9c3lM?usp=sharing)! +- 📣 NEW! [Llama 3.1 8b, 70b](https://colab.research.google.com/drive/1Ys44kVvmeZtnICzWz0xgpRnrIOjZAuxp?usp=sharing) & [Mistral Nemo-12b](https://colab.research.google.com/drive/17d3U-CAIwzmbDRqbZ9NnpHxCkmXB6LZ0?usp=sharing) both Base and Instruct are now supported - 📣 NEW! `pip install unsloth` now works! Head over to [pypi](https://pypi.org/project/unsloth/) to check it out! This allows non git pull installs. Use `pip install unsloth[colab-new]` for non dependency installs. -- 📣 NEW! [Gemma-2-9b](https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing) and Gemma-2-27b now supported -- 📣 UPDATE! [Phi-3 mini](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) model updated. [Phi-3 Medium](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) 2x faster finetuning. - 📣 NEW! Continued Pretraining [notebook](https://colab.research.google.com/drive/1tEd1FrOXWMnCU9UIvdYhs61tkxdMuKZu?usp=sharing) for other languages like Korean! -- 📣 NEW! Qwen2 now works -- 📣 [Mistral v0.3 Base](https://colab.research.google.com/drive/1_yNCks4BTD5zOnjozppphh5GzMFaMKq_?usp=sharing) and [Mistral v0.3 Instruct] -- 📣 [ORPO support](https://colab.research.google.com/drive/11t4njE3c4Lxl-07OD8lJSMKkfyJml3Tn?usp=sharing) is here + [2x faster inference](https://colab.research.google.com/drive/1aqlNQi7MMJbynFDyOQteD2t0yVfjb9Zh?usp=sharing) added for all our models +- 📣 [2x faster inference](https://colab.research.google.com/drive/1aqlNQi7MMJbynFDyOQteD2t0yVfjb9Zh?usp=sharing) added for all our models - 📣 We cut memory usage by a [further 30%](https://unsloth.ai/blog/long-context) and now support [4x longer context windows](https://unsloth.ai/blog/long-context)! --
## 🔗 Links and Resources @@ -69,7 +63,7 @@ All notebooks are **beginner friendly**! Add your dataset, click "Run All", and |   **Twitter (aka X)** | [Follow us on X](https://twitter.com/unslothai)| | 💾 **Installation** | [unsloth/README.md](https://github.com/unslothai/unsloth/tree/main#-installation-instructions)| | 🥇 **Benchmarking** | [Performance Tables](https://github.com/unslothai/unsloth/tree/main#-performance-benchmarking) -| 🌐 **Released Models** | [Unsloth Releases](https://huggingface.co/unsloth)| +| 🌐 **Released Models** | [Unsloth Releases](https://docs.unsloth.ai/get-started/all-our-models)| | ✍️ **Blog** | [Read our Blogs](https://unsloth.ai/blog)| ## ⭐ Key Features diff --git a/docs github button.png b/docs github button.png new file mode 100644 index 0000000000000000000000000000000000000000..6b129e0db728a4ffac63419eb054653930dee372 GIT binary patch literal 11805 zcmXw9bySp17iURn1rd>!5|9w28$?n`=@3cj?pRPjx=T`NN$C!eMnJj*1c4>BzmP4@r2I>|A6_7MMsX#;oKINhI&W zA1vWKzN6_7iAmzHhx3`t!H6NI&IsR9os0F2xPWZN3C_a>U zPd|^aAo_8do#y~gS>M?CQo6G~bbh!oiBNPIkJ>2xVkdSvw!h`tqau_2(#UTyE3q5K z7A2o0wEK2ZjV(&VUkOzou7u)6+M+HO8{dB3mgye_uk~+5v8xuoLbt}KL9we6_MQBD zBK2G0)mQ_gKD?rWR2laG(-Rh4ElhyXz@U{xFBdv7?>+aS#_!(HEM^l<)5u>f!eX?K zaKg`zmG_=xk(Mc;GQoWOv#}*lf1iAmi_JN5K|8Aw);TBN<&x0ygK41^KeR~KjG6wy z&Mv)tJ3xHCb;X<`!1hrry`00zhpa#!lJbg`H9MjbSvX8YE)%r{70>ciJKcf@Y$(S* zAuC5e!PxCK>1S5LJovVuaJvWVPF3uzzH}@-NfDv{HRG*Kgzu(_4braC*7efbgfO zG~7J5O&$fBkS5uCqE4bDp`0`7hSJNa_Ii*eW6Tj)%htp%nd5qP6QExayJO!E`>hvC z&rEubs!43}<(j@n%Q6_&Y=ecefMx(=@3>L}oT^O4(dafyMpqNa)}$?&h%fxSYAb28 zyCu5fv78T1J~(>Vqti;;WAH1EEx&H-YJJDo9@apFmvC~+{z17+#R`x}LZApd4j$%j z)AnF7`xz?>YE4!LO~5C3-f})4cvE_3rk*!|vC0aKLD(hd>nFQvS(-PZWbQE$!|+v% z9`92Xz&vP%r@~47D~U?M>&P_<#?ci0dPX)+tYr(kycrlWnwthdppFCOric zK?U(^;az@vxsgC1rou&bTEzUtX%23Pr1>p`3%a2lMbgc2@&ie?xQz5i0{;|^p`G!; zGhqpGq!2^jUlEyf*wO3Dipuk3Az^beoZkm`{@t_mRY>A_Qxpw%Vwx%*A`{&gPcQA} zFcuwBVv{Zk z7TpcW!Ht2Ga1-+}qXi1PPAA3i5SH()t&|&jHl7}VRZ+fm$r-g`*ro6yjA9w6rQFY1 z;$(%Gpo2Gfhz|ejcaW;e?M3=XXC9w($*EsA(U%~o(N?I6rQIk z09lqFp|8Twh)Rtc-(Vw}{R73N|H*hu9pzd&M)89-9_&(lkqO;cO&**ecS0}0D8jie z8B2hWuB3z5J_`N(o4*&{!nFHH`tPd%u=FhY1vNwa`NKFogc}Tvc&-JkjV1cG9pvz~ zoaZ-EM}K!MLt~`>Y9t4Y>b)^g_b-xm{2})a1Q&&{tu?_leO+$wQJm(Vl2`}tZgr(N zKLl?`Pty3yAIpD;)G<2{58-vkxvuQ-A}`Rc9B*~hsM{vf0gp_?GYPy4;j?kz!q2-5 zIPRkspHR;x{S|Pk*r2;ClWv>LOp!dyqQ;)nhi?N_l5_Y-KtXc+e2D~mJZsv1#C6S#Ir^SW*K4~gt(`JeX3tK(7fsTNOqtIBU4@4;a}Z0uTr86 zeqw~bZCe$8e z|M)_Dfw+#P$Pp~j>&r#wO7${kWn!tD>qmkW_4zOUcZ^5}8M6$NeAvK2vG6@3QatkE zC@jx4jWB4(kteaqM))_(0|lot$cqxSZ+#{U#73N>uB~eR+i{gbqwDraQJmUeiWPd# z(-|y&u~A7Q)U$#MD@ci+eXvev&N{}749mL*alhpgg7>b>1P4Kux(L*k-ZZx({FjRO zLd0WCb?|brcl&My6#vSLL_G@;fAI&32XZSVnEZv!Vy<_GgJle%Yv~uaj;@9c#24`I z-&2A}SDik8u z_)cqxmp!~Mt`D8yeTIs)bLKsC$QTlyW6e=h->9K9q1MqQ}tD5snF$5I8YlL+mrIw z@7FX9FY&D=cp3B})a{wju9+^ygs=|caCN~6ESe%;itdXY$p;ZkJDert4T&A)VrSlA z#sR%t$cBo3Ct9W1aAjWJFVmI&LHOslE|KJ;Is~phP><^v|SMiyK+Gg|Ly! zRW0`V8e-b7$?0@77htUxn^T2_Y)x7|r%ZCF|GhRbyCS`^q_~)0Zfte>wRh|@y}G)( zy?5(!^02+gW`=&}pAh_mZM9Tq?bsZy;aVk|0(%0^)e4m-nsg))+uBVo;>dJyS{76i z5)wUSv4B&;AEtl5V|uDhp+VrG~q5u?59j8P3?MpSFmG#b6XMifk;r;6O!-x(q<;YsU^we;bBAQPT0 z`JB&==n{+zq62R@ULSw&Q&ACYS#S{Mw8E_QUTb`;|CMXqk)GMEBN%@bx4KzS&+PZ- z?!K$M?{&wc6(mtkr^)@2Y>lqJ8q#WL6S7LtQ!b{R#cSUs&5Tv%1qtQ7Fx@)AlxoO{ zJr0~P06cx}Fg38&F+75a>|9ov*%0HBvdx=b?5F3dGCx43q>S?B5RH?&^Zx40U5@Hy zu>y3@J+WZ{7?nvs6!?X@8Y7|?FE(c{y;u_7Y@GyET$krmjN*#!EIGOd1U^!5S{*e0 z8(;V-2??Gxgy~=zkj%fd+aXD4Gp-|;W9a@fH{&sbT+Ds03vWgx>(|3%`ABToTuLU^EjGWKL>?)vh)|_w~9G06DKDY8dSx#CSW8pGNw{G|IPwTlV zfl@~XIa3pGrD6tG0@e5{3YR~3T=UiIh44hFI#;66aaH_n=BiS9)$$yV>JJA}ggA=U zj*1J7E~Z$GzbMA4m-5=qSo0_8aCwlv_RG?(sD6aKo+auz@Jbusn8&&2br9j^Nf#9` z6Vnp%;bz?3(;QZnio{Z_-s!c8%1`wI2Tp08+8|=x!*YcY$T#3H*e&P%&~yKZ7^ZDD ze}TVTMn&xO3fQIT+P*%#FQ-V^VXE#e_oUDAbJarif|VgMU)KQ=aC|~~Un#oJFau4p z@RFC?3vu_6!sGOs#OtHdT1 zoK}6=?6BJ|Qdm$hWhz#7G5#gZj#r^Iqnpa7#kx`KhLqB!e@aehpy*b*b^gg3_fXrJt5>szQm@aGr-nqXQOjDdJpZgE zzKQdtj1pP3fNRYU&AckqH=bE?7+d!-tCflrJkwZg00O;n5zAcDk?%6XfQj&pxvwjg z_t{R zJACha-;llk^t+h%)a$%?N2lQaXQ!eemR|Lx<|pOtdRk%uGk9uIwKYI;R2kQ|fUm); z_SQ9iBGVk<4ID#%a{u!c=WuHa+f^npZ6dk!)21N9^L$$EA05%6HIobG|J@Gp(ya+E1yZ;1}!mAbkz->&)wW`?}lsHg4OM6Iz9cCM7_uQs)P9i@hO z+)bI?wmA>!X-PFa;D#JVzmq%8EF>iSs`nl+^*)Wi3a_`+YefE3q1z;L*^}1B*iWx@ z7~HK)Ien-7wMzXEu*7%0ZMZM^+;d{g9{7s;r-!)7KG$gh-dVE!YdqFI)})_}9=`nN z)BD2JW^0))A9S4>Y=+BkKNzKo%C6W?Ip7Q%A($w(d9z-z@+naSnO3$^712vfp8V5Y z(|~C(eSG5R&6Si>ittz33YWYW8B=W+!KQf?o+;w8*%SaOW)yz7N`T!dWYMrQr&x{~T}i^1ZLtxeHDzlys0o z7G34K_rou`#mACP+g#?eCq9h5fVzEz0axKKzZ^1HufxM?{dgvPy;eq9driY~GWp1z zA|dy(Sx$Xh=kkF2Q=-t{9=Zx0yQmG8M0V{S;Iti?S>tgxWXdhB-o>Q7$5BBJ)4ua~ zJrN%8K2KGdFH6i#G4-RH{mY_)g2mZ|hPUVEp8%zV10V0K2eaKW=zbKSHixXAnK2ON z+?>~1K;i2RHok5yy$#>Fi?hD=g)F8i<_GB>)P=(5DhF)83v|-39hQ z?$IyJC{%lpnEgo#v)fT3U@;>^^8G@Ce?FI`iOHM^_P_{s}>wD@@VScIrl0pd@BNi%k1y3_CJ@3%`h46u-c0Gl! z=tLN8tV+6z@rxig-EGiYjUOk;Cg!`95HmjO`FOkB(#q?})y5MHN}KE{8w*>rl`@UJ z9UTUitGQ#0Hc zc+1IGH)Wg?jVw{ImY4KhUc|n##HnxWjv=Vcvin+>vGH8^@tyGJNb`* zwlW$93P#2kld6z=mWo&m3Nyz`NYg>%rbJees&FMa=qkx@6M z0r;M_W5~SgU{+ys%Z}<};;A~&3(zRQoKhmHd_(!gwBt51(o4~53JA#4?M7K`S0#c962Q$cf>rURv zZRZz2%xMOS0_JS>V@t(5*(R|+jC2`_Fgh2l9+M+~&jiV7+c*J45#I8KV!^J7Y3whs z4Izr^hdZ9Mb&qN^k>8}l$ebrQJ3LrAr}pr*13M#9d7XYsXHQp{T4~Il`7VoSBaz_2 zW<}Fo8Jn*rRVc4S2E1(h6H5dF_p5SQHMm^8R~=GWSsT}!ovgf63-99dz3kiIPZ989 zH}2+p|3Cs~5s`iVN9|1xd$WlNLOg!+l>4jiWcRd9bm;#1-| zxX{C8E!oxF7(r|y!{c)rZ=bFGOdqLMlxcQSe<+||E|fieJkL9pSPlvd}Hw&JH zwEH@j#)k+Wm10Xm0?`tl1s#kuc9|4WMA2d3AvTmnJIcoK5!7EA+X23$TelL~9(;a} zz0aFyQ(J9xwYA7Db4J;aoI|gQf5emAHtqYVVe8nHAKS-|6KmCeYaCh>Ur2nUoSY06 z)Qh!ezXJ{i>-1!hg`C~8YNKa8EnI$x-!k0nP9VK_I6VQRFW}Q7k9+Qq$`$qS)h!U{ zf-QR^HZEWivV{9oTg4!r#)UZwMOOXt(udnG_!T_%IUCLFJFUM|5@lZIDpU`(dwKeD zaCU0CN@$1RFu_2#8lTcpqiYR7^SoaUZZE5=SAaf9{|v577Cyef^(Zs$9rBBDNU=fM z^m_>niJt2j3=ukoKla?}_OQvW25%d5wi}Bqw*nY{kX1d0XP%pT%+c_JJVE3uC%Q;8b zDFHBFjloFCcegF7q2G8VyhO$G&}KTcd;eoNDn21dxO`BSSpkSCDIa8Cgfd3;$qT7r?X!=sB>n!Ttz}r(;DlVwAydt{vVF)ze z%%W%5Ak_C4gQDP&%W^4p^&E$9{nIrY3Xn?(i1CHEbZgsD0CKI z)1p^$ycG4Z~)6v0a7J`T3I@&y?hS#Yq>pT&2T zrEJBVo4%i4O{V2+Mo1+)q$H$q5=UEm+36ZorhnlSTt3a7VrU+!Z0?fPt_TZred;~6 z7{1(?$HP;Cw7*rPSP4PDd|;Ku8@SovUpO_VYT+1roF@G%JH3YOQMPrWc!@l15i2V% zl5u4lOG9%ZzxV(U9c8so-tmQ4HbTJq?b7?_r;5~*Y zNAt|mPH1fa_9e3;@Yy*f+!VfmFn<{>5o~IT){e9a&?EtBX8b?p{^ZNd-j*2rWPM zu!ez$F>#^oJvd}<%i?}HbiSm{m*rQ08(VSQ5lsD~PSM7O1MKDy=aIib!oDw)Q$1K& zSWZ@3FRj=as#*K?&1)Q|l_0|jj}KgMfz_bS#+E({oXla zkK@#ObQIE!0@8+gUoYEKPwwtxA@*g*?HBdUps?_Q_IVjU$aw~meZ5}t)Icfcq@D}L zZM|m@YG3a6?E*+W*wPetpTyGC)U7M3nQQHV7f5lI@h43Wuaw=P^dAy>Qse3u)`wOY zTeJCzE<>eDAel%qB`t7U=oRK$2w|egJZ#Z!cw^e-g+p`FM$&nT_8N-c>HCg%CVfLy zA;pcZf$26@T_~*zSdG60IoKOe(@7Mrfq#gUR)pV4bWMkjNuFOl!aM?l*Rj4%=Rt!$ppS|0w@u!w4WnQPs)$3Ul_GY zQVN=If}}srB?5pmvtDCdgtl7Tp%aNgvRyX7e(oTlo$!6|XAY3I!H0NBPzy5rMO4jaBGzuj33 zvJvPOzz$W@wE>mk5jP+|#-#`W;k8zyHyT4JAP^VaBe)gt=kqC{*|<66TZD9VHU}`QQ)B_+T#k2p%rv>u#_7M_R zDHp>2mDGy2ZoEO3(!p~>)E&VX)Wu79YqydAJ<)9gOvKEYpZWv9z#Y#=-wU7={9D-b zwWrqV%Oc#|k(#L*y0c+WK@R9dvF{m~*Nkw-@U1+7B0GW&OAdAj|okCzC0{dwm*roHcHxqy>Rt_ktf>nyiN=+*G*;CAPA))GDtdmbDq}6f&wtX;FTHmDs%~f(gl(vRgcxmf zLlD0ISHGmn8Vx?kymr(^ERKWfA~yac_ov@y+Y6<+LsC$3#^C<4u`-4ub409Yx#jTG zc+GBC^n7~A?C_JQyu&{%|Ge+D2;hsGu&byd-i}|X5fl_EsAlDO_b^YD+4gkMjhmfsd$22@_6su=(AqyebWuHDI!i$YGDwXw z3mB~*4R*SPs-I^^GDGBg`b0AVdSAzlhxb-CQ_AZ8ldQ{YO7A%=ACX-n$LfOnY#@bb zkdUHF#oq?{7QDlHKNqB7P+4VSsPQT&BxDSi^ZZKSQ(gd;B*v+C;Ck4Xi-?vtdI&F? z&x!K6bvxvIj1}AT-pHMkGi*ChYW%uPF4*ZnyyhFK134dFZ&?LgPr`;&M5iKEKUGa z0&MN@jXA$_B8uGymF*ddc7wIYJkuZm-)ejo76mH2e-q?$?~AN{Nqde0!FFq&!4^LP z4#u%Bx{;tZIsia%d|(*PXp#$X7*IxV)s@E`vR~(aB6S8}b>)L2UQGw&O`i6AruT?_ zO@HPrx33>Cn!@cy^=|NiMs%EyARN-kL|o0YcU=h9E@{^#3ZG8?E1!*$ulg35I+stu zzC=yg2R@y3vbv-Hk`Tw9^7nePoFPHvOo3E29HaonLLMQ>6ciTD>ZYfGtm=0^E6^bu z;zhejfj&Y0E67QqOVaH7-h%pY^&-U|N^XB* z_mPD0>Q+6Tpp{k*(Pb3}J z*ep+Sw0_$xYaOf{63KL*f+Cn^tK$PKDtDU9B2hTn7uWQdz4|r7#b0jozfY?bwT%p8412Yw)W~@e^$&sJpeYNwsJ=a$V`0Ey0)l)G1b*nfn2_8?231!}?NR7@xe{py>=MfhP+MU+s04+@(^L@LX`C>Pr;ZW^0?2Ek-PYT|}i z!o>PFJxFb@AR6OZRyA>~WU?3kx{{3?qo|-2`mW9E6q+cZPpCeI*QeaWAs$}9@Qrs9 z06eRDoZpeC*-=i06L8YC{-fBWx%|4rx= z>FD&zSOsyc6W(2k$e{$VPq2{?&Rj42WD?Yz)AZHBC6+_Y=qUnL(~(OfIy}29yCHg- zG~jXihHioPjy4kcrte1g-ERgX$X+l-VTCHL zB!5N&UiJsICaej9<($E#)IWCyX|Le~?PIyu#OWqtmm$Fw+(&PQc;Id1}NbnW$Hx1YT;dP6g~3szR!3Fj<{uOXrC%} zD(Z#cmxL9efu+GClnmI}vwBr%t858c%dHEZFC@>W0|+FHvf+eu24Ey8R^3_@ot@#E zJ;6^bXL!E9kg?Xys^Si6MyO}5ikpMn=JVk>KYI#RsLM@s^kA5L>fT}KvV}x&)^Nve1Gi;_As2`-Tt$0{u{5Dy;WiRcxhuPwuWh!ewc4C0%}HM{1vuGZJ%2E z`k@koa*?C$)*H(!=x7`Za*3J6+EU9w3tC;t0@4@cAnc242p;^D{?xPB}>Ab!lE#Gh( zP@Rx|SK^+J=x;&z5;g**-~SH%)B+?6(=`rsGG~vGz#I}_tXl7h$)Jbj93j5LCJ&7T zv1OqRV6nGFLvt)A-m=jGYfSn*gP~8rG8+5$n`Oi$$;kqGN+rViMFX5bShjqB71fJg z4{{!Y!Gv%pc&8lcNX8o*2_>)l{N(Op|7M{6sI(PB1GKYYAZuU((6OB2=WuN zZJKaHFk;1+dwUqBfs&7!N>nNZD9v5Ne@(Mnl!J?%tqxdoYQt^|`!~uJyZ6vuSR(N; zcJAF_m@jUcBDOaVEui=NWh{-*74N!ag^DFm@hd5F)XIN@R?=4YB+R_Qlvc&7Q1Hud zQVvYpwyz`L2@bvzVskBZqNjZQ6K^T5l!}9gTElol8BuKl4f)F#4bk55p5nn-c0Wn| z&HtSi;t@ncoOsb3nogS~HN4j4+3=BbBCt^F>}+@#+>Q8V3UALV ziRN5ilIf#I3z&1Onie-)EPTHi4NYHY5Sz42V^7|xAG|*dGhn#C?}!um7r4x^6z~9h zSN=5nE`sSiK+Eux(Y?vs*?vQwGQM^4R>d#+fZ|DJmsNPi(So?$B7yY_plU4A)OSqk zMbW2A)mVe!1m*bmvC`0s73v^oA6QGkA4AjMoMg|2p~4OmvEgYyNsdE2b2|gd>mrFH zHmTttGO4Rju?1pcHq1wnK~wl|icklYLMnRt>Z3Jtpk^dYWP7qr)8D9#_1_2`O&mU5 zj|B&Vp2C|xfvo&XFutmX)s$et03LE>oh{(8hC=j@zL}lO9bfSgp1bb|{h8qBaFJV> z&B%`yH|D=^-uA0apiPy+fv_CI$6VLn%AXGVn`b_yA(%sAmhhwN(UBnvAFSY+b_@*z zqnE81&tWpKO3ngcjMBgD(5E*Z1r#67g89^!mWDQ_` z3pA{+y%HTiiSP|yqU`3`0RdL!zLdV&Gtb6^{o$55YfN`={ED zC@A>k&>t#FLJ~C!3ip8GV;N1iq^%jxq!-Q&H+!*3!nrJtldla_n1rKPqziRHS@w02hfv zi$*(rI!+~m^9H>f{kYFNFM@?5=)^A>(T1o)d_;(chBu+=`f($sLc@DP!))M6r(j^rm(+B)=J?|TMf6x@VgvtO>HA_ zlKaY{*clj(t23U8onn1i!!#OOWh?fHqQK=`7ihF4L5+v-o)=PwWq2=Tobm!jD|9dHH&k|_5iE+V-{@qH&(UQlt?$^0p8bRqt)dpANg>LF0oEc;EbXt0$xO>4 zUUs53pq*l>;nR{ka)VQN0@~baa$iQC!8ouEui(1y5yaVFE@UoDZ%x!d{ z5J(zW%nUt^P-U>DrB-zAYH*=8;O}zTEB`yRTEv8$oo3yZO;;m8y|TtIwDWqnEem0t zSWII2tGGq~X*QljS~zA3idc*OC-C9*SPh*`=cWm@q202_=+#SVIJ~eP@yEzxiSGxcY0cAz^TG1 zVbW45M$+dg4piV#GyMn1Un%n0ddS?1g8PX5AUO5j{$hh9h7-03_^rEXDa%++PsX>Ts1yWOW448 zY8vY4LN&gnVk6d}68yp;#1ZgG-p+8IeaZ4^UmN_I(15X(t%;Se^f{V9g;o6?K0n z*X@Oa{D>Y-8lubr0Jr>M^ndQ*HXZPruK&PX9=u@Q(z1NBLlUaD;e1m^WHA}wD2xBw z^SR${`leu&(*9*gS1edeaHQOQUrkA`C4EE0_bA&8+^yI_z}#LSTGbc0R1G6rY2 zFMzOLt_p`Az}X#Z`t_>PT-m5C2JLwOgS}2TjQIOmWAS^;r02rZn!R&qc{HHsc+76Z zArsRRq=$9LfMVG3XCGXbdCx0=*5EeI#D}XD%b=&`WqH|64({k}$vVO@a zd~iCV6e;09diZcc?`frO{|RgLz2U#N$-1Kal+V!K3QyUBOVS{tm(2jNiZz3u85hl! zh3cx4@VJ_pZUL~GdXjgmr;ue6aqg8m{lqBS&414?9B>#h$*hFxRid4n&VSw4T1(}$L;JF-}(w&@)B+FscD{+Ja zY2RN;wc1ckGLnp=^!38f4A76mtpWj17^+(;V{UQiYjgF=w7H-^z$hwaS@`;M(FWzB zor@yQNP8e=-+!=5Zn_X6Puf34_J`&@xI{B;`)+$HP0-!6aFno6BS{DrlQ>qjM%pHu2?Y40r|DodO)v5PSh~KW4wG zL3al>ZX`KBBwUJ~R@l#DH3LA?+;-OQ@ zuL0f}Pv44KzvCH01chc!^3jIslvy~wPVP50C5z52&xaR}q zs?IXdR@_NcC#+fA3x7@Beg&hyKz~`O2D?%zXJL99rqPRVRV09mG9!*De>x{I1jdU* zQywbu{lzm+jDVWVWkD0|+!aK-K?wvgSDanqduc zEiAay^ycY&R6+DOm$(Xc&=!Fb^wUQ_8Y+T*m8TK^goJ>_EMJ1}!Ng-@wDTX`*3h2E zpWY^O0X1v9tqj?0SjPpPB9@^dWq+j5Pl#d96u;BdRlpmGIS6tg031!3@iozmc*Fw6z;)1bSyK33@6Kcl0@noo-|)>Y{--=`j;Jhz3zk05}tb5{^Qwf z6RGL0{xJbEXQ>D>>57OBWnca@rHIV<&xFgc0YrGo{?yuuPFdn{%b9(d&D_XLA!0 zil4PaD8%#aXV0WGIi0En8aU%BoFM5n%wWA+qfZ+ldlBTzfc7?PIgyG3yOxgnsyDaK zvLR2T)}LoNS#)!^S!6%gcKeV;vBc`lct`aHyINU!Ws4=HX5aFmPtMAZ`y4_H=yY&b zqZ7CR?juT7-SZGzxB9104L{{jE+?Kl-=Xn3CfQDU zG&Jg@2wR$D7?(UO_g2|RZh4kjzmMtvywN(Y#RRU}@-l3HaC2(sPtVcO^yv)06UPp% zt7t@2$;0m(ubN!sz7HjhSGjf=M#XI%Ic>&<;Yw_2uO`Qj3S^=U$1p}VCE*~CrL@Yd zVn+8bwC0SAj~>a@X8Kq8Y+*gom#iJJr1$*uHG&v9VW3JU?rb!<$>AC5+nBQ-T%_mJ z0WXhXx|N%P=nCTD`J-y~;@$d~%~Ge|*pECPM~JTA-z{1MD}#UJbT2)@TTs62<2;LG zjZQL|kL2M|XnFMs_F&^WUX!qZK*C$DSw+ZZ{P%gqom^dgDvvkcv&opQa+0Qv@;pvFVHz_ zY`ZJCJQvm)Tt0OS7f-bp3agl(&kWt;w2o~FW>w5ew=!Y{+bn@RjW)fl5YKw1kLe{4 ztW-sfuO1d1U6BF4JH~b!k6sCCc-Zs|>iCo_4Qx`dDc;vMGut9qg_*F~p*x`veX62t zVImz)^||VZ3M@(B+h-dsDC&A`2DF294JFOwPMXqm^gRRpPH{ok4%&RIfSs}H_czZP z80vt^MTpfC&Yviz7ZlJ9?W9nOWsY$bF2}Q{cD;K6ozGFalcq8K;+|Agbj4zfxNUZQ zx^?c_PJ^{d3@7-0Uo?4=>`*2aKEFchI(;C4l`6rqR?eBK-_pft_~(OUn}Uwq0+EaJ z_YJqzWLNDf^O>Rn^Yf(9#NZ50k<r=|a zb>V|58{Z6bOKy^o#i^45js5&t*-GawAp^Zet1%0cGYw84_725I(`8!W<-9^l^{Ko( zh80tu@u!>496Q~{9~oi>1O!GVgYfmgS$Gv7vos$1mTs#jRA+-D>FDE;II= zp+mSX6g~E{#b%o$%@iZ@5CeK5IPA8@+!i)JB^=)C=5GCRti)>qkFw9#eoTd}0LUvr z!#P=R&HQE(-S0}wC){Ow@~2JMlT04U2|lwPDxn`s3>l)O}Rabqs0` zw82UXKE3HcQp+DaHj#b<&t`j-9apQ|HS2t94SJ6UBW9+rH)3oqFMoaHK} zQ?@d6lZmf^w@TvG>^Oojetz50`g`g?tXAW%{`@X#=QCu~FZUrcg2LJmuof|B-8@-q zb_cqC$Cf>(A#@msXue;w$oDZ((I%J7HP5(pLq9PK&oSehF+1^070Z^l z!h%HiibWyytx5iDR$B$u)v_RC%B&e+b^U@g+5!@eh5MSKMg^?$twl2|nxfuoSyT3C zCsib?DLZc!eW8e!ccg@nbw@U`(j+4(GL(-JX@uO|S~onwgFqHkXvA)I>+CnG-R^`O zjA+GE=WO#nP=gO2w{;^1ypE$zyX7lFf%;c(WU^8*G3TF<7!5CH&^Rw6{ijvPHjDwA z1X8pX%7k1;FLCa3+6K6USO!h0E3VJWr(3BP7yF0~J6YfS5Xw3eda!9a91V`1A%{QX zSpq1Ll}UlopFP+^>Vq7{R6k{zu`$@>&vi$m7k7w=utPTI)J}Q&$oZa4*+>uRF5O;Ypv**CK43#sI-g;o|-<{z3=w z``XEqD5*hpc*{BD=d=t~k7IM_4JkcRfSL+!pX1i?)OTeAURl(WKP%T;N}o;0s(`?I zRvgA8(D}~&1UMSX+~wtn;Uf~eKk z`t{NdlNT8e-cLbQ$+j2hAI)s8HP}_cFLe`TwSXFqR?|h7zb8Oijhq?oq&g);_Z8t) zyodZUAh)|COpqxu#=lUvQhxT`*X|VVhOB#$jaMDrjaMm2ALjDNb8<#hMZ)@~0bqF_ zn27O`vi6k}kg1*vqx1bvtc|lA65>C&`+2tSM>W-cq%|`g3REr( zdm8bkqT6iu>C?@EoH>cm(o%bdlfb~_X^+LF0?d}t0xy?ab9HAfwuuf(y|Mau?~Z8d z9e%sf>+jjWlyZzEV(1L=_S`w14r|&mIxD@!5fH_RZ4e17JTn5)a5{+Fw!BQ@1~}`u z_-W4}RviXm!n^M-e&6Au3iCJ=JVG&Z_&U(f6PNn2^*O3OPsYNRp}*tGl^@cT2^9d| zA4(ih@HD=*&t3BcPNRL@Ml8nV)#nwi73*k=Wz*9}S#@9&53rJyF3_hAYUFOHe0?r> zH^&>_w=wwK9i%Li%sDbg2=gxW`-(rvScW92ls$z=;<4ViO=f*MrwQ8(B7QYx;5Bvi zY4FeKcWph8`H?#+D>keuVN}JV;&tQWwQgLj6C4z)DZZ`QL8=h5V?{l&cS3i0T_Nmp zXHlnzv)tslWXIT6?(eNJ%2TK57rDC)SEH%Ams22$ z)Q?^CgBuf=)vUt`hd2+2@n))zk_Sbmq;$)N;VOKi*T8C3j@2sHhXcQQHtPDseyW;- zrXuA9A>K;a>T1T?JKLH?^TwNoiEj!4aSl`Z-_Se&K~`#Na&aXg;x6lmAL?2|C*2y= z*QM%;RgplT1&wwwwHxv}fWJo`jqnZ~wHMW0F+k>f$I$z|;h1Ff;HNtfW2M@{r>(ua zYkkc>7rQ&0JZ%uzx<52#ms z=ZH(@s|p+Zs1{&)wPJ1c1yb5<7PCL3xPWL!kMHy{(~#T+Sgd@%=p^@T8m^QRX=L&N z*zu!!rA@TB3OH{-aNg}#iptJ5-!t%8i0CI8X0X`sx;VMX>TK_dg6S?!O);p7V z!UZ}5{LcNH6Y*%&_q_A1q>FMN(+#DLhYbp2I%c~}|FF&^etHOORI80^K#Jt%HM;-j zV^yT%BPx9X#;nNSqrWRs(jhPEwCo`cDH5JQv z5ZzDas2Si^5mrcRW}eG;y?7KKDT4oEV?E-h35Nd|P-_7ez72&qkM3 zNylL%t@ZNi?ZBYg5I49_-8F3JBKful_N0z-Q<3-m;YaC3u1n*j1(Um8I@p`9ox13R zbDWj7N=sdyC(t)~4i;QkWM4^MubzJWmVI$$ZC8GN!Z+2Pxh8COS%ZYliVpX*rJRkF z(QvafIYdOO;9HNA& zkZ}Yeu(+3x2e!ki@H1>T+_LD9x(9)+EQwZv8re9GH3~F2z65etWw%w z;bhvx#l*;X+G3+|wW7vdqrN4@h_hxa`MxC|$o1O2imj)0n{Ut`{D$PZIqXo1zWBz) zdC(HDUEnZ*$&%MZm5M-8YFNvoXXJQce+hdG3hRNums?r)0%7R=ESYp>+GmVDw71h7 z;VLDX*~~=4yZRwgr5$->T=^9UV*48!Wf1Pst}n3;iHE2pO0Z_AeYGE8MVD0uDMVmM zY;+z~igvtHLhcCz%2;TP(BRpv2-*MmwzaRWJI~;t_3iR#wd#hBZF95%}U;t;kMS=DrwvBZ;T3{a>wHg&VJ zr8X2@G3Uvbwyet9gk%XG9duf$dDFYl(CQuf-QZTy^E%dmHP{FULua|oJ zwo9PcqOGmMt(Di2#fKdQH@4P|PQM%!b1ZdC;&?C=Xth%m6$fqW899^moV7Y1$DIsb zDU<`BMP4O49#)=6hvig(1Ys71Rc)h1O2p`8okdBsVE4=xEV{>o^_K7nLk;li)OkT$ zHgIm}Eu41r9c7iNvRP$de?Gll!cl}xR+ghyLdLNV_=az-hupU6!~0hvvbD$rhFCJ%Jz)A$7&Il6Pcg*vhj~ zV_hW_5yl?b?7hw(;4HsV;J;UH5;FYTaOyP`)z-J{S|QC1QN-AJ>d5ASvCfF0&e^+X z&RKW6>+760N;l>l#u5pQTg_g2{c>W47c&O!bJGIjfzAXL1Iht7wP# zV!yWgt&qq#dehb#N{&Xk2YfGq4~aR%$f!@>Fc2>O^nLd2*;Hjyi@txvQx~m1;CWU@ z)NHJ@OSYu&WWLdvWKz66xU0TZ3gYlFL!tYJhjPWWnIlan5}ap9o(XC@(e6iy-?^VR zH2G}im0JbNfpoE1KSg|_UNBOH+~oml%lGkPhJZAnl`uTOOrbXYfQ2dmOpBrc&s8@O_{Fb*?I4_ioNngOLoMAnkDcVRjCNg9s9eyw!MGVv1KGyV;Nwc5 zVoz5xX6Qg(6`^vlp(CeXbal4XlzEv@eazsH@*$@zC0!G|E(c~JA#!0kx;T@fw~Mp( z9Z=8+6)JP=7ogZ7mE>TTyBxUJOmh-=?wfh1e*+}(*9_Mpg_8*fAR?m%Wr$uWGn9uT zc@=K#7amme?2dvvcS}HypNk#=I~j>=if~E*EQ2<`Qwt>{jyko?XpOov@5s(bif(Rw zv6%BAemK&x^D3cR?u6U;cmB~8EGO$O;*LtSz=C|8wcM}MpTRoVZX+MT@b z(V1T7!y;{cs`5?Vo=Ze_Ou(5?K8WF4^`)2=`G)&{IF7B_ftExXI<=D#dZ(0SpoeD&*dp|O8Y60plAeK)N*yoAI|8x&(e6(b|NHb$Ji zFSBj6u{*a*A!qu&4v1SfTJ;hKt^X#$lteiGOCbNu9#(J!xhIrKBGF%rq&PGScw&Xdp`z{{tm>P`ZNbd%_PnYt54%PK_Mm|J`(T1r;F) z?mT+*{SR$v-G9RWp<-82I=qTG9wq<^Z%*PrJdq0FectgP)o+LBx^jL4aL%5S;4U=I zXgo?L{g$Kuz=9v*K6oYFE!PVm*>U35lF&-J=x63av+>;Ta(#Av9=dpy45B0SJfSdglOlQS8d>`yYi!5+C0& zkjq05)wRO10mlLxRpqlwYyP7+$Qi}vj_+O=o0U4-A6kD>h3uY`2$*X^HfqVgx+)q@ z>#<5zS4;VXhHR+MU{<|g*>)<8$0N5lH3p1X$(|Z7-rtXOVLxxpdJofKINA`p-FTn| z!jkIcu*4dx`5$US?2?K9rBaSa-p zvk8Z%S)980plDeFB5C!v!V`L`>L4o%84d}5^HxR`<1l%gSU(gh*MVq}Wq%Y70$9;G zKl6P;Y1$(+DAFF3e*$udvNR9d(RB`xY{V%Fo7!AY&HzkkM1`uZUI7k3VY(tAAytdZ z&?FseU6gCD;?q>;o9=>Msj2_WSAx32mkw7}jot@u zAd!n)enFUTYJ61RKQ~Sh`PNqW%heZ20IcA6OQrqTBNztxR*B8^ceB|w>@Rd6Vzp2D zmOzTJYHE>+!w3+Gk4D#8cx(a`j(N&~=#KM4C7&bdQKE9F)CcxiP(%cV)`|^|mV?;! z(eW>4A{xp_NI*<@ACY;CH#$8}ln7XJRpsokPa!#SDa!$-{A`Vv;Zywv>@`CyC#Y+y zgRm=PIClbWaC^}j7srK{MSa%HfYKF`KI8zfw0M>fWE?haudy5Ggt zsn0AQ2LO}@Kvqt$;-0x?u?<4W-+h$a%GQ~|zg^zFo||PI8GBR!xLRY=%1zL)P8P$f z`*Y!p_=!(O(ev51uOd_q41RfGpX>R<9P6vRcBe=4MQk9B=l3Z)uHHk0X9H>VPh zu`5?U1@9-N(mpIKuK8R+w)n<(ubS6mmxIJu(lLm99~5k!fXv>@Zs>=(Gu?mX7L%&g zDTeVBlP0IqJ=qsEUUNca4S$nm>gbRW>Av5*o}dwpmN(O-4~p(Ewr$KJ zrOQ$a*O*mDU8rxSx&QLPm02IW5e;gI)eKun8xmh~xK=P zb{}`nf~K3{N*W2YzsGcE1?@Z%(Ny}k38y)DE&rv#bx2Rz_s*%!XxHCf-;^D0>4~jQ ztu?Z*OeYvznQ8do#8e5jf$j6ZPMt!6XK5DvTGzeSQzzm+oMyO+q*=4gNQ##$ITAGoE z+UhG;pC0@Y&7AWm`ss$;@N-$~{o7cV1&wvwRw-3&&@O+#vx`krZvZ{e6Dh_Xd!qP? zG|*OBzwHb4XtTD95ma!SwO+OmAN|BVFe-zXBFbWEq@Ng&Etn<_jt%`)`t*E7|DweVXsfb?9q0^bIqyE@Wn6! z2+<{j-5LQ1SmPPaDe+x`RT6J$>{G(sE3R%Wu-wUB2B*qMu^iUdBO_vbvn_S^jvzE7 z@cNjopWKaioyH;qlg?kZ_;(Qxgoh*px#>)Mv#2R=8u}yr>G+m2NDdsTBN;c$7c1F7kla*f`bB6KlCF?^4o3nKbi3x@w@ zf`$X&`^{bC8(si)IU@i;=fEXW1SjG{bGzTV4AIr&j#n*I6wpS~FNM)aR8Mb02wkO7 zOl~?%ym+6n7f!Ux)6Z~f((Mcx6knG1?44Ha$J3~E{UPwQIXVPQ%I2bkTMAq7}| zvhd___;3BE8eqf-)dx#2kO{JP69=R+E0@~S0h=5QfIM`>GW8%30g47+lvpy;-ufY- zZ8V*dIHTn&j8ms>$mr#Xc%cwr`H;FYad(VS9)4g&FwS578+rO^jnc~=-YmN0EB(<3 z^Cg;ZcpYQ}Zxy)uIr_0L^036q;vi8}(Jj#ITJ$~wRPp+z5ZZ|SYJA)bHGnFoG=q`z zzcDPM0xT7l_6W>xU%)zB8ZYMV$OmxX^$ClN%QS^6Q{KIDIFZcFw`+v|zq@C3q`%P2 zfsrxITd%?Gr!C+8_C#LRf-{nPq%u$H6Bq?4lVk~zxI=yi@WOvHVB?e;4DqI6^ z7JxMI=W?Z$spPcEBI!q+MYly!!7>Q{JBAkkRF_O9W{T92TU7l%ICksjX`HaeW36*w zaBh7&!4;Uo#h@uNjna|+I3H820p~>x|A+qZvB<8OfA?{?n^H#ui~xyygRY5&?&th( zD3LLekfr?%_(7HkBSPXzVTWNhqSupKvjTjw*694XT)1R}I4!Xb!AvZ{{JnXe%~UWD z6NSG+9@6`cLWc5!Z`5l40~oE;^|M2t@pGrxJ52yb`On#9M%PvN#JWb^x;l8V4p(f@ z0#Oyv@%f;+7WqG^7!&Jn{0k~L{tjj~IB)oU_v1w=!)et3MgV3t%iIn#WNd%AvB=Hd zv;|tn9R`evVSQ+0s2OlMlOo`PIOZCZNxFS;E+4Mw;02;eNL--G;haCGqxw69dA8#0 z|5?fx&5Oj2BcKhQQqzdQF(Q$%2f**kGoUE>dBT_Z$6);g$bmC1K8osy`IvHu;CVA6 zrXSRGC69l>Q|Ki8KKt2`g=~OA&&TB4J## zGgUvW*F{;mH2sbnoYH2C8N$1HZcrFtHK2gFpH{iI2diI>B0~4@lEhCUOu|mWr=|Qm zWm=E<2=x)sCEi6;8@N;|jD?zXWvR7|-;EjE8zsSjZimt*EE%&%_+j54Rl?VsQUv!@ z;ODE2Hc~x%t14=+CqEpAXVa^2Z%=$4^DR@K64v3bQYm|`pZECN&~8arwWgZ!j{?=- zmqY(SG~R!DaDpLBcrLN)6XMK)iFLRzKq^9uM5CQNMGl`{{#2&F5W1emJ|(DZSBi)V zm&BUIyvD3Y_ZmMH+#APxi#luT^%m`V{Ini#`UacNg?oX)@=p(#_(oBbS9x3{Yvlib Dgg^(d literal 0 HcmV?d00001 diff --git a/images/documentation lighter.png b/images/documentation lighter.png new file mode 100644 index 0000000000000000000000000000000000000000..0545ee331d0794a3dcfdc52f3aac7b15ea82da8a GIT binary patch literal 11777 zcmYLvby$_n^EU?$T`D3S(jgtvWx%0Jkw#Jk>26Tu0VxrrOOWnv5EPJZq(SP?-SzH` z-{1BA!3FHSyE8jGJM)5IN z?TCVcO9cJEP!f_zQBXL_5l^Kx-IBIuypvu!C*SPFK4qhS+5AyjDPFzwHNSwf-aRv( zE*gI0e2RN@?&WDqHQ$#0bAK;#f41V$*SK^RxUrm^S@4Gq^LxQj??YVaDb@w8bGZ-s zVx&2?PCDXoG2`Ml^v0e0oQo325B6%PTY75G?20y$4w{-)J)10I>mIhLA#>P7TM~ zZZBY0em7`d1&fvJyJ~491DJorZ@$Y%W%AErRH5#m`NC$=4R+kFB-8N!z#As_CI|K= zlWx?BD`ycTX`!1Kl&F+Alz4cXZd6#CpN;gR7+7`c52WdJ%rFk#!3^MIt_>-eDX@uy7OtW~^n*1y3eKR9!Wu-K&oL@@7~wGuDQcKePV}c;NvX_s7Oo zdlYF}n#sM@AMhDf21Jy@C`HbeTiDv^#08vVGv69Rje z+$R;iN#0I)&CgWfGXtV)vb!Zh)Ifz%SKg8@tz(g|?i$tWgXI_i~Il%tt>ZHa^=$M^01WK0>*MPlqn(MKQ$0W?=CS zr25rkm3?+FlIrg(YSD%-{&)T0l>En&58!NFqc$zgcT6)X;o6qtPubbjF%EmY@J^Y2 zDBU~wbM_Dg?2f}bBo@$fs1$(rJrWPO;DvL_Ie;^b;zc+37HkmG#*9^_!EQ$+ouy8W zahS4&IgL|*#su?Z(g|ZgsM}8x)kM;2%*aQUj^Lwtj2|?%@12!ur&fH8~JPOo@sg6%RZPltsTv-!u(r zeq}(g(beBIt*V5R6H=en$pkHkb}_s_Q%Qvxig z58>9_cZu{>b4xyVK3VGSteB1>>`E>GwnnIQ zOM*D;iGK%HyAl`9uX_VQux1R2&rS~kOKc(+cT9gTMsYHP8e<`uVa;%fsDB@HXwbi+ z87kIAI^l??MPPhI@s7GZ#4}sMd5Hjw`G6Oluq!8{yCqit$eL~{51?F*X5y|Xr(+zJ zwj>L+*8)DjRkQEXN&UI zcR(xpL&QDPp(}8hH9rGx3K>AkteBQO{0knk9#!(Y6elCN9t3>iFq^^6&@>8mWp3ma zun`MZ*6~u-Fy_WE6Ay5~m>4Q-z@@MzJ7V!#%`)B{XhxZiZ1G?O>?nc<$TP>@>ncSf z#od=lNC&o4PrtZRawn&6RsMi%z?B|pXBgArEDt`^=Ain>M4BDYrNXWrq;W=Ue1XKF zDobrk-m4`giA$QA?F{H3)F@OUElpx0{0z{%BndabV~P6fgv-sc9rKfFz?HFvwZgXN z-)I%)52cLKW7!DNG+zoKx)o>WIA;!U>Gm+CB$zkL0O{gB#3q9t-ld5iZUlSU4y>nR zuMdoAdPu-~=sO!ee_{t~|43$;0f{3JQ>k%vbwBS8L||}+zODr$>JMupX)m6$tph!r zC)k{X9>LJb!+_0;1r`!=XQ^hm(4)U*(D4G-x#x#iNJ!13B(Z8lT9=`t{4&Ckeh~gN z8g}*E=maa48H^FXpTRD(WB7)Hydfba9xMZ=%QWT-p(R><6yi6Vh}Fyk9pc$~J$yol zSq*-OF&~Nbl^$AH{@(wuBw$jdQ81Dr5# zTe>%`&`rwsG{Y`TMIAX|S2$^OL1&OW)JHID#HQlQ@sSrWqB8wlFirC>6AD^cJkOM$ z1o|J2Zui@1)f9GSf_&yg1Cr>W-R5S1FLg39%+1s_#R>S~Cx1cXY5vIL$N{ct=l zK9UCQT3|Q^I)K@_AvUkmn|O>vj5OPi2hh-R9v2ybQVK$9jyNt(O&E+sang1UrJLRx z=!k0b+km!@pTrd6*!&p}KUCl?eg;PF4UoFDmcYEZ?cTUR9tdH!qZ*)?{PK^VkInJz z0G(Ui8=cpv=P4~(j``3MHcu0*GUtd^_~EeKn`c0BtXkgw!^2AVnbK0fcyfRlQCOb~ zhtUt8_-AC~{#)UkN#@06xm^^qrIX>#S)sXkNVg z4$#p4F~UgBwRw{@Eh4-LGQo%zt-6b6t}t(s`DnMdkk)q1N)9VMj@?E*zuS`iH}5&b zn`0_N=d|b=XId($;VmmjCP!D20hN$a$nYWT9JC}i+0u5Nar@m8KBa_hHZBs@;~)() zgY3z`tuQ<@CJESVOfEt_|L+v?BvJ6oIV#_Q7g9Ge)Ht{Qx4B;a2!~yv%oCmc?|2vY zOn2Y8;2}F!ivO`1*>{6%Y~#X5g2V2t%fhwJ46jpP>Wmet)R438grjZCUzv3H`RMsT z6qfURUohemwe#fpEV)S9y1%CS^nmO21R5cuW_+~klt9#u`eRew=EEDbiovV4PtlkW zlKc6cu@hre@k;OBH5^YqH}KvP&@p(mDyo?Es-fj^_gnYb!qlmLCRlG|b zUx)V$Cl?#ZQ481Wi?RJf&tUoI&z{k&hl@nDBZ##3=GDo$-lkQV8x5I(+mL$K+oHNa zX*8{E_^FmW*T{N`RRr_!2@%Uv^t5l^2u(UdSNdwJ2i?@ruJYNw0Bft+x-Oe?$78ae z(!nk?Mh@Lf_rVV30HH$K9pboBljn~8t!JU)M@yLspBaLEuB`dhC;qUeoxmiL6*<`W ziHW`6B=LXN=F6T+;6#K>C)zd1^-GZNxho6mxK z(R0pXbbTeW`*UW}XauS9R$2mewszI!kFsV~p6#uQ{m-((u9p+(S>DIBtK~XDnjbvC zMgqBhHaCvWxpf{W<0@8v_gR%bIw77FyQt!WGi}=^d?gu+@s^746s;`K?H@Ahb|t^@ zlHJ^wph-fq)1q3<#&&LZRl)QTuSlWpK0-Y>&ST37T-^B9&eT-3 z!&!7z*-cqynIpH?z^~;rIIh~q2OI{ip^4^!DNR5}oMov2%j6i;Zc@nfLJ@_)!voH^ zr61~1_k;O5QdJh8;d|Miu2$ER9WD^F1L~apkJ)6=7T){$7zG5fm{UA{XAZRLou|i} zn>_hGA@3e#%%4txCGJ8?9DEYNGW#!&x8$N69xrBT_taQE0M|t3)^hEVtyxQfMHBkh zK?>~BFK9E_130oD6UKj})4-77o}Z67!2McJF$Th!?MjLGtLm~$JLsCIo(#I5q9SqD zN%Sv!LqC?1zmSYNzuRU~pcuuF`)CVxxPh+8tREaz`PP}wB6(QKYUAU;L8#yX& zm9y+G#znx8ToA)-o2MRl^JXLJSXrdw>|LkO%!V*81`47I>F3RyA z6T-anqbb2jMcpZL!OF@~v+f6D&8PP`xC*8OMU>RRZ7s(2Q}xbzZv8UwJhF%g!O6>i zkE;Q$`0R<5mGr1?&s!@Db;9 z3~H(3DIJ%%!t=cxX8%m1i=(ve4Yr)u_P6&+sVp}Z?Qr^am9ymS1w~%jXt50X!Kjdh zs2DaCA@C41?@gk9+i**54rMZo*Ivf6gpHRAC>c`8@$ z6r!NYoYv%dYU}P|keMDA+~o4w}C=*M-MuZ zs$aNdExryC4uAeI(fiSqiLA=b>$sbp!498?`Dfn-W?Z^f?F$C_cz%`8u&Uh*Qk=RC zy~x?^X;>EAPV&^9T!Wr8uyooHpCnA}s8*FoW7T#i)qr&hrxA=^48s4u9O$ffv-7y! z<&Jz{CK%|o6wdYN2q)|P_Dcb$EHLZNVR4@6{$nQd%EpVM{H5tV3&-iw4!hrGTFw__ zRG#aXaa@ z0z84O-^`;HmErB5$ZOWxcb1&yr`5nC2dAV6BN2F7)e{YUaom0<9CXn47IT#QXq~-w zrR(=$v4hs8b?3au${Y{->XyD}&PUG&eHNYbbz45Ofco0Fz@U(&@ymsVa-SuFxd0ZBpZyY`*IDh1H!warz&c@VCKMvxDTW~}P!3@Hy68=B>yX#!hY zRQGqNfl^4r!ICwnkc`y5`slB|bErYGC>!7++6qu@S}w(9+TAI%k)9sJD%Ct^0e z+&#_Vvrbu$raOFR6h!53N9Dp0Q`)OMR&9Aj)KYC9>=hebJaX13m_k1l%=VdB2E71E zioMv^x$%Y~u&B}*EuUJq7dY-;H`jX+o7Fo_o3$2(AH8hsGC9oOy-Rf8bd5OZVlQ5) zNC^p%NU}eV8=z7znf^=%pD#P`=T>_5R)g{?RYR%x>S$%9 z-X=Lozck#|)qMhH@U@U<9*{%==SYc>OEQrSC&7`wfgE3=q)3MIvk?W`* zz+6wBOY`*Xp#^aA$x`i2$k|#eJBU8Z1v04Z8u!V3dbEwN34AS%)&Y@~qe1)6gMM7` zEdXWW1w3sg?MHRT3bV~-JRUsKOB(!BLewwGwe^E9+~e^-D8K{o0>8n?z#(agg(||k z8-_1_$I2#?dYqSZ5MtR(8Uh>eIrLL%)QC%nPHD19S|%LjGhGq?5tqW!s(bUQ#=EZ}wEAR;Jwc`#XUyw7joO!Hg>QVUN;uyxr3}CtutZ)| zzu6~7duDnv`S`d{@1W3%Z&KyKj3XELh9eMgzvXqRFhlG;dS{?TGML}2BWQBDg2gL+ zV!wbVZkjQDOplI-3>p28$i=6@Ny{dck1KndvV4^j%Nu1YH)}CmoN0vjTT!)pFHw^4 z_J($HiNavS_O<=Xh>=-vuE#g?V*|rn0He zGs*gFbZp#Z!&-rxKLS$hfg+rbCAW;2Ys}wm=eU;Q)?hCnY+zq;V(`$f9lth(g z)MTHR6?JFxn;kK-Gj5XL$VrB1N&wSDlE?TsZxIvzP`t;YL=(6JCW_@M8q36L57-uj z43A19%p?`Bc~V71w)i1Hem#>|ZqN~K-kUcBTB#w_||zMI{^zZ6(w)2un3 za-CBhb{?fU*=-|G5}o7vI!j|28y=P@cmPaWw)am*tx86<;=gxKa_+LS)420z2MvBW z>uKC;r}!MMF+Dp=a5z1rE1JWU%@uCjc=U4Yzy&fNK$`pK^YPZ-H@e_^l#*R%Zkqp} z_8NXeB4oBNHp^Xe)(m{F!jgTCf3L4s9B=?j*DesKZC>}@LW=_(F`EB3#;Q7#o8zTM zaT9O0^ZoklRPQ2|fM;y zJb3A7jt0aKG$q z2CqvlxnTl#*Q&A3sY^2X{@c2a#POec!I@qjTa{smSX#ugwFXz3mn}@I8kl=Rfh=3k zi?MIW9d=eRT}(z+wM0)eU*$R+;*{XkRFb!KLDUAHk3^u@Z0vxkbW>;}?8#h{_ZLoS zapc>j464d18>3N^`~%hA!v0q zM#QG^4~dVKyQnHjM5@EHly$(f4RNk^XDEmAKZWol_)Ae#U#l`O*Y$L- zjn4Kn89+Z|w(myUo}`{%6871FNHaGQ8vo3 zwR}QZ4Pb6S7azlk2SQdjtN)A2D~1Eze>wmNoMqswopbb&=P0jR=1Q_IkHfl=XaA&)EW*!__atkl9m~>zZyn;ZP7&0v@faC`uUDhteJ4o3Ab^icAD>IMDoRoWCNL|xs9x3vxKNt6haq(t;ikZgPDIxv zgL~yy6vD((rh_sp*PbMNS9Mvch1Rb*(~mc_5CDZ~kN*KUupL2SdU&Sl2C@JvkkFOK zxfhZ;bzoLj7-fHdeAjbaUMkx}Jz7azqjA3`lpQx~c=xionJ0vMmU_YGrPTPPJ^TD}CxACZNz?X|pb#u1o-A_8zq zveBW5!;J))8$*x0wdx2l^0#CFJtqt!TtF0)C7`s%drB>7&EneE7zWZh8j{9t)!kU0 z*v^_)asd0&FOqM_;;p6x9^@8LNA5M?aP^Gl0N@8OKK&{;h_@W3R&Bsk5Dw*c^uJ>D zIIjH)Lg1Xb@Y?B~Vw`%f!^85hGrfK40!ln27ge5z3BFsUUyPb6Ketm~%)v0MW5;yn|=repQiG@uPY>hyHAZk;Ijvt5r{31xhpl-o-Ur@(am2fn4G| zj$*2kR;9b^j?#4op2#kgh zOwdRI6J>EtRc?&GdS}NaL>(ZX%sW1W1N8@znv@`_qw-++nNv|^leW;>Uhpw2fp4d; zVsSKUgdlGLB<1#YLt+Zt_gMihv5g%L6F)Ky7x`OGmYTWJQ_r`>U)`6|t@PN3d1T+E z;3Pdr%cVB#|8gSGdpIOLxZ}jUZj)H7GouJ)aX6-d`u-NjT|KDpNYp-9(r5r~>yhXD z!*i_{>aI%*5Y(sKn`kqxH!a%kP4*?Zv2)U&-~6gg{sufDDQno#uC$bj3ugKyVdU z0@9*kUzkv%z2t6=LgqdY7yt=CSxsPfs`d31sDc2bFVomf0wB19%2Vi1zE1a2?ny#><3F+HV)nOSdJm*x&MFi=ApZ8f`|Cl>a9{YQ z+;ot5+xq-Nb$j>*%-W+=f@DvMzpVwdYe|3c%lxg)QYUoT^_c%YS2V@YG;@0>ml+ zTE>hH*r{zkhJvSv?tWQ0m6NBddcXefc(@x_yk}K@1SNv;I+`rNd6oecZ8}sjG2W1k z74ci4_k$2{A7&qf-@wTU1iGFVfoTqdzvg=0!u^1Jf4*FVaJg||-7?D`S*CKQiT#Fx zVh{sEae())tmvP$b5fTNuYhUhBR>|@wYU9eWhZ}BajrJjou_vxfb1lc4$jR^jfs8l z4dPP(>%S6dt$%6<5eS{AopRH+V6wS-;VPnc2MNqVM~CGOI0z8ggj5I=HhQx|2s{W+ zp>xD4+p)L02w&XVQ1C0Ku5w#om>F50#K?sL`Vn&>tCgw7wjN}AhN z^=P(!nDD?;=|%8c+qt@THI11zIEnVdk3dM+vt+9nYOOd21*zw*w&BZZwa64tcjQ70 z2-!)SK%6Vrjs>b3`rM`lPqiL_`1g}6jb{?@M@vqNZvdxp?%nUqMwip?Z~zmLkot;Z z&4M7{$`l}KFJIBj-@Kb?`FiYP;I(Ep>#@N>?Y+4dV97Ie8|z_FzcU2c+RzT{p!u8X z_0rKtuWE#j->-n=bv4ytzmPZH6N$${i?q|A!eWu?nUZ?tW%+2QEpN29KxE7M))y_z zQX7;;85W<0JqaX^>`SfHYj0KFVt(DB!u2@+<>PW7NRUT3ID36V`d05;ad}@tu-HA| zV*AGI?}fQUl}0qc&>3aN61%U~Y3fJtC$@R|`%#0I>gjUJfkVd=KPEk%F6Tlc6!Q}Y z`RV#O+ljKkBd(^OkSVcB^%2}F83aCkp@YzDPGz(0f0^ky%LK^WKpBr7RZnf3( zx;kxkfp8aBLT&X0?1Oj(?}t!<3B=?_jlD7;Kk>h`%iNaF^-7{`DX4%rN*p^$jBVA} zx=8d9pEr@6{0pQt1yj(gsIWkls~Xj?-xaFssgu>?3vnGqP}P?gblF1XHuVY5%mYa< zCxRBL^QC&=A-Pa}yKYNy>e?4XyXvbnP6Ec=#rqncKxs|$*rjV$AN}yA=Lyst5LMTC z`yc8xweRh2+83hI$Fs^>u&WpIzyEaK7X7Baz$ZLKXT${#XJ3KJ5F*OS5Y+TI)8=SQ zZugic(Z4FG_q%4;4Lbc-CY+=}X6d@S!mf5&R7}`qq2hwA?K)f}iezv$RIz9N11@30#@CfT^i zew5z9QBb&Mh^aIW2%RZ(f>F(fzxiK@GRcYP12z)ZzuMup-pXcW3zL+8tQO~Y%~T?^ z>HuhE?pMXp>YlJDN@2z&>;BugUcBe&YoN?SPUcXNd8E+7ucK9wR4@&#r?coEzcZk4 ztB86&)50DPwY31^TAlM}GN82U>c3;57Y>$b&iY72et+_tAuapCcH}4M5JI+}U(>s0 z!LFR=gUI@zx_3-I%{^1(0H`jb%%NW={PKdP37@e`yY$Hk!miAWY#tndX^7>eAozqZ z{RG^}&&uDmEJ+KDu{rOc4Bu9@n_#J&Y+#!6s!KpmQmHkXNXt>n80E@siG1+oe4VvL zA7ffG>U31wB}9O=E|8LXH(o&svj+AjX^|I_gZiFDKxk5>9@@G2ysj|k6Nmt@#_v*? z=(r!jkKt#!bgynt)uV$|X0QAW3+ao-^sIyuP|c0=eosrW$`Im5u$$U)b(aHbHeeGx zK_h%xnwEnq0-qzD4YzZL2=lmM)(GVVJ|i3ORr#s{&5H{8!$04gq4O#l+}oUgHoZX> zB)gmxSQF7QY6bz)7rhTQ`^oLLT_N3_P$-rbO_pgLkP_nN`?!4Do3LbVnO8y!xSQ};aZca)5zgKvYwzA!e*8KH9IxB7~Myw>`UK86M<9g zDzs4i>*o4)L}CQrHAIj^Df0k<1^M@*6*K4sK`Pgu-(ojkjUc)AycB$4_UniBGwJ^a zeQ7UmF$#F3eLsXX4XpyTLk%9w1csw)Zsw#<<8khctPZK5_rV@k)}RGt?JR z7pX{WA{AtG`pdq>Oz0)l-XO6{E&ZHQHj?ap>#%Efgr z)Lj@~)*W!b#t77_!33Mv@HguyxZcO}Y(}KJ4!2#9tgc){ruF$)Wd;bgWxtetEIXE6 zacd=DO4y$75SiAtVU_74*n=s>h)`C543+S`R6tbTuTz(%q*)05I1 zu*oBp4S|y1I~0^Q7Y(6piwSc8@)O&4CbttR5xZSkQNG+_fz3N*_=SVgc6HV%&Zd(Xh9A^achr^duL0_d%NO^t%{#%UOFPp~73LEB?=TJv4gLh^t??a3&P z>=M#}`Z1g&G(1$F|D>TdR~%pZ&X@18%3fN5rbs3L)xP^?O7HJ2h>HJg`7Bd@vVCUDy{8Pb{$pU6$1RcK7I7XJ0aFC|_QZ5)=0dQI#m za@){}GtAfG7SQ8FCH+xd4&!jM0j-DL?a!d9(d&+WAk3~&QF#0t7CTaIk^{Auc*xod zj$N)6X20N;1k-MCeO>tia55wjJRVv7`>V#KOD+Yk*c!R~V0w#|KM#D-0~p1as5!I~ zlu!PZ*yHLBc*s4`DB7F|*s%)KW|(Q+sCcK71WkoOPXWP~wyH(7`#pSI5`{*-3Yhau zn?}q_f>X1Oe5k!P2qrE}gQ<-29F^YBJBkCi?_e!IOuyxp_d@6Lba=?XMX7lqBe{+Z zqlRGMlYXsw&$}DF=-8+}MM)1B5M4v9=>0@hd`5M1Eo6tcs~wOYLO@@$Fk-_4q^z-X zclytnQ^yADyqfhxE+B~3Ba8X#aILQGmWS|+pOmP0a029MIo%vO=vfXJCMdu(SU)`w z4hX^!$81P8((m$vHP$pJagV*xGATSkJEnJw-kzl@&I?cVEf}^M50`X>Nd$20Lq_t% zzZu0B(*SdJ>y+>4Et(*#*rh?o|7Pp7He~t+6N|QZ#Bhsn2lRBKAf72dEs`+`_ Date: Thu, 21 Nov 2024 11:25:12 -0800 Subject: [PATCH 5/6] Delete docs github button.png --- docs github button.png | Bin 11805 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 docs github button.png diff --git a/docs github button.png b/docs github button.png deleted file mode 100644 index 6b129e0db728a4ffac63419eb054653930dee372..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11805 zcmXw9bySp17iURn1rd>!5|9w28$?n`=@3cj?pRPjx=T`NN$C!eMnJj*1c4>BzmP4@r2I>|A6_7MMsX#;oKINhI&W zA1vWKzN6_7iAmzHhx3`t!H6NI&IsR9os0F2xPWZN3C_a>U zPd|^aAo_8do#y~gS>M?CQo6G~bbh!oiBNPIkJ>2xVkdSvw!h`tqau_2(#UTyE3q5K z7A2o0wEK2ZjV(&VUkOzou7u)6+M+HO8{dB3mgye_uk~+5v8xuoLbt}KL9we6_MQBD zBK2G0)mQ_gKD?rWR2laG(-Rh4ElhyXz@U{xFBdv7?>+aS#_!(HEM^l<)5u>f!eX?K zaKg`zmG_=xk(Mc;GQoWOv#}*lf1iAmi_JN5K|8Aw);TBN<&x0ygK41^KeR~KjG6wy z&Mv)tJ3xHCb;X<`!1hrry`00zhpa#!lJbg`H9MjbSvX8YE)%r{70>ciJKcf@Y$(S* zAuC5e!PxCK>1S5LJovVuaJvWVPF3uzzH}@-NfDv{HRG*Kgzu(_4braC*7efbgfO zG~7J5O&$fBkS5uCqE4bDp`0`7hSJNa_Ii*eW6Tj)%htp%nd5qP6QExayJO!E`>hvC z&rEubs!43}<(j@n%Q6_&Y=ecefMx(=@3>L}oT^O4(dafyMpqNa)}$?&h%fxSYAb28 zyCu5fv78T1J~(>Vqti;;WAH1EEx&H-YJJDo9@apFmvC~+{z17+#R`x}LZApd4j$%j z)AnF7`xz?>YE4!LO~5C3-f})4cvE_3rk*!|vC0aKLD(hd>nFQvS(-PZWbQE$!|+v% z9`92Xz&vP%r@~47D~U?M>&P_<#?ci0dPX)+tYr(kycrlWnwthdppFCOric zK?U(^;az@vxsgC1rou&bTEzUtX%23Pr1>p`3%a2lMbgc2@&ie?xQz5i0{;|^p`G!; zGhqpGq!2^jUlEyf*wO3Dipuk3Az^beoZkm`{@t_mRY>A_Qxpw%Vwx%*A`{&gPcQA} zFcuwBVv{Zk z7TpcW!Ht2Ga1-+}qXi1PPAA3i5SH()t&|&jHl7}VRZ+fm$r-g`*ro6yjA9w6rQFY1 z;$(%Gpo2Gfhz|ejcaW;e?M3=XXC9w($*EsA(U%~o(N?I6rQIk z09lqFp|8Twh)Rtc-(Vw}{R73N|H*hu9pzd&M)89-9_&(lkqO;cO&**ecS0}0D8jie z8B2hWuB3z5J_`N(o4*&{!nFHH`tPd%u=FhY1vNwa`NKFogc}Tvc&-JkjV1cG9pvz~ zoaZ-EM}K!MLt~`>Y9t4Y>b)^g_b-xm{2})a1Q&&{tu?_leO+$wQJm(Vl2`}tZgr(N zKLl?`Pty3yAIpD;)G<2{58-vkxvuQ-A}`Rc9B*~hsM{vf0gp_?GYPy4;j?kz!q2-5 zIPRkspHR;x{S|Pk*r2;ClWv>LOp!dyqQ;)nhi?N_l5_Y-KtXc+e2D~mJZsv1#C6S#Ir^SW*K4~gt(`JeX3tK(7fsTNOqtIBU4@4;a}Z0uTr86 zeqw~bZCe$8e z|M)_Dfw+#P$Pp~j>&r#wO7${kWn!tD>qmkW_4zOUcZ^5}8M6$NeAvK2vG6@3QatkE zC@jx4jWB4(kteaqM))_(0|lot$cqxSZ+#{U#73N>uB~eR+i{gbqwDraQJmUeiWPd# z(-|y&u~A7Q)U$#MD@ci+eXvev&N{}749mL*alhpgg7>b>1P4Kux(L*k-ZZx({FjRO zLd0WCb?|brcl&My6#vSLL_G@;fAI&32XZSVnEZv!Vy<_GgJle%Yv~uaj;@9c#24`I z-&2A}SDik8u z_)cqxmp!~Mt`D8yeTIs)bLKsC$QTlyW6e=h->9K9q1MqQ}tD5snF$5I8YlL+mrIw z@7FX9FY&D=cp3B})a{wju9+^ygs=|caCN~6ESe%;itdXY$p;ZkJDert4T&A)VrSlA z#sR%t$cBo3Ct9W1aAjWJFVmI&LHOslE|KJ;Is~phP><^v|SMiyK+Gg|Ly! zRW0`V8e-b7$?0@77htUxn^T2_Y)x7|r%ZCF|GhRbyCS`^q_~)0Zfte>wRh|@y}G)( zy?5(!^02+gW`=&}pAh_mZM9Tq?bsZy;aVk|0(%0^)e4m-nsg))+uBVo;>dJyS{76i z5)wUSv4B&;AEtl5V|uDhp+VrG~q5u?59j8P3?MpSFmG#b6XMifk;r;6O!-x(q<;YsU^we;bBAQPT0 z`JB&==n{+zq62R@ULSw&Q&ACYS#S{Mw8E_QUTb`;|CMXqk)GMEBN%@bx4KzS&+PZ- z?!K$M?{&wc6(mtkr^)@2Y>lqJ8q#WL6S7LtQ!b{R#cSUs&5Tv%1qtQ7Fx@)AlxoO{ zJr0~P06cx}Fg38&F+75a>|9ov*%0HBvdx=b?5F3dGCx43q>S?B5RH?&^Zx40U5@Hy zu>y3@J+WZ{7?nvs6!?X@8Y7|?FE(c{y;u_7Y@GyET$krmjN*#!EIGOd1U^!5S{*e0 z8(;V-2??Gxgy~=zkj%fd+aXD4Gp-|;W9a@fH{&sbT+Ds03vWgx>(|3%`ABToTuLU^EjGWKL>?)vh)|_w~9G06DKDY8dSx#CSW8pGNw{G|IPwTlV zfl@~XIa3pGrD6tG0@e5{3YR~3T=UiIh44hFI#;66aaH_n=BiS9)$$yV>JJA}ggA=U zj*1J7E~Z$GzbMA4m-5=qSo0_8aCwlv_RG?(sD6aKo+auz@Jbusn8&&2br9j^Nf#9` z6Vnp%;bz?3(;QZnio{Z_-s!c8%1`wI2Tp08+8|=x!*YcY$T#3H*e&P%&~yKZ7^ZDD ze}TVTMn&xO3fQIT+P*%#FQ-V^VXE#e_oUDAbJarif|VgMU)KQ=aC|~~Un#oJFau4p z@RFC?3vu_6!sGOs#OtHdT1 zoK}6=?6BJ|Qdm$hWhz#7G5#gZj#r^Iqnpa7#kx`KhLqB!e@aehpy*b*b^gg3_fXrJt5>szQm@aGr-nqXQOjDdJpZgE zzKQdtj1pP3fNRYU&AckqH=bE?7+d!-tCflrJkwZg00O;n5zAcDk?%6XfQj&pxvwjg z_t{R zJACha-;llk^t+h%)a$%?N2lQaXQ!eemR|Lx<|pOtdRk%uGk9uIwKYI;R2kQ|fUm); z_SQ9iBGVk<4ID#%a{u!c=WuHa+f^npZ6dk!)21N9^L$$EA05%6HIobG|J@Gp(ya+E1yZ;1}!mAbkz->&)wW`?}lsHg4OM6Iz9cCM7_uQs)P9i@hO z+)bI?wmA>!X-PFa;D#JVzmq%8EF>iSs`nl+^*)Wi3a_`+YefE3q1z;L*^}1B*iWx@ z7~HK)Ien-7wMzXEu*7%0ZMZM^+;d{g9{7s;r-!)7KG$gh-dVE!YdqFI)})_}9=`nN z)BD2JW^0))A9S4>Y=+BkKNzKo%C6W?Ip7Q%A($w(d9z-z@+naSnO3$^712vfp8V5Y z(|~C(eSG5R&6Si>ittz33YWYW8B=W+!KQf?o+;w8*%SaOW)yz7N`T!dWYMrQr&x{~T}i^1ZLtxeHDzlys0o z7G34K_rou`#mACP+g#?eCq9h5fVzEz0axKKzZ^1HufxM?{dgvPy;eq9driY~GWp1z zA|dy(Sx$Xh=kkF2Q=-t{9=Zx0yQmG8M0V{S;Iti?S>tgxWXdhB-o>Q7$5BBJ)4ua~ zJrN%8K2KGdFH6i#G4-RH{mY_)g2mZ|hPUVEp8%zV10V0K2eaKW=zbKSHixXAnK2ON z+?>~1K;i2RHok5yy$#>Fi?hD=g)F8i<_GB>)P=(5DhF)83v|-39hQ z?$IyJC{%lpnEgo#v)fT3U@;>^^8G@Ce?FI`iOHM^_P_{s}>wD@@VScIrl0pd@BNi%k1y3_CJ@3%`h46u-c0Gl! z=tLN8tV+6z@rxig-EGiYjUOk;Cg!`95HmjO`FOkB(#q?})y5MHN}KE{8w*>rl`@UJ z9UTUitGQ#0Hc zc+1IGH)Wg?jVw{ImY4KhUc|n##HnxWjv=Vcvin+>vGH8^@tyGJNb`* zwlW$93P#2kld6z=mWo&m3Nyz`NYg>%rbJees&FMa=qkx@6M z0r;M_W5~SgU{+ys%Z}<};;A~&3(zRQoKhmHd_(!gwBt51(o4~53JA#4?M7K`S0#c962Q$cf>rURv zZRZz2%xMOS0_JS>V@t(5*(R|+jC2`_Fgh2l9+M+~&jiV7+c*J45#I8KV!^J7Y3whs z4Izr^hdZ9Mb&qN^k>8}l$ebrQJ3LrAr}pr*13M#9d7XYsXHQp{T4~Il`7VoSBaz_2 zW<}Fo8Jn*rRVc4S2E1(h6H5dF_p5SQHMm^8R~=GWSsT}!ovgf63-99dz3kiIPZ989 zH}2+p|3Cs~5s`iVN9|1xd$WlNLOg!+l>4jiWcRd9bm;#1-| zxX{C8E!oxF7(r|y!{c)rZ=bFGOdqLMlxcQSe<+||E|fieJkL9pSPlvd}Hw&JH zwEH@j#)k+Wm10Xm0?`tl1s#kuc9|4WMA2d3AvTmnJIcoK5!7EA+X23$TelL~9(;a} zz0aFyQ(J9xwYA7Db4J;aoI|gQf5emAHtqYVVe8nHAKS-|6KmCeYaCh>Ur2nUoSY06 z)Qh!ezXJ{i>-1!hg`C~8YNKa8EnI$x-!k0nP9VK_I6VQRFW}Q7k9+Qq$`$qS)h!U{ zf-QR^HZEWivV{9oTg4!r#)UZwMOOXt(udnG_!T_%IUCLFJFUM|5@lZIDpU`(dwKeD zaCU0CN@$1RFu_2#8lTcpqiYR7^SoaUZZE5=SAaf9{|v577Cyef^(Zs$9rBBDNU=fM z^m_>niJt2j3=ukoKla?}_OQvW25%d5wi}Bqw*nY{kX1d0XP%pT%+c_JJVE3uC%Q;8b zDFHBFjloFCcegF7q2G8VyhO$G&}KTcd;eoNDn21dxO`BSSpkSCDIa8Cgfd3;$qT7r?X!=sB>n!Ttz}r(;DlVwAydt{vVF)ze z%%W%5Ak_C4gQDP&%W^4p^&E$9{nIrY3Xn?(i1CHEbZgsD0CKI z)1p^$ycG4Z~)6v0a7J`T3I@&y?hS#Yq>pT&2T zrEJBVo4%i4O{V2+Mo1+)q$H$q5=UEm+36ZorhnlSTt3a7VrU+!Z0?fPt_TZred;~6 z7{1(?$HP;Cw7*rPSP4PDd|;Ku8@SovUpO_VYT+1roF@G%JH3YOQMPrWc!@l15i2V% zl5u4lOG9%ZzxV(U9c8so-tmQ4HbTJq?b7?_r;5~*Y zNAt|mPH1fa_9e3;@Yy*f+!VfmFn<{>5o~IT){e9a&?EtBX8b?p{^ZNd-j*2rWPM zu!ez$F>#^oJvd}<%i?}HbiSm{m*rQ08(VSQ5lsD~PSM7O1MKDy=aIib!oDw)Q$1K& zSWZ@3FRj=as#*K?&1)Q|l_0|jj}KgMfz_bS#+E({oXla zkK@#ObQIE!0@8+gUoYEKPwwtxA@*g*?HBdUps?_Q_IVjU$aw~meZ5}t)Icfcq@D}L zZM|m@YG3a6?E*+W*wPetpTyGC)U7M3nQQHV7f5lI@h43Wuaw=P^dAy>Qse3u)`wOY zTeJCzE<>eDAel%qB`t7U=oRK$2w|egJZ#Z!cw^e-g+p`FM$&nT_8N-c>HCg%CVfLy zA;pcZf$26@T_~*zSdG60IoKOe(@7Mrfq#gUR)pV4bWMkjNuFOl!aM?l*Rj4%=Rt!$ppS|0w@u!w4WnQPs)$3Ul_GY zQVN=If}}srB?5pmvtDCdgtl7Tp%aNgvRyX7e(oTlo$!6|XAY3I!H0NBPzy5rMO4jaBGzuj33 zvJvPOzz$W@wE>mk5jP+|#-#`W;k8zyHyT4JAP^VaBe)gt=kqC{*|<66TZD9VHU}`QQ)B_+T#k2p%rv>u#_7M_R zDHp>2mDGy2ZoEO3(!p~>)E&VX)Wu79YqydAJ<)9gOvKEYpZWv9z#Y#=-wU7={9D-b zwWrqV%Oc#|k(#L*y0c+WK@R9dvF{m~*Nkw-@U1+7B0GW&OAdAj|okCzC0{dwm*roHcHxqy>Rt_ktf>nyiN=+*G*;CAPA))GDtdmbDq}6f&wtX;FTHmDs%~f(gl(vRgcxmf zLlD0ISHGmn8Vx?kymr(^ERKWfA~yac_ov@y+Y6<+LsC$3#^C<4u`-4ub409Yx#jTG zc+GBC^n7~A?C_JQyu&{%|Ge+D2;hsGu&byd-i}|X5fl_EsAlDO_b^YD+4gkMjhmfsd$22@_6su=(AqyebWuHDI!i$YGDwXw z3mB~*4R*SPs-I^^GDGBg`b0AVdSAzlhxb-CQ_AZ8ldQ{YO7A%=ACX-n$LfOnY#@bb zkdUHF#oq?{7QDlHKNqB7P+4VSsPQT&BxDSi^ZZKSQ(gd;B*v+C;Ck4Xi-?vtdI&F? z&x!K6bvxvIj1}AT-pHMkGi*ChYW%uPF4*ZnyyhFK134dFZ&?LgPr`;&M5iKEKUGa z0&MN@jXA$_B8uGymF*ddc7wIYJkuZm-)ejo76mH2e-q?$?~AN{Nqde0!FFq&!4^LP z4#u%Bx{;tZIsia%d|(*PXp#$X7*IxV)s@E`vR~(aB6S8}b>)L2UQGw&O`i6AruT?_ zO@HPrx33>Cn!@cy^=|NiMs%EyARN-kL|o0YcU=h9E@{^#3ZG8?E1!*$ulg35I+stu zzC=yg2R@y3vbv-Hk`Tw9^7nePoFPHvOo3E29HaonLLMQ>6ciTD>ZYfGtm=0^E6^bu z;zhejfj&Y0E67QqOVaH7-h%pY^&-U|N^XB* z_mPD0>Q+6Tpp{k*(Pb3}J z*ep+Sw0_$xYaOf{63KL*f+Cn^tK$PKDtDU9B2hTn7uWQdz4|r7#b0jozfY?bwT%p8412Yw)W~@e^$&sJpeYNwsJ=a$V`0Ey0)l)G1b*nfn2_8?231!}?NR7@xe{py>=MfhP+MU+s04+@(^L@LX`C>Pr;ZW^0?2Ek-PYT|}i z!o>PFJxFb@AR6OZRyA>~WU?3kx{{3?qo|-2`mW9E6q+cZPpCeI*QeaWAs$}9@Qrs9 z06eRDoZpeC*-=i06L8YC{-fBWx%|4rx= z>FD&zSOsyc6W(2k$e{$VPq2{?&Rj42WD?Yz)AZHBC6+_Y=qUnL(~(OfIy}29yCHg- zG~jXihHioPjy4kcrte1g-ERgX$X+l-VTCHL zB!5N&UiJsICaej9<($E#)IWCyX|Le~?PIyu#OWqtmm$Fw+(&PQc;Id1}NbnW$Hx1YT;dP6g~3szR!3Fj<{uOXrC%} zD(Z#cmxL9efu+GClnmI}vwBr%t858c%dHEZFC@>W0|+FHvf+eu24Ey8R^3_@ot@#E zJ;6^bXL!E9kg?Xys^Si6MyO}5ikpMn=JVk>KYI#RsLM@s^kA5L>fT}KvV}x&)^Nve1Gi;_As2`-Tt$0{u{5Dy;WiRcxhuPwuWh!ewc4C0%}HM{1vuGZJ%2E z`k@koa*?C$)*H(!=x7`Za*3J6+EU9w3tC;t0@4@cAnc242p;^D{?xPB}>Ab!lE#Gh( zP@Rx|SK^+J=x;&z5;g**-~SH%)B+?6(=`rsGG~vGz#I}_tXl7h$)Jbj93j5LCJ&7T zv1OqRV6nGFLvt)A-m=jGYfSn*gP~8rG8+5$n`Oi$$;kqGN+rViMFX5bShjqB71fJg z4{{!Y!Gv%pc&8lcNX8o*2_>)l{N(Op|7M{6sI(PB1GKYYAZuU((6OB2=WuN zZJKaHFk;1+dwUqBfs&7!N>nNZD9v5Ne@(Mnl!J?%tqxdoYQt^|`!~uJyZ6vuSR(N; zcJAF_m@jUcBDOaVEui=NWh{-*74N!ag^DFm@hd5F)XIN@R?=4YB+R_Qlvc&7Q1Hud zQVvYpwyz`L2@bvzVskBZqNjZQ6K^T5l!}9gTElol8BuKl4f)F#4bk55p5nn-c0Wn| z&HtSi;t@ncoOsb3nogS~HN4j4+3=BbBCt^F>}+@#+>Q8V3UALV ziRN5ilIf#I3z&1Onie-)EPTHi4NYHY5Sz42V^7|xAG|*dGhn#C?}!um7r4x^6z~9h zSN=5nE`sSiK+Eux(Y?vs*?vQwGQM^4R>d#+fZ|DJmsNPi(So?$B7yY_plU4A)OSqk zMbW2A)mVe!1m*bmvC`0s73v^oA6QGkA4AjMoMg|2p~4OmvEgYyNsdE2b2|gd>mrFH zHmTttGO4Rju?1pcHq1wnK~wl|icklYLMnRt>Z3Jtpk^dYWP7qr)8D9#_1_2`O&mU5 zj|B&Vp2C|xfvo&XFutmX)s$et03LE>oh{(8hC=j@zL}lO9bfSgp1bb|{h8qBaFJV> z&B%`yH|D=^-uA0apiPy+fv_CI$6VLn%AXGVn`b_yA(%sAmhhwN(UBnvAFSY+b_@*z zqnE81&tWpKO3ngcjMBgD(5E*Z1r#67g89^!mWDQ_` z3pA{+y%HTiiSP|yqU` Date: Thu, 21 Nov 2024 17:46:22 -0800 Subject: [PATCH 6/6] Update pyproject.toml --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index e0c5d93562..ab2de2b732 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -33,7 +33,7 @@ exclude = ["images*"] [project.optional-dependencies] huggingface = [ - "unsloth_zoo>=2024.11.1", + "unsloth_zoo>=2024.11.7", "packaging", "tyro", "transformers>=4.46.1",