Updated Home (markdown)

Home.md

@@ -18,7 +18,8 @@ trainer = Trainer(...)
 trainer.train()
 ```
 
-### Finetuning the `lm_head` and `embed_tokens` matrices:
+### Continued Pretraining & Finetuning the `lm_head` and `embed_tokens` matrices
+Add `lm_head` and `embed_tokens`. For Colab, sometimes you will go out of memory for Llama-3 8b. If so, just add `lm_head`.
 ```python
 model = FastLanguageModel.get_peft_model(
     model,
@@ -29,6 +30,20 @@ model = FastLanguageModel.get_peft_model(
     lora_alpha = 16,
 )
 ```
+Then use 2 different learning rates - a 2-10x smaller one for the `lm_head` or `embed_tokens` like so:
+
+```python
+from unsloth import UnslothTrainer, UnslothTrainingArguments
+
+trainer = UnslothTrainer(
+    ....
+    args = UnslothTrainingArguments(
+        ....
+        learning_rate = 5e-5,
+        embedding_learning_rate = 5e-6, # 2-10x smaller than learning_rate
+    ),
+)
+```
 
 ### Finetuning from your last checkpoint
 You must edit the `Trainer` first to add `save_strategy` and `save_steps`. Below saves a checkpoint every 50 steps to the folder `outputs`.