Updated Home (markdown)

Daniel Han 2024-07-14 22:46:21 -07:00
commit 66d6beb526

19
Home.md

@ -47,6 +47,25 @@ trainer = UnslothTrainer(
)
```
### Train on completions / responses only (Do not train on inputs)
We now support it! Try the following:
```python
from trl import SFTTrainer
from transformers import TrainingArguments, DataCollatorForSeq2Seq
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset = dataset,
data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
...
args = TrainingArguments(
...
),
)
from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(trainer)
```
### Finetuning from your last checkpoint
You must edit the `Trainer` first to add `save_strategy` and `save_steps`. Below saves a checkpoint every 50 steps to the folder `outputs`.
```python