Updated Home (markdown)
parent
872398d86d
commit
66d6beb526
1 changed files with 19 additions and 0 deletions
19
Home.md
19
Home.md
|
|
@ -47,6 +47,25 @@ trainer = UnslothTrainer(
|
|||
)
|
||||
```
|
||||
|
||||
### Train on completions / responses only (Do not train on inputs)
|
||||
We now support it! Try the following:
|
||||
```python
|
||||
from trl import SFTTrainer
|
||||
from transformers import TrainingArguments, DataCollatorForSeq2Seq
|
||||
trainer = SFTTrainer(
|
||||
model = model,
|
||||
tokenizer = tokenizer,
|
||||
train_dataset = dataset,
|
||||
data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
|
||||
...
|
||||
args = TrainingArguments(
|
||||
...
|
||||
),
|
||||
)
|
||||
from unsloth.chat_templates import train_on_responses_only
|
||||
trainer = train_on_responses_only(trainer)
|
||||
```
|
||||
|
||||
### Finetuning from your last checkpoint
|
||||
You must edit the `Trainer` first to add `save_strategy` and `save_steps`. Below saves a checkpoint every 50 steps to the folder `outputs`.
|
||||
```python
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue