ros/LLaMA-Factory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

drafting ray integration

Browse Source 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>

Former-commit-id: 19c12ddae9350f6e25a270fe3372f5b9094cf960
This commit is contained in:
Kourosh Hakhamaneshi
2024-12-30 16:48:52 -08:00
committed by hiyouga
parent 5ccc607222
commit 8683582300
9 changed files with 143 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
examples/train_lora/llama3_lora_sft.yaml
View File

@@ -9,6 +9,7 @@ finetuning_type: lora
lora_target: all
### dataset
dataset_dir: /home/ray/default/lf/data/
dataset: identity,alpaca_en_demo
template: llama3
cutoff_len: 2048
@@ -38,3 +39,10 @@ val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500
### ray setup
resources_per_worker:
GPU: 1
num_workers: 4
# placement_strategy: ...