ros/LLaMA-Factory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix initializing data arguments

Browse Source 
Former-commit-id: e6b83c8b87cb93358086121a6f9ccaba5dfa7497
This commit is contained in:
hiyouga
2023-06-27 22:50:23 +08:00
parent 640f774d30
commit 2d22961c7d
3 changed files with 35 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@@ -202,6 +202,33 @@ accelerate config # configure the environment
accelerate launch src/train_XX.py # arguments (same as above)
```
<details><summary>Example configuration for full-tuning with DeepSpeed ZeRO-2</summary>
```yaml
compute_environment: LOCAL_MACHINE
deepspeed_config:
gradient_accumulation_steps: 4
offload_optimizer_device: none
offload_param_device: none
zero3_init_flag: false
zero_stage: 2
distributed_type: DEEPSPEED
downcast_bf16: 'no'
machine_rank: 0
main_training_function: main
mixed_precision: fp16
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false
```
</details>
### Evaluation (BLEU and ROUGE_CHINESE)
```bash