ros/LLaMA-Factory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[infer] set env for vllm ascend (#7745)

Browse Source
This commit is contained in:
hoshi-hiyouga
2025-04-17 01:08:55 +08:00
committed by GitHub
parent 2e518f255f
commit d222f63cb7
5 changed files with 28 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/README.md
View File

@@ -181,7 +181,6 @@ llamafactory-cli export examples/merge_lora/llama3_full_sft.yaml
#### Batch Generation using vLLM Tensor Parallel
```
export VLLM_WORKER_MULTIPROC_METHOD=spawn
python scripts/vllm_infer.py --model_name_or_path path_to_merged_model --dataset alpaca_en_demo
```

1
examples/README_zh.md
View File

@@ -181,7 +181,6 @@ llamafactory-cli export examples/merge_lora/llama3_full_sft.yaml
#### 使用 vLLM+TP 批量推理
```
export VLLM_WORKER_MULTIPROC_METHOD=spawn
python scripts/vllm_infer.py --model_name_or_path path_to_merged_model --dataset alpaca_en_demo
```