Add ROCm support

Former-commit-id: cf9df10a24936efd420b0fdac541fd6c0808a327
This commit is contained in:
HardAndHeavy
2024-07-25 21:29:28 +03:00
parent 21adb09730
commit 5b67ccd1c6
4 changed files with 128 additions and 4 deletions

View File

@@ -422,16 +422,24 @@ For CUDA users:
```bash
cd docker/docker-cuda/
docker-compose up -d
docker-compose exec llamafactory bash
docker compose up -d
docker compose exec llamafactory bash
```
For Ascend NPU users:
```bash
cd docker/docker-npu/
docker-compose up -d
docker-compose exec llamafactory bash
docker compose up -d
docker compose exec llamafactory bash
```
For ROCm users:
```bash
cd docker/docker-rocm/
docker compose up -d
docker compose exec llamafactory bash
```
<details><summary>Build without Docker Compose</summary>
@@ -493,6 +501,34 @@ docker run -dit \
docker exec -it llamafactory bash
```
For ROCm users:
```bash
docker build -f ./docker/docker-rocm/Dockerfile \
--build-arg INSTALL_BNB=false \
--build-arg INSTALL_VLLM=false \
--build-arg INSTALL_DEEPSPEED=false \
--build-arg INSTALL_FLASHATTN=false \
--build-arg PIP_INDEX=https://pypi.org/simple \
-t llamafactory:latest .
docker run -dit \
-v ./hf_cache:/root/.cache/huggingface \
-v ./ms_cache:/root/.cache/modelscope \
-v ./data:/app/data \
-v ./output:/app/output \
-v ./saves:/app/saves \
-p 7860:7860 \
-p 8000:8000 \
--device=/dev/kfd \
--device=/dev/dri \
--shm-size 16G \
--name llamafactory \
llamafactory:latest
docker exec -it llamafactory bash
```
</details>
<details><summary>Details about volume</summary>