ros/LLaMA-Factory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add models

Browse Source 
Former-commit-id: b9eaadde8b5f4b9f89fa7bb910b325fcf9c84434
This commit is contained in:
hiyouga
2023-11-30 19:16:13 +08:00
parent 08d5340bd8
commit 7ef8f46591
4 changed files with 53 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@@ -92,7 +92,7 @@ Compared to ChatGLM's [P-Tuning](https://github.com/THUDM/ChatGLM2-6B/tree/main/
| [LLaMA-2](https://huggingface.co/meta-llama) | 7B/13B/70B | q_proj,v_proj | llama2 |
| [Mistral](https://huggingface.co/mistralai) | 7B | q_proj,v_proj | mistral |
| [Phi-1.5](https://huggingface.co/microsoft/phi-1_5) | 1.3B | Wqkv | - |
| [Qwen](https://github.com/QwenLM/Qwen) | 7B/14B | c_attn | qwen |
| [Qwen](https://github.com/QwenLM/Qwen) | 1.8B/7B/14B/72B | c_attn | qwen |
| [XVERSE](https://github.com/xverse-ai) | 7B/13B/65B | q_proj,v_proj | xverse |
> [!NOTE]
@@ -199,8 +199,8 @@ huggingface-cli login
| Full | 16 | 140GB | 240GB | 520GB | 1200GB |
| Freeze | 16 | 20GB | 40GB | 120GB | 240GB |
| LoRA | 16 | 16GB | 32GB | 80GB | 160GB |
| LoRA | 8 | 10GB | 16GB | 40GB | 80GB |
| LoRA | 4 | 6GB | 12GB | 24GB | 48GB |
| QLoRA | 8 | 10GB | 16GB | 40GB | 80GB |
| QLoRA | 4 | 6GB | 12GB | 24GB | 48GB |
## Getting Started