ros/LLaMA-Factory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

support DoRA, AWQ, AQLM #2512

Browse Source 
Former-commit-id: 6614cc1f08aa944db083e27e451bbdd733f7dd97
This commit is contained in:
hiyouga
2024-02-28 19:53:28 +08:00
parent 1e7962dfc4
commit b392e6cfb9
9 changed files with 40 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -56,6 +56,8 @@ Compared to ChatGLM's [P-Tuning](https://github.com/THUDM/ChatGLM2-6B/tree/main/
## Changelog
[24/02/28] We supported weight-decomposed LoRA (**[DoRA](https://arxiv.org/abs/2402.09353)**). Try `--use_dora` to activate DoRA training.
[24/02/15] We supported **block expansion** proposed by [LLaMA Pro](https://github.com/TencentARC/LLaMA-Pro). See `tests/llama_pro.py` for usage.
[24/02/05] Qwen1.5 (Qwen2 beta version) series models are supported in LLaMA-Factory. Check this [blog post](https://qwenlm.github.io/blog/qwen1.5/) for details.