Added Mixture of Depths

Former-commit-id: 75dd98b9abc847e22cb263c17ebcd2ca5dd98345

examples/README.md

@@ -41,6 +41,9 @@ examples/
     ├── llama_pro/
     │   ├── expand.sh: Expand layers in the model
     │   └── sft.sh: Fine-tune the expanded model
+    ├── MoD/
+    │   ├── freeze_sft.sh: Freeze finetune a model, updating only the MoD router
+    │   └── sft.sh: Fine-tune the MoD model
     └── fsdp_qlora/
         └── sft.sh: Fine-tune quantized model with FSDP+QLoRA
 ```