mirror of
https://github.com/hiyouga/LlamaFactory.git
synced 2026-01-31 06:42:05 +00:00
* Shard the dataset when streaming to allow multiprocessing * Allow user to not set dataset_shards to ensure backward compatibility
* Shard the dataset when streaming to allow multiprocessing * Allow user to not set dataset_shards to ensure backward compatibility