数据并行（Data Parallelism）

参考文献： Team D, Majumder R, President V, et al. DeepSpeed: Extreme-scale model training for everyone[J]. Microsoft, 2020. Rajbhandari S, Rasley J, Ruwas