pipeline-parallelism 8 data-parallelism 7 deep-learning 6 inference 5 large-scale 4 pytorch 4 machine-learning 4 billion-parameters 3 compression 3 gpu 3 mixture-of-experts 3 trillion-parameters 3 zero 3 self-supervised-learning 2 pretraining 2 paddlepaddle 2 paddlecloud 2 unsupervised-learning 2 ai 2 big-model 2 distributed-computing 2 foundation-models 2 lightning 2 fleet-api 2 elastic 2 distributed-algorithm 2 cloud 2 benchmark 2 heterogeneous-training 2 hpc 2 graph-partitioning 1 parallelism 1 gpipe 1 checkpointing 1