Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "3d-parallelism" keyword

pipegoose 0.2.0

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in ...
2 versions - Latest release: 7 months ago - 12 downloads last month - 68 stars on GitHub - 1 maintainer

Related Keywords

data-parallelism 1 distributed-optimizers 1 huggingface-transformers 1 large-scale-language-modeling 1 megatron 1 megatron-lm 1 mixture-of-experts 1 model-parallelism 1 moe 1 pipeline-parallelism 1 sequence-parallelism 1 tensor-parallelism 1 transformers 1 zero-1 1