Provides differentiable communication primitives such as Send/Recv/Gather etc for pytorch. Autograd can backpropagate through those, making distributed model parallel models easy to implement.