Torchcomms: New Experimental Communication API for PyTorch Distributed Training at Scale

Torchcomms is a new experimental, lightweight communication API intended for use with PyTorch Distributed (PTD). In addition to the core API, we are open-sourcing NCCLX, a new backend we developed to…