Search
In a groundbreaking study published in the prestigious IEEE Transactions on Computers journal, Professor Liu Bin's team from Northwest A&F University has introduced GroPipe, a novel hybrid parallel training method. GroPipe is designed to tackle the pervasive challenges of load imbalance and excessive communication overhead encountered during the training of large-scale deep convolutional neural network models. By seamlessly integrating a "pipeline within groups + data parallelism between groups" architecture with an advanced automatic model partitioning algorithm, GroPipe substantially enhances GPU resource utilization and training throughput.
Experimental data underscores GroPipe's remarkable performance enhancements across diverse models, heralding a significant milestone for Northwest A&F University in the realm of computer architecture.
-
C114 Communication Network -
Communication Home
