Search
Revolutionizing Large Model Training Costs: Microsoft Unveils FP4 Framework Rivaling BF16 in Performance
2025-01-30 /
Read about 0 minute
Author：小编
Microsoft Research has pioneered the introduction of the first FP4 large model training framework, which boasts training results on par with FP8 and BF16 under identical hyperparameters, while demanding significantly less storage and computing resources. This groundbreaking framework leverages FP8 to emulate FP4, enabling the training of models with up to 13 billion parameters and handling an impressive 100 billion tokens.
-
C114 Communication Network -
Communication Home
7 X 24 Track global technological trends
