SkyMem Semiconductor Enables Single-Card Processing for Models with Trillions of Parameters
1 day ago / Read about 0 minute
Author:小编   

Recently, SkyMem Semiconductor, a Shenzhen-based AI chip company specializing in in-memory computing, introduced a dedicated inference deployment solution for extremely large Mixture-of-Experts (MoE) models. Utilizing a setup that combines a single chip card with four solid-state drives (SSDs), this innovative approach allows DeepSeek-V4 to achieve decoding speeds of around 20 transactions per second (TPS), providing a throughput that is highly practical for real-world applications. It has been reported that since last year, SkyMem Semiconductor has been working closely with leading storage partners to lead the way in developing AI inference chips capable of handling models with trillions of parameters. These chips are based on a stack of Hybrid Bonding Fabric (HBF) technology and Static Random-Access Memory (SRAM) in-memory computing, marking SkyMem Semiconductor as the first domestic company to embrace this cutting-edge technological strategy.

  • C114 Communication Network
  • Communication Home
7 X 24 Track global technological trends
Hot Topic