NVIDIA-backed AI inference chip company Groq plans to adopt 3D stacked DRAM technology in its next-generation LPU, replacing the existing SRAM solution. This move aims to enhance chip storage capacity to meet the growing demands of large-scale AI models. While SRAM offers speed, its capacity is limited, making DRAM more suitable for large-capacity designs. The 3D stacking technology reduces surface area and improves energy efficiency by vertically stacking chips, providing a new pathway for enhancing the capacity and performance of AI chips.
