The JEDEC Solid State Technology Association is developing a way to increase overall high bandwidth memory (HBM) capacity with longer channel distances to enable more HBM stacks per GPU.
JEDEC is the Joint Electron Device Engineering Council. Standard Package High Bandwidth Memory (SPHBM4) is similar to an HBM4 device with its silicon substrate, but with fewer pins in the different base interface die design, which can be mounted on standard organic substrates.
Mian Quddus, chairman of the JEDEC Board of Directors, said: “JEDEC members are actively shaping the standards that will define next generation modules for use in AI data centers, driving the future of innovation in infrastructure and performance.”
HBM3 has 1024 pins. An HBM4 device has 2,048 pins, through which digital signals pass, doubling its performance. The planned SPHBM4 device has a quarter of this number, 512, and operates at a higher frequency, with 4:1 serialization, so that it supplies the same aggregate throughput as HBM4. It means one SPHBM4 pin can do the same amount of work as four HBM4 pins in bandwidth terms.
Because there are fewer pins, they can be set wider apart. JEDEC says this change allows the relaxed bump pitch required for connection to organic substrates.
Silicon substrates support a high interconnect density, with lines having a more than 10 μm pitch, whereas organic substrates have a less than 20 μm pitch. They also cost less.
HBM4 and SPHBM4 devices are expected to have the same memory capacity. Organic substrate-mounted devices can be further away (have a longer channel length) from HBM4 devices. This potentially increases the number of permissible SPHBM4 stacks and so increases overall memory capacity over HBM4.
SPHBM4 memory stacks, with a 4:1 pin count reduction, will need a redesigned base logic die.
Eliyan, a base logic die semiconductor company, posted a LinkedIn comment about SPHBM4: “Our #NuLink D2D/D2M #interconnect solution has demonstrated the ability to achieve 4TB/s of bandwidth in standard packaging, which is up to 2x the bandwidth required by the recently published HBM4 standard, so we look forward to leveraging the work JEDEC has done with SPHBM4 to allow organic substrates to be a viable implementation option in addressing the memory wall challenges of high-performance compute.”
The interposer connecting the SPHBM4 memory stack and its base logic die to an accelerator would change from a silicon design to an organic substrate design.
Quddus is also VP Standards and Technology Enabling at Samsung. HBM suppliers Micron, Samsung, and SK hynix are JEDEC members and are developing HBM4E and technologies. They would need to support SPHBM4 for it to become a real standard. More HBM capacity and lower manufacturing costs seem like good ideas but will the gains be enough to convince the three HBM4 suppliers? JEDEC’s SPHBM4 announcement concludes: “JEDEC standards are subject to change during and after the development process, including disapproval by the JEDEC Board of Directors.”