![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
erek
[H]F Junkie
- Joined
- Dec 19, 2005
- Messages
- 10,939
HBM3 finally
"Either way, for the high-end processor market that Samsung is targeting with Shinebolt, chipmakers are unlikely to be fazed by the power increase. Like the rest of the high-end processor space, Samsung has the AI market set square in its sights – a market segment where both memory bandwidth and memory capacity are limiting factors, especially with massive large language models (LLMs). Along with the traditional supercomputer and networking market segments, Samsung should have little trouble selling faster HBM in the middle of a booming AI market.
Like the other major memory vendors, Samsung expects to ship Shinebolt at some point in 2024. Given that the company just started sampling the memory – and that HBM3 Icebolt itself just hit mass production – Shinebolt’s likely not shipping until the later part of the year.
For HBM4, Samsung is looking at employing more advanced fab and packaging technologies that are currently the domain of logic chips. On the fab side of matters, the company wants to move to using FinFET transistors for their memory, as opposed to the planar transistors still used there. As with logic, FinFETs would reduce the drive current required, which would help to improve DRAM energy efficiency. Meanwhile on the packaging side of matters, Samsung is looking at moving from micro-bump bonding to bumpless (direct copper-to-copper) bonding, a packing technique that’s still on the cutting-edge of development even in the logic space. Embracing cutting-edge technologies will be critical to keeping HBM bandwidth growing as it has over the last decade, but the costs and complexities of doing so also underscore why HBM remains an exclusively niche high-end memory technology.
Back in July of this year, Samsung announced that they completed initial development on their GDDR7 memory. The next generation of GDDR memory, GDDR7 brings with it several major changes versus today’s GDDR6, the most significant of which is a switch to PAM3 encoding. PAM3 allows for 1.5 bits to be transferred per cycle (or rather 3 bits over two cycles), opening the door to improving memory transfer rates without employing more costly means of further improving the frequency of the memory bus.
As a quick recap from Samsung’s July announcement, Samsung will be rolling out 16Gbit (2GB) modules, which will be able to run at up to 32Gbps/pin. That’s a 33% improvement in bandwidth per pin over current GDDR6 memory, and would bring the aggregate bandwidth of a 256-bit memory bus to a cool 1TB/second. GDDR7 should also deliver a 20% improvement in power efficiency over Samsung’s GDDR6 (in terms of pJ/bit), thanks in part to the use of Samsung’s 3rd generation D1z (10nm-class) fab node.
Today’s event from Samsung is largely a recap of July’s announcement, but in the process we have learned a couple of new technical details on GDDR7 that Samsung hasn’t previously disclosed. First off, GDDR7 isn’t just improving active power consumption, but the tech will also improve on stand-by power consumption to a significant degree. Thanks to additional clock controls, GDDR7 will consume 50% less stand-by power than GDDR6.
Second, in discussing why Samsung (and the industry as a whole) went with PAM3 encoding for GDDR7 instead of even denser PAM4, the company confirmed some of our technical suppositions on the new technology. In short, PAM3 has a lower average bit error rate (BER) than PAM4, largely thanks to the wider margins on the eye window. None of which makes PAM4 unworkable (as Micron has already proven), but Samsung and the rest of the memory industry are favoring the relative simplicity of PAM3, given the trade-offs.
Besides the usual video card/gaming customers, Samsung is expecting GDDR7 to be adopted by AI chip makers, and perhaps a bit more surprisingly, the automotive industry. In fact some of these non-traditional customers may be the first to adopt the memory; since the traditional GPU vendors are still mid-cycle on their current generation of products, it will still be quite some time before they ship any GDDR7-capable silicon.
At this point Samsung has not announced a projected date for when their GDDR7 memory will go into mass production. But the company is still expecting that they will be the first vendor to ship the next-generation memory, presumably in 2024."
Source: https://www.anandtech.com/show/2110...-hbm3e-memory-hbm-hits-36gb-stacks-at-98-gbps
"Either way, for the high-end processor market that Samsung is targeting with Shinebolt, chipmakers are unlikely to be fazed by the power increase. Like the rest of the high-end processor space, Samsung has the AI market set square in its sights – a market segment where both memory bandwidth and memory capacity are limiting factors, especially with massive large language models (LLMs). Along with the traditional supercomputer and networking market segments, Samsung should have little trouble selling faster HBM in the middle of a booming AI market.
Like the other major memory vendors, Samsung expects to ship Shinebolt at some point in 2024. Given that the company just started sampling the memory – and that HBM3 Icebolt itself just hit mass production – Shinebolt’s likely not shipping until the later part of the year.
A Brief Teaser on HBM4: FinFETs & Copper-to-Copper Bonding
Finally, looking even farther into the future, Samsung is briefly talking about their plans for HBM4 memory. While that technology is still a few years off (there’s not even an approved specification for it yet), we know from previous disclosures that the memory industry is aiming to move to a wider, 2048-bit memory interface. Which, as Samsung likes to frame things, is the only practical choice when further HBM clockspeed increases would blow out power consumption.For HBM4, Samsung is looking at employing more advanced fab and packaging technologies that are currently the domain of logic chips. On the fab side of matters, the company wants to move to using FinFET transistors for their memory, as opposed to the planar transistors still used there. As with logic, FinFETs would reduce the drive current required, which would help to improve DRAM energy efficiency. Meanwhile on the packaging side of matters, Samsung is looking at moving from micro-bump bonding to bumpless (direct copper-to-copper) bonding, a packing technique that’s still on the cutting-edge of development even in the logic space. Embracing cutting-edge technologies will be critical to keeping HBM bandwidth growing as it has over the last decade, but the costs and complexities of doing so also underscore why HBM remains an exclusively niche high-end memory technology.
GDDR7 Update: 50% Lower Stand-By Power Than GDDR6
Besides HBM3E, Samsung’s other big bandwidth memory update of the day is a brief status update on their GDDR7 memory.Back in July of this year, Samsung announced that they completed initial development on their GDDR7 memory. The next generation of GDDR memory, GDDR7 brings with it several major changes versus today’s GDDR6, the most significant of which is a switch to PAM3 encoding. PAM3 allows for 1.5 bits to be transferred per cycle (or rather 3 bits over two cycles), opening the door to improving memory transfer rates without employing more costly means of further improving the frequency of the memory bus.
GDDR Memory Generations | |||
GDDR7 | GDDR6X | GDDR6 | |
| B/W Per Pin | 32 Gbps (Projected) | 24 Gbps (Shipping) | 24 Gbps (Sampling) |
| Chip Density | 2 GB (16 Gb) | 2 GB (16 Gb) | 2 GB (16 Gb) |
| Total B/W (256-bit bus) | 1024 GB/sec | 768 GB/ssec | 768 GB/ssec |
| DRAM Voltage | 1.2 V | 1.35 V | 1.35 V |
| Data Rate | QDR | QDR | QDR |
| Signaling | PAM-3 | PAM-4 | NRZ (Binary) |
| Packaging | 266 FBGA | 180 FBGA | 180 FBGA |
Today’s event from Samsung is largely a recap of July’s announcement, but in the process we have learned a couple of new technical details on GDDR7 that Samsung hasn’t previously disclosed. First off, GDDR7 isn’t just improving active power consumption, but the tech will also improve on stand-by power consumption to a significant degree. Thanks to additional clock controls, GDDR7 will consume 50% less stand-by power than GDDR6.
Second, in discussing why Samsung (and the industry as a whole) went with PAM3 encoding for GDDR7 instead of even denser PAM4, the company confirmed some of our technical suppositions on the new technology. In short, PAM3 has a lower average bit error rate (BER) than PAM4, largely thanks to the wider margins on the eye window. None of which makes PAM4 unworkable (as Micron has already proven), but Samsung and the rest of the memory industry are favoring the relative simplicity of PAM3, given the trade-offs.
Besides the usual video card/gaming customers, Samsung is expecting GDDR7 to be adopted by AI chip makers, and perhaps a bit more surprisingly, the automotive industry. In fact some of these non-traditional customers may be the first to adopt the memory; since the traditional GPU vendors are still mid-cycle on their current generation of products, it will still be quite some time before they ship any GDDR7-capable silicon.
At this point Samsung has not announced a projected date for when their GDDR7 memory will go into mass production. But the company is still expecting that they will be the first vendor to ship the next-generation memory, presumably in 2024."
Source: https://www.anandtech.com/show/2110...-hbm3e-memory-hbm-hits-36gb-stacks-at-98-gbps
