How Chiplet Bandwidth Allocation Breaks Down When Multiple Dies Share a Single HBM Stack

Shared HBM stacks look elegant on a block diagram. One memory stack, two or three compute dies, a clean interposer connecting everything. The bandwidth numbers add up on paper. What happens in silicon is considerably messier.

Photo by Brett Sayles on Pexels.

The problem starts with a mismatch between how HBM is physically wired and how bandwidth-hungry workloads actually behave. An HBM3 stack exposes 16 channels, each 8 bytes wide, running at speeds that yield aggregate bandwidths above 800 GB/s. Those channels are hardwired to specific physical channel controllers at the base die. When you place two compute chiplets on the same CoWoS interposer and route them to a shared HBM stack, the interposer routing determines which chiplet owns which channels. That assignment happens during interposer layout, weeks or months before any software ever runs.

Split the channels evenly and you get symmetry with a cost. Neither chiplet can burst beyond its half-stack allocation even when the other chiplet sits idle. This is the static partition problem. It shows up brutally in heterogeneous workloads where an AI compute die needs 90% of memory bandwidth for 20 milliseconds, then drops to near zero while a signal processing die picks up. Static channel ownership means the idle allocation is simply wasted.

Dynamic bandwidth arbitration sounds like the fix. Give a shared bus controller the ability to redirect channel ownership based on real-time demand. In practice, the latency overhead of that arbitration often cancels out the bandwidth gain. HBM channels are not designed for rapid ownership transfers; the training and calibration state associated with each channel controller adds switching overhead measured in hundreds of nanoseconds. For latency-sensitive inference pipelines, that overhead is poison.

Some packaging teams approach this by dedicating specific HBM pseudo-channels to a shared pool, leaving the remainder statically assigned. It is an honest engineering compromise. You give up some peak bandwidth to buy flexibility on the shared portion, and you accept that the shared pool becomes a contention point under worst-case load.

The interposer routing graph illustrates why this gets complicated fast:

graph TD
    A[Compute Die A] --> C{Interposer Router}
    B[Compute Die B] --> C
    C --> D[HBM Channels 0-7]
    C --> E[HBM Channels 8-15]
    D --> F((HBM Stack))
    E --> F

Every arrow in that diagram represents physical RDL traces with fixed impedance, fixed capacitance, and fixed routing delay. The router in the middle is not software. It is copper. Changing the allocation model after tape-out means a new interposer revision.

That physical rigidity is why the bandwidth partitioning decision has to be made during co-design, not after logic is finalized. Architects who treat HBM channel assignment as a packaging detail and not an architectural input end up with dies that structurally cannot feed their compute units at peak demand. The compute die may have 512 TOPS of throughput and the bandwidth budget to sustain only 60% of that.

The most careful implementations today use profiling data from early RTL simulation to estimate per-chiplet bandwidth demand curves across representative workloads. Those curves drive the channel assignment during interposer floorplanning. When the workload mix changes after product launch, which it always does, the channel assignment is already fixed. This is not a solvable problem so much as a risk to be characterized and accepted with open eyes.

There is also a thermal coupling effect that rarely gets discussed in bandwidth papers. HBM stacks generate substantial heat under sustained high-bandwidth access. When two compute dies are both hammering the shared stack simultaneously, the thermal load concentrates at the HBM base die and spreads laterally through the interposer toward both compute dies. The thermal profile changes the leakage characteristics of the compute dies in ways that affect timing closure. A joint electro-thermal simulation is the right tool here, and most teams are still running thermal and electrical analysis in separate silos.

The cleaner long-term path is disaggregating memory controllers into a dedicated memory interface chiplet with its own direct HBM connection, then connecting compute dies to that chiplet over a die-to-die interface like UCIe. Each compute die gets a well-defined bandwidth contract enforced at the interface protocol level, not at the interposer routing level. The tradeoff is latency: adding a die-to-die hop to every memory access costs somewhere between 10 and 30 nanoseconds depending on interconnect technology and hop distance.

Whether that latency is acceptable depends entirely on the memory access pattern of the target workload. Streaming workloads with large tiles tolerate it well. Sparse, irregular memory access patterns do not. The answer, as with most packaging decisions, lives in the workload data you collect before the package design review begins.