Summary: Samsung presented LPDDR5X-PIM at Hot Chips 2026 as its LPDDR-based processing-in-memory solution for AI inference, discussed by Karam Hwang. Each chip has 16 banks with PIM blocks, and the package reaches 2.4 TOPS while eight chips reach 9.6 INT8 TOPS. Internal compute bandwidth totals 614 GB/s, eight times ordinary external access, though adoption is limited by caching and software constraints.
Hot Chips 2026 took place on August 24–25. If a single thread ran through the program, it was memory. The Sunday tutorials devoted a whole morning to “feeding AI’s voracious hunger for data,” with HBM base dies, advanced packaging, and 3D DRAM inference accelerators taking turns at the podium. Tuesday morning’s Memory session then belonged to Samsung, whose talk carried the title “Samsung LPDDR5X-PIM: World’s First LPDDR based Processing in Memory (PIM) Solution for AI Inference,” delivered by Samsung’s Karam Hwang. Chips and Cheese followed on August 29 with Chester Lam’s detailed analysis of the design, and that piece rewards a full read.
How LPDDR5X-PIM Works
The premise is straightforward: push multiply-accumulate (MAC) units inside LPDDR5X chips while leaving standard memory controllers untouched:
- A chip carries 16 banks, and every bank gets its own PIM block built around a MAC tree plus register files — a 1,024-bit instruction register holding up to 64 instructions, a 4-kbit source register for activation vectors, and a 2-kbit scale register.
- Compute turns on through special row addresses. Sending mode-control rows flips the chip between states, which is nearer to memory-mapped I/O (MMIO) than to an instruction stream, and toggles single-bank versus multi-bank modes.
- Writes are broadcast across all 16 banks, giving the programming model a SIMD-like shape, while Address Align Mode reads column addresses to derive source-register indices so that controller reordering does not break the setup.
Put differently, no NPU has been bolted next to the memory here; the DRAM itself performs an extra layer of arithmetic, and the external interface stays exactly as it was.
Context helps here: the same Memory session also hosted the XCENA MX1, a CXL computational memory device, and the Sunday tutorial ran through HBM base dies, advanced packaging, and 3D DRAM accelerators for generative inference. Compute-capable memory is no longer one vendor’s experiment; it is becoming the industry’s default answer to the bandwidth wall.
The Headline Numbers: 8x Internal Bandwidth
Every PIM block sustains four INT8 or FP8 MACs per data clock, doubling when 4-bit weights are used; a full package lands at 2.4 TOPS, and eight chips reach 9.6 INT8 TOPS — about the level of Intel’s Meteor Lake laptop NPU. Bandwidth is where the real story sits: across the 16 banks, internal compute bandwidth totals 614 GB/s, exactly eight times the 76.8 GB/s reachable through ordinary external accesses. For a workload that streams a weight matrix once per token, that difference is the entire product. “Move the compute to the data” is a slogan, but here the arithmetic checks out — the catch is written in the next section.
Three Engineering Realities
The analysis goes past the spec sheet and states the limits plainly. First, PIM compute and ordinary memory accesses cannot proceed together, which forces software to carve PIM regions out on their own; interleaving is therefore limited, and locks or OS-level blocking become necessary. Second, PIM accesses have to be uncacheable and non-speculative — a direct collision with CPU caching, prefetching, and out-of-order execution. Third, no path exists for one PIM block to exchange data with another. Each of these is a software-model problem as much as a hardware one: the memory controller, the OS, and the allocator all have to know which address ranges are special, and standard toolchains assume none of that. The author’s verdict: absent deeper system changes — dedicated compute commands, cache-coherent memory controllers, perhaps CPU instructions in the spirit of rep macb — the adoption barrier stays high.
What It Means for AI Inference Hardware
LLM inference is the canonical bandwidth-bound workload: moving weights dominates energy and latency relative to the arithmetic itself, which is precisely why memory vendors are pushing compute toward the data. Two signals stand out here. First, PIM descending from HBM and the server room to LPDDR points at a target of on-device inference — phones and laptops running models locally rather than calling a data center. Second, TOPS in the range of a Meteor Lake NPU say the role is assistive compute, not a replacement for the main NPU. Reading the PIM talk alongside a CXL computational memory device from the same session points at the broader direction: memory is becoming a place where compute lives, and the software model — not the silicon — will decide who can actually use it. For developers, the near term is a watching brief: isolation and uncacheable access requirements mean this will not behave like a transparent accelerator, and until tooling and the ecosystem mature, real gains depend on whether an application can be redesigned around this programming model.