Samsung's LPDDR5X-PIM puts compute inside DRAM for local LLM inference

Samsung's LPDDR5X-PIM delivers 614 GB/s of internal bandwidth from a 16 GB package, matching an Apple M5 Max's memory bandwidth. The eightfold gap vs. external pins makes the case for processing-in-memory, though quantization and runtime support remain hurdles.