New: As presented at the IEEE HPEC Conference (High Performance Extreme Computing) today, the IBM AIU NorthPole Chip has been incorporated into a compact, rugged 3U VPX form factor module (NP-VPX), delivering high-performance and energy-efficiency for edge AI inference. NP-VPX processes 965 frames per second (fps) with a Yolo-v4 network with 640×640 pixel images at 73.5 W at full-precision accuracy, achieving 13.2 frames/J (fps/W). NP-VPX processes over 40,300 fps with a ResNet-50 network with 224×224 pixel images at 65.9 W at full-precision accuracy, achieving 611 frames/J.
NorthPole is a brain-inspired, silicon-optimized chip architecture suitable for neural inference that was published in October 2023 in Science Magazine. Result of nearly two decades of work by scientists at IBM Research and a 14+ year partnership with United States Department of Defense (Defense Advanced Research Projects Agency, Office of the Under Secretary of Defense for Research and Engineering, and Air Force Research Laboratory).
Today, high-performance AI runs primarily in the data center and—while training may remain there—great opportunity exists to migrate inference out to the edge, reducing transmission energy as well as bandwidth, mitigating concerns regarding privacy as well as security, and enabling previously impossible applications. To enable inference outside the data center, users need AI accelerators with both high performance and high energy efficiency, embodied in a form factor optimized for deployment at the edge.
PDF of the Accepted Version.
