Guest Blog by Steven K. Esser
Today, to seek feedback from fellow scientists, my colleagues and I are very excited to share a preprint with the community.
Title: Convolution Networks for Fast, Energy-Efficient Neuromorphic Computing
Authors: Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeff rey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner, and Dharmendra S. Modha
Abstract: Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classi cation accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1100 and 2300 frames per second and using between 25 and 325 mW (effectively > 5000 frames / sec / W) and iii) can be specifi ed and trained using backpropagation with same ease-of-use as contemporary deep learning. For the fi rst time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
At this juncture, I hope that a personal retrospective will help you in sharing my enthusiasm.
In the Winter of 2008, I graduated school in Wisconsin and moved to California over the New Year to begin working in what was to become IBM’s Brain Inspired Computing lab. The Team had just won DARPA SyNAPSE contract. Arriving, I joined a handful of researchers whose enthusiasm quickly pulled me into sharing their lofty vision — to build a computer designed from the ground up along the lines of the mammalian brain, and to have that system make a beneficial impact on society. Our goal was clear; our route was not. From the beginning, we chose to take a different approach from rule-based artificial intelligence, whose algorithms, though able to do some of the things our brain can do, under-the-hood work nothing like the brain. We also chose to take a different path from traditional artificial neural networks, which use neurons and synapses — the basic computational elements of the brain — but ignore limits on precision and connectivity that are critical for low power operation. Energy-efficiency is, after all, critical to creating scalable or embedded systems (and indeed our own brain uses only as much power as a typical lightbulb). At the start, this decision left us with no suitable existing approaches capable of tackling real world problems, as many of our scientific peers were more than willing to remind us! This gave me pause, but the challenge was fascinating, and so with the optimism of a recent graduate, I dove in.
As work progressed, the lab had the fortune to build an incredible team of hardware researchers. They created a prototype brain-inspired core in 2011 and the four thousand core TrueNorth chip in 2014, giving us the hardware needed for fast, low energy “neurosynaptic” computation. On the algorithm front, we began to internalize the architecture and created a number of small scale demos in 2012 and 2013 and also built a programming language that provided the necessary foundation for later work. These showcased certain capabilities, but were custom solutions to specific problems. To have a major impact, we needed a truly general purpose, scalable approach for network creation.
To this end, our path led back to neural networks in the form of modern deep learning, which can achieve state-of-the-art accuracy across a broad range of perceptual tasks. Though the traditional methods of deep learning are not directly compatible with our chip, to our great surprise and relief, we found that they are extremely resilient to reduced precision and connectivity, as necessitated by TrueNorth. From this insight, we were able to adapt the canonical backpropagation learning rule used in deep learning for compatibility with TrueNorth. At NIPS 2015 last year, we demonstrated near state-of-the-art accuracy on a canonical handwritten digit recognition challenge, using orders of magnitude less energy than the best previous results. As this work was ongoing, we were very encouraged to see research from a number of other laboratories demonstrating the power of deep learning for low precision computation.
Today, this preprint builds on upon many previous efforts. In this work, we demonstrate a method for adapting convolutional neural networks, a powerful tool for deep learning, to create networks that run on the TrueNorth chip. We achieve near state-of-the-art accuracy on 8 datasets spanning color image and speech, while running on real hardware at between 1100 and 2300 frames per second and using between 25 and 325 mW, which is effectively > 5000 frames / sec / W. This is important because approaching state-of-the-art accuracy within neuromorphic constraints was previously believed to be difficult, if not, impossible, and because the ensuing speed and energy efficiencies open up an entirely new operating regime not accessible via conventional computing.
This work is a major personal milestone in a journey that began over 7 years ago and I cannot wait to see the breakthroughs that the next 7 years will bring!