Brain-inspired Computing Boot Camp Begins
Today, August 3, 2015 at 9am PST, we launched a three-week-long Boot Camp. Read my thoughts here. Representing five continents, sixty+ participants from the following institutions are attending including IBMers from IBM Research - Australia, IBM Research - Tokyo Lab, IBM Brazil, and IBM UK:
- Air Force Research Lab, Rome, NY
- Argonne National Lab
- Arizona State University
- Army Research Lab
- California Institute of Technology
- Cornell University
- Johns Hopkins University
- Imperial College, London
- Institute of Neuroinformatics, ETH Zurich
- Lawrence Berkeley National Lab
- Lawrence Livermore National Lab
- National University of Singapore
- Naval Research Lab
- Pennsylvania State University
- Riverside Research
- Rensselaer Polytechnic Institute
- Syracuse University
- Technology Services Corporation
- University of California, Davis
- University of California, Los Angeles
- University of California, San Diego
- University of California, Santa Cruz
- University of Dayton
- University of Pittsburgh
- University of Tennessee, Knoxville
- University of Western Ontario
- University of Wisconsin-Madison
- Pennsylvania State University
- University of California, Irvine
- University of Geneva
- University of Heidelberg
- University of California, San Diego
- University of Ulm
Here is a rough transcript of my opening remarks.
When history looks back and asks "When did Brain-inspired Computing reach critical mass?", I believe the answer will be 9am PST on August 3, 2015.
Thank you for investing three precious weeks of your life, we will strive to make it worth it. Special welcome to Telluride participants who have come back for another 3 weeks. All of you are a very special group of people -- incredibly talented, excited, and engaged researchers who have the right background. Because of the space limitations, many could not be admitted. Today is the first time that IBM is opening the SyNAPSE Ecosystem. In fact, you are getting access to these boards before our own team. According to the dictionary, to pioneer is the one to "develop or be the first to use or apply (a new method, area of knowledge, or activity)". So all of you are pioneers. It takes two hands to clap and and you are our partner in this quest to bring brain-inspired computers to the society.
Where are we?
With 1 million neurons, 256 million synapses, 4096 cores interconnected via a network-on-chip at less than 100mW of power consumption, TrueNorth is literally a supercomputer the size of a postage stamp consuming the power of a hearing aid battery.
The chip has an entirely novel parallel, distributed, modular, scalable, fault-tolerant, event-driven architecture that breaks path with nearly 70 year dominance of the von Neumann blueprint. Today, the effort is much broader than the chip. It is an end-to-end ecosystem consisting of development boards; a simulator; a programming language; an integrated programming environment; a library of algorithms as well as applications; firmware; deep learning tools; a teaching curriculum; and cloud enablement.
We got here via a 10 year sustained effort with cumulative effort of 250 person years.
Where we are headed is ever more useable and useful ecosystem and what is forthcoming is a sequence of ever-denser and energy-efficient chips with the end goal to literally build a brain-in-a-shoebox.
What is the best possible outcome?
The best possible outcome is to map the entirety of existing cache of neural network algorithms and applications to this energy-efficient substrate. And, to invent entirely new algorithms that were hereto before impossible to imagine. Then, to compose these algorithms to exhibit mind-like behavior on top of the brain-like substrate.
What is the challenge?
You have all heard about deep learning. Deep learning and neuromorphic computing are two sides of the same coin. Deep Learning is about capability in terms of application performance and accuracy. Brain-inspired Computing Is about energy-efficiency, volume-efficiency, speed-efficiency, and scalability. Deep Learning is used to program neuromorphic computing. Neuromorphic computing used to deliver deep learning. The goal is to bring together these two complimentary revolutions.
To exploit the full potential of the substrate, the challenge is to shift the thinking. Energy-efficiency was realized by architecture shift to spiking neurons, limited precision synapses, and connectivity within cores and between cores.
The reason it is possible to shift the thinking is because brain itself is able to achieve its remarkable function with similar constraints and because neural networks are remarkably forgiving structures.
The first thought of nearly everyone is to train a classical network, and as an afterthought map it to hardware. This approach leads to inefficient and inelegant solutions. Thinking has to shift from "train and constraint" to a mindset of "constraint and train". The challenge is to design efficient algorithms from the start. As a motivational example in historical context, let us go back to "sorting algorithms" for today's computers, where quadratic-time algorithms were upturned by Tony Hoare's beautiful and efficient quicksort. Tony Hoare was awarded the Turing Prize for his many contributions and perhaps one of you, someday, might receive the same honor, perhaps for Brain-inspired Computing. Neural networks are remarkably versatile structures -- there are often multiple paths to the solution. Amongst these, the challenge is to seek the most efficient and most elegant and most beautiful algorithms. The opportunity for innovation is huge and unprecedented. Supervised learning, unsupervised learning, reinforcement learning as well as feedforward networks, recurrent networks all are up for grabs. The frontier is open. And, it is yours to capitalize. I predict that by December 2016 there will be at least one major breakthrough -- inventing a fresh new algorithm for neuromorphic computing -- from one of you. My hope is that each of you will be part of at least one such breakthrough.
What are we teaching at the Boot Camp?
We are teaching an overview of the ecosystem. We are teaching end-to-end examples with simple datasets so as to completely and comprehensively touch on the entire tool set. In particular, we are teaching how we are leveraging Caffe (a deep learning tool) to produce native-TrueNorth programs.
In the final week, we will provide clinics and help from both infrastructure and algorithmic perspectives to ensure that you are successful after the bootcamp which is really the whole point of this exercise.
How are we teaching it?
The approach is to teach the building blocks. We are looking forward to seeing what you produce with them. The possibilities are endless because permutations and combinations of the underlying build blocks are limited only by your imagination and creativity, which I know is infinite.
Boot Camp is about teaching the techniques of fishing, and not about catching one big, fat fish right away. it is about bricks, and not about buildings. It is end-to-end integration, and not fragmentation. It is simple, and not complex. It is practical, and not theoretical. It is about breadth across tools and techniques, and not about depth in a particular application. It is about ingredients and recipes and not tasty dishes. It is atomic and not molecular. It is sequential, where each step leads to the next level. It is slow, and not fast. It is flexible, and not a priori cast in stone.
Now, our team has worked very hard but much is to be done. You are experiencing a living and growing set of tools and not something that is polished and dead. We welcome all feedback, and will humbly accept it.
Sixty years ago, IBM released FORTRAN to the world. At that time, there were only a few hundred or at best low thousands of programmers. Today, we a team of 30 people, feel that our ecosystem has tripled in size instantaneously by adding 60 new partners. The hope is that each of you will become an ambassador for the project. A seed carrier. And, together we will draw in ever greater number of collaborators to create a spiral of value co-creation.
On our part, we are committed to continue to provide systems and tools to enable you to build more algorithms and applications.
This is a historic moment. Let us partner and together we can bring brain-inspired computers to society.
In conclusion, I invite the whole Brain-inspired Computing Team to join with me in welcoming our friends.
Finally, for all our guests, please join me in thanking the Brain-inspired Computing Team for countless nights, weekends, and their blood, sweat, and tears in making this event a reality.