Dharmendra S. Modha

 Today, we had a beautiful talk by Dr. Marc-Oliver Gewaltig. The quality of research on-going at Honda Research Institute is absolutely world-class. Previously, we have had the honor to host Prof. Dr.- Ing. habil. Edgar Koerner who is now the Chairman of the Honda Research Institute.

TITLE: Structural Principles of Cortical Memory and Information Processing

ABSTRACT: The performance, efficiency, and robustness of neural systems are unrivalled even by modern super-computers. The human brain comprises of 10^15 neurons and synapses, all operating concurrently and each at a dedicated position. But even insect brains with less than 10,000 neurons are unrivaled by modern technology. We propose that the structural and architectural principles of the brain are a main ingredient to the brain’s unattained performance. In this talk we present results, which illustrate at different levels, how structural and architectural principles of neural structures enable and improve information processing and memory performance. At a global level, architectural principles like modularity and laminarization enable both, bottom-up driven and top-down mediated information processing. We substantiate these ideas in a dynamic columnar cortex model with spiking neurons, which robustly recognizes composite objects.   Next, we discuss an extension of this model that uses so-called Liquid-state machines to equip neurons with spatio-temporal receptive fields. Here, the structure of the recurrent connectivity can be optimized to reduce connections that do not contribute significantly, thus reducing the metabolic (and computational) costs of the network. We then extend this idea to a model of correlation based structural plasticity which optimizes the formation and consolidation of memory traces, using cortico-hippocampal interactions. Finally, we give a short overview over our neural simulation tool NEST which has been chosen as priority target for the next super-computer in Kobe, Japan.

BIO: Dr. Marc-Oliver Gewaltig studied Physics and Computational Neuroscience at the University of Bochum. After his graduation he went to the University of Freiburg to pursue his Ph.D. with Prof. Ad Aertsen. After his dissertation in 1999 he joined Honda R&D Europe GmbH as Senior Scientist. In 2003, Marc-Oliver Gewaltig was appointed Principal Scientist and Project Leader of the newly founded Honda Research Institute Europe GmbH in Offenbach, Germany. His research focuses on information processing in the visual cortex of mammals in general and on spike-based processing in columnar cortical architectures in particular. His second research focus is on efficient algorithms and methods for the simulation of large neuronal systems. In 2001, Dr. Gewaltig co-founded the NEST Initiative, a consortium, dedicated to advancing our understanding of neural simulation technology, for which he is also the coordinator. Since 2004, the NEST Initiative releases the popular simulation software NEST.

Today, we had a genuinely great talk by Professor Bob Knight.

TITLE:  Neural Oscillations Track Behavior in the Human Cortex

ABSTRACT: Since the discovery of the EEG in 1920’s, neurophysiological dogma for the ensuing 80 years stated that the human cortex did not generate reliable rhythms above 50-60 Hz. However, findings over the last decade  reliably report neural activity up to 250 Hz in the human cortex. This high frequency activity is the key neural response measuring cortical activation in humans. We record the human electrocorticogram (ECoG) from subdural electrodes implanted directly on the cortex of neurosurgical patients. We have observed that every cognitive process examined including language, attention, memory and motor control generates high frequency oscillatory activity in the range of 70-250 Hz (high gamma, HG). Importantly, the HG band of the human ECoG has the most precise spatial localization and task specificity of any frequency observed. For instance, during linguistic processing, HG precisely tracks the spatio-temporal evolution from comprehension in posterior temporal areas to production structures in the left frontal region. HG a precisely tracks the time course of the behavior needed to comprehend the word, select a noun and articulate a response all occurring within a second. Similar findings of HG activity are observed for working memory, contextual processing and a host of other human behaviors.  Importantly the HG response can be reliably extracted at the single trial level providing a powerful tool for studying the physiology of human behavior. HG is also phase locked to the trough of theta rhythms in the human neocortex parallel to findings of HG-theta coupling  observed in animal hippocampus and cortex. HG-theta coupling occurs in a task specific manner with different cognitive tasks eliciting unique distributed spatial patterns of HG-theta coupling.  These results indicate that transient coupling between low- and high-frequency brain rhythms provide a mechanism for effective communication in distributed neural networks engaged during cognitive processing in humans. Taken together the results indicate that HG activity provides a powerful new tool for understanding the real-time cortical dynamics subserving cognition in humans.

BIO: Dr. Knight received his BS in Physics from the Illinois Institute of Technology in 1970 and his MD from Northwestern University Medical School in 1974.  He did Neurology residency at the University of California at San Diego followed by post-doctoral work at the Salk Institute for Biological Studies in the area of human neurophysiology. He was a faculty member in the Department of Neurology at the University of California at Davis School of Medicine from 1980-1998. In 1998 Dr. Knight moved to UC Berkeley to lead a program in human neuroscience. He is currently the Evan Rauch Professor of Neuroscience and Director of the Helen Wills Neuroscience Institute at UC Berkeley.  His laboratory utilizes electrophysiological and MRI techniques in neurological and neurosurgical patients to delineate the role of prefrontal cortex in human cognitive and social behavior. His laboratory also records electrocorticographic activity  directly from the human cortex from neurosurgical patients in an effort  to delineate the cortical mechanisms supporting behavior. His laboratory is also involved in developing neural prosthesis devices for motor and language restoration in patients with stroke or spinal cord injury using electrocorticographic recordings. Dr. Knight received the Jacob Javits Award from the National Institute of Neurological Disorders and Stroke for distinguished contributions to understanding neurological disorders, the IBM Cognitive Computing Award and the 2008 German Humboldt Prize in Neurobiology.

Today, I had the good fortune to spend time with Manfred Warmuth who is a leading scientist in Computational Learning Theory.

ABSTRACT: Multiplicative updates multiply the parameters by nonnegative factors. These updates are motivated by a Maximum Entropy Principle and they are prevalent in evolutionary processes where the parameters are for example concentrations of species and the factors are survival rates. The simplest such update is Bayes rule and we give an in vitro selection algorithm for RNA strands that implements this rule in the test tube where each RNA strand represents a different model. In one liter of the RNA soup there are approximately 10^20 different strands and therefore this is a rather high-dimensional implementation of Bayes rule.

We investigate multiplicative updates for the purpose of learning online while processing a stream of examples. The “blessing” of these updates is that they learn very fast because the good parameters grow exponentially. However their “curse” is that they learn too fast and wipe out parameters too quickly. We describe a number of methods developed in the realm of online learning that ameliorate the curse of these updates. The methods make the algorithm robust against data that changes over time and prevent the currently good parameters from taking over. We also discuss how the curse is circumvented by nature. Some of nature’s methods parallel the ones developed in Machine Learning, but nature also has some additional tricks.