IBM Israel Research Seminars

A fundamental issue for almost all aspects of pattern recognition, machine learning and data mining, is the extraction simple task relevant representations of complex data. The specific solutions to this very general problem obviously vary from one setting to another and are extremely context dependent. Yet, we can formulate powerful general information theoretic principles that lead to concrete practical algorithms for prediction, clustering, feature extraction, dimension reduction, and other similar problems. These algorithms are optimal in a well defined sense and provide a unified theoretical framework for unsupervised as well as many aspect of supervised learning. The key idea is in formulating the proper tradeoff between compression and prediction, also known as the Information Bottleneck method. This method amounts to finding approximate sufficient statistics - properly defined - and has interesting connections to other general frameworks, such as the Minimum Description principle (MDL). When applied to temporal processes and dynamical systems it provides an intriguing new approach for quantifying adaptive behaving systems.

About the Speaker
Professor Naftali Tishby is an active researcher at the School of Computer Science and Engineering and the Interdisciplinary Center for Neural Computation at The Hebrew University, Jerusalem. He is a founding member of the Interdisciplinary Center for Neural Computation (ICNC) and one of the key teachers of the well known computational neuroscience graduate program of the ICNC. He received his PhD in theoretical physics from the Hebrew university in 1985 and has been a research member of staff at MIT, Bell Labs, AT&T, and NECI since then. His current research is on the interface between computer science, statistical physics, and computational biology. He introduced various methods from statistical mechanics into computational learning theory and machine learning and is interested in particular in the role of phase transitions in learning and cognitive phenomena. More recently he has been working on the foundation of biological information processing and has developed novel conceptual frameworks for relevant data representation and learning algorithms based on information theory, such as the Information Bottleneck method and Sufficient Dimensionality Reduction.