AISTATS 2015 Keynote Presentations

May 9 (Saturday) 17:00 - 18:00

The Ebb and Flow of Deep Learning: a Theory of Local Learning

Abstract In a physical neural system, where storage and processing are intertwined, the learning rules for adjusting synaptic weights can only depend on local variables, such as the activity of the pre- and post-synaptic neurons. Thus learning models must specify two things: (1) which variables are to be considered local; and (2) which kind of function combines these local variables into a learning rule. We consider polynomial learning rules and analyze their behavior and capabilities in both linear and non-linear networks. As a byproduct, this framework enables the discovery of new learning rules and important relationships between learning rules and group symmetries.

Stacking local learning rules in deep feedforward networks leads to deep local learning. While deep local learning can learn interesting representations, it cannot learn complex input-output functions, even when targets are available for the top layer. Learning complex input-output functions requires instead local deep learning, where target information is transmitted to the deep layers, thereby raising two fundamental issues: (1) the nature of the transmission channel; and (2) the nature and amount of information transmitted over this channel. This leads to the class of deep targets learning algorithms, which provide targets for the deep layers, and its stratification along the information spectrum, illuminating the remarkable power and uniqueness of the backpropation algorithm. The theory clarifies the concept of Hebbian learning, what is learnable by Hebbian learning, and explains the sparsity of the space of learning rules discovered so far and the unique role backpropagation plays in this space.

Bio Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Chancellor's Professor in the Department of Computer Science, Director of the Institute for Genomics and Bioinformatics, and Associate Director of the Center for Machine Learning and Intelligent Systems at the University of California Irvine. The long term focus of his research is on understanding intelligence in brains and machines. He has pioneered the development and application of deep learning methods to problems in the natural sciences such as the detection of exotic particles in physics, the prediction of reactions in chemistry, and the prediction of protein secondary and tertiary structure in biology. He is the recipient of the 1993 Lew Allen Award at JPL, the 1999 Laurel Wilkening Faculty Innovation Award at UCI, a 2006 Microsoft Research Award, the 2010 E. R. Caianiello Prize for research in machine learning, and a 2014 Google Faculty Research Award. He is and Elected Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.

May 10 (Sunday) 08:30 - 09:30

How Deep Learning Transforms Chinese Internet Search

Abstract Since 2012, the Chinese internet giant Baidu has launched a significant effort to develop deep learning technologies to transform its search engine to the next stage. The company established Institute of Deep Learning (IDL), the first research team of Baidu and later expanded the research force to a bigger organization Baidu Research. By leveraging big data and massive parallel computation on CPU and GPU servers, Baidu researchers have achieved transformative improvements to Baidu's core business, such as web search, online advertising, speech recognition, computer vision, natural language processing, smart data center, robotics, etc.. In this talk I will showcase some of the highlights, and introduce the underlying technologies. These technologies have been shaping up a foundation for imaginative long-term innovation of Baidu. In the end I will discuss some thoughts about the future directions of deep learning.

Bio Dr. Kai YU is the Deputy Managing Director of Baidu Research, the Head of Baidu Institute of Deep Learning (IDL), and a Senior Engineering Director of Baidu. In 2013 and 2014, his team has won 3 Highest Achievement Awards of Baidu, for significant contributions to speech recognition, computer vision, online ads, and web search ranking. Dr. Yu is elected to China "Top 1000 Talents", a prestigious group of technology leaders of the country. Dr. Yu has published a number of papers in NIPS, ICML, AISTAT, CVPR, ICCV, ECCV, SIGIR, SIGKDD, etc., with more than 8000 citations. He and his colleagues have received many awards including the Best Paper Runner-up Award of ICML-2013, the First Place of PASCAL VOC 2009, and the First Place of ImageNet Challenge 2010. He is an Adjunct Professor of Nanjing University, Beijing University of Posts and Telecommunications, and Institute of Computing Technology, Chinese Academy of Sciences. Before joining Baidu, he worked at Microsoft, Siemens, NEC Labs America, and was an Adjunct Faculty at Department of Computer Sciences at Stanford University, teaching "CS121: Introduction to AI". He received his B.Sc and M.Sc degrees from Nanjing University, China, and his Ph.D. degree from University of Munich, Germany.

May 11 (Monday) 08:30 - 09:30

Data and/as Journalism

Abstract As statisticians know well, data and data technologies have profound social and political relevance -- almost every aspect of our lives can "rendered" in data. These data tell us stories about who we are and how we live, but, as products of human attention, innovation and memory, their perspective is not neutral and the stories they tell are often incomplete, open-ended and biased. Data can either reinforce or challenge systems of power and so, in a very real sense, our democracy depends on the public being able to think critically about these technologies, telling the good stories from the bad. As "truth-tellers, sense-makers and explainers of last resort," to quote my colleague Emily Bell, journalists needs to understand the working of data in society, and the ways in which data, code and algorithms function. In this talk, I will look at how journalism is adapting to a world of "Big Data" and "Data Science." From a pedagogical perspective, journalism acts as a kind of "integrator" of real world problems with more technical statistical and computational perspectives. The practice of journalism fills a gap between lived experience and data, its analysis and its visualization -- and might even expand the way we teach statistics.

Bio Mark Hansen is a Professor in the Columbia Graduate School of Journalism, where he heads the Brown Institute for Media Innovation. Prior to that he was a Professor in the Department of Statistics at UCLA and a Member of the Technical Staff at Bell Laboratories. He is also co-founder of the Office for Creative Research, a design group that has exhibited work at the Museum of Modern Art in New York, the Whitney Museum, the Centro de Arte Reina Sofia, the London Science Museum, the Cartier Foundation in Paris, and the lobbies of the New York Times building and the Public Theater (permanent displays) in Manhattan. He received a B.S. in Applied Math from UC Davis and a Ph.D. in Statistics from UC Berkeley.

May 12 (Tuesday) 08:30 - 09:30

Scaling Machine Learning and Statistics for Web Applications

Abstract Scaling web applications like recommendation systems, search and computational advertising is challenging. Such systems have to make astronomical number of decisions every day on what to serve users when they are using the website and/or the mobile app. Machine learning and statistical modeling approaches that can obtain insights by continuously processing large amounts of data emitted at very high frequency by these applications have emerged as the method of choice. However, there are three challenges to scale such methods : a) scientific b)infrastructure and c) organizational. I will provide an overview of these challenges and the strategies we have adopted at LinkedIn to address those. Throughout, I will illustrate with examples from real-world applications at LinkedIn.

Bio Big data analyst with 15 years of experience developing and deploying state-of-the-art machine learning and statistical methods for improving the relevance of web applications. Have worked in various positions: chief scientist of large projects, managed small and highly technical teams and also experienced in managing large teams (almost all PhDs). Most recently, leading a team of 60+ scientists and developers at LinkedIn. Also experienced in conducting new scientific research to solve notoriously difficult big data problems, especially in the areas of recommender systems and computational advertising. Fellow of the American Statistical Association, Member Board of Directors for SIGKDD, program chair of KDD in the past, associate editor of two top-tier journals in Statistics, regularly serves on senior program committees of top-tier conferences like KDD, NIPS, CIKM, ICDM, SIGIR, WSDM.