From Associative Memories to Deep Networks and from Associative Memories to Universal Machines

Panelists: Profs. Christos Papadimitriou (Columbia), Tomaso A. Poggio (CBMM, MIT) and Santosh Vempala (Georgia Tech) Moderator: Kenneth Blum Abstract: About fifty years ago, holography was proposed as a model of associative memory. Associative memories with similar properties were soon after implemented as simple networks of threshold neurons by Willshaw and Longuet-Higgins. It turns out that the recurrent Willshaw networks were very similar to today’s deep nets. Thinking about deep learning in terms of associative networks memories a more realistic and sober perspective on the promises of deep learning and on its role in eventually understanding human intelligence.