The spawn of SPAUN

SPAUN (Semantic Pointer Architecture Unified Network) is a model of a functioning brain out of Chris Eliasmith’s group at the University of Waterloo. I first met Chris almost 15 years ago when I visited Charlie Anderson at Washington University, where Chris was a graduate student. He was actually in the philosophy department (and still is) with a decidedly mathematical inclination. SPAUN is described in Chris’s paper in Science (obtain here) and in a forthcoming book. SPAUN can perform 8 fairly diverse and challenging cognitive tasks using 2.5 million neurons with an architecture inspired by the brain. It takes input through visual images and responds by “drawing” with a simulated arm. It decodes images, extracts features and compresses them, stores them in memory, computes with them, and then translates the output into a motor action.  It can count, copy, memorize, and do a Raven’s Progressive Matrices task. While it can’t learn novel tasks, it is pretty impressive.

However, what is most impressive to me about SPAUN is not how well it works but that it mostly implements known concepts from neuroscience and machine learning. The main newness was putting it all together. This harkens back to what I called the Hopfield Hypothesis, which is that we already know all the elementary pieces for neural functioning. What we don’t know is how they fit and work together. I think one of the problems in computational neuroscience is that we’re too timid. I first realized this many years ago when I saw a talk by roboticist Rodney Brooks. He showed us robots with very impressive capabilities (this was when he was still at MIT)  that were just implementing well-known machine learning rules like back-propagation. I recall thinking that robotics was way ahead of us and that reverse engineering may be harder than engineering. I also think that we will likely construct a fully functioning brain before we understand it. It could be that if you connect enough neurons together that incorporate a set of  necessary mechanisms and then expose it to the world, it would start to develop and learn cognitive capabilities. However, it would be as difficult to reverse engineer exactly what this constructed brain was doing as it is to reverse engineer a real brain. It may also be computationally undecidable or intractable to a priori determine the essential set of necessary mechanisms or the number of neurons you need. You  might just have to cobble something together and try it out. A saving grace may be that these elements may not be unique. There could be a large family of mechanisms that you could draw from to create a thinking brain.




2 thoughts on “The spawn of SPAUN

  1. a bit off topic but i remember being a bit amazed when hopfield showed spin glasses could represent neural nets, and you could have a ‘potential landscape’, attractors etc. And these are basically the same as ‘fitness landscapes’ of s wright in genetics. so you can map these systems into each other, and even look at the relations of stochastic processes to newtonian mechanics (forces and potentials). (then you get the questions in genetics of whether fisher’s theorem is the same as the entropy law, or whether its a newtonian mechanical law; same in economics—is ‘general equilibrium’ some sort of minimization of potential forces, or a maximum entropy solution).

    later hornik and others showed neural nets are universal computers, or approximators (weirstrauss, fourier series…) and ross et al in pnas you could use simple chemical reactions to implement them. azimuth blog of baez shows related approaches.

    so what does this mean? dunno. presumably with AI one can get robots to download JSTOR articles and such to avoid emotional trauma—unless they develop ’empathy’ and are liable for prosecution too.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s