The standard pop story about connectionism in philosophical circles goes somewhat as follows: connectionism is an alternative to computationalism, or at least to classical computationalism, that emerged in the 1980s. This story is largely a myth due in part to the ambiguity of the term “connectionism”.
“Connectionism” has come to mean several different things.
1. In its original sense, “connectionism” meant explanation of behavior in terms of changing connections between stimuli and behavior (Thorndike 1932) or between neurons (Hebb 1949). In this sense, connectionism is a close cousin of associationism.
2. Some people use “connectionism” for a “neurally inspired” version of computationalism, according to which behavior is explained by neural computation. This idea goes back to McCulloch and Pitts 1943 (who did not use the term “connectionism”). Someone who is a connectionist in this second sense need not be a connectionist in the previous one (McCulloch and Pitts were not).
3. Probably the most common meaning of “connectionism” these days is explanation of behavior in terms of neural networks. This is more general than the previous sense because it doesn’t make the assumption that the neural networks in question perform computations. This idea (though not the term “connectionism”) goes back to the discovery that the cognitive functions of the brain are fulfilled by networks of neurons (beginning of the 20th century). In a mathematically sophisticated form, this version of connectionism goes back at least to the mathematical biophysics of Nicolas Rashevski (ca 1930s).
Morals:
(A) Connectionism is not necessarily an alternative to computationalism, as all versions of connectionism that do no explicitly reject computationalism are consistent with computationalism.
(B ) Connectionists need not be associationists, although the people who originally introduced the term “connectionism” did so for something relatively similar to associationism, and today many people who are connectionists in sense 2 or 3 are also connectionists in sense 1.
(C) Connectionists need not be committed to computationalism either, as connectionism in its most general sense (3 above) is independent of computationalism.
(D) Connectionism (in any reasonable sense) originated WAY BEFORE the 1980s. What happened in the 1980s is that connectionism became more popular in psychology and AI, and so philosophers noticed it.
(E) Connectionism in its most general sense (3 above) is a truism. No one in their right mind should deny it.
The Ambiguity of “Connectionism”
10 Comments
Comments are closed.
This is a very nice corrective – it can be frustrating dealing with the various misunderstandings that you mention! Here’s another common misunderstanding: often people seem to think that connectionism is just backprop nets, or perhaps backprop nets “plus or minus a bit”. In fact, backprop nets are a minute part of connectionism, in any reasonable sense. This is a case of a standard example mistaken for the whole kit’n’kaboodle.
One quibble about your (E) – it’s probably OK in the most general sense of “explanation” and the most general sense of “behaviour”. But some (of course) maintain that neural networks are not at the right level to explain behaviour. E.g. perhaps the explanation of behaviour must make reference to content, and content attaches only to a higher level of explanation than the neural network level.
Hi, Gualtiero,
A couple of thoughts:
(A) Back in the 1980’s philosophers recognized that the kinds of
networks they were thinking about could implement what people called
GOFAI computations, and so I think it was pretty quickly recognized
that connectionism wasn’t an alternative to computationalism, just a
different kind of computation.
(B) That said, the “different kind” is very akin to associationism
(hence the debate between (Pinker and Pylysyn) and (Fodor and
McLaughlin).)
I’m interested in (C), if there can be a network that involves some
causal process that isn’t a computation, in such a way that the result
can do things that computational connectionist networks can’t do.
(D) Here, the pop story I remember is that (i) Minsky proved that
perceptrons couldn’t do anything interesting (since they couldn’t
compute XOR?) but (ii) in the late 1970’s it turned out that a pretty
trivial variation on perceptrons could do all kinds of cool things. So
while the structure of the idea was old, in the 1980’s what was new
was all this cool stuff you could do.
(E) Agreed, brains cause behavior, and brains are made of networks of
neurons. We explain behavior in lots of ways other than by appeal to
the brain, so I’d want to say: “given what we know about brains and
behavior, you should agree that neural networks can explain
behavior”.
Cheers,
Tony
You should read William Uttal’s book, “Neural theories of mind: Why the mind brain problem may never be solved.” It’s a very long way from neural networks to behavior.
Thanks for the comments.
Dan, I agree with everything you say.
Tony:
(A) You are right but there are still people who haven’t learned that lesson.
(B ) Like I said many (most?) contemporary connectionists in sense 2 and 3 are also quasi-associationists, though their associationism is more sophisticated than classical associationism. In addition, you can be a connectionist in sense 2 or 3 without being a connectionist in sense 1.
Connectionism in the third sense goes much farther back than Rashevsky.
One relatively well-known example is Freud’s “Project for a Scientific Psychology” from about 1895. I think one of his colleagues, Breuer, may also have had a take on connectionism.
There was also some Italian guy in the late 19th Century who had a version of this. Alas, he wrote in Italian and 15 years ago, or whenever I found this, I didn’t know anyone who read Italian. I can probably dig that up out of my files, if you’re interested.
Then, there is a less well-known treatment of this third kind of connectionism in Herbert Spencer’s third edition of his Principles of Psychology. There is even a nice drawing of a completely connected two layer network and a discussion of things like size invariance of perception.
Ok. The name is Eugenio Tanzi. I think this is it:
Tanzi, E. (1898). Sulle modificazione morfologiche funzionali dei dendriti delle cellule nervose. Rivista di Patologia Nervosa e Mentale, 3, pp. 337–359.
I don’t really read Italian, but it looks like this paper is on the morphological and functional modifications of the dendrites of nerve cells.
Right after the neuron doctrine came out, around what 1891, about a billion neuroscience types came up with the idea that synaptic modification was somehow the neurobiological basis of learning and memory. All one billion of them had the idea that synaptic modification of one sort or another might, as we would say, realize the psychological processes of the association of ideas.
Right, that’s what I had in mind when I said connectionism in its most general sense goes back to the beginning of the 20th Century (it was a rough approximation). Thanks for making my point explicit and more precise.
I had never heard of Tanzi, though.
Oh, I forgot to note that Spencer’s 3rd edition was 1872. So, the connectionist neural network/associationism kind of project was, one might say, “in the air,” for a while.
Moi,
and greetings from the 6th International Conference on Conceptual Change. Anyway, I am interested in (B ).
Of course, connections do not “need” to be associationists… Gualtiero, what do you mean by “associationism”, “quasi associationism”, the “classical associationism” and the more “sophisticated versions of associationism”? Descriptions of neural networks models (and the features of their learning algorithms), theories of learning at some “higher level” of description etc.?
I am not sure, whether this is an innocent question:).
Also, I´d like to emphasize (as someone already did) that not all neural network models of brains are meant to be explanatory ones. Perhaps Dan Ryder´s SINBAD is – I have always thought it might be – and perhaps some other models are meant to be biologically “realistic enough”, but most aren´t – for several reasons.
Anna-Mari,
Thanks for your questions. Classical associationism is the idea that cognition is explained by the association of ideas. This goes back to the British empiricists (and to some extent Aristotle). In the 19th century, people started speculating that such associations were embodied in neural mechanisms. Starting with Thorndike, a similar view, with emphasis on learning, was dubbed “connectionism”. Starting with Rosenblatt in the 1960s, specific neural network models were proposed. Beyond that, neural network models became more powerful and sophisticated by adding backpropagation, recurrent connections, etc. Since such “associationism” relies on more powerful mechanisms, it is more “sophisticated” than classical associationism.