I just got back from the Eastern APA, where I chaired an interesting author-meets-critics session on Paul Smolensky and Geraldine Legendre’s book, The Harmonic Mind (MIT Press 2006). The critics were Bill Ramsey and (jointly) Terry Horgan and John Tienson.
We all went to lunch after the session. Smolensky expressed surprise at having been selected as “the enemy” by Fodor and associates. The reason for his surprise is that unlike most connectionists, Smolensky does not try to eliminate classical symbolic representations and their recursive structures. Rather, he tries to implement them in connectionist networks, albeit in a non-classical way. (The book is all about that.)
(Background: in the late 1980s and early 1990s, there was an exchange of papers between Smolensky on one side and Fodor + Pylyshyn/McLaughlin on the other side. The debate was over whether connectionist architectures were a viable alternative to classical architectures within a theory of cognition. Smolensky said that the exchange simply ended with neither party being persuaded by the other.)
Is Smolensky right to be surprised? As far as I know, the debate started when Fodor and Pylyshyn 1988 argued that connectionism is insufficient to explain cognition (because of productivity, systematicity, etc.). Smolensky did respond, quite forcefully, to Fodor and Pylyshyn, so it’s not too surprising to me that Fodor would not be persuaded by Smolensky’s reply and would respond further.
However, Smolensky is right about a deeper point. Classicists like Fodor and Pylyshyn argued that connectionism is at best a theory of the implementation of classical representations and computations. But as far as I know, they have given few if any details on how such a connectionist implementation is supposed to occur in the brain. Their excuse seems to be that we don’t know enough about the brain. But that excuse, which was lame to begin with, becomes lamer and lamer every year. The fact is, we know an enormous (and increasing) amount about the brain, and what we know does not fit well (to put it mildly) with the way symbolic representations are implemented in standard (classical) computers. So an alternative is needed. Smolensky’s book offers precisely such an alternative! So shouldn’t classicists embrace Smolensky? I actually don’t know, because I don’t know enough about the details of Smolensky’s proposal. But I do agree that classicists need to develop a genuine theory of implementation.
Smolensky also noted that most connectionists ignore his proposal too. His explanation was that he never gave a practical demonstration, using a computational model and empirical data, that his approach is superior to tradititional connectionist approaches. Maybe so. Another possible explanation is that at least at first glance, his work does not really seem to be motivated by neuroscientific considerations, and the trend in connectionist circles is to look for neurological grounding for their theories. But this is just a speculation on my part.
To the connectionists out there: any comments on Smolensky’s proposal?
Gualtiero wrote, “Smolensky expressed surprise at having been selected as “the enemy” by Fodor and associates. The reason for his surprise is that unlike most connectionists, Smolensky does not try to eliminate classical symbolic representations and their recursive structures. Rather, he tries to implement them in connectionist networks, albeit in a non-classical way.”
Well, it didn’t seem that way to me back in the ’90’s. Then, he did appear to accept symbolic representations, but also to reject various other elements of the Classical picture, e.g. items having context independent content, having a combinatorial syntax that mirrors a combinatorial semantics. It sure didn’t sound as though he was merely trying to implement Fodor and Pylyshyn’s Classicism.
Happy new year Gualtiero!
I disagree with your criticism of Fodor and Pylyshyn’s reference to our ignorance of brain mechanisms.
Frankly, what do we know about how beliefs, desires etc. are realized in the brain? Nil. Zero, Nothing. What do we know about the brain realization of our higher cognitive competences (reading, mate choice, etc.). A bit: We know that some of these competences preferentially involve some parts of the brain rather than other, that they are disrupted following some brain injuries, etc. Really not that much.
More important, the claim that psychologists committed to a specific theory of mental states (say, LoT) “need” a theory of implementation strikes me as curious. I do not know exactly what you mean by “need”, but I take it you mean something like this: our credence in a psychological theory would be substantially decreased if we could not show how it is implemented in the brain.
Well, supposing that this is what you mean, I do not see any reason to grant the point. Scientists’ credence in (say, Mendelian) inheritance was not weakened by their ignorance of the molecular basis of this phenomenon (etc., for hundreds of sundry examples). Rather, if we have good reasons to endorse a classical theory of mental states, then it is the job of brain scientists to show us how the brain can implement this type of mental states.
Edouard
Edouard said:
I might agree with this conditional but the antecedent is not true. I’m reminded of Bechtel’s story about the guy who had the crazy theory of digestion based on the inputs and outputs of the digestive system: he was just way off but because he made certain assumptions about how food is turned into usable energy. And he knew more chemistry than we know neuroscience.
The few neuroscientists that care tend to look upon these debates (LOT
vs non-LOT) with bemusement, as the confidence and emotion are so high
but the data are so impoverished. That said, they do tend to think eliminativism about Fodor’s LOT is a reasonable hypothesis and don’t quite understand why that would be considered controversial (as they don’t really care all that much one way or the other) .
Neuroscientists largely look at connectionism as a strange fad from the 80s that was not helpful as a model of brain function. Hodgkin-Huxely style models, on the other hand, are ridiculously useful as they connect so directly with experimental predictions. I use connectionist nets, but only as classifiers for my neural data, not as any model of how the nervous system works. This is the norm, I believe.
Guys, Thanks for these great comments!
Ken, you are right and Smolensky is still not just trying to implement classicism. He is trying to implement classical representations and processes defined over them, though. This is better than what classicists themselves have done, and it is more classically inclined than what other connectionists have done.
Edouard, we do know a lot about the implementation of “beliefs” and “desires” in the brain, and they don’t look classical! Or if you prefer, we do know a lot about how the brain represents and processes information, and neural representations don’t look much like “classical” beliefs and desires. So if classicists want their theory to remain viable (i.e., not eliminated), they need to make some serious proposal on how to reconcile their theoretical constructs with what we know about the brain.
Eric, I take it that by “connectionism” you mean the kind of psychological modeling using neural networks made popular by Rumelhart and McClelland’s PDP volumes. But of course there is a broader sense of “connectionistm”, according to which it is the explanation of cognition in terms of neural networks (at some level). In this sense, all neuroscientists are connectionists. In fact, in this sense even classicists are (or ought to be) connectionists! It’s really just a debate about which kind of neural network explains cognition. The problem with classicism is that they have yet to provide any serious details on what kind of neural networks they think are doing the job, let alone grounding their theory in neuroscientific evidence.
OK, if you want to define connectionism to include biophysical models ok, but that is a strange use of the term for me. I use the term to refer to most nonspiking artificial neural network models as you’d find in PDP volumes. So in that sense of connectionism I meant that it hasn’t been very useful for neuro. If in your post you also meant to include a broader class that’s your perogative I guess (though it would be weird to call a network model of Hodkin-Huxely neurons a connectionist model–a pretty clear distortion of typical usage of terminology). At any rate, Smolensky is also clearly connectionist in the sense I (and most others) mean, the type that is looked at as a fad that wasn’t useful for neuroscience.
Operationally, we can define connectionist nets as the types of nets you find in Matlab’s neural network toolbox.
From my own sampling of the literature, people who call themselves “connectionist” use all kinds of different models, some more neurologically oriented than others. And some of the people who did McClelladn/Rumelhart PDP connectionism in the 1980s, including at least McClelland himself, seem to have drifted to some extent towards neuroscience. Plus, neurophysiologically realistic neural networks have been on the rise. Plus, a lot of people seem to use “connectionist system” and “neural network” interchangeably. By this usage, connectionism includes all neural network models. This is why I think it’s convenient to keep a generic notion of connectionism and then distinguish more specific versions.
What I take your substantive point to be is that Smolensky’s work falls into a non-neuroscience-based version of connectionism. From this point of view, it’s not too surprising that in a field that is looking more and more closely at neuroscience, his work gets ignored.
I don’t know if I had a substantive point.
I was just responding to some of the claims in the comments.
I have always been intrigued by Smolensky’s work, and quite surprised that it hasn’t received more attention. It may be a Grossberg-like effect, where the stuff is esoteric so very few people have taken the time to really understand it. He is my go-to guy when people want an example of someone working with ANNs that takes grammar quite seriously.
(Incidentally, I tend to not use the term connectionism, and don’t know anybody that does in my field, but I tend to use the term ‘ANN’ (artificial neural networks) for the sigmoid firing rate type models of yore, and ‘conductance-based models’ for the HH type models that try to reach toward the biophysics.
Of course they aren’t separate. ANN models are reducible to HH type models of a certain type.
Gualtiero, an example might be useful to support your claim that we know a lot about how the representations involved in higher cognition are implemented in the brain! Seriously, what do we know about their implementation?
Hi Gualtiero (and guys),
hope you´re all doing great.
Gualtiero, you wrote: “Smolensky is still not just trying to implement classicism. He is trying to implement classical representations and processes defined over them, though.”
Well, I haven´t read 2006 book, but in those “old papers” it was not so clear, whether Smolensky really tried to implement “classical representations”. At least his interpretation of classical representations did not perfectly match with the fodorian interpretation – and it was probably one source for that debate.
Ansku
(Sorry for the delayed response.)
For starters, we know that higher cognition is implemented (by and large) in the activity of neural networks, and that neural networks in the areas of the brain implementing higher cognition do not look like the circuitry of a digital computer (which would easily lend itself to implementing classical computations) but rather look a lot like the neural networks of areas of the brain we understand well, such as, say, the visual system and the hippocampus. Given this, the burden is on the classicist to make some explicit suggestions as to how classical processes might be implemented in the brain.
For more on why the brain is more relevant than classicists have tended to suppose, see some of my recent papers, such as “Computers” and “The Mind as Neural Software?” (available on my website).
Thanks, I agree.
These generalities really fall short of a concrete example. Sure we know our mind is realized in our brain…
Edouard,
In *The Cognitive Brain*, Ch.3, “Learning, Imagery, Tokens, and Types: The Synaptic Matrix” (see in particular p. 46), I give a concrete example of what constitutes a concept in the neuronal mechanisms of the brain. How does this fit in with your critique of concepts?
In Smolensky’s latest work, I do not take him to be showing how neural/connectionist network “implement” LOT representations (and processes defined over LOT representations). In terms of Marr’s 3-levels of analysis, the representations processed at the “algorithmic” level are vectors, but the structure of these vectors is not anything LOTish (and the processes are definitely not GOFAI-ish). According to Smolenksy, “symbolic” descriptions are relevant to explanation insofar as they figure in proofs regarding the capabilities of connectionist networks, capabilities that classicists claim are best explained by positing LOT, etc.. However, LOT representations have no “psychological reality.” As you know (and take exception to!), I think there is a different way to interpret Smolensky’s proofs, but this is how I think he sees matters.
I will be really thankful towards the author in this post for making this lovely and informative article live to put us. We really appreciate ur effort. Keep up the excellent work.