How Do You Define Connectionism, and How Do You Relate Connectionism to Associationism?

Some people, usually classicists, stress assimilate connectionism to associationism.  They do have a point:  “connectionism” was historically introduced and popularized by authors, such as Thorndike and Hebb, who were closely linked to associationism.  But as I explain in a recent review article  it seems to me that the assimilation of connectionism to associationism is also misleading.

In my experiene, most contemporary connectionists define “connectionism” more or less as the explation of cognition in terms of “neural networks”.  Such an explanation may or may not be associationist (and if it is, it may be so in varying degrees).  In fact, often contemporary connectionists identify Warren McCulloch and Walter Pitts as the founders of connectionism.  While this is historically incorrect, it’s interesting and relevant here because McCulloch and Pitts were hostile to associationism.  So it seems to me that connectionism in its general contemporary form is not committed to any particular version of associationism, although many contemporary connectionists remain sympathetic to some form of associationism.

Now a referee for a paper I submitted to a journal asked me to provide references backing up my distinction between generic connectionism (which is not committed to associationism) and associationist connectionism.  The problem is, off the top of my head I don’t know what any particular author says about this.

To complicate matters, I’m out of the country until June 24 and for reasons beyond my control I am supposed to turn in the revised paper within a few days.

Question 1: Is my distintion between generic and associationist connectionism on the right track, or am I missing something?

Question 2: Does anyone know of any helpful quotes defining/discussing either connectionism or associationism or both (with full references, of course)?  Examples of recent connectionists who are either pro or against associationism would be great; references to recent discussions of these matters might help too.

5 Comments

  1. Mark Collier may know about this. His dissertation title was ‘Newton of the Mind: an Examination of Hume’s Science of Human Nature’ and had a heavy connectionist focus (under the Churchlands).

    Recent stuff he’s done:
    “Hume’s Causal Psychology and the Limits of Associationionism”, Central Division Meeting of the American Philosophical Association, Chicago, April 18, 2008.

    “Why History Matters: Associations and Causal Judgment in Hume and Cognitive Science”, Journal of Mind and Behavior, 28, 2007, 175-188.

    I just send Mark a message on Facebook hopefully he’ll find this call for help. 🙂

  2. Thanks for the note, Eric. And great question, Gualtiero. Off the top of my head, I’m not aware of any explicit discussion of this issue by connectionists. But there was some discussion of this point in an exchange between Fodor and Churchland in the book ‘The Churchlands and Their Critics’.

    I have explored this issue to some degree in my work. I see connectionism as a development of classical associationism (as developed by figures such as Hume). In short (too short): most connectionist models update weights according to statistical dependencies in the data set. That is enough to make them associationists (in the classic sense) to me.

    Hope this helps.

  3. Hi Gualtiero,

    I think your distinction between generic and associationist connectionism is spot on.

    You could use Fodor’s definition of associationism (The Modularity of Mind, p. 27), which is any theory that accepts the following principle bits of explanatory apparatus (and not much else, if anything):
    a) A set of elements to be associated (e.g. reflexes, ideas)
    b) A relation of association defined over those elements
    c) Laws of association = principles for how experience determines which particular elements become associated
    d) Parameters of (a) and (b) (e.g. associative strength)
    (This is a paraphrase. Email me if you want the whole shebang.)

    On that definition, it seems pretty clear that e.g. connectionists who are primarily engaged in modeling actual neural hardware aren’t associationist (nor are they anti-associationist). Some of Grossberg’s work would fall in this category, for instance (e.g. doi:10.1016/j.cogpsych.2009.07.002 ), or even models of orientation selectivity and the like. (But maybe this work isn’t “connectionist”? Well, I’d agree with you that there’s a generic “connectionism” which just means neural network modeling.)

    Interestingly, Fodor goes on to postulate an intermediate sort of “computational associationist” who allows that the relation of association defined in (b) can be more complex than the classical “mutual attraction”, e.g. it could be Boolean (pp. 30-35). He ultimately repudiates this intermediate position as lacking a plausible learning theory, but some current connectionist models could be thought of as filling this gap (e.g. dynamic liquid association, Morse & Aktius doi:10.1016/j.neunet.2008.10.008 , or our SINBAD model http://homepage.mac.com/ancientportraits/drsite/associationism.pdf , see esp. section 7.) This stuff is associationist on Fodor’s broad definition, but it’s very different from classical associationism. So within your connectionist associationism category, one could distinguish between classical (pairwise, linear) and computational (multivariate, nonlinear) associationism.

    This isn’t recent, but in responding to Fodor and Pylyshyn, Smolenksy makes remarks of the sort that might be helpful for you. For example: “On the PTC [Proper Treatment of Connectionism] account, simple associationism is a particularly impoverished and impotent corner of the connectionist universe…. To equate connectionism with simple associationism is no more appropriate than equating Classical symbolic theory with Aristotelean logic.” (p 165 of “Connectionism, Constituency and the Language of Thought” in Connectionism: debates on psychological explanation, ed. Cynthia & Graham Macdonald, Blackwell 1995.) I’m not exactly sure what he means by “simple” associationism, but I think it’s classical associationism. (And of course, F&P don’t agree with Smolensky’s denial of the dichotomy between inadequate associationism and “mere” implementation.)

    Hope that helps a bit. I’ll stop because I’m out of characters.

  4. Martin Roth

    Hi Gualtiero,

    Along the lines of Dan’s comment, perhaps this will be useful to you. It’s from Smolensky’s “Principle-Centered Connectionist and Generative-Linguistic Theories of Language,” in The Harmonic Mind (vol. 2.):

    “Finally, consider the most controversial of the PDP principles, (1c): Knowledge acquisition results from the interaction of innate learning rules, innate architectural features, and modification of connection strengths with experience. At first glance, this principle would seem to imply that knowledge acquisition consists entirely in the statistical associations gathered through experience with some task. And this does characterize the early connectionist tabula rasa models with simple Hebbian-like learning rules (e.g., Kohonen 1977; Stone 1986) and input-output representations that have no built-in domain structure (e.g., local representations as in Hinton 1986).

    However, much of the most noteworthy progress in connectionist learning is more appropriately characterized by another claim: connectionist knowledge acquisition consists in fitting to data the parameters of task-specific knowledge models. To varying degrees, this describes somewhat more recent networks (e.g., Rumelhart et al. 1996; Smolensky 1996c) in which more sophisticated error functions that embody Occam’s razor force closeness-of-fit to data to compete with simplicity of generalizations or knowledge….

    Thus, a commitment to the PDP principles…does not per se constitute a commitment regarding the degree to which…innate learning bias applies to human cognition…the indeterminism of the basic connectionist commitments towards most central issues of cognitive theory forces a major choice of research strategy” (p. 479).

Comments are closed.