In a recent post, I criticized some passages from Alva Noe’s book, Out of Our Heads. I’d like to clarify some details.

First, it was pointed out to me that my tone was disrespectful. I am truly sorry about that. My comments were only aimed at the quoted claims, not the book as a whole, let alone Noe himself.

Second, I’ve spent the last decade of my life studying and publishing on cognition and computation, information processing, and related matters. When I encounter dubious claims on these topics, I feel it is my duty to correct them and try to raise the level of the debate. I’ve done it before and I will continue to do it.

Third, since I found some of Noe’s statements ambiguous, he was kind enough to email me and clarify what he meant, which in turn allows me to clarify what I object to. I greatly appreciate his help and integrity in clearing things up.

Here is the quote I commented about in my previous post: “For Hubel and Wiesel, … cells were understood to be specialized in order to be able to “stand for” and thus represent features. This application of information theory to the brain was not new when Hubel and Wiesel set to work… Rafael Lorente de No had represented neural relations as networks already in the thirties, and his treatment had a direct influence on the work of Warren McCulloch and Walter Pitts, and through them, John von Neumann. (Walter Freeman, the neuroscientist, likes to say that in a way Lorente de No is the godfather of the computer)” (p. 156).

In the book, this quote is followed by a reference to Shannon and the “mathematical theory of information.”

Noe: When I mention Shannon by name I explicitly use a different phrase to refer to his theory; I speak of him as the developer of the *mathematical* theory of information and thus I contrast his contribution that of the more general ideas about information processing that have been my main focus in passages in question.

GP: Point taken. Then my objection becomes that few people will perceive the intended contrast between “theory of information” (which Noe uses for the “more general ideas about information processing”) and “mathematical theory of information” (which he uses for Shannon’s theory). When “theory of information” is used without further qualification, what is generally understood is Shannon’s theory. But this by itself is a minor point – just an example.

Noe: I am actually very interested in the historical question. In 1929 Lorente de No wrote a paper (“Studies in the structure of the cerebral cortex: I. The area entorhinalis”) in which he represented neurons as standing in feedback loop relations to each other. This was brought to my attention by Walter Freeman. Ramon y Cajal forbade him to publish the paper; Ramon y Cajal thought the feedback notion was incoherent, for a cell couldn’t tell input from output. Lorente de No respected his professor’s wishes; he only published the paper in 1934, in a German physiology journal (not in an anatomy journal). According to Freeman, Pitts made use of this idea when developing his Boolean approach [to neural networks]… What I say — and you have given me no reason to retract — is that Lorente de No is the GODFATHER of the whole approach, in that he recognized that it was reasonable to think of feedback systems.

GP: Noe’s clarification of what he meant allows me to explain why his suggestion is untenable. There are several problems.

Problem 1: Feedback is not relevant to the topic at hand. I knew Lorente de No talks about closed loops of neuronal activity. It surprises me that he talked about “feedback” between neurons. But whether Lorente de No uses the word “feedback” (I haven’t checked, because I don’t have the primary source) doesn’t change my point. In fact, if he does talk about “feedback”, it makes the connection with Pitts less plausible, since McCulloch and Pitts 1943 do not talk about feedback between neurons. They talk about nets “with circles” (i.e., closed loops). But feedback certainly became a central concept in the cybernetic movement, and McCulloch discusses its importance in later works. However, there were many other people talking about feedback systems in the 1930s. For instance, McCulloch cites a 1934 paper by H.S. Black entitled “Stabilized Feedback Amplifiers”. So the fact that Lorente de No talked about feedback is not enough to conclude that he influenced McCulloch (let alone Pitts) on feedback. McCulloch cites Lorente de No as influencing him on the idea that loops of neurons are functionally important, but not on the idea of feedback (cf. “Recollections of the Many Sources of Cybernetics”). At any rate, McCulloch and Pitts 1943 do not talk about feedback, so feedback is not especially relevant here.

Problem 2: The idea of neural networks is more general than the idea of closed circuits of neurons, although networks with recurrent connections are surely an important case. So you can’t just go from “Lorente de No talked about feedback between neurons” to “Lorente de No talked about neural networks”.

Problem 3: Talking about feedback between neurons, or even talking about neural networks informally, is a far cry from developing a mathematical theory of neural networks (with or without closed loops), which is what McCulloch and Pitts did and what is relevant here (and relevant to the design of computers). Before McCulloch and Pitts, there were other people who developed mathematical theories of neural networks; notably, Rashevsky and his mathematical biophysics group. They are the people who influenced Pitts, and from whom Pitts took the question of characterizing the behavior of closed loops of neurons (as he says in one of his papers).

Problem 4: Noe has yet to provide evidence for the implausible claim that Pitts knew much if anything about Lorente de No and his ideas about closed loops of neurons (or feedback between neurons, if he talked about that) when he developed his mathematical work on neural networks. For starters, neither McCulloch and Pitts 1943 nor Pitts’s earlier publications, in which he formulated theories of neural networks, cite Lorente de No. For more details on the origin of McCulloch and Pitts’s theory, see my paper.

Problem 5: Even if Pitts had known that Lorente de No talked about feedback between neurons, that would not have helped Pitts develop his mathematical theory of neural networks. (McCulloch says that McCulloch did know about Lorente de No’s idea, although McCulloch also says that he had thought about the role of closed loops of nervous activity before reading Lorente de No (see McCulloch’s “Recollections of the Many Sources of Cybernetics”). Of course, McCulloch did not actually develop the mathematical theory of loops of neurons; Pitts did.)

Problem 6: Insofar as the “whole approach” is relevant to the design of computers, it involves the use of Boolean functions and other logic-inspired formalisms to characterize networks of simplified neurons (with or without loops). That was McCulloch and Pitts’s great contribution, and the contribution that was relevant to computer design, and there is no evidence that it owes anything to Lorente de No (cf. my paper for more details).

Problem 7: In his published passage, Noe implies that Lorente de No influenced von Neumann via his (putative) influence on McCulloch and Pitts, and thus, according to Walter Freeman, was “the godfather of the computer”. But as far as I know, the only part of McCulloch and Pitts 1943 that had an influence on von Neumann and early computer design was their theory of networks “without circles” (i.e., without closed loops). The theory of networks “with circles” – which is the part at least thematically related to Lorente de No’s ideas – is very obscure and it probably had no impact on computer design until after Kleene published his paper on finite state automata in 1956. (As far as I know, the definitive history of the impact of McCulloch and Pitts 1943 on computer design has yet to be written; it might be a nice project for someone to pick up.) Even if we suppose for the sake of the argument that Lorente de No’s writing helped McCulloch think about closed loops of neural activity (McCulloch says it did) and that McCulloch’s consequent thinking played a role in the development of McCulloch and Pitts’s mathematical theory of neural networks (a dubious claim, especially since by McCulloch’s admission, Pitts did all the technical work), it would still be a serious stretch to conclude that therefore, Lorente de No was “the godfather of the computer” (Freeman’s words according to Noe) or of “the whole approach” (Noe’s words).

Conclusion: Based on the available evidence, Lorente de No probably played a role in convincing McCulloch of the importance of closed loops of neural activity, although it appears that McCulloch discussed closed loops of neural activity before reading Lorente de No’s work on this topic. None of this had any impact on the mathematical theory of neural networks developed by McCulloch and Pitts 1943, let alone on von Neumann and computer design.

A couple of things. Where Lorente do No’s 1933 “Entorhinalis” paper does discuss closed, self-exciting neuron chains, there does remain the question of how, if at all, either McCulloch or Pitts might have been influenced by it. As GP notes, Pitts never refers to this in his early papers (or probably ever). Nor do McC& P, 1943 refer to it (although they also did not refer to Turing, 1937). A cursory review does not show me that McCulloch refered to the “Entorhinalis” paper, although I know that McC does refer to others by Lorente de No, in particular later papers. So, they could have known of the “Entorhinalis” paper, but the most obvious sort of evidence is not there.

Regarding McC&P, 1943, contra GP, the final section contains a brief allusion to feedback:

“Moreover, systems which so respond to the difference between afferents to a regenerative net and certain activity within that net, as to reduce the difference, exhibit purposive behavior; and organisms are known to possess many such systems, subserving homeostasis, appetition, and attention.” This is a vague reference to negative feedback, although the word does not appear.

McCulloch appears to have met Wiener while working on “A Logical Calculus,” so there is a chance that he got this from Wiener. But, I could be wrong on this list. I can’t remember if Lorente de No was present when McC met Wiener.

Ken,

Thanks for your helpful comments.

On feedback, I meant to say that McCulloch and Pitts do not use the word “feedback”, and specifically, they do not use it to describe relations between neurons. The passage you cite is very likely motivated at least in part by the famous paper Rosenblueth, A., N. Wiener, and J. Bigelow: 1943, ‘Behavior, Purpose, and Teleology’, Philosophy of Science 10, 18–24. As I wrote in my paper on McCulloch and Pitts, McCulloch was pretty excited about the Rosenblueth et al paper before publishing “A Logical Calculus”, because he thought it helped solve the mind-body problem by giving an account of “purposive behavior”.

I’ve never seen evidence that Lorente de No was present when McCulloch met Wiener. From what I remember, McCulloch simply describes meeting Wiener. I think McCulloch says the meeting was arranged by Rosenblueth.

So, I am guessing we are on the same page about the following.

The cyberneticians brought together (at least) four different big ideas. 1) The formal theory of effective computation, 2) the formal theory of information, 3) interest in closed loops of neurons, and 4) feedback. Maybe there is a fifth informal notion of information. It seems to me plausible that Lorente de No had some role in promoting 3). In fact, Dusser de Barenne and McCulloch invoked Lorente de No’s work in a 1937(?) paper on excitation and facilitation. So, the case for this contribution is pretty strong, at least to my mind. But the case for his contributing the other three or four notions to McCulloch, Pitts, or the cyberneticians seems to be attested only by Freeman’s recollections.

I am hoping that in the coming months I’ll be able to finish a paper on what McCulloch brought to the table when he and Pitts sat down to write “A Logical Calculus”. (I’ve had a large part of it for a few years now, just never the drive or time to finish.) I think the short, perhaps flip, answer is “mostly enthusiasm and drive”.

Yes, I agree. And I look forward to your paper on McCulloch. Feel free to send me a draft any time.

On second thought, simplification of McC seems wrong. McC also brought the “big picture” to the paper. Still oversimplifying, McC seems to have been responsible for the first and the last sections, where Pitts was responsible for the middle two.

Thanks for the offer of a read, Gualtiero.