Information Processing, Computation, and Medium Independence

In response to a previous thread, Jonathan Livengood asked some very good questions about, roughly, what should count as information processing and computation in physical systems.  Perhaps it will help to take a step back.

In my early work on computation, I argued that, roughly, only physical processes that take strings of digits as inputs and return strings of digits as outputs by following a rule defined over the inputs (and possibly internal states) count as computing systems .  My reason had to do with the centrality of the formalisms of computability theory to the notion of computation.  As to analog computers, which do not manipulate strings of digits but are still called computers, I argued that they are “computational” only by courtesy and for contingent historical reasons .

I later realized that, although there was something right about my early purism about computation, it was unhelpful to try to restrict the notion of computation to digital computation in the face of important yet broader uses of the term “computation” in many sciences, including neuroscience.  (BTW, I had this realization in time to dodge Ken Aizawa’s criticism that I was insufficiently pluralistic; Ken’s criticism does apply to my former self, though.).  

After that, I needed to characterize a notion of computation more general than that of digital computation (without appealing to representation, of course, otherwise I would have gone against one of my core views about computation ).

What came to my rescue is the notion of medium independence.  Medium independence was introduced by Justin Garson in his 2003 MA thesis, part of which was published in a beautiful and underappreciated article in Philosophy of Science on “The Introduction of Information in Neurobiology “.

Justin pointed out that the first person to talk about neural systems transmitting information was Edgar Adrian (1928), on the grounds of his groundbreaking discovery of some crucial properties of neural signals (“all or none”, “rate coding”, and “adaptation”).  Justin reconstructed Adrian’s notion of information as involving medium independence:

“Medium independence: The structure S–for example, the structure relation that obtains between the units of a sequence of action potentials–can be instantiated across a wide range of physical mechanisms.” (Garson 2003, p. 927)

While Justin’s medium independence is not necessary for carrying (natural) information in the usual sense, a slightly modified version of it seems well suited for characterizing the general notion of computation.  So in my we’ve been discussing.  If Jonathan insists in calling this type of thing information processing, so be it.  But information processing can also be done in a medium independent way, and IMO that’s what most people mean when they talk about information processing.  In any case, if you care about what does and does not compute it’s important to notice the difference between the two cases, because medium-dependent information processing does not entail computation whereas medium-independent information processing does entail computation.

17 Comments

  1. Ken Aizawa

    Gualtiero,
    By my lights you are still non-sufficiently pluralistic or not pluralistic in the way I encourage. What you still seem to be aiming for is a single generic account of computation that does justice to lots of things that have been called “computation” with other notions as species. I’ve been suggesting that such a generic notion is what one might call a “philosopher’s fiction”. This generic notion plays little role in scientific practice. Instead, what matters are one or another specific notion designed for relatively specific ends. So, Turing’s notion of computation as effective computation has essentially nothing theoretically interesting to do with have been called “computed maps” in developmental neuroscience. Maybe you can come up with a generic account that links these usages, but that is largely an artificial construction on your part. It’s just like human memory and computer memory being described by the same term. There really is nothing theoretically deep relating them. There are just some vague and superficial similarities. That’s the idea I broach in this forthcoming paper, “Computation in Cognitive Science: It is not all about Turing-equivalent computation”. I’ve got a draft on my web page, but it’s past proofing for Studies in History and Philosophy of Science.

  2. Joshua Stern

    Medium independence was introduced by Justin Garson

    In what way is that different from multiple realizability, which is something like Putnam 1965?

    Actually, in what way is that different from Turing 1936, in which the UTM can pretend to be any other TM?

  3. gualtiero piccinini

    Good question.  As Justin defined it, medium independence seems to me very similar to multiple realizability, though I’d love to hear Justin’s opinion about this.  As I define it, however, medium independence is not the same as multiple realizability.  A medium independent process is defined in terms of certain degrees of freedom of the medium, regardless of what the medium actually is.  So except for certain weird cases (such as a medium independent process as large as the universe), medium independence entails multiple realizability.  But multiple realizability does not entail medium independence. Many of the classic examples of multiply realizable kinds, such as mousetrap, are not medium independent (e.g., they are defined in terms of their ability to catch mice, which are a specific concrete kind).  BTW, I suspect that my definition of medium independence is closer to what Justin (following Adrian) intended than his own definition, although again, I’d be interested in his opinion on this.

    As to universal TMs, that’s an entirely separate notion.  It’s the notion of one computing machine simulating the behavior of another.

  4. Martin Roth

    Hi Gualtiero,

    As I was reading this (nice) post, I was struck by the parallels between your general characterization of computation and John Haugeland’s discussion of automatic formal systems (for example, he also mentions medium independence). In what ways do you think your characterization is importantly different from his?

  5. gualtiero piccinini

    Does he use the term “medium independence,” and if yes, where?

    Leaving that question aside, I do think there are similarities between Haugeland’s account and mine.  I think the main differences are these: (1) I don’t rely on the notion of formal system (whose only explication I know of is in terms of computation, which makes Haugeland’s account circular), (2) I emphasize that computing systems are a special type of mechanism, relying on recent work on mechanistic explanation (which appeared after Haugeland published his work on this topic).

  6. Martin Roth

    As far as I can remember (don’t have AI in front of me; thanks for providing the reference, Eric!), Haugeland does not invoke the notion of computation to explain the notion of a formal system. Rather, he introduces the notion of an automatic formal system and then says that computers are *interpreted* automatic formal systems. But the stuff about interpretation/semantics is not a part of he characterization of a formal system.

  7. gualtiero piccinini

    Right.  I didn’t mean to say that Haugeland invokes computation to explain formal systems; only that I don’t know of any other way to explain what a formal system (of the relevant kind) is.

  8. Joshua Stern

    As to universal TMs, that’s an entirely separate notion. It’s the notion of one computing machine simulating the behavior of another.

    My current interest in computation is propounding an aggressively physicalist theory. Any realization of a TM or UTM is for my a physical realization. I then see no difference between simulation and realization. The benefit is that it takes all the mystery out of simulation – realization carries all the weight. One realization may have different characteristics from another – color, or weight, or speed, or efficiency. These are all familiar issues in physical systems.

  9. gualtiero

    I just read Haugeland’s discussion of medium independence (thanks Eric for the reference!). It seems to be pretty much the same idea as mine (which in turn was a modified version of Garson’s), except that Haugeland’s is talking about the medium independence of digital formal systems whereas I am talking about a more general notion of computation (digital, analog, or possibly neither).

    So either Garson had read Haugeland, forgotten about it, and then reformulated a similar idea under the same name, or approximately the same idea was stumbled upon by different people at different times and given the same name.

    Either way, the convergence between me (via Garson) and Haugeland on this point is more evidence that medium independence is an important and useful notion for characterizing computation.

  10. gualtiero

    “Any realization of a TM or UTM is for my a physical realization.”

    Right.

    “I then see no difference between simulation and realization.”

    This is true in one sense and false in another. A machine that simulates another computes the same function, so it’s the same in that sense. But it will typically have a different architecture and will undergo a different dynamical process than a direct realization of the original machine would, so it’s different in that sense.

    Consider this: a Dell computer can simulate an Apple, an HP, a Toshiba, etc. It doesn’t follow (in the second sense) that a Dell computer is a realization of an Apple computer, an HP computer, a Toshiba, etc.

  11. Martin Roth

    Hi Gualtiero,

    I’ve a question about how you are thinking of the relationship between analog computation and the “rules” of a computational system. As I am understanding you, the rules are specified over medium independent vehicle types, and so a system computes just in case it manipulates tokens of such types in accordance with the rules (for that type). However, given the bounds placed on distinguishing the inputs of analog systems (in Haugeland’s terminology, there is not a positive and reliable technique for producing and re-identifying tokens of pre-specified types), won’t it turn out that either (a) analog systems aren’t computational after all, or (b) analog systems (as you are using the term) are a case of digital systems (as Haugeland is using the term)? If not, what am I missing?

  12. Eric and Martin – thanks for the reference to Haugeland. I did read the book years ago and used the same term, “medium independence”, in the paper that Gualtiero cites. Embarrassingly, I had forgotten that Haugeland uses the same term – otherwise I would have cited him, though I did point out in the paper that others have used the same concept. At the time I wrote it I was preoccupied with the concept of information as it’s used in genetics and early 20th century neuroscience, rather than the context of computation or AI (given that computational analogies weren’t in vogue in neuroscience in the 1920s and 30s and have not played a significant role in genetics). I use it specifically to describe one way that spike trains can “carry information about” certain general features of stimuli, namely, by implementing a similar structure.

    In response to Joshua’s post regarding the difference between medium independence (as I use it in 2003) and multiple realizability, multiple realizability is a *functional* concept and medium independence a *structural* one. (This is just putting Gualtiero’s distinction in different words.) In other words, what makes a mousetrap multiply realizable is that there are alternate physical structures that can “do the same thing”, whereas what makes a string of binary digits “medium independent” is that the same pattern can be implemented by many different physical systems (action potentials, marks on paper, electrical impulses) quite irrespective of their function. What makes my concept of medium independence different, in turn, from Haugeland’s is that Haugeland seems to be concerned with rule-governed systems, rather than simple binary patterns. My view doesn’t presuppose any definition of a “rule” or “rule-governed system”.

    I do agree with Martin that it would be good to elaborate the notion of a “rule” as used in Gualtiero’s definition of generic computation.

    Gualtiero, thanks for the nice comments!

  13. Josh

    Consider this: a Dell computer can simulate an Apple, an HP, a Toshiba, etc. It doesn’t follow (in the second sense) that a Dell computer is a realization of an Apple computer, an HP computer, a Toshiba, etc.

    This is the thing about computers, they are pretty boring if you don’t plug them in. They are dynamic devices. They do stuff. Or, if you don’t plug them in, or give them enough time, or don’t program them right, they don’t do something.

    If a Dell computer *can* simulate an Apple, that may be well and good, but if and when it *does* simulate an Apple, then I’d say it does follow, that is it is true, that the Dell is a realization of the Apple.

    Any two realizations of anything differ somehow in their positions and momentums, among other features. This is a matter of what it is you think you are realizing. Is there only one, or some ideal, realization of X? Is realization a primary property at all? Or are all things things first, and realizations only second? Maybe all realizations are rather more vague and contingent to any ideal or abstraction, making the difference between simulation and realization vanishingly small to the point of disappearing entirely.

  14. gualtiero

    Martin, thanks for your question and sorry for the slow response.

    I hope I’m not committed to either (a) or (b)! Actually, I used to incline towards (a). Maybe that’s where your question comes from; from looking at some of my older works, in which I basically maintained that computation requires the manipulation of digits. I no longer maintain that :-).

    Now I would say that analog computation is a different type of computation than digital computation precisely because the system cannot reliably distinguish different portions of its vehicle types (in Haugelan’s terms, there is no positive and reliable technique for producing and reidentifying tokens of such types). But any physical realization of an analog computer does process its vehicles (to some level of precision), which are defined in a medium-independent way, in accordance with rules defined over the vehicles (or at least idealizations thereof), so analog computers are still computers.

    Does this sound reasonable?

  15. Martin Roth

    Hi Gualtiero,

    Here’s my worry about the kind of line you want to defend: lacking positive and reliable techniques for identifying tokens of vehicle types, the physical system–in the course of its “normal” processing–will fail to honor certain distinctions captured in the specification of the computational system (in terms of medium independent vehicles and rules). If that’s right, however, then I don’t see how it can be correct to say that the system is an instance of such a computational system.

    Does this make sense?

Comments are closed.