Shea: Representation and Agency

Realist Representational Explanations of Agency Should Require Some Unity of Purpose

Nicholas Shea

My paper is about what it takes for an AI system to be an agent. It is relevant to Sprevak and Fallon’s PhiMiSci Special Issue on Representation in the Neurosciences and AI because conditions on AI agency are often articulated in representational terms.

Approaches to AI agency can be instrumentalist, for example when being an agent is understood in terms of representing goals and means to achieving goals, where representation is in turn understood to be a matter of displaying a certain pattern of behaviour. This often inspired by Dennett’s intentional stance. Or it can be a matter of having real internal representations of certain kinds, calculated with in a variety of ways, what Long et al. (2024) call ‘robust’ agency.

Instrumentalist approaches to agency require a certain amount of coherence between the representations, grounded as they are in patterns of behaviour. Realist approaches to representation do not. This paper argues that it is characteristic of agents that they have various mechanisms for representational coherence, and do in fact display a certain amount of coherence. Thus, the coherence requirement which is built into instrumentalist accounts of agency needs to be added when accounting for agency in realist representational terms.

More precisely, taking a dimensional approach to what it is to be an agent, where the sophistication of mechanisms for producing goal-directed behaviour is one dimension of agential sophistication, the sophistication of coherence mechanisms is another important dimension. In fact, as the mechanisms for choosing and controlling behaviour become more complex, there is a greater threat of behavioural incoherence: of the system acting so as to undermine in one output what it is was aiming to achieve with another. We see that clearly in human action, where substantial incoherence is normal. Thus, the need for coherence mechanisms increases as goal-directedness mechanisms become more complex. It is this requirement, implied by some representational accounts of AI agency but rarely made explicit, that this paper aims to highlight.

I use the term ‘coherence’ as a deliberately loose way of talking about the general idea that an agent should not act in a way that is self-undermining. A human agent has motivations both to get food and to achieve social status. These goals can pull in different directions. Often actions have to be chosen in a way that reflects their relative importance to the agent. I avoid gorging myself on the buffet because my friends will think less of me if I do. To achieve behavioural coherence, I need to weigh goals against one another in a relatively consistent way. I should not perform an action that advances the first goal at the expense of the second at one moment, and then shortly after choose an action that has the opposite effect. Behaviour like that is self-undermining. Without mechanisms which have the effect that goals are traded off against one another in a relatively stable way, the ability of the system to achieve any goals is compromised, however good it is at calculating, at a moment, the best way to bring them about.

The paper starts by arguing that instrumentalists about representation subscribe to a coherence requirement, and that representational realists do not and need not (section 2). However, the paper is not about the merits or otherwise of realism over instrumentalism about representation. The point of the discussion of instrumentalism is to show that, when purely behaviour-based approaches to agency are advanced in the AI literature, they thereby build in a coherence requirement. The main focus of the AI literature on agency is on the way that computational systems are able to achieve certain goals or produce certain outcomes. Section 3 distinguishes between approaches to that question that are purely based on behaviour, ‘outcome-directedness’, and approaches that are realist about the internal representations involved, ‘goal-directedness’. Section 4 shows that accounts of AI agency in terms of outcome-directedness do ensure a certain amount of coherence. Section 5 then discusses various accounts of agency in terms of types of goal-directedness mechanisms. While these do not explicitly build in coherence requirements, more sophisticated goal-directedness mechanisms do promote some degree of behavioural coherence.

Section 6 pins down coherence somewhat more carefully – without aiming to give an exhaustive definition – and argues for the main claim of the paper: accounts of what it is to be an agent in terms of types of representation and how they are processed should include coherence mechanisms. One important dimension along which systems become more agentive, in addition to having more sophisticated mechanisms for representing various kinds of goals and calculating how to achieve them, is by having more sophisticated principles or mechanisms for achieving representational consistency and behavioural unity of purpose. Conversely, engineers seeking to design AI agents will need to give their systems capacities,  by means of architectural design or through training, directed at achieving consistency and unity of purpose. This requirement has largely been overlooked in existing accounts of AI agency.

Nicholas Shea

nicholas.shea@sas.ac.uk

3 April 2026