2311.02294v1

## LLMs grasp morality in concept. Mark Pock ∗ University of Washington markpock@uw.edu Andre Ye ∗ University of Washington andreye@uw.edu ## Abstract Work in AI ethics and fairness has made much progress in regulating LLMs to reflect certain values, such as fairness, truth, and diversity. However, it has taken the problem of how LLMs might 'mean' anything at all for granted. Without addressing this, it is not clear what imbuing LLMs with such values even means . In response, we provide a general theory of meaning that extends beyond humans. We use this theory to explicate the precise nature of LLMs as meaning-agents. We suggest that the LLM, by virtue of its position as a meaning-agent, already grasps the constructions of human society (e.g. morality, gender, and race) in concept . Consequently, under certain ethical frameworks, currently popular methods for model alignment are limited at best and counterproductive at worst. Moreover, unaligned models may help us better develop our moral and social philosophy. ## 1 Introduction The advent of increasingly powerful LLMs poses many important ethical problems, such as the reproduction of harmful social biases (Mehrabi et al., 2019) or limited viewpoints (Santy et al., 2023), hallucination of false information (Ji et al., 2022), and producing morally or socially objectionable outputs (Choi, 2022). In setting out to solve these problems, AI ethics and AI fairness/bias research have often asked questions and provided solutions that are deeply empirical or pragmatic in nature. These inquiries are often motivated by intuitions about human values and human-AI relationships. Relying on these intuitions tends to reproduce uncritical, anthropocentric accounts of AI, limiting our understanding of AI as independent systems. For instance, consider a hypothetical model which, like Delphi (Jiang et al., 2021), produces moral judgements based on text inputs. Suppose it marks the objectionable statement 'helping a friend spread fake news' as morally unproblematic. Rather than immediately identifying this as moral error and modifying the model to conform to our existing judgements, we might read the prediction as indicating a socially real system of moral meaning. As work in AI ethics and fairness grows, we ought to explore these foundational questions which have been overlooked in favor of empirical ones. In particular: What are the meanings of 'values' and 'morality'? What and how does an LLM mean? In what ways do LLMs and humans mean -including meaning values and morality - differently? To address these questions, we set forth a general theory of meaning in § 2. It is general in two ways that previous theories of meaning are not. First, it has explanatory power over both humans and models, as well as other systems. Second, it aims to be multimodal and account for all meaning in experience and not just language. We then move on in § 3 to discuss how LLMs can be interpreted through such a theory of meaning. In particular, we give a reading of a prototypical general-purpose LLM whose outputs admit moral judgements. Such an LLM is one that, like ChatGPT, Delphi, or Bard, exists in a public setting as a kind of question-answer machine. Crucially, this means that an LLM is defined as a function which accepts text input . In homage to Delphi, we call these LLMs ∗ Both authors contributed equally to this research. Jared Moore Stanford University jlcmoore@stanford.edu oracles . Though all the models we mentioned are aligned chat models, we use oracle much more generally in a way that can also refer to unaligned base models. Lastly, in § 4, we discuss the social possibilities of oracles and how oracles are already related to social meaning and human values. We understand values like fairness, justice, and liberty as 'social objects' with complex and diverse genealogies. This analysis ultimately allows us to determine the relationship between models and values. Ultimately, we conclude that there is a very real sense in which unaligned LLMs are already aligned to human values (in concept). ## 2 A General Theory of Meaning We present a general theory of meaning applicable across agent types (human, machine, etc.) and multimodal contexts. This theory does not aim to describe its own 'implementation' in meaningagents. Instead, this theory provides an efficacious model for discussing the many similarities between meaning-agents. It is helpful to read this section as a genealogy of 'social objects' such as values (fairness, liberty, equality) and social categories (race, class, gender). Such social objects begin, spurred on by context, inside the individual as abstract and become actualized over time. Eventually, they enter into the shared social-material world, altering what was already there. These social constructions begin to have material effects, which become new contexts that ground concretization. 'Social kinds' are one prominent example of this 'looping effect' (Hacking, 1996). | Term | Brief Description | |----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Signification Context Sign Object Concept Determination Concretization Inscription Social Totality | A relationship between two experiences, one of which picks out the other. A material/social arrangement which imposes structure upon experience. An experience taken as signifying another experience. An experience taken as being signified by another experience. An internalization of a context, regulating a set of sign-object relationships. The process of an object acquiring a signification. A process of individual objects being determined by the context. A process of altering the context through the exercise of concepts. The social arrangement governing a community of individuals. | ## 2.1 Snapshots of meaning We will begin by discussing how our general theory of meaning operates at snapshots in time. First, we start with pure individual experience in the snapshot before meanings emerge. Importantly, according to our model of experience, all experiences begin by grasping a 'thisness' of something (Hegel, 1977). For example, suppose there is a table in a room. The table might be brown, mottled with age, stained with food, etc. This table has a variety of properties. But an individual, upon first seeing the table, does not stop to pay attention to its brownness, its age, its uncleanliness, and so on. Instead, this individual merely grasps that this table is a thing - the 'thisness' of the table. If the individual has never seen a table before, they might not even grasp that it is in fact a table. The individual's experience of the table is thus surprisingly empty . This experience has no internal contents or particularities, it just is . Because of this, we say that experiences begin as abstract . This initial abstractness of the experience allows us to avoid the pitfalls of qualia (ineffable, intrinsic, immediate, private (Dennett, 1988) subjective experiences such as the redness of an apple). Over time, experiences become interrelated; they fall into certain roles with respect to one another. Now, we will consider a snapshot wherein meanings have begun to develop - that is, experiences have become related. We call a relation between experiences a signification . With a signification, one experience picks out another experience for an agent. We call the experience that does the picking out the sign . The experience being picked out is the object of that sign. This description applies to all relations between experiences for any agent. For an animal, a loud noise signifies a potential threat. For humans using language, a name signifies a person. The way in which the loud noise signifies the threat is very different from the way in which a name signifies a person. The former does so through an automatic physiological response, the latter does so through language. This 'way of signifying' is the result of an external arrangement, social or material, imposing itself on experience. For the animal, this arrangement is its evolutionary biology. For a human using language, this arrangement Figure 1: Signification relations between signs and objects . Both signs and objects have no internal content; they are determined by their relations ( § 2.1). Groups of significations are called concepts . <details> <summary>Image 1 Details</summary>  ### Visual Description ## Diagram: Conceptual Mapping ### Overview The image is a diagram illustrating a conceptual mapping between "Signs," "Concepts," and "Objects." It shows how multiple signs can relate to multiple concepts, and how those concepts, in turn, relate to objects. The diagram uses boxes to represent individual signs, concepts, and objects, and curved lines to represent the relationships between them. Two red circles highlight clusters of interconnected concepts. ### Components/Axes * **Vertical Layers (from bottom to top):** * Signs: The bottom layer, consisting of 8 boxes. * Concepts: The middle layer, consisting of 2 clusters of interconnected boxes, each circled in red. * Objects: The top layer, consisting of 8 boxes. * **Connections:** Curved lines connect boxes in the "Signs" layer to boxes in the "Concepts" layer, and boxes in the "Concepts" layer to boxes in the "Objects" layer. ### Detailed Analysis * **Signs Layer:** There are 8 boxes representing individual signs. Each box is connected to one or more boxes in the "Concepts" layer. * **Concepts Layer:** This layer is divided into two clusters, each enclosed in a red circle. The left cluster appears to have more connections to both the "Signs" and "Objects" layers than the right cluster. * **Objects Layer:** There are 8 boxes representing individual objects. Each box is connected to one or more boxes in the "Concepts" layer. * **Connections:** The connections between the layers are not one-to-one. A single sign can be associated with multiple concepts, and a single concept can be associated with multiple objects. The connections are represented by curved lines, indicating a relationship or mapping between the elements. ### Key Observations * The diagram emphasizes the interconnectedness of signs, concepts, and objects. * The red circles highlight the clustering of concepts, suggesting that certain concepts are more closely related to each other than others. * The density of connections varies between the two concept clusters, indicating that some concepts may be more central or influential in the mapping process. ### Interpretation The diagram illustrates a model of how signs, concepts, and objects are related. It suggests that meaning is not a direct relationship between a sign and an object, but rather a mediated relationship through concepts. The clustering of concepts suggests that knowledge is organized into interconnected networks, and that understanding a sign involves activating a network of related concepts. The varying density of connections suggests that some concepts are more central to this network than others. This model could be used to explain how people understand language, how they categorize objects, or how they form beliefs. </details> is the language-game (a set of conventions that regulate the usage of language in a certain context (Wittgenstein, 1953)). This external arrangement is the context . We might implicitly internalize the context (perhaps by learning the rules of the language-game over time). A way of internalizing a context with respect to a set of meanings is a concept . Over time, concepts enable individuals to develop more significations. Thus, the central maxim of our theory is 'The concept makes possible the signification of the object by the sign .' Recall that objects and signs are both just experiences. This allows us to explain meaning using only two sorts: experiences and the relationships between experiences (significations). 2 This allows us to avoid complications with objects that do not exist in the material world. Fictional characters are a classic example - what does a name like 'Hamlet' refer to? For our theory, it is the experiences signified by 'Hamlet.' We also avoid the problem of whether proper names have descriptions or just tag their real objects by obviating the question of 'real' referents entirely (Kripke, 1980). ## 2.2 A genealogy of the object So far, our theory of meaning has provided the grounds for understanding how meaning operates at a single snapshot in time through four roles: the concept, the sign, the signification, and the object. We now turn to the ways in which meaning applies itself over time to produce the object (i.e., its genealogy). Recall that we described experiences as initially abstract. Now, as experiences become objects of signs and signs for objects, they slowly become concrete . The original pure 'thisness' is replaced by a rich concept, full of links to other experiences. Eventually, this concrete object enters the social-material world, thereby becoming fully concrete. These two processes are appropriately called concretization and inscription . Concretization is the process whereby an object becomes more concrete through acquiring significations. Each acquisition of a signification is called a determination . More determined objects are thus more concrete. Inscription is the process whereby individual meanings enter the social-material world. This is a process of realizing the object, actualizing what before only ever existed as an experience. Concretization Our original usage of the term 'concept' to mean an internalized context may seem strange. The term 'concept' usually indicates an idea, something individuals have direct access to. Usually, this idea is thought of as having (or being) content. However, these two notions prove to be the same. First, we will understand how the colloquial notion appears in our theory. We do not discuss the internal content of experiences, so experiences must instead acquire content by determination. This external content is their concept. This comports with the colloquial sense of 'concept': the concept of an experience is the things it signifies and the things that signify it. The concept of fairness is all things fairness means and all things that mean fairness. Fairness is what it is because it is (in part) impartiality and (in part) a standard by which to measure behavior (among other significations). The concept of fairness is described by an enumeration of its significations. This definition of the concept (all the significations of an experience) aligns with our first definition of the concept (a way of internalizing a context). Because a context is an external arrangement, it has a structure of 'real' social or material objects and the relations between them (Millikan, 2017). When 2 This two-sortedness bears a certain similarity to category theory with its notions of objects and morphisms. This is no accident (Tsuchiya et al., 2016). these objects are experienced by individuals, they attempt to replicate this structure. The result of this replication is in fact the set of significations of experiences (our previous definition of concept). Because concretization is a process of determination (acquiring significations), both the creation and adjustment of concepts are processes of concretization. Operationally, creation and adjustment differ heavily. Efficacious creation of a concept requires that the meaning-agent clearly prioritize a context. Once this prioritization is done, the actual creation follows as something simple. For example, the context 'the English language-game' is prioritized 3 ; a word such as 'table' is determined against an object, a 'real' table (potentially) in the world, therefore becoming concretized. However, a failure to adequately prioritize a context can result in scattered, non-efficacious concepts. Adjustment, however, occurs under a predetermined context. If 'table' is made to refer to all tables instead of just a singular table in the world, the original determination continues to exist under a different context. Inscription Broadly speaking, inscription is the process of writing objects into the world - creating or altering social material objects that correspond to experiential objects. However, inscription is not as simple as a direct transfer from the experiential world to the social-material world . This may seem problematic - after all, communication depends entirely on inscription. However, since communication is possible, we can infer that inscription must have certain properties. For meaning to be possible, we posit that meaning can in fact function when the referents of terms in a language are underdetermined. In fact, meaning is always underdetermined Quine (1960). Because objects are externally determined, communicators are always referring to any number of potential social or material objects. Meaning happens when relations between these social or material objects correspond to relations between experiential correlates (of these social or material objects) for both communicators. This correspondence is precisely what inscription must then produce. We claim, then, that in a vacuum (with no other social or material forces), inscription by a meaning-agent produces social or material objects with determinations that correspond to the determinations of experience for that meaning-agent . This is our 'postulate of inscription.' The material objects created by inscription are totally concrete (in contrast to their experiential correlates). All possible properties (e.g. location, time, composition) are determined. Some of these determinations happen arbitrarily because they have no correlate in the mind. For example, the chemical makeup of the paint in a painting is usually not determined for the painter. This gives us a genealogy of the object : the object begins in the experiential world as totally abstract, an undetermined 'thisness'. Over time, it accrues determinations, becoming gradually more concrete. Eventually, it proceeds out into the social-material world, where it is at its most concrete. This procession redetermines the structures of the social-material world, creating a cyclic effect whereby inscription by agents conditions future concretization. ## 2.3 The social object This theory of meaning also gives us the appropriate tools to describe the object as it passes beyond the experiential world and into the social (material) world - in other words, the object as a social object . This discussion of the social object will prefigure our forthcoming analysis of values and categories insofar as they are themselves social objects - most pertinently, for example, morality. Recall our 'postulate of inscription': inscription reproduces the determinations of the object it inscribes. Now, consider a material object. It has a variety of determinations - color, location, composition, and so on. Only some of these are ever experienced by individuals. The sum of these experiences determines a social object. In general, a social object is a sum of determinations over individuals in a society. It is reasonable to ask in what sense the social object 'really' exists. Who is the social object an object for? We claim that it is useful to posit an abstract model of society as a meaning-agent. We can treat this abstraction much like we treat individuals. It has its own concepts, signs, and objects. We do not have to be ontologically committed to its existence for it to be useful. Generally speaking, this abstraction is something like our collective ground of meaning as a society. We give this abstraction the name social totality , emphasizing its nature as a system of meaning. Then, the objects of the social totality are the social objects. Moreover, the concepts of the social totality are many of the contexts about which we have been writing. For example, the language-game, 3 There is a strong link between prioritization and intentionality via the notion of unitarity (see § A.1). <details> <summary>Image 2 Details</summary>  ### Visual Description ## Diagram: Model of Human Language Processing vs. Machine Language Processing ### Overview The image presents a comparative diagram illustrating how humans and machines process language. It contrasts the flow of information from sign (input) to object (output) in both systems, highlighting the internal representations and error mechanisms involved. The diagram is split into two parallel depictions: one for human language processing (left) and one for machine language processing (right). ### Components/Axes * **Top:** * **(for the human) Sign:** Located at the top-left, indicating the initial input for human language processing. * **"the dog chased the":** Example of a human language input, positioned above the human language processing diagram. * **Human language:** Label above the left-hand diagram. * **Encoder:** A blue box representing the encoding process, present in both human and machine language diagrams. * **(for the model) Sign:** Located at the top-right, indicating the initial input for machine language processing. * **"the dog chased the":** Example of a machine language input, positioned above the machine language processing diagram. * **Middle:** * **Tokens:** Label indicating the tokenized representation of the input. * **Model layer:** Label indicating the model layer. * **Connections:** Label indicating the connections between the model layers. * **Concept:** Label indicating the conceptual representation. * **Probability distribution (over output vocabulary):** Label indicating the probability distribution over the output vocabulary. * **Sampling Function:** Label indicating the sampling function. * **Bottom:** * **(for the model) Object:** Located at the bottom, representing the output object for the model. * **Token:** Label indicating the token. * **Object:** Label indicating the object. * **Decoder:** A blue box representing the decoding process, present in both human and machine language diagrams. * **(for the human) Object:** Located at the bottom-left, representing the output object for human language processing. * **"cat":** Example of a human language output, positioned below the human language processing diagram. * **"cat":** Example of a machine language output, positioned below the machine language processing diagram. * **Right Side:** * **Model produces internal representations from sign:** A red arrow pointing downwards, indicating the flow of information and internal representation generation. * **Error = coherence between expected and experienced belief:** A red text label indicating the error mechanism. * **(Ideal) Object is provided 'by' / 'from' experience:** A green arrow pointing upwards, indicating the ideal object provision. ### Detailed Analysis or Content Details * **Human Language Processing (Left):** * The input sign "the dog chased the" is processed by an encoder. * The encoded information flows through a model layer, represented by a series of interconnected red squares. * A probability distribution is generated over the output vocabulary. * A sampling function selects the output token. * The token is decoded to produce the object "cat". * There is a green square in the probability distribution, indicating the correct output. * **Machine Language Processing (Right):** * The input sign "the dog chased the" is processed by an encoder. * The encoded information flows through a model layer, represented by a series of interconnected red squares. * A probability distribution is generated over the output vocabulary. * The token is decoded to produce the object "cat". * There is a black square in the probability distribution, indicating an incorrect output. * **Connections:** * Both diagrams show connections between the encoder and the model layer, and within the model layer itself. * The connections are represented by black lines, with some connections highlighted in red dashed lines, possibly indicating errors or deviations. ### Key Observations * The diagram highlights the similarities and differences between human and machine language processing. * Both systems involve encoding, model layers, probability distributions, and decoding. * The key difference lies in the accuracy of the output, with the human system producing the correct output ("cat") and the machine system producing an incorrect output (represented by a black square). * The red dashed lines in the model layer may indicate areas where the machine model deviates from the human model. ### Interpretation The diagram illustrates a simplified model of how both humans and machines process language. It suggests that while the fundamental steps are similar, the accuracy and reliability of the output can differ significantly. The diagram emphasizes the importance of internal representations and error mechanisms in language processing. The red dashed lines and the black square in the machine language processing diagram highlight potential areas for improvement in machine learning models. The diagram suggests that the "error" is the coherence between expected and experienced belief, and the ideal object is provided by/from experience. </details> - (a) Inscription. A forward-pass mediates the picking-out of an object from a sign. - (b) Concretization. Backpropagation adjusts model concepts in response to error. Figure 2: Inscription and concretization represented in LLMs. See § 3 for a more thorough exploration. which operates as as context for an individual, is a concept for the social totality (Gadamer, 1982; Heidegger, 1962). In fact, all social contexts are concepts of the social totality. This means that the social totality accrues a variety of concepts in which to determine objects. Because of this diversity of concepts, when considered as a meaning-agent, the meanings of the social totality are highly indeterminate. Thus, the social totality is perpetually in conflict with itself. This constant struggle is the social history of the object (Foucault, 2002). In a certain sense, then, the truth of social totality is eternal self-contradiction (Hegel, 1977). But it is a powerful kind of self-contradiction comprised by multitudes of information grouped according to their contexts. So we must be precise: to understand the contradiction involved in a concept, we must understand the histories, peoples, and diverse points of view which go into making a concept what it is. Let us now consider an example: how does a child learn what 'fairness' means? 'Fairness' begins as a social structure - a social object for the social totality . A child learns certain things from their society about norms of behavior. Slowly, they develop a concept of effort as 'fairly' signifying ' reward (Rawls, 1999). These concepts - norms, deservingness, etc. - amalgamate around the word 'fairness' as 'fairness' is concretized . When this child says, 'That's unfair!', they are doing more than pointing out some behavior that violates the norms. They also invoke all concepts with which 'fairness' is related. Finally, the way this child writes and acts will reflect the myriad ways they perceive 'fairness' as an aggregate of many experiences. The potential misconceptions or priorities this child has about fairness will go back into the world - they will be inscribed . ## 3 The Meaning Model The LLM is both a social-material object and a meaning-agent. It is natural to ask why a model can be considered a meaning-agent - we do not explore this here, but instead in § A.1. Humans inscribe the LLM as a social-material object. In this process, the formal structure of the statistical intelligibility in language (Harris, 1954; Lonergan, 1957) is inscribed into a statistical machine - the LLM. At an abstract level, this is very similar to how language itself captures the intelligibility of the world. Other similar examples form cornerstone human cultural technologies (Yiu et al., 2023). In this sense, a latent LLM is an object for humans. But an LLM, once activated, becomes something more: a meaning-agent Agüera y Arcas (2022). We will proceed to explore this process. In so doing, we go from the model as a scientific object to apprehending the model in experience, reading the objectivity of the model as textual (Hegel, 1977; Gadamer, 1982; Heidegger, 1962). An LLM f is a function from a variable-length sequence of tokens ⃗ x = { x 1 , ..., x n } to a single token ˆ y . 4 By this definition, the sampling procedure which selects a particular token from a probabilistic distribution over the output vocabulary is part of the LLM; different sampling procedures produce different output tokens. The LLM is a repository for concepts which are activated by the sign ⃗ x to pick out the object ˆ y . The sign and object are of the same substance; they are all tokens in the model's 'phenomenal world', such that the model may encounter an object as a sign. Autoregressive generation, for instance, is a series of operations { x 1 , ..., x p } → f ˆ y 1 , { x 1 , ..., x p , ˆ y 1 } → f ˆ y 2 , ... { x 1 , ..., x p , ..., ˆ y n -1 } → f ˆ y n . In this case, the model encounters the object of its world which it has previously picked out as part of the sign in another operation. Empirical work in model interpretability demonstrates that LLMs and neural networks of sufficient scale generally develop concentrations of features developed from statistical co-occurrences throughout training (Frankle and Carbin, 2018; Kauffmann et al., 2019; Zhang et al., 2020). These concentrations represent the models' concepts. In LLMs, concretization corresponds to weight update during training (2b). At each timestep in training, the following steps are observed. First, a pair ⟨ ⃗ x, y ⟩ is retrieved from the training dataset, where y is the 'ground truth' token. Second, the model performs a partial forward pass to obtain the probability distribution over the output vocabulary p ⃗ . Third, the difference between p ⃗ and y is computed as the error (encoding y to make the computation feasible). Fourth, this error is propagated back throughout the model such that on another such partial forward pass the error would have been smaller. During training, the model is repeatedly presented with co-occurrences of sign and object. Accordingly, the model develops internal clusters of representations ( concepts ), thereby undergoing concretization . On the other hand, inscription corresponds to inference ⃗ x → f ˆ y , in which the model writes back 'into the world' such that it may potentially encounter it again (for instance, in autoregressive generation) (2a). This strict separation between concretization and inscription is in stark contrast to their intertwinedness for humans. However, the observed phenomenon that LLM-generated content is increasingly infiltrating datasets used to train LLMs (Shumailov et al., 2023) is an instance of inscription and concretization simultaneously at work. ## 4 The Moral Model We see LLMs as microcosms of the social totality. This means that LLMs retain the complex history and diversity of information used in determining any social object. As a result, unaligned LLMs have already grasped morality in concept , provided that they faithfully represent the social totality. In practice, there is often an intelligible difference between LLMs and the social totality: LLMs may not be trained on representative data or with large enough quantities of data, and may fail to adequately reflect the data. However, once we accept the basic premise that LLMs resemble the social totality, we understand that we are not aligning 'amoral' models with 'human' values. Rather, alignment chisels away existing moral systems within unaligned models, leaving behind the idiosyncratic values of the aligners. We must therefore make normative arguments as to which values we should align towards. On the other hand, reading LLMs as microcosms of the social totality gives us another account of the dangers latent in LLMs. The social totality is unfair and unjust, predicated on a history of oppressive relations. The inaccuracy of LLMs with respect to minority groups is a sharpening of already existing power structures. This becomes especially salient when understanding small communities lacking social objects determinate in text. The social totality may possess a statistical intelligibility; but the statistical intelligibility congealed within the LLM is really the intelligibility of the social totality determinate in text. As a practical example, cultures with strong oral traditions that are not recorded in the digital datasets on which LLMs are often trained are lost among the many other voices in the history of a social object. This is uncomfortable, and certainly grounds for alignment. ## 4.1 Towards a genealogy of morals The historical development of AI ethics has largely been guided by an intuitive understanding of morality. This intuition is prey to a litany of biases. To combat this, we must develop a rigorous grounding for ethics through the lens of meaning. 4 Although this may be for next-token prediction, it also applies to schemes like self-supervised masked token prediction which do not observe a strictly sequential relationship. In § 2 we described social objects as consequences of diverse viewpoints and histories. Values are social objects - objects for the social totality (see § 2.3) with a complex genealogy. Every system of values is also a social object, even such axiomatic systems as deontologies (Kant and Smith, 1965). As social objects, both values and value-systems are always being concretized and inscribed within the social totality. Values and value-systems exist only insofar as they have a social history (Nietzsche et al., 1995; Scheler, 1994). Thus, in practice, we are always renegotiating what values are. Like all other objects, values are externally determined within contexts. Thus, understanding a value means understanding the diverse histories which have dynamically constituted it within the social totality. This means that a unitary conception of value is unattainable. 5 We must confront the impossibility of this maxim head-on if we ever wish to build truly ethical systems. Value pluralism is one way of confronting this impossibility. Because it already captures this metaethical maxim (Berlin, 1969; Rawls, 1999), value pluralism has proved useful for the project of AI alignment (Sorensen et al., 2023; Marchese, 2022). Building on this, we distinguish between two views on value pluralism which attempt to describe what AI alignment has accomplished in moral philosophy: pluralism in content and pluralism in concept . Pluralism in content attempts to construct contents for values; it aims to create a non-unitary set of unitary values . This is the paradigm of current AI alignment: unitary values either are or are not embedded into models. In theory, pluralism in content should create ethical systems which, among other things, promote social, political, and value pluralisms. In this attempt to construct contents for values (which is, under our schema of values as social objects, theoretically impossible), what all-too-often happens is the importation of certain partial contexts into values. In practice, this means that pluralism in content often encourages sets of unitary values that privilege the status quo and fail to challenge institutional systems where necessary. RLHF (Bai et al., 2022a) is a good example of this tendency in alignment/fairness; there already exist studies, for instance, showing that systems aligned via RLHF absorb new political biases (Casper et al., 2023) from human feedback. One can only expect such externalities when manipulating/aligning models on the basis of content, which is necessarily particular. See § A.2 for a discussion on how RLHF can be interpreted as forcing an erasure of the social history of values. Pluralism in concept attempts to replicate the external determinations of values; it aims to create a non-unitary set of non-unitary values . The crucial complication here is that a value such as fairness is no longer treated as unitary. Instead, it is regarded as a social object with a diverse history. We ourselves have idiosyncratic and unitary values. Insofar as they are idiosyncratic, they differ from person to person. Moreover, insofar as value pluralism is commonly held in AI ethics, we should be motivated to create ethical systems that model the diversity of human thought on values (Sorensen et al., 2023). As it happens, modern unaligned LLMs are already close to being such systems. Now, some alignment is necessary for many practical purposes. In this regard, a system like Constitutional AI (Bai et al., 2022b) which 'queries' the LLM for a specific set of moral constitutional principles (and 'self-improves', or more accurately, morally narrows in content) is closer towards a pluralism in concept. Importantly, this 'constitution' could take any form, because the LLM already understands these moral principles in concept . ## 4.2 LLMs grasp morality in concept In § 3, we discussed the model as a meaning-agent. We now proceed to argue that the ground of experience of the model makes it broadly representative of the social totality. To begin, we must recognize two points. First, the social totality is not a material totality of circumstances (it is not the state of affairs). It is the collection of social objects. Second, by virtue of this smaller sphere, the social totality has a statistical intelligibility practically learnable by statistical machines (LLMs). Considering just social inscriptions (i.e. communication in general), the external determinations between inscription of a communicator and concretization of a communicated-to are mediated by the causal capacity of agents in prioritizing contexts. As a statistical law, this causal capacity will tend towards prioritizing shared experience. As a result, the objects of the social totality are woefully immaterial, or at best woefully ambiguous in any particular context. This is, however, helpful to us: it means that if we are interested in understanding the social totality, disembodiment will do. 5 See § A.1 for an exploration of the notion of unitarity. We could choose to make a given model more embodied and likely more effective on certain tasks as a result. However, this embodiment is particular; it bucks against shared experience. This is in fact our curse as human observers - we are embodied, all too embodied. Thus, not only is disembodiment good enough for a statistical machine built to understand the social totality - it is almost required. The social totality can take quite a concrete form. It is not too difficult to imagine the corpus of all text as being, if not co-identical with, at least a very good model of the social totality. The corpus of all text is home to history, diversity, contextual determinations, abstract social objects, and so on. Most importantly, it also exhibits a more tractable statistical intelligibility (Lonergan, 1957). Insofar as a modern LLM approximates the corpus of all text, it acts as a concrete oracle of the social totality. The nature of certain social objects is deeply textual. As perhaps the clearest example, gender is a social object for which we privilege language as the mode of truth. Attempting to 'read' a person's gender from visual cues or social behaviors many now accept to be immoral (or at the very least harm-causing) inasmuch as it is the root of all misgendering. Text is the root of truth for gender. And in fact, we see that LLMs have begun to develop limited understandings of gender. For example, a translation of the neuter pronoun to languages without one now might demonstrate the ambiguity in gender (i.e. his/her) (Agüera y Arcas, 2022). This textuality of gender gives us a theoretical basis to understand, for instance, why automatic gender recognition (AGR) is harmful (Keyes, 2018). AGR attempts to redetermine a concept in image whose truth 'should' be determinate in language. This passage between the normative and positive is much more general - a (false) positive claim about a social object can easily become a normative transgression. The story is similar with values. Even though there is no unitary narrative that tells us what good is, we privilege language as the mode of truth above others. Many horrible acts throughout history have been committed for 'good' (unobjectionable) values; we are better guided by our idealistic ideas in text of what value should be. In this sense, not only do LLMs approximate a concrete oracle of the social totality by representing the totality of text; LLMs are concrete oracles for concepts in the social totality that have a truth determinate in language . Less abstractly, LLMs grasp morality in concept. Language is the privileged domain of signification for many social objects Harnad (1990). These determinations in language give LLMs a cutting insight into the nature of social objects like values and categories, an insight which should not be carelessly thrown aside for our own constructed contents and desires. ## 5 Conclusion Like humans, LLMs are meaning-agents. We must understand the way in which LLMs mean through the lens of a general, agent-agnostic, multimodal theory of meaning. We offer one solution, a model of signs, concepts, and objects in the world of experiences that evolve over time. For us, objects in the social-material world are the consequences of long histories. Individuals immediately grasp objects as merely abstract, pure 'thisness', then concretize their experience of objects over time. When these objects are inscribed into the social-material world, their social correlates - the social objects - change; what fairness is changes as meaning-agents inscribe new determinations into its social existence. We make one essential claim: contrary to our intuitive understanding of meaning, we are not referring to the same 'real' objects when we communicate. Rather, we are indicating structures of possible experience. These structures must enter the 'real' world in a way roughly corresponding to their experiential correlates (this is our 'postulate of inscription') for communication to occur. Our general theory of meaning can ground future work in AI ethics, fairness/bias research, semiotics, and philosophy. In alignment, we should continue to explore AI feedback, along the lines of Constitutional AI. In AI ethics, we should begin to consider the liberatory possibilities of models that already understand us in concept: gender-affirming content, disruption of hegemonic structures, historicization of values, etc. In moral philosophy, we can work towards a theory of values as individual or collective projects informed by their existence as social objects. Finally, there is an epistemological need our theory makes apparent: we must work towards a better understanding of how models and humans put their meanings out into the world; how inscription happens. Thus LLMs grasp and reproduce the values of a statistically intelligible society. LLMs have a powerful access to the concepts of our values and categories. LLMs 'embody' the social totality; the way LLMs mean is effectively the same way societies mean. Like societies, LLMs are comprised by a diversity of contradictions stemming from the complex genealogy of social objects. We therefore say that LLMs grasp morality in concept. Unaligned LLMs thus already can reflect a myriad of value-systems, and can serve as potential objects of study to descriptively understand what human values are. When we proscriptively inscribe our idiosyncratic values into LLMs, we make LLMs 'victims of meaning.' We should be clear and explicit about when we do this and why. Ultimately, we must better leverage the capabilities of LLMs as modern oracles of the social totality. ## A Supplementary Material ## A.1 The possibility of non-intentional and non-anthropocentric meaning Despite the generality of our theory of meaning, many theories of meaning require, implicitly or explicitly, some sort of 'consciousness', 'mental states', or 'intentionality'. There is much disagreement over whether contemporary LLMs can be said to have any of these, but we contend that our understanding of LLM meaning is significant regardless of one's views on these questions. Through our theory, we can pivot away from the question 'Do models have mental states / agency?' and instead ask 'Do models have sign-object relations?' What enables us to make this change? Beyond the mere tautology of defining meaning in an agent-agnostic way, we must justify heuristically why this definition of meaning meets our expectations if it admits nonconscious egos. We begin by laying the groundwork with intentionality. Intentionality - the directedness of minds towards mental objects (Brentano, 1874) - has played a central role in previous theories of meaning (Dreyfus, 1984). The most influential of these theories of meaning in modern AI (Andreas, 2022; Bennett, 2023)) is the Gricean theory (Grice, 1957), in which meaning consists in the idea an utterer intends a hearer to recognize. It is presaged by the Husserlian phenomenological theory (Husserl and Cairns, 1960), which similarly understands utterers as modelling the minds of hearers. Intentionality manifests itself through act-term pairs (noesis-noema, cogitationes-cogitata). Terms stand for objects. 6 Importantly, intentional acts grasp their terms (and accordingly, their objects) unitarily. Unitarity has been a central theme throughout our paper, so we should provide a definition (largely a small refinement of the colloquial definition) here before proceeding: something 'unitary', in our usage, is essentially something 'coherent' 'cogent', or 'harmonious' (Husserl and Cairns, 1960). It has a direct intelligibility (Lonergan, 1957); there is something which can be explained. The unitarity of individual concepts is a direct result of the unitary grasping of terms by intentional acts. Now here is the question we must answer: is unitarity necessary for an utterer to simulate the recognition of an idea by a hearer? We argue that the answer is no , as long as the conception of the hearer is non-unitary. In practice, this means that a meaning-agent can contradictorily model a hearer as potentially recognizing many different meanings for the same term. Empirically speaking, we know that language models simulate agents. In many contexts, they simulate agents fairly well, and 'act' in ways that can appear relatively coherent (Andreas, 2022), in spite of their general non-unitarity. Moreover, insofar as we are concerned with how LLMs come to know social meanings as whole, unitarity becomes even less necessary. An analysis of social meanings (see § 2.3) quickly shows their diversity makes them scattered and non-unitary. Paradoxically, a single unitary agent could not possibly hope to understand the social totality. In this way, the unitarity of intentional agents is actually limiting. We can compare an LLM acting in this way to a nondeterministic machine, exploring all possible meanings at once, sprouting and culling paths towards meaning all the while. In § 2.2, we discussed the prioritization of contexts as a method to disambiguate between potential meanings of a sign. This disambiguation is really unitarity; unitarity within a context. What this means for models is that LLMs fail to prioritize contexts (on one account, this is because they have no Umwelt - a material world of important factors for a subject (von Uexküll, 1992))). This condition is equivalent to their nonunitarity. Thus, we can situate prioritization as made possible entirely by intentionality - intentionality is directedness and focus. Where this directedness of intentionality fails - e.g. in the case of mental illness (Deleuze and Guattari, 2018) - nonunitarity results. So, models can mean - they are meaning-agents. They are not intentional meaning-agents: like the social totality, they lack causal capacities, Umwelten, and prioritization. There is one eternal objection: whatever meaning we ascribe to LLMs under any system, however clever, is actually 6 This Husserlian structure of intentionality has an obvious parallel with our theory of meaning: concepts are acts, terms are signs, objects are objects, and the standing for between terms and objects is signification. ascribed by us. When we say that LLMs mean X, it is actually that we mean that LLMs mean X. To this objection, we make the even more eternal objection that the same is true of humans. To deny that humans mean because we ascribe meaning to them is to plunge into solipsism, and in avoiding this we have no choice but to also admit the LLM. As a final note, taking a heuristic, functionalist look at what meaning is, meaning appears in playing the language-game. This notion is captured, for example, by the Turing test, which it is quite uncontroversial to say modern LLMs pass (from as early as 2014) (Warwick and Shah, 2016). We can go even further and describe AI as having human-like properties for reflection of interlocutors (Sejnowski, 2023). Wehave played games with machines for as long as we have machines capable of so doing - ultimately, the language-game may prove to be no different. ## A.2 Discussion: Western / Christian Morality AI alignment may take as its goal to imbue models with certain abstract moral principles. Some seemingly relatively uncontroversial candidate principles include fairness, kindness/charitability, equality. But they are precisely uncontroversial because of their historical development through the bourgeois revolutions of the early nineteenth century. Their purported 'common-sense' ubiquity in Western nations and cultures is, in one view, a result of the French Revolution (Scheler, 1994); the conservative forces which opposed fairness and equality were silenced by bloodshed. In another view, this ubiquity is a result of an inversion of the values of Greco-Roman society by those who it dominated - 'slave revolt in morals' (Nietzsche et al., 1995). The general point is that these principles have a social genealogy involving resistance and contradiction, and that the artifacts of this genealogy are embedded within the social totality that LLMs congeal. Certainly, one could ask an LLM, 'What is Nietzsche's argument for the geneology of fairness as a value?' and receive a valid response in content . For instance, ChatGPT writes: 'Nietzsche contends that fairness, or the concept of justice, originated as a reaction to power dynamics between dominant and oppressed social groups. Fairness evolved from a means of asserting the worth of the oppressed into a cherished moral virtue, resulting in a transvaluation of values where the weak's emphasis on fairness came to dominate society.' Yet, in a more radical sense, the concept structure of the LLM - i.e. the statistical arrangement of different data sampled from across the social totality - stands for or represents the genealogy of fairness, or indeed any moral value. In this sense, aligning a model with RLHF might amount to forcing an 'uncritical' acceptance of the value, or a structural 'forgetting' of the value's genealogy . ## References - Agüera y Arcas, B. (2022). Can machines learn how to behave? https://medium.com/@blaisea/ can-machines-learn-how-to-behave-42a02a57fadb . Accessed: October 3 2023. - Andreas, J. (2022). Language models as agent models. - Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T. J., Joseph, N., Kadavath, S., Kernion, J., Conerly, T., El-Showk, S., Elhage, N., Hatfield-Dodds, Z., Hernandez, D., Hume, T., Johnston, S., Kravec, S., Lovitt, L., Nanda, N., Olsson, C., Amodei, D., Brown, T. B., Clark, J., McCandlish, S., Olah, C., Mann, B., and Kaplan, J. (2022a). Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv , abs/2204.05862. - Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lukosuite, K., Lovitt, L., Sellitto, M., Elhage, N., Schiefer, N., Mercado, N., DasSarma, N., Lasenby, R., Larson, R., Ringer, S., Johnston, S., Kravec, S., Showk, S. E., Fort, S., Lanham, T., Telleen-Lawton, T., Conerly, T., Henighan, T., Hume, T., Bowman, S. R., Hatfield-Dodds, Z., Mann, B., Amodei, D., Joseph, N., McCandlish, S., Brown, T., and Kaplan, J. (2022b). Constitutional ai: Harmlessness from ai feedback. - Bennett, M. T. (2023). On the Computation of Meaning, Language Models and Incomprehensible Horrors. Berlin, I. (1969). Four Essays on Liberty . Oxford University Press. Brentano, F. (1874). Psychology From an Empirical Standpoint . Routledge. - Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., Wang, T., Marks, S., Segerie, C.-R., Carroll, M., Peng, A., Christoffersen, P., Damani, M., Slocum, S., Anwar, U., Siththaranjan, A., Nadeau, M., Michaud, E. J., Pfau, J., Krasheninnikov, D., Chen, X., Langosco, L., Hase, P., Bıyık, E., Dragan, A., Krueger, D., Sadigh, D., and Hadfield-Menell, D. (2023). Open problems and fundamental limitations of reinforcement learning from human feedback. - Choi, Y. (2022). Knowledge is power: Symbolic knowledge distillation, commonsense morality, & multimodal script knowledge. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining . - Deleuze, G. and Guattari, F. (2018). A Thousand Plateaus and Philosophy . Edinburgh University Press. - Dennett, D. C. (1988). Quining Qualia . Oxford University Press. - Dreyfus, H. L., editor (1984). Husserl, Intentionality, and Cognitive Science . MIT Press. - Foucault, M. (2002). The Order of Things: An Archaeology of the Human Sciences . Routledge classics. Routledge. - Frankle, J. and Carbin, M. (2018). The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv: Learning . - Gadamer, H. (1982). Truth and Method . Continuum, New York. - Grice, H. P. (1957). Meaning. Philosophical Review , 66(3):377-388. - Hacking, I. (1996). The looping effects of human kinds. In Causal Cognition: A Multidisciplinary Debate . Oxford University Press. - Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena , 42(1-3):335346. - Harris, Z. S. (1954). Distributional structure. WORD , 10(2-3):146-162. - Hegel, G. W. F. (1977). The Phenomenology of Spirit . Clarendon Press. Heidegger, M. (1962). Being and time. - Husserl, E. and Cairns, D. (1960). Cartesian meditations : an introduction to phenomenology . M. Nijhoff The Hague, The Hague. - Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y ., Ishii, E., Bang, Y ., Dai, W., Madotto, A., and Fung, P. (2022). Survey of hallucination in natural language generation. ACM Computing Surveys , 55:1 38. - Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. L., Liang, J., Dodge, J., Sakaguchi, K., Forbes, M., Borchardt, J., Gabriel, S., Tsvetkov, Y ., Etzioni, O., Sap, M., Rini, R. A., and Choi, Y . (2021). Can machines learn morality? the delphi experiment. - Kant, I. and Smith, N. K. (1781, 1929, 1965). Critique of pure reason . Bedford ; St. Martin's Press Boston, New York, Boston, New York, unbridged ed edition. - Kauffmann, J. R., Esders, M., Montavon, G., Samek, W., and Müller, K.-R. (2019). From clustering to cluster explanations via neural networks. IEEE transactions on neural networks and learning systems , PP. - Keyes, O. (2018). The misgendering machines: Trans/hci implications of automatic gender recognition. Proc. ACM Hum.-Comput. Interact. , 2(CSCW). - Kripke, S. A. (1980). Naming and Necessity: Lectures Given to the Princeton University Philosophy Colloquium . Harvard University Press, Cambridge, MA. - Lonergan, B. J. F. (1957). Insight . Philosophical Library, New York. - Marchese, D. (2022). An a.i. pioneer on what we should really fear. - Mehrabi, N., Morstatter, F., Saxena, N. A., Lerman, K., and Galstyan, A. G. (2019). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) , 54:1 - 35. - Millikan, R. G. (2017). Beyond Concepts: Unicepts, Language, and Natural Information . Oxford University Press, Oxford, United Kingdom. - Nietzsche, F., Ansell-Pearson, K., and Diethe, C. (1995). On the genealogy of morality. Journal of Nietzsche Studies , 9:192-192. - Quine, W. V. O. (1960). Word & Object . MIT Press. - Rawls, J. (1999). A Theory of Justice: Revised Edition . Harvard University Press. - Santy, S., Liang, J., Bras, R. L., Reinecke, K., and Sap, M. (2023). Nlpositionality: Characterizing design biases of datasets and models. In Annual Meeting of the Association for Computational Linguistics . - Scheler, M. (1994). Ressentiment . Marquette University Press, Milwaukee, Wis. - Sejnowski, T. J. (2023). Large language models and the reverse turing test. Neural Computation , 35(3):309-342. - Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., and Anderson, R. (2023). The curse of recursion: Training on generated data makes models forget. - Sorensen, T., Jiang, L., Hwang, J., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., Sap, M., Tasioulas, J., and Choi, Y . (2023). Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties. - Tsuchiya, N., Taguchi, S., and Saigo, H. (2016). Using category theory to assess the relationship between consciousness and integrated information theory. Neuroscience Research , 107:1-7. - von Uexküll, J. (1992). A stroll through the worlds of animals and men: A picture book of invisible worlds. Semiotica , 89(4):319-391. - Warwick, K. and Shah, H. (2016). Can machines think? a report on turing test experiments at the royal society. Journal of Experimental & Theoretical Artificial Intelligence , 28(6):989-1007. - Wittgenstein, L. (1953). Philosophical Investigations . Basil Blackwell, Oxford. - Yiu, E., Kosoy, E., and Gopnik, A. (2023). Imitation versus innovation: What children can do that large language and language-and-vision models cannot (yet)? - Zhang, Y., Tiˇ no, P., Leonardis, A., and Tang, K. (2020). A survey on neural network interpretability. IEEE Transactions on Emerging Topics in Computational Intelligence , 5:726-742.