2102.10717v2

## Abstraction and Analogy-Making in Artificial Intelligence Melanie Mitchell Santa Fe Institute, Santa Fe, NM, USA Address for correspondence: Melanie Mitchell, Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501 USA. mm@santafe.edu Abstract: Conceptual abstraction and analogy-making are key abilities underlying humans' abilities to learn, reason, and robustly adapt their knowledge to new domains. Despite of a long history of research on constructing AI systems with these abilities, no current AI system is anywhere close to a capability of forming humanlike abstractions or analogies. This paper reviews the advantages and limitations of several approaches toward this goal, including symbolic methods, deep learning, and probabilistic program induction. The paper concludes with several proposals for designing challenge tasks and evaluation measures in order to make quantifiable and generalizable progress in this area. ## Introduction Without concepts there can be no thought, and without analogies there can be no concepts. -D. Hofstadter and E. Sander 50 In their 1955 proposal for the Dartmouth summer AI project, John McCarthy and colleagues wrote, 'An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.' 77 Now, nearly seven decades later, all of these research topics remain open and actively investigated in the AI community. While AI has made dramatic progress over the last decade in areas such as computer vision, natural language processing, and robotics, current AI systems almost entirely lack the ability to form humanlike concepts and abstractions. For example, while today's computer vision systems can recognize perceptual categories of objects, such as labeling a photo of the Golden Gate as 'a bridge,' these systems lack the rich conceptual knowledge humans have about these objects-knowledge that enables robust recognition of such objects in a huge variety of contexts. Moreover, humans are able to form abstractions and apply them to novel situations in ways that elude even the best of today's machines. Continuing with the 'bridge' example, humans can easily understand extended and metaphorical notions such as 'water bridges,' 'ant bridges,' 'bridging one's fingers,' 'bridge of one's nose,' 'the bridge of a song,' 'bridging the gender gap,' 'a bridge loan,' 'burning one's bridges,' 'water under the bridge,' and so on. Indeed, for humans, any perceptual category such as bridge is understood via the rich conceptual structure underlying it. This conceptual structure makes it easy for humans to answer commonsense questions like 'what would happen if you drove across a raised drawbridge?' or 'what is on each side of a bridge across the gender gap?' Moreover, conceptual structures in the mind make it easy for humans to generate 'bridges' at different levels of abstraction; for example, imagine yourself forming a bridge from a couch to a coffee table with your leg, or forming a bridge between two notes on your piano with other notes, or bridging differences with your spouse via a conversation. Most, if not all, human concepts can be abstracted in this way, via analogy (for example, as an interesting exercise, consider the possible abstractions of everyday concepts such as mirror, shadow, ceiling, driving, and sinking ). Douglas Hofstadter goes so far as to define concepts by this property: 'a concept is a package of analogies.' 54 Humans' abilities for abstraction and analogy are at the root of many of our most important cognitive capacities, and the lack of these abilities is at least partly responsible for current AI's brittleness and its difficulty in adapting knowledge and representations to new situations. Today's state-of-the-art AI systems often struggle in transferring what they have learned to situations outside their training regimes, 81 they make unexpected and unhumanlike errors, 1 and they are vulnerable to 'adversarial examples' in a very unhumanlike way. 26,103 Concepts, abstraction, and analogy are foundational areas of study in cognitive psychology. Psychological theories of concepts have focused on ideas such as core knowledge, 10,101 exemplars and prototypes, 84,90 the 'theory theory' of concepts, 44 perceptual simulations, 6 experience-based metaphors, 67 and stochastic functions in a probabilistic 'language of thought.' 43 Some cognitive scientists have postulated that all concepts, even the most abstract ones, correspond to mental models of the world-grounded in perception-that can be simulated to yield predictions and counterfactuals. 6,28,61,66,68 Others have argued that concept formation, abstraction, and adaptation are all undergirded by processes of analogy-the act of perceiving essential similarities between entities or situations. 41,54,56 Understanding what concepts are-how they are formed, how they can be abstracted and flexibly used in new situations, how they can be composed to produce new concepts-is not only key to a deeper understanding of intelligence, but will be essential for engineering nonbrittle AI systems, ones that can abstract, robustly generalize, resist adversarial inputs, and adapt what they have learned to diverse domains and modalities. The purpose of this paper is to review selected AI research-both old and very recent-on abstraction and analogy-making, to make sense of what this research has yielded for the broader problem of creating machines with these general abilities, and to make some recommendations for the path forward in this field. The remainder of the paper is organized as follows. The next section gives a brief discussion of the role of abstraction and analogy-making in human intelligence, and the need for such abilities in AI. The following sections review several approaches in the AI literature to capturing abstraction and analogy-making abilities, particularly using idealized domains. The paper concludes with a discussion that appraises the domains and methods discussed here and proposes several steps for making generalizable progress on these issues. ## Abstraction and Analogy-Making in Intelligence One makes an conceptual abstraction one extends that concept to novel situations, including ones that are removed from perceptual entities, as in the examples of 'bridge' I gave earlier. The process of abstraction is driven by analogy , in which one mentally maps the essence of one situation to a different situation (e.g., such as mapping one's concept of a bridge across a river to a bridge across the gender gap). We make an analogy every time our current situation reminds us of a past one, or when we respond to a friend's story with 'the same thing happened to me,' even though 'the same thing' can be superficially very different. In fact, the primary way we humans make sense of novel situations is by making analogies to situations we have previously experienced. Analogies underlie our abilities to flexibly recognize new instances of visual concepts such as 'a ski race' or 'a protest march.' Analogies drive our conceptualizations of wholly new situations, such as the novel coronavirus pandemic that erupted in early 2020, in terms of things we know something about-the pandemic has been variously described as a fire, a tsunami, a tornado, a volcano, 'another Katrina,' or a war. Recognizing abstract styles of art or music is also a feat of analogy-making. Many scientific insights are based on analogies, such as Darwin's realization that biological competition is analogous to economic competition 97 and Von Neumann's analogies between the computer and the brain. 106 (These are just a few examples; see Ref. 50 for many more examples at all levels of cognition.) These examples illustrate two important facts: (1) analogy-making is not a rare and exalted form of reasoning, but rather a constant, ubiquitous, and lifelong mode of thinking; (2) analogy is a key aspect not only of reasoning but also of flexible categorization, concept formation, abstraction, and counterfactual inference. 41,50,54,105 In short, analogy is a central mechanism for unlocking meaning from perception. (See Ref. 7 for a broad discussion of analogy and its relation to other mechanisms of cognition and reasoning.) Analogy-making gives rise to human abilities lacking in even the best current-day AI systems. Many researchers have pointed out the need for AI systems that are more robust and general-that is, those that can perform sophisticated transfer learning or 'low-shot' learning, that robustly can figure out how to make sense of novel situations, and that can form and use abstract concepts. Given this, it seems that analogy-making, and its role in abstraction, is an understudied area in AI that deserves more attention. In the following sections I review several selected approaches to studying abstraction and analogy making in AI systems, ranging from older symbolic or hybrid approaches such as the structuremapping approach of Gentner et al. and the 'active symbol' approach of Hofstadter et al., to more recent recent methods that employ deep neural networks and probabilistic program induction. This is not meant to be an exhaustive survey of this topic, but rather a review of some of the more prominent approaches, ones that I will analyze in the Discussion section below. Note that I will focus on concepts and analogies that involve multipart situations, rather than the simpler single-relation 'proportional' analogies such as 'man is to woman as king is to ?' 79,92 While the term 'analogy' often brings to mind proportional analogies like these, the range of human analogymaking is far more interesting, rich, and ubiquitous. ## Symbolic Methods Symbolic approaches to abstraction and analogy typically represent input as structured sets of logic statements, with concepts expressed as natural-language words or phrases. Early examples include Evans' geometric-analogy solver, 25 Winston's frame-based system for analogy-making between stories, 113 and Falkenhainer et al.'s Structure-Mapping Engine (SME). 27 In this section I will describe SME, as well as the Active Symbol Architecture of Hofstadter et al., which combines symbolic and subsymbolic elements. ## The Structure-Mapping Engine The Structure-Mapping Engine (SME) 27 is based on Gentner's structure-mapping theory of human analogy-making. 40 In Gentner's theory, analogical mapping of one entity to another depends only on 'syntactic properties of the knowledge representation [describing the entities], and not on the specific content of the domains.' 40 Furthermore, according to the theory, mappings are primarily made between relations rather than object attributes, and analogies are driven primarily by mappings between higher-order relations (the 'systematicity' principle). SME's input consists of descriptions of two entities or situations, a base and a target , where each description consists of a set of logical propositions. In Figure 1, adapted from one of Falkenhainer Figure 1: An example of SME's input, in the form of logical propositions. Adapted from Ref. 27. <details> <summary>Image 1 Details</summary>  ### Visual Description ## Diagram: Solar System vs. Rutherford Atom ### Overview The image presents two diagrams side-by-side, illustrating analogies between the Solar System and the Rutherford model of the atom. The left diagram, labeled "Base: Solar System," breaks down the causes and properties of the solar system. The right diagram, labeled "Target: Rutherford Atom," similarly breaks down the properties of the Rutherford atom. Both diagrams use a tree-like structure to represent relationships and properties. ### Components/Axes **Left Diagram (Base: Solar System):** * **Title:** Base: Solar System * **Top Node:** CAUSES * **Second Level:** AND, REVOLVES-AROUND(planet, sun) * **Third Level:** ATTRACTS(sun, planet), GREATER * **Fourth Level:** MASS(sun), MASS(planet) * **Bottom Level:** GREATER, YELLOW(sun) * **Bottom Level:** TEMPERATURE(sun), TEMPERATURE(planet) **Right Diagram (Target: Rutherford Atom):** * **Title:** Target: Rutherford Atom * **Top Node:** GREATER * **Second Level:** MASS(nucleus), MASS(electron) * **Third Level:** ATTRACTS(nucleus, electron) * **Fourth Level:** REVOLVES-AROUND(electron, nucleus) ### Detailed Analysis **Left Diagram (Base: Solar System):** * The diagram starts with "CAUSES" at the top. * "CAUSES" branches into "AND" and "REVOLVES-AROUND(planet, sun)". This suggests that the solar system exists because of some combination of factors, and that planets revolve around the sun. * "AND" branches into "ATTRACTS(sun, planet)" and "GREATER". This suggests that the attraction between the sun and planets is a factor, and that something is "GREATER" than something else. * "GREATER" branches into "MASS(sun)" and "MASS(planet)". This indicates that the mass of the sun is greater than the mass of the planet. * The diagram also includes "GREATER" branching into "TEMPERATURE(sun)" and "TEMPERATURE(planet)", indicating the temperature of the sun is greater than the temperature of the planet. * Finally, "YELLOW(sun)" is present, indicating the sun is yellow. **Right Diagram (Target: Rutherford Atom):** * The diagram starts with "GREATER" at the top. * "GREATER" branches into "MASS(nucleus)" and "MASS(electron)". This indicates that the mass of the nucleus is greater than the mass of the electron. * The diagram then states "ATTRACTS(nucleus, electron)", indicating the nucleus attracts the electron. * Finally, "REVOLVES-AROUND(electron, nucleus)" is present, indicating the electron revolves around the nucleus. ### Key Observations * Both diagrams use a hierarchical structure to represent relationships. * The diagrams highlight the similarities between the solar system and the Rutherford atom model, specifically in terms of mass relationships, attraction, and revolution. * The left diagram includes the property "YELLOW(sun)", which is a specific characteristic of the solar system not directly analogous to the Rutherford atom. ### Interpretation The diagrams illustrate an analogy between the structure of the solar system and the Rutherford model of the atom. The solar system, with planets revolving around a massive sun due to gravitational attraction, is presented as analogous to the Rutherford atom, with electrons revolving around a massive nucleus due to electromagnetic attraction. The diagrams emphasize the relationships between mass, attraction, and revolution in both systems. The inclusion of "YELLOW(sun)" in the solar system diagram suggests a specific, non-analogous property. The diagrams likely serve to explain the Rutherford model by relating it to a more familiar concept, the solar system. </details> et al.'s illustrative examples, the base gives propositions about the solar system and the target gives propositions about the Rutherford atom. The descriptions in Figure 1 are logical statements represented as trees. The statements include entities (e.g., planet ), attributes (e.g., yellow ), first-order relations (e.g., revolves-around ), and higher-order relations (e.g., causes ). SME's job is to create a coherent mapping from the base to the target. The program uses a set of 'match rules' to make candidate pairings between elements of the base and target. Examples of such rules are: 'If two relations have the same name, then pair them'; 'If two objects play the same role in two already paired relations (i.e., are arguments in the same position), then pair them.' The program then scores each of the pairings, based on factors such as having the same name, having the same type (e.g., object, attribute, nth-order relationship), and being part of 'systematic structures,' that is more deeply nested propositions rather than isolated relations. Once all plausible pairings have been made, the program makes all possible sets of consistent combinations of these pairings, making each set (or 'global match') as large as possible. 'Consistency' here means that each element can match only one other element, and a pair is allowed to be in the global match only if all the arguments of each element are also paired up in the global match. After all possible global matches have been formed, each is given a score based on the individual pairings it is made up of, the inferences it suggests, and its degree of systematicity (relations that are part of a coherent interconnected system are preferentially mapped over relatively isolated relations). The output of the system is the set of possible global matches, ranked by score. It is important to note that SME is considered to be a model solely of the mapping process of analogy-making; it assumes that the situations to be mapped have already been represented in logical form. SME has often been used as a mapping module in a system that has other modules for representation, retrieval, and inference. 8,18,32 One notable example of such a system is the work of Lovett et al. 72 on solving Ravens Progressive Matrices (RPMs). I'll use this work as an illustrative example, since RPMs have more recently been studied extensively in the deep learning community as a domain for studying abstraction and analogy. RPMs, originally created by John Raven in the 1930s as way to assess 'fluid intelligence,' have long been given to both children and adults as nonverbal intelligence tests. Figure 2 gives a sample Figure 2: A sample RPM problem. <details> <summary>Image 2 Details</summary>  ### Visual Description ## Abstract Reasoning Puzzle ### Overview The image presents two abstract reasoning puzzles. The first puzzle is a 3x3 grid with eight cells containing geometric shapes. The ninth cell is empty, implying a missing element. The second puzzle presents a series of eight numbered cells, each containing geometric shapes. The task is likely to identify the pattern in the first grid and determine which of the eight options in the second puzzle best completes the pattern. ### Components/Axes **Puzzle 1 (3x3 Grid):** * **Cells:** Arranged in a 3x3 grid. * **Shapes:** Each cell contains a combination of two shapes: a square, a circle, or a diamond. Inside each of these shapes is a smaller shape, either a filled circle, a filled square, or a filled diamond. * **Empty Cell:** The bottom-right cell is empty. **Puzzle 2 (Numbered Cells):** * **Cells:** Eight cells numbered 1 through 8. * **Shapes:** Each cell contains a combination of two shapes: a square, a circle, or a diamond. Inside each of these shapes is a smaller shape, either a filled circle, a filled square, or a filled diamond. Cell 6 contains a white diamond inside a black square. ### Detailed Analysis or ### Content Details **Puzzle 1 (3x3 Grid):** * **Row 1:** * Cell 1: Filled circle inside a square. * Cell 2: Filled circle inside a circle. * Cell 3: Filled circle inside a diamond. * **Row 2:** * Cell 4: Filled square inside a circle. * Cell 5: Filled square inside a diamond. * Cell 6: Filled square inside a square. * **Row 3:** * Cell 7: Filled diamond inside a diamond. * Cell 8: Filled diamond inside a square. * Cell 9: Empty. **Puzzle 2 (Numbered Cells):** * **Cell 1:** Filled circle inside a circle. * **Cell 2:** Filled square inside a diamond. * **Cell 3:** Filled circle inside a diamond. * **Cell 4:** Filled circle. * **Cell 5:** Filled diamond inside a circle. * **Cell 6:** White diamond inside a black square. * **Cell 7:** Filled square inside a square. * **Cell 8:** Filled circle inside a diamond. ### Key Observations * Puzzle 1 presents a matrix of shapes, with the outer shape changing across columns (square, circle, diamond) and the inner shape changing down rows (circle, square, diamond). * Puzzle 2 provides a set of options, one of which should logically complete the pattern in Puzzle 1. * The empty cell in Puzzle 1 suggests the need to identify the missing combination of outer and inner shapes based on the established pattern. ### Interpretation The image presents a visual reasoning problem. The goal is to identify the underlying pattern in the 3x3 grid and select the appropriate shape combination from the numbered cells to complete the grid. The pattern likely involves the systematic variation of the outer and inner shapes across rows and columns. By analyzing the existing shapes, one can deduce the missing shape combination and find the corresponding option in the numbered cells. The solution requires careful observation and logical deduction. </details> problem. The 3 × 3 grid on the left shows patterns of change along the rows and columns; the challenge is to decide which figure from the eight choices on the right fits the blank square. Here the answer is 5. This example is one of the easier RPM problems; often they involve more than two shapes per square and changes in multiple attributes (shape, color, position, number, etc.) Lovett et al. 72 used SME as one component of a system to solve RPM problems. Their system did not address lower-level vision at all; the input was a vectorized image, pre-segmented (by humans) into objects placed in the grid squares. The input to SME was a set of predicate-logic descriptions of each square, generated by the CogSketch system. 33 To form these descriptions, CogSketch is programmed with a repertoire of possible object attributes (e.g., size, position, degree of symmetry) and object relations (e.g., inside/outside, intersection, rotation) that it looks for in each box in a given RPM problem. The system can also create qualitative descriptions of the edges making up objects (e.g., relations such as relative orientation, relative length, etc.) as well as descriptions of object groupings based on proximity and similarity. These predicate-logic representations are input to SME, which returns the highest-scoring mapping between descriptions of pairs of images in a row. This mapping is used by the system to determine a higher-level description of the pattern of change across each of the top two rows, from an existing vocabulary of possible changes between corresponding objects (e.g., identity, deformation, shape change, addition/removal). These higher-level descriptions of the top two rows are themselves mapped by SME to produce a description of more general pattern. SME scores each of the eight possible answers to see which one best completes the third row according to this pattern. (This brief description leaves out some additional details, such as special-purpose operations for certain kinds of problems.) Lovett et al. ran this system on the Standard Progressive Matrices test. Their system was able to solve 56 out of the 60 problems. Lovett et al.'s experiments provide an evaluation of SME as part of a larger system for making analogies in the context of RPMs. I will say more about the RPM domain in the next section. SME captures an important aspect of analogy-making: people tend to prefer systematic analogies involving more abstract concepts rather than less systematic mappings involving more superficial concepts. 40 However, I believe the SME approach is limited in its ability to capture analogy-making more generally, for three main reasons, as I describe below. Focus on syntax rather than semantics. The goal of SME is to give a domain-independent account of analogy-making; thus, true to its name, SME focuses on mapping the structure or syntax of its input representations rather than domain-specific semantics. However, this reliance on syntactic structure requires that representations of situations be cleanly partitioned into objects, attributes, functions, first-order relations, second-order relations, and so on. The problem is that humans' mental representations of real-world situations are generally not so rigidly categorized. Building on the 'bridge' example from the introduction, consider a mapping between the Golden Gate bridge and President Joe Biden's statement that he is a 'bridge' to a future generation of leaders. 85 Should 'bridge' be represented as an attribute of the object 'Golden Gate' ( bridge(GoldenGate) ), an object itself, with its own attributes (e.g., golden(Bridge) ), or a relation between the areas on either side: bridge(GoldenGate, San Francisco, MarinCounty) ? In the latter case, if the Joe Biden 'bridge' is represented only as a two-argument relation ( bridge(JoeBiden, FutureGeneration) ), it would not be paired with the three-argument Golden Gate relationship. Or what if 'future generation' is not a unitary object but an attribute (e.g., FutureGeneration(Leaders) )? Then it could not be matched with, say, the 'object' MarinCounty . These examples illustrate that real-world situations are not easily reducible to unambiguous propositions that can be mapped via syntax alone. The distinctions between 'object,' 'attribute,' and 'relation,' not to mention the order of the relation, depend heavily on context, and people (if they assign such categories at all) have to use them very flexibly, allowing initial classifications to slide if necessary at the drop of a hat. However, a major tenet of SME is that the mapping process is performed on existing representations. This naturally leads into the second issue, the separation of representation-building and mapping. Separation of representation and mapping processes. The SME approach separates 'representation-building' and 'mapping' into two distinct and independent phases in making an analogy. The base and target entities are rendered (by a human or another program) into predicate logic descriptions, which is then given to SME to construct mappings between these descriptions. Some versions of SME enable a limited 're-representation' of the base or target in response to low-scoring mappings. 31 Such re-representations might modify certain predicates (e.g., factoring them into 'subpredicates') but in the examples given in the literature, SME almost entirely relies on another module to build representations before mapping takes place. In the next section I will discuss a different view, arguing that in human analogy-making, the process of building representations is inextricably intertwined with the mapping process, and that we should adopt this principle in building AI systems that make analogies. This argument was also made in detail in several previous publications 12,36,55 and counterarguments were given by the SME authors. 31 Semi-exhaustive search. Finally, SME relies on semi-exhaustive search over matchings. The program considers matches between all 'plausible pairings' (defined by its match rules) to create multiple global matches. While this is not a problem if representations have been boiled down to a relatively small number of propositions, it is not clear that this semi-exhaustive method will scale well in general. In short, the structure-mapping theory makes some very useful points about what features appealing analogies tend to have, but in dealing only with the mapping process while leaving aside the problem of how situations become understood and how this process of interpretation interacts with the mapping process, it leaves out some of the most important aspects of how analogies are made. In addition to symbolic approaches such as SME, several hybrid symbolic-connectionist approaches have also been explored, such as the ACME and LISA systems, 57,59 neural networks to perform mapping between symbolic representations, 17 as well as systems based on cognitive models of ## memory. 63 ## Active Symbol Architecture In the 1980s, Hofstadter designed a general architecture for abstract perception and analogy-making that I'll call the 'active symbol architecture,' based in part on Hofstadter's notion of active symbols in the brain: 'active elements [groups of neurons] which can store information and transmit it and receive it from other active elements' 51 -and in part on inspiration from information processing in other complex systems such as ant colonies and cellular metabolism. The active symbol architecture was the basis for several AI programs exploring abstract perception and analogy-making, many of which were described in Ref. 53. Here I'll focus on Hofstadter and Mitchell's Copycat program. 55,80 The name 'Copycat' is a humorous reference to the idea that the act of making an analogy is akin to being a 'copycat'-that is, understanding one situation and 'doing the same thing' in a different situation. A key idea of the Copycat program is that analogy-making should be modeled as a process of abstract perception. Like sensory perception, analogy-making is a process in which one's prior concepts are activated by a situation, either perceived via the senses or in the mind's eye; those activated concepts adapt to the situation at hand and feed back to affect how that situation is perceived. Hofstadter and Mitchell developed and tested Copycat using the domain of letter-string analogy problems, created by Hofstadter. 52 The following are some examples from this domain: If the string abc changes to the string abd , what does the string pqrs change to? If the string abc changes to the string abd , what does the string ppqqrrss change to? If the string abc changes to the string abd , what does the string srqp change to? If the string abc changes to the string abd , what does the string xyz change to? If the string axbxcx changes to the string abc , what does the string pzqzrzsz change to? While these analogy problems are idealized 'toy' problems, they are, similar to Ravens' matrices, designed to capture something of the essence of real-world abstraction and analogy-making. Each string is an idealized 'situation' containing objects (e.g., letters or groupings of letters), relations among objects, events (a change from the first to second string), and a requirement for abstraction via what Hofstadter termed conceptual slippages 52 (e.g., the role of 'letter' in one situation is played by 'group' in another situation, or the role of 'successsor' in one situation is played by 'predecessor' in another situation). In the Copycat system, the process of analogical mapping between two situations (here, letter strings) is interleaved with the process of building representations of those situations, with continual feedback between these processes. This is achieved via four interacting components: a concept network , which contains the system's prior knowledge in symbolic form; a workspace , which serves as a working memory in which representation of and mappings between the input situations takes place; a set of perceptual agents , which-competitively and cooperatively-attempt to adapt the system's prior knowledge to the input situations over a series of time steps; and a temperature variable, which measures the quality and coherence of the system's representations and mappings at a given time, and which feeds back to control the degree of randomness of the perceptual agents. When the system is far from a solution, the temperature is high, and the perceptual agents' actions Figure 3: (a) Illustration of part of Copycat's concept network. (b) Illustration of Copycat's workspace, during a run of the program. <details> <summary>Image 3 Details</summary>  ### Visual Description ## Diagram: Conceptual Models of Letter Processing ### Overview The image presents two conceptual diagrams, labeled (a) and (b), illustrating different aspects of letter processing. Diagram (a) depicts a network of relationships between letters, letter groups, and associated concepts like "first," "last," "identity," etc. Diagram (b) shows a sequence of letters and a process involving a "thermometer" and letter manipulation. The two diagrams are linked by a "two-way feedback" arrow, suggesting an interaction between the two models. ### Components/Axes **Diagram (a): Network of Letter Relationships** * **Nodes:** Represented as ovals containing terms such as "letter," "letter-group," individual letters (A, B, Y, Z), and concepts like "identity," "rightmost," "leftmost," "successor," "predecessor," "first," "opposite," and "last." * **Edges:** Represented as lines connecting the nodes, indicating relationships between them. * **Letter Sequence:** A sequence of letters A, B, ... , Y, Z is shown at the top. * **Positioning:** The "letter" and "letter-group" nodes are positioned at the top of the diagram. The sequence of letters (A, B, ..., Y, Z) is below them. The remaining concept nodes are positioned below the letter sequence, connected to the letters "A" and "Z". **Diagram (b): Letter Sequence Processing** * **Letter Sequence:** A sequence of letters "a b c a b d" is shown at the top. * **Thermometer:** A vertical thermometer-like graphic is present on the left side. * **Letter Group:** A group of letters "i i j j k k" is shown below the letter sequence. The letters "k k" are enclosed in a dashed rectangle. * **Arrows:** Arrows indicate the flow and manipulation of letters. Curved, dotted arrows show relationships between letters in the top sequence. A solid arrow connects "c" in the top sequence to "k" in the bottom sequence. Solid arrows also indicate the direction of processing in both sequences. **Link between Diagrams** * **Two-way feedback:** A double-headed arrow labeled "two-way feedback" connects diagram (a) and diagram (b), indicating a reciprocal relationship between the two models. ### Detailed Analysis or ### Content Details **Diagram (a):** * The "letter" node is connected to the "letter-group" node. * The letters A, B, ..., Y, and Z are connected in a sequence. * "A" is connected to "first". * "Z" is connected to "last". * "rightmost" is connected to "leftmost". * "successor" is connected to "predecessor". * "identity" is connected to "opposite". * "first" is connected to "leftmost" and "opposite". * "last" is connected to "rightmost" and "opposite". * "identity" is connected to "A" and "Z". * "successor" is connected to "Z". * "predecessor" is connected to "A". **Diagram (b):** * The letter sequence "a b c a b d" is present. * Curved, dotted arrows connect "a" to "b", "b" to "c", and "c" back to "a". Another curved, dotted arrow connects "a" to "b". * A solid arrow points from "c" in the top sequence to "k" in the bottom sequence. * The letter sequence "i i j j k k" is present. * Curved, dotted arrows connect "i" to "j" and "j" to "k". * The letters "k k" are enclosed in a dashed rectangle, and a solid arrow points to the right, indicating further processing. * The thermometer is filled to approximately 25% of its capacity. ### Key Observations * Diagram (a) represents a static network of relationships, while diagram (b) represents a dynamic process. * The "two-way feedback" arrow suggests that the static relationships in diagram (a) influence the dynamic process in diagram (b), and vice versa. * The thermometer in diagram (b) might represent a level of activation or some other internal state. * The dashed rectangle around "k k" in diagram (b) suggests a specific operation or selection. ### Interpretation The diagrams likely represent different levels of abstraction in a cognitive model of letter processing. Diagram (a) describes the conceptual relationships between letters and related concepts, providing a framework for understanding letter properties and their roles within words or sequences. Diagram (b) illustrates a specific process involving letter manipulation, potentially representing a reading or writing task. The "two-way feedback" suggests that the conceptual knowledge in diagram (a) informs the processing in diagram (b), and the results of that processing may update or refine the conceptual knowledge. The thermometer could represent the level of attention or cognitive load during the processing. The selection of "k k" might indicate a focus on specific letter combinations or patterns. </details> are more random; as the system zeroes in on a coherent solution, the temperature falls, and the perceptual agents are more deterministic. Figure 3 illustrates the architecture of the Copycat program. Figure 3(a) illustrates part of the program's concept network , which contains the program's prior (symbolic) knowledge about the letter-string domain, corresponding to long-term memory. The concept network models a symbolic 'semantic space,' in which concepts are nodes (ellipses) and links (lines) between between concepts represent semantic distance, which can change with context during a run of the program. A concept (e.g., letter-group ) is activated when instances of that concept are discovered in the workspace, and in turn, activated concepts (the program's 'active symbols') trigger perceptual agents that attempt to discover additional instances. Activation can also spread between conceptual neighbors. Activation decays over time if not reinforced. Figure 3(b) illustrates the program's workspace , a short-term memory, inspired by blackboard systems, 23 in which perceptual agents construct (and sometimes destroy) structures (relations, groupings, correspondences, and rules) that form the program's current representation of the input situations and the analogy between them, at any given time during a run. Dashed lines or arcs represent structures with low confidence; solid lines or arcs represent structures with high confidence; the confidence of a structure can change during the run and structures can be destroyed depending on their confidence. A temperature variable (represented by the thermometer in the bottom right of the workspace) measures the quality of the current structures and feeds back to affect the randomness of the perceptual agents. Figure 4 gives the state of the workspace at selected timesteps during a run of the program, illustrating how the program constructs representations of, and analogies between, its input situations. The workspace serves as a global blackboard on which agents explore, build, and destroy possible structures. The actions of agents are probabilistic, and depend on the current state of the workspace, concept network, and temperature. Perceived correspondences between objects in different situations (here letters and letter-groups) lead to conceptual slippages (e.g., letter slips to letter-group ) that give rise to a coherent analogy. Details of Copycat's operations are described in Ref. 80. In summary, the Copycat program is an example of Hofstadter's active symbol architecture, in <details> <summary>Image 4 Details</summary>  ### Visual Description ## Diagram: String Transformation Over Time ### Overview The image depicts a series of diagrams illustrating the transformation of a string "a b c -> a b d" over several timesteps. Each diagram shows the string transformation, the state of a secondary string "i i j j k k", and a thermometer indicating a value that decreases over time. The diagrams also include a rule that governs the string transformation at each step. ### Components/Axes * **Timestep:** The title of each diagram indicates the timestep (45, 165, 225, 255, 495, 605). * **String Transformation:** Each diagram shows the transformation of the string "a b c" to "a b d" using arrows. * **Secondary String:** Each diagram shows a secondary string "i i j j k k" which is also transformed over time. * **Thermometer:** A thermometer is present in each diagram, indicating a value that decreases over time. The values are approximately 98, 68, 59, 59, 38, and 20. * **Rule:** Each diagram includes a rule that governs the string transformation. ### Detailed Analysis **Timestep: 45** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k ->". * Thermometer: Approximately 98. * Rule: No rule is explicitly stated. **Timestep: 165** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k ->". The 'k' is highlighted. * Thermometer: Approximately 68. * Rule: "Replace letter-category of rightmost letter by 'd'". **Timestep: 225** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k ->". The 'k' is highlighted. * Thermometer: Approximately 59. * Rule: "Replace letter-category of rightmost letter by successor". **Timestep: 255** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k ->". The 'k' is highlighted. * Thermometer: Approximately 59. * Rule: "Replace letter-category of rightmost letter by successor". **Timestep: 495** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k ->". The 'k' is highlighted. * Thermometer: Approximately 38. * Rule: "Replace letter-category of rightmost letter by successor". **Timestep: 605** * String Transformation: "a b c -> a b d". The 'c' is being replaced with 'd'. * Secondary String: "i i j j k k -> i i j j l l". The 'k' is being replaced with 'l'. * Thermometer: Approximately 20. * Rule: "Replace letter-category of rightmost letter by successor". * Additional Label: "letter -> letter-group" * Translated Rule: "Translated Rule: Replace letter-category of" ### Key Observations * The string "a b c" is consistently transformed to "a b d". * The thermometer value decreases over time. * The rule changes at timestep 165. * The secondary string "i i j j k k" is transformed to "i i j j l l" at timestep 605. ### Interpretation The diagrams illustrate a process where a string transformation is influenced by a rule that changes over time. The decreasing thermometer value might represent a resource or energy level that is being depleted as the process progresses. The transformation of the secondary string suggests that the rule is affecting not only the primary string but also other related elements. The change in the rule at timestep 165 indicates a shift in the transformation process, potentially triggered by the depletion of the resource represented by the thermometer. The final transformation of the secondary string at timestep 605 suggests a culmination of the process, where the rule has led to a significant change in the secondary string. </details> TranslatedRule:Replaceletter-category of rightmost letter-group by successor Figure 4: State of the workspace at six different timesteps during a run of Copycat (adapted from Ref. 80). which symbolic concepts become activated via bottom-up perceptions, spread activation to semantically related neighbors, and influence perception in a top-down manner by triggering probabilistic perceptual agents to find instances of the associated concepts in a blackboard-like workspace. In this way, processing in the system consists of a continual interplay between bottom-up and topdown processes. A temperature variable controls the degree of randomness in the system and in turn is dynamically determined by the quality of perceptual structures constructed by the system. Coherent representations of input situations, and analogies between them, result from the perceptual structures constructed by these probabilistic agents. A central idea underlying the active symbol architecture is that, in analogy-making, the mapping process cannot be separated from the representation-building process-these must be interleaved. This is a central point of disagreement with the structure-mapping engine approach described in the previous section (see also Ref. 12). As described in Ref 55, Copycat's emergent dynamics show a gradual transition from a largely bottom-up (perception-driven), random, and parallel mode of processing-in which many possible representations of the input are explored--to one that is largely top-down (concept-driven), deterministic, and serial. Copycat does not fit neatly into the traditional AI dichotomy between symbolic and neural systems; rather it incorporates symbolic, subsymbolic, and probabilistic elements. The architecture resonates with several ideas in psychology, psychophysics, and neuroscience, such as the Global Workspace hypothesis of Baars et al., 3,98 in which multiple, parallel, specialist processes compete and cooperate for access to a global workspace, and the proposal that visual cortex areas V1 and V2 work as ''active blackboards' that integrate and sustain the result of computations performed in higher areas.' 42,89 Copycat also resonates with the idea of neural 'object files' 62 -temporary and modifiable perceptual structures, created on the fly in working memory, which interact with a permanent network of concepts. The system's dynamics are also in accord with Treisman's 104 notion of perception as a shift from parallel, random, 'pre-attentive' bottom-up processing and more deterministic, focused, serial, 'attentive' top-down processing. Mitchell and Hofstadter showed how Copycat was able to solve a wide selection of letter-string problems; they also described the program's limitations. 55,80 The Copycat program inspired numerous other active-symbol-architecture approaches to analogy-making, some of which are described in Ref. 53, as well as approaches to music cognition, 82 image recognition, 87 and more general cognitive architectures. 3 It bears repeating that Copycat was not meant to model analogy-making on letter strings per se. Rather, the program was meant to illustrate-using the letter-string analogy domain-a domainindependent model of high-level perception and analogy. However, the program has several limitations that need to be overcome to make it a more general model of analogy-making. For example, Copycat's concept network was manually constructed, not learned; the program illustrated how to adapt pre-existing concepts flexibly to new situations, rather than how to learn new concepts. Moreover, the program was given a 'source' and 'target' situation to compare rather than having to retrieve a relevant situation from memory. Finally, the program's architecture and parameter tuning were complicated and somewhat ad hoc. Additional research on all of these issues is needed to make active symbol architectures more generally applicable. ## Deep Learning Approaches At the other end of the spectrum from symbolic approaches are deep learning approaches, in which knowledge is encoded as high-dimensional vectors and reasoning is the manipulation of these representations via numeric operations in a deep neural network. Moreover, in deep learning, knowledge representation, abstraction, and analogy-making abilities must be learned, typically via large training sets. Here, I won't survey the substantial literature on pre-deep-learning connectionist modeling of analogy-making (though see, e.g., Refs. 4,36). Probably the best-known result of the (post-2010) deep learning era on analogy are the proportional analogies (e.g., 'man is to woman as king is to ?') that can arise from vector arithmetic on word embeddings. 79 More recently, numerous papers have been published applying deep learning methods to make analogies between words or simple images 73,86,92 as well as on simplified Ravens-Progressive-Matrix-like problems. 47,111 In addition, one group demonstrated a deep neural network that can learn to make mappings on symbolic representations (like the ones in Figure 1) that roughly agree with the mappings made by SME. 17 Attempts to get deep learning systems to learn abstract concepts and create analogies are fascinating as ways to explore what these architectures are capable of learning and what kind of reasoning they are able to do. However, these investigations can be complicated by the amount of data needed to train deep learning systems, as well as by their lack of transparency. In this paper, as an illustrative example of the promise and pitfalls of deep learning approaches to abstraction and analogy, I'll describe in detail a series of attempts to use deep neural networks on Ravens Progressive Matrices. This domain has became a popular benchmark in the deep learning community. As I described above, the symbolic Structure Mapping Engine was tested on part of a standard 60problem RPM test. However, 60 problems is orders of magnitude too small a dataset to successfully train and test a deep neural net. In order to apply deep learning to this task, researchers need an automated way of generating a very large number of distinct, well-formed, and challenging RPM problems. In 2015, Wang and Su 110 proposed an RPM-problem-generation method, guided by an earlier analysis of RPM problems. 11 Their method generated a problem by repeatedly sampling from a fixed set of object, attribute and relation types. Building on this approach, Barrett et al. 5 proposed a related method to produce a large dataset of RPM problems, which they called Procedurally Generated Matrices (PGMs). Each PGM is a set S of triples, where each triple consists of a specific relation, an object type, and an attribute. The generating process first samples from a fixed set of relation, object, and attribute types to create between one and four triples. The generating process then samples allowed values for the object types and attributes. All other necessary attributes (e.g., number of objects) are sampled from allowed values that do not induce spurious relationships. The PGM S is then rendered into pixels. See Ref. 5 for more details on this process, though the paper did not give details on how the eight candidate answers were generated. Barrett et al. used this generation method to produce their PGM corpus of Raven-style problems. They split the corpus into 1.2M training problems, 20K validation problems, and 200K test problems. They used these splits to train and test several deep learning methods, including their own novel method, the Wild Relation Network (WReN), which utilized Relation Network modules. 96 For each network, the input was the eight matrix panels and the eight candidate-answer panels (i.e., 16 grayscale images). The output was a probability distribution over the eight candidate answers. Each network was trained to maximize the probability of the correct answer for a given input. Random guessing would yield 12.5% accuracy. Among their many experiments, the authors found that their WReN method achieved the best performance of the various networks, an impressive 63% accuracy on the test problems for the so-called 'neutral split' (training and test set can contain problems generated with any triples). Other training/test splits that required extrapolation from a restricted set of triples or attribute values to a different set produced notably poorer performance. Subsequent to the work of Barrett et al., several other groups developed new deep-learning-based methods that improved state-of-the-art accuracy on the PGM corpus. 45,60,102,118 In a 2019 paper, Zhang et al. 115 questioned whether the PGM dataset was diverse enough to be a good test of abstract reasoning abilities. They noted that even though the PGM corpus is large (over 1.2 million problems), the PGM generation procedure is limited in the possible kind of problems that can be generated. Zhang et al. noted that 'PGM's gigantic size and limited diversity might disguise model fitting as a misleading reasoning ability, which is unlikely to generalize to other scenarios.' In other words, the PGM corpus might make it too easy for learning systems to overfit to its particular, limited types of problems. To remedy this limitation, Zhang et al. devised a new stochastic image-grammar method for generating a more diverse, comprehensive set of Ravens-like problems. They generated a new 70,000problem corpus called RAVEN-smaller in size than PGM, but more diverse. Using a split with 42,000 training examples and 14,000 test examples, Zhang et al. compared some of the same deep learning methods examined by Barrett et al., plus their own novel 'dynamic residual tree' method, which dynamically builds a tree-structured computation graph. They found that, whereas Barrett et al.'s WReN method had relatively high accuracy on the PGM dataset, its accuracy on the RAVEN dataset was 15%, barely above chance. The highest scoring method, with 60% accuracy, was a Residual Network 46 (ResNet) using features computed by a dynamic residual tree. The authors also tested humans (college students) on the RAVEN corpus and found human accuracy to be about 84%. Other groups were able to improve on the machine-learning accuracy on RAVEN with larger, pretrained networks 119 and contrastive learning, 116 among other novel mechanisms. However, Hu et al. 58 discovered a major flaw in the RAVEN dataset. Recall that there are eight candidate answers, only one of which is correct. In RAVEN, the seven incorrect answers were generated by randomly modifying a single attribute of the correct answer, as illustrated in Figure 5. Figure 5: Illustration of answer-generation method for RAVEN dataset. Given the correct answer (here, black pentagon at top), each incorrect candidate answer is generated by changing one attribute of the correct answer. Adapted from from Ref. 58. <details> <summary>Image 5 Details</summary>  ### Visual Description ## Diagram: Hierarchical Diagram of Shapes ### Overview The image is a hierarchical diagram showing a single shape at the top branching down to eight different shapes at the bottom. Arrows indicate the relationship between the top shape and each of the bottom shapes. ### Components/Axes * **Top Shape:** A black pentagon inside a square. * **Bottom Shapes (left to right):** 1. Black pentagon inside a square, labeled "1". 2. Black triangle inside a square, labeled "2". 3. Gray pentagon inside a square, labeled "3". 4. Pentagon with vertical stripes inside a square, labeled "4". 5. Black circle inside a square, labeled "5". 6. White pentagon inside a square, labeled "6". 7. Small black pentagon inside a square, labeled "7". 8. Shape with horizontal stripes inside a square, labeled "8". * **Arrows:** Eight arrows pointing from the top shape to each of the bottom shapes. ### Detailed Analysis or Content Details The diagram illustrates a one-to-many relationship. The top shape is connected to each of the eight shapes below it. The shapes at the bottom vary in color, size, and internal design. ### Key Observations * The top shape is a black pentagon. * The bottom shapes include variations of pentagons, a triangle, and a circle. * The arrows indicate a hierarchical relationship, possibly representing categories or derivations. ### Interpretation The diagram likely represents a classification or derivation process. The top pentagon could be a parent category, and the bottom shapes represent different subcategories or variations derived from it. The diagram does not provide specific labels or context, so the exact meaning is open to interpretation. It could be used to illustrate concepts like inheritance, diversification, or categorization in various fields such as computer science, biology, or design. </details> In the figure, answer #1 is the correct answer, and each of the other answers differs from it in exactly one attribute. But this allows a possible shortcut to determining the correct answer: just take the majority vote among the figures' attributes, here, shape, fill-pattern, and size. Indeed, when Hu et al. trained a ResNet only on the eight answer panels in each training example in RAVEN (i.e., leaving out the matrix itself), the ResNet achieved an accuracy of about 90%, competitive with the state of the art reported in the original RAVEN paper. This discovery casts doubt on previous claims that deep neural networks that perform well on the RAVEN dataset are actually solving the task that the authors intended them to solve. Hu et al. proposed a more complex sampling method to create what they claimed was an 'unbiased answer set' for each problem; the result is what they call 'Impartial-RAVEN.' The authors showed that some deep learning methods reported to do well on RAVEN did much more poorly on ImpartialRAVEN. Hu et al. also described a new method that outperformed other methods. ImpartialRAVEN (previously called 'Balanced-RAVEN') has become the subject of further competition for state-of-the-art accuracy. 114,117 In addition, 112 demonstrated a mechanism for variable-binding in a neural-network system that performed well on simple abstraction problems including simplified versions of Ravens Progressive Matrices. In this section I've reviewed some recent work on using deep learning for abstraction and analogymaking in Ravens-like problems. While this is only one example of work on deep learning in this area, it serves as an illustrative microcosm, and highlights some of the advantages and limitations of such approaches. On the advantages side, deep learning avoids some of the issues I discussed with SME and Copycat. Deep learning approaches learn end-to-end from pixels, so avoid relying on built-in knowledge. Unlike SME, there is no separation of representation-building and mapping, and no semi-exhaustive search over possible matchings. However, these advantages are at the expense of two major limitations: the need for a huge training set, and the lack of transparency in what has been learned and how it is applied to new problems. The procedural generation of problems leaves open possibility of biases that allow shortcuts -the networks learn ways of performing well on the task that do not require the abstraction and analogy-making mechanisms that we assume humans bring to the task, and that we are trying to capture in machines. This can be seen in a simple way with biases in the RAVENS set, but such biases can be more subtle and hard to perceive, and have been identified time and again in machine-learning systems. 37,39,76 It's not clear that benchmarks such as Balanced-Raven are free from such biases. This, coupled with the lack of transparency about how the networks accomplish their tasks, makes it unclear what these networks have learned, and what features they are basing their decisions on. All this leads to the conclusion that accuracy alone can be a misleading measure of performance. I will propose additional evaluation metrics that might be more informative in the discussion section below. Even setting aside these problems with biases and lack of transparency, the approach of training such a system with a large set of examples is questionable, given the original intention of RPMs as a way of measuring general human intelligence. Humans don't need to train extensively in advance to perform well on RPMs; in fact, doing so would invalidate the test's ability to measure general intelligence. Rather, the idea is that humans learn how to do successful abstraction and analogymaking in the world, and intelligence tests such as Raven's are meant to be a measure of these general skills. In contrast, the neural networks that do well on Raven's test after extensive training are unable to transfer any of these skills to any other task. It's not clear that such approaches bring us any closer to the goal of giving machines general abilities for abstraction and analogy-making. A potentially more promising (though less explored) set of approaches to abstraction uses metalearning 109 -enabling a neural network to adapt to new tasks by training it on a training set of tasks rather than only on examples from a single task. Several meta-learning approaches for few-shot learning have been evaluated on Lake et al.'s Omniglot dataset, 64 among other benchmarks 30,88,95,100 . Meta-learning methods produced substantially improved generalization abilities for agents in a grounded-language task in a simple simulated environment, 49 and improved fewshot learning accuracy in a simplified version of Bongard problems. 83 However, at the time of this writing, meta-learning approaches have not yet been explored for other abstraction and analogy domains. Another proposed approach is that of 'meta-mapping' 69 which directly maps a representation of one task to a representation of a related task. ## Probabilistic Program Induction So far I have covered methods illustrating symbolic and deep learning approaches. In this section I'll describe a different family of methods for concept learning and analogy-making: probabilistic program induction methods, in which a space of possible programs is defined (often via a program grammar) and a probability distribution is computed over this space with respect to a given task. Solving the task amounts to sampling from the space of possible programs guided by this probability distribution. Here, the task is concept induction, and a concept is identified with a program. I will illustrate probabilistic program induction by describing its application to two task domains: (1) recognizing and generating handwritten characters, and (2) solving Bongard problems. As I will discuss, program-induction methods can combine both symbolic and neural-network representations. ## Recognizing and Generating Handwritten Characters Lake et al. 64 applied probabilistic program induction to the task of learning visual concepts of handwritten characters from 50 different writing systems-the 'Omniglot' task. The goal was to study one-shot learning -learning a concept from a single example. Here, learning a concept means seeing one example of a handwritten character and being able to not only recognize new examples of this character, but also being able to generate new examples. In this work, a concept is represented as a program that generates examples of the concept. In Omniglot, a program representing a concept consists of a hierarchical character-generation procedure. A character consists of a number of parts (complete pen strokes), each of which itself is made of sub-parts (movements separated by pauses of the pen). The generation procedure consists of sampling from a hierarchy of generative models describing the probabilities of the sub-parts, parts, and relationships among them. See Figure 6 for an example. Figure 6: Illustration of the generative model used by Lake et al. for generating Omniglot characters, given a set of primitive pen strokes. To generate a new character, first sample the number of parts; then sample the number of sub-parts that will make up each part; then sample these sub-parts from the primitives set; then sample possible sequences of the sub-parts to make a part; then sample relations between parts (from a library of possible relations). This sequence of samples is the program for generating the character concept. To generate a token of the character concept (i.e., a rendered character), run the character-concept program with motor noise added to selected points. Adapted from Ref. 64. <details> <summary>Image 6 Details</summary>  ### Visual Description ## Diagram: Character Generation Process ### Overview The image illustrates a process for generating character tokens from primitive shapes. It outlines a series of steps involving sampling parts, sub-parts, and their relationships to form a character concept, which is then used to generate a character token. ### Components/Axes * **Primitives:** A set of basic shapes used as building blocks. The primitives shown are a vertical line with an arrow pointing down, a horizontal line with an arrow pointing right, a curved arrow pointing right, a curved shape resembling a "2", a curved shape resembling a "q", and an ellipsis (...). * **Steps:** A numbered sequence of operations. ### Detailed Analysis 1. **Sample number of parts:** The number of parts is given as 2. 2. **Sample number of sub-parts for each part:** The number of sub-parts is given as 2 for the first part and 1 for the second part (2, 1). 3. **Sample sub-parts:** The sub-parts are shown as a curved arrow pointing right and a curved shape resembling a "2". 4. **Sample sub-part sequences to create parts:** The sequences are shown as a shape resembling a "3" and a curved shape resembling a "2". 5. **Sample relation between parts:** The relation between parts is illustrated by a shape resembling a "3" connected to a curved shape resembling a "2" with a dotted arrow. 6. **Return program for character concept:** The character concept is represented by a shape resembling a "32". 7. **Run character concept program to generate character token:** The final character token is a stylized shape resembling a "32". ### Key Observations * The process starts with simple geometric primitives and builds up to a more complex character token. * The steps involve sampling and combining sub-parts to create larger parts, and then defining relationships between these parts. * The final character token is a result of running a program based on the character concept. ### Interpretation The diagram demonstrates a generative approach to character design, where complex shapes are constructed from simpler primitives through a series of sampling and combination steps. This process could be used in computer graphics or handwriting recognition to generate or interpret characters. The use of sampling suggests a stochastic element, where different combinations of sub-parts could lead to variations in the final character token. </details> How is this hierarchy of generative models learned? In Ref. 64 the authors describe a system that obtains prior knowledge from a training set of human-drawn characters spanning 30 alphabets (taken from the Omniglot dataset). The training set included the human-drawn pen strokes for each character, as well as the final character images. The learning system collected a library of primitive pen strokes and learned probability distributions over features of these pen strokes (e.g., starting positions). It also learned transition probabilities between strokes and probabilities over types of relations among the strokes, among other probabilities. All in all, this collection of probabilities is the prior knowledge of the system. The system used this prior knowledge in a Bayesian framework for the tasks of one-shot classification and generation of characters not contained in the training set. (The authors also explored related tasks but I won't cover those here.) In the one-shot classification task, the system was presented with a single test image I ( t ) of a new character of class t , along with 20 distinct characters I ( c ) in the same alphabet produced by human drawers. Only one of the 20 was the same class as I ( t ) . The task was to choose that character from the 20 choices. The Omniglot system computed an approximation to the probability P ( I ( t ) | I ( c ) ) for each of the 20 I ( c ) , and chose the I ( c ) that yielded the highest probability. By Bayes rule, <!-- formula-not-decoded --> The term P ( I ( t ) ) is approximated by a probabilistic search method to generate a program to represent I ( t ) of the form shown in Figure 6; the prior probabilities learned from the original training set can be used to approximate P ( I ( t ) ). The term P ( I ( c ) | I ( t ) ) can be approximated by attempts to 'refit' the program representing I ( t ) to I ( c ) . See Ref. 65 for details. Figure 7: Four sample Bongard Problems (adapted from Foundalis' website. 34 ) <details> <summary>Image 7 Details</summary>  ### Visual Description ## Abstract Shapes: Visual Discrimination Task ### Overview The image presents a visual discrimination task, displaying sets of abstract shapes within a grid format. There are four distinct blocks, labeled "BP #2", "BP #7", "BP #57", and "BP #91", each containing a 3x4 grid of individual cells. Each cell contains one or more simple geometric shapes or abstract line drawings. ### Components/Axes * **Labels:** The blocks are labeled as "BP #2", "BP #7", "BP #57", and "BP #91". These labels are located at the top of each block. * **Grid:** Each block is organized as a 3x4 grid, with 12 cells in total. * **Shapes:** Each cell contains one or more abstract shapes or line drawings. These shapes vary in form, size, and arrangement. ### Detailed Analysis **Block BP #2:** * **Row 1:** * Cell 1: A triangle. * Cell 2: A circle. * Cell 3: A square. * Cell 4: A small filled circle. * **Row 2:** * Cell 1: An amoeba-like shape. * Cell 2: An irregular polygon. * Cell 3: A small square with a corner marker. * Cell 4: A small curved line. * **Row 3:** * Cell 1: A semi-circular shape. * Cell 2: A filled irregular polygon. * Cell 3: A triangle. * Cell 4: A shape resembling a stylized letter "H". **Block BP #7:** * **Row 1:** * Cell 1: An oval shape. * Cell 2: A series of short parallel lines stacked vertically. * Cell 3: A rectangle. * Cell 4: A rectangle with extensions on both sides. * **Row 2:** * Cell 1: A spring-like shape. * Cell 2: A shape resembling a stylized letter "I" with extensions. * Cell 3: A wavy line. * Cell 4: A series of short parallel lines stacked horizontally. * **Row 3:** * Cell 1: A long, thin oval shape. * Cell 2: A wavy line. * Cell 3: An oval shape. * Cell 4: A wavy line. **Block BP #57:** * **Row 1:** * Cell 1: Three triangles. * Cell 2: Two circles. * Cell 3: A triangle and a square. * Cell 4: A square. * **Row 2:** * Cell 1: A square. * Cell 2: Two bean-like shapes filled. * Cell 3: An upside-down triangle and a triangle. * Cell 4: A shape resembling a stylized letter "P". * **Row 3:** * Cell 1: A shape resembling a stylized letter "P". * Cell 2: Two triangles. * Cell 3: A circle. * Cell 4: A triangle and a square. **Block BP #91:** * **Row 1:** * Cell 1: A three-pronged shape. * Cell 2: A triangle. * Cell 3: A square. * Cell 4: A shape resembling a stylized flower. * **Row 2:** * Cell 1: A square and a diamond. * Cell 2: A shape resembling a stylized gear. * Cell 3: A curved line with small line segments. * Cell 4: Multiple circles. * **Row 3:** * Cell 1: Multiple short vertical lines. * Cell 2: A curved line with filled circles. * Cell 3: An "X" shape. * Cell 4: Multiple ovals. ### Key Observations * Each block contains a unique set of shapes and arrangements. * The shapes are simple geometric forms or abstract line drawings. * The arrangement of shapes within each cell varies. * Some cells contain multiple shapes. ### Interpretation The image likely represents a visual discrimination task used in psychology or cognitive science. The "BP" labels might refer to different "block patterns" or experimental conditions. Participants would likely be asked to identify differences or similarities between the patterns, assess their complexity, or match them to target stimuli. The task could be designed to assess visual perception, pattern recognition, or cognitive processing speed. The variety of shapes and arrangements suggests a focus on testing different aspects of visual discrimination abilities. </details> In the experiments reported by Lake et al., 64 the Omniglot system's performance on one-shot classification matched or exceeded that of the humans tested on this task. The one-shot generation task was, given an example image I ( c ) of a hand-drawn character of class c, to generate a new image of a character of class c . The Omniglot system did this by first searching to find a program representing I ( c ) as above, and then running this program as in Figure 6 to generate a new example. Lake et al. enlisted human judges to compare the characters generated by their system with those generated under the same one-shot conditions by humans-what they called a 'visual Turing Test.' The judges were typically not able to distinguish between machine-generated and human-generated characters. Feinman and Lake 29 proposed an interesting 'neuro-symbolic' extension of Lake et al.'s system that integrated neural networks with probabilistic programs to learn generative models of handwritten characters. ## Solving Bongard Problems A second example of recent work on probabilistic program induction is Depeweg et al.'s system for solving Bongard programs. 19 Here the idea is to induce a rule rather than a runnable program, but the general idea is the same. Bongard problems are visual concept recognition tasks, first presented in M. Bongard's 1970 book Pattern Recognition . 9 Figure 7 shows four sample problems from Bongard's collection. For each problem the task is to identify the concept that distinguishes the set of six frames on the left from the set of six on the right (e.g., large vs. small or three vs. four ). Often the concept is simple to express, but is represented quite abstractly in the figures (e.g., BP #91). Bongard devised 100 such problems as a challenge for artificial intelligence systems. Harry Foundalis created a web site 34 to make available the original 100 problems plus many additional Bongard-like problems created by other people. It is notable that in the decades since Bongard's book was published, no artificial vision system has come close to solving all of Bongard's original problems. Figure 8: (a) Human-designed grammar for rule induction. (b) Sample Bongard problem, with rule derivation and corresponding rule expression ('the left side has objects that contain triangles.') (Adapted from Ref. 19.) <details> <summary>Image 8 Details</summary>  ### Visual Description ## Grammar and Shape Examples ### Overview The image presents a set of grammar rules and examples of shapes generated based on those rules. Part (a) shows the grammar rules, while part (b) illustrates the application of these rules to generate specific shape configurations. ### Components/Axes **Part (a): Grammar Rules** * **R (Relation)**: * `R -> LEFT : S` * `R -> RIGHT : S` * **S (Scene)**: * `S -> ∃(L)` * `S -> #(N, L)` * `S -> =(L, L)` * `S -> >(L, L)` * `S -> >(L, A)` * `S -> >(L, L, A)` * `S -> (L, A)` * `S -> (L, L, A)` * **L (Location/Object)**: * `L -> ∩(L, L)` * `L -> ∪(L, L)` * `L -> \(L, L)` * `L -> INSIDE(L)` * `L -> CONTAINS(L)` * `L -> ALIGNED(L)` * `L -> GET(L, T)` * `L -> SOLID(L)` * `L -> OUTLINE(L)` * `L -> LARGE(L)` * `L -> SMALL(L)` * `L -> HIGH(L, A)` * `L -> LOW(L, A)` * `L -> FIGURES` * `L -> CIRCLES` * `L -> TRIANGLES` * `L -> RECTANGLES` * **A (Attribute)**: * `A -> XPOS` * `A -> YPOS` * `A -> DISTANCE` * `A -> ORIENTATION` * `A -> NCORNERS` * `A -> COLOR` * `A -> SIZE` * `A -> COMPACTNESS` * `A -> CONVEXITY` * `A -> ELONGATION` * **T (Thing)**: * `T -> HULLS` * `T -> HOLES` * **N (Number)**: * `N -> 1` * `N -> 2` * `N -> 3` * `N -> 4` **Part (b): Shape Examples and Production Rules** * **Shape Grid**: A 3x3 grid displaying different arrangements of circles and triangles. * **Production Rules**: * `R -> Left: S` * `S -> ∃(L)` * `L -> Contains(L)` * `L -> Triangles` * **Expressions**: * `Left: S` * `Left: ∃(L)` * `Left: ∃(Contains(L))` * `Left: ∃(Contains(Triangles))` * **Rule**: * `Left: ∃(Contains(Triangles))` ### Detailed Analysis or ### Content Details **Part (a): Grammar Rules** The grammar rules define how different elements (relations, scenes, locations/objects, attributes, things, and numbers) can be combined. For example, a scene (S) can be defined as the existence of a location/object (∃(L)), or a relation between two locations/objects (= (L, L)). Locations/objects (L) can be defined by their spatial relationships (INSIDE, CONTAINS, ALIGNED), attributes (LARGE, SMALL, HIGH, LOW), or the types of figures they represent (CIRCLES, TRIANGLES, RECTANGLES). **Part (b): Shape Examples and Production Rules** The shape grid provides visual examples of how the grammar rules can be applied. Each cell in the grid contains a different arrangement of circles and triangles. The production rules and expressions show how a specific rule (e.g., "L -> Contains(L)") can be expanded to generate a more complex expression (e.g., "Left: ∃(Contains(Triangles))"). The top-left shape in the grid shows a circle containing a triangle. The top-middle shape shows a triangle and a circle, with the circle partially overlapping the triangle. The top-right shape shows a circle and a triangle, with the triangle partially overlapping the circle. The middle-left shape shows a triangle containing a circle. The middle-center shape shows a circle containing a triangle, with another circle partially overlapping the triangle. The middle-right shape shows a triangle and a circle, with the triangle partially overlapping the circle. The bottom-left shape shows a triangle and a circle, with the triangle partially overlapping the circle. The bottom-middle shape shows a triangle. The bottom-right shape shows a circle and a triangle, with the circle partially overlapping the triangle. ### Key Observations * The grammar rules provide a formal way to describe and generate spatial arrangements of objects. * The shape grid illustrates how these rules can be used to create different visual patterns. * The production rules and expressions show the step-by-step application of the grammar rules. ### Interpretation The image demonstrates a system for generating and describing visual scenes using a formal grammar. The grammar rules define the relationships between different elements, while the shape grid provides concrete examples of how these rules can be applied. This approach could be used in various applications, such as computer vision, image generation, and visual communication. The system allows for the creation of complex visual patterns from a set of simple rules, providing a powerful tool for generating and understanding visual information. </details> Douglas Hofstadter has written extensively about Bongard problems, starting with his 1979 book G¨ odel, Escher, Bach: an Eternal Golden Braid , 51 in which he sketched a rough architecture for solving them. (Notably, this sketch became the basis for the Active Symbol Architecture that I described above.) In contrast to the attention garnered in the AI community by Ravens Progressive Matrices, Bongard problems have seen less attention. In a 1996 paper, Saito and Nakano 93 showed that an inductive logic programming approach could solve 41 of the original 100 problems, starting not from the raw pixels but from logic formulas created by humans to represent each problem. In his 2006 PhD dissertation, Foundalis 35 described the construction of a 'cognitive architecture inspired by Bongard's problems,' whose input was raw pixels of the 12 frames and whose output was an English phrase describing one or both sides of the problem. Foundalis's architecture was meant to model human concept induction in general rather than Bongard problems specifically, and was able to reliably solve about 10 problems. Here I'll describe Depeweg et al.'s 19 probabilistic rule-induction approach, which was inspired by the Omniglot system described above. Depeweg et al.'s system's task was to input the raw pixels of the 12 frames, and to output a rule R -in a logic-like language-that is true of all frames on one side (left or right) and none of the frames on the other side. (Note that this differs from Bongard's original task, which was to output English-language expressions contrasting the left and right sets of boxes.) In Depeweg et al.'s system, the space of possible rules is given by a human-designed grammar, given in Figure 8(a). A rule can be derived from this grammar by choosing one of the sides (right or left), starting with the start symbol S and probabilistically choosing one of the possible expansions of that symbol (e.g., S →∃ ( L )), then probabilistically choosing possible expansions of the symbols in that expression (e.g., L → Contains ( L )) and so on until all the variables have been expanded. Given a Bongard problem, the goal is to search the possible space of rules to find the most probable rule. The authors define a probability distribution P ( R | E,G ) over possible rules. Here R is a rule, E = ( E LEFT , E RIGHT ) is the set of 12 input frames, and G is the grammar. By Bayes rule, this probability distribution can be factored into prior probability P ( R | G ) and likelihood P ( E | R ): <!-- formula-not-decoded --> The authors define the likelihood P ( E | R ) as equal to 1 if two conditions are met, and equal to zero otherwise. The conditions are (1) R is true of all the frames on one side (left or right) and none of the frames on the other side, and (2) the frames in E are 'informative' about R , meaning that the frames must contain the objects or relationships that are mentioned in the rule. For example, the rule 'there exist triangles or squares inside circles' is true of all the frames on the left side of Figure 8(b), but since no squares actually appear, these frames are not considered to be informative about the rule, and the rule would be given zero likelihood. The prior probability term P ( R | G ) is defined by the authors in a way that favors shorter rules, along with some other structural properties of rules (see Ref. 19 for details). Given the raw pixels of a Bongard problem, Depeweg et al.'s system applies simple image processing operations to extract objects, attributes, and relationships in each frame. The system was given a small repertoire of object, attribute, and relation types that it was able to extract. The authors note, 'Our aim was not to build a general vision system, instead we decided to focus on these visual shapes, properties and relations that appear in many of the Bongard problems and hence seem to be likely candidates for a natural vocabulary from which relevant visual concepts can be built.' The limitations of the system's repertoire means that their system is able to deal with only a subset of 39 out of the 100 original problems. Once the image processing has been completed on the input Bongard problem, the system searches for a rule by sampling (using a form of the Metropolis-Hastings algorithm) from possible rules under the probability distribution described above. The system is allowed 300,000 samples to find a compatible rule for each Bongard problem. Of the 39 problems the grammar could deal with, the system was able to find compatible rules for 35 of them. ## Summary of Program Induction Approaches The Omniglot and Bongard-problem examples illustrate the promise and challenges of probabilistic program induction approaches. Framing concept learning as the task of generating a program enables many of the advantages of programming in general, including flexible abstraction, reusability, modularity, and interpretability. Bayesian inference methods seamlessly combine prior knowledge and preferences with likelihoods, and enable powerful sampling methods. Such advantages have fueled strong interest and progress in probabilistic program induction in recent years, including combining program induction with neural networks, reinforcement learning, and other methods, 13,21,22,38 , as well as incorporating program induction with methods inspired by neuroscience 70 and psychology. 24,91 The work of L´ azaro-Gredilla et al. 70 is notable in that it combines Figure 9: A sample ARC task (adapted from Chollet 14 ). Best viewed in color. <details> <summary>Image 9 Details</summary>  ### Visual Description ## Diagram: Task Demonstrations and Test ### Overview The image presents a series of grid-based environments, visually demonstrating tasks and a test scenario. The left side of the image shows six "Task Demonstrations" arranged in a 3x2 grid. The right side shows a single "Test" environment. Each environment is a grid with colored borders and colored squares within the grid. ### Components/Axes * **Titles:** * "Task Demonstrations" (top-left) * "Test" (top-right) * **Grid:** Each environment contains a grid of black squares. The grid appears to be approximately 16x16. * **Borders:** Each environment has a colored border. The colors vary between environments. * **Squares:** Each environment contains several colored squares within the grid. The colors and positions of these squares vary. ### Detailed Analysis **Task Demonstrations (Left Side):** * **Top-Left:** Border colors: Top - Yellow, Right - Green, Bottom - Light Blue, Left - Red. Contains colored squares: Green, Orange, Yellow, Light Blue, Red. * **Top-Right:** Border colors: Top - Yellow, Right - Green, Bottom - Light Blue, Left - Red. Contains colored squares: Light Blue, Yellow. * **Middle-Left:** Border colors: Top - Blue, Right - Yellow, Bottom - Orange, Left - Red. Contains colored squares: Red, Orange, Yellow, Light Blue. * **Middle-Right:** Border colors: Top - Blue, Right - Yellow, Bottom - Orange, Left - Red. Contains colored squares: Blue, Orange. * **Bottom-Left:** Border colors: Top - Pink, Right - Light Blue, Bottom - Green, Left - Red. Contains colored squares: Red, Yellow, Light Blue. * **Bottom-Right:** Border colors: Top - Pink, Right - Light Blue, Bottom - Green, Left - Red. Contains colored squares: Yellow. **Test (Right Side):** * Border colors: Top - Yellow, Right - Red, Bottom - Light Blue, Left - Blue. Contains colored squares: Red, Orange, Yellow, Light Blue, Green, Pink. ### Key Observations * The "Task Demonstrations" show different configurations of colored squares within the grid environments. * The "Test" environment has a unique border color configuration and a different arrangement of colored squares. * The colors used for the squares and borders are consistent across all environments. ### Interpretation The image likely represents a reinforcement learning or pathfinding problem. The "Task Demonstrations" could be examples of successful states or actions within the environment. The "Test" environment presents a new scenario where an agent must navigate or interact with the colored squares based on the patterns learned from the demonstrations. The different border colors might represent different types of environments or constraints. The colored squares likely represent objects or goals within the environment. </details> probabilistic program induction with a neurally inspired model of visual perception and action in a physical robot. The work of Evans et al. 24 notably focuses on unsupervised program induction constrained by domain-independent properties of cognition. Many challenges remain to make probabilistic program induction a more general-purpose AI method for concept learning, abstraction, and analogy. These methods currently need substantial builtin knowledge, structured by humans, in the form of the program primitives and grammar (the 'domain-specific language') for a given problem. Moreover, these methods require humans to define prior probability and likelihood distributions over possible programs, which is not always a straightforward task. Perhaps most important, solving a given task can require an enormous amount of search in the space of possible programs, which currently limits scaling up such program induction methods to more complex problems (though some of the hybrid methods cited above are focused on dealing with this combinatorial explosion of possibilities.) Finally, it remains to be seen whether the concept-as-program notion allows for the flexibility, extensibility, and analogy-making abilities that are the hallmark of human concepts. ## Abstraction and Reasoning Corpus This section does not describe a new AI method for abstraction and analogy, but rather a new promising benchmark-the Abstraction and Reasoning Corpus (ARC)-for evaluating these abilities. ARC was developed by Fran¸ cois Chollet as part of a project on how to measure intelligence in AI systems. 15 ARC is a collection of visual analogy 'tasks.' Figure 9 gives a sample task from the corpus. The left side presents three 'task demonstrations'; each of these consists of two grids with colored boxes. In each demonstration, you can think of the first (left) grid as 'transforming' into the second (right) grid. The right side of Figure 9 presents a 'test'; the task is to transform this grid analogously to the demonstrations. This task could be thought of as one of 'few-shot' learning-the demonstrations on the left are the training examples, and the grid on the right is a test example. The ARC domain features visual analogies between idealized 'situations' that can express abstract concepts in unlimited variations. In this way it exhibits some of the combined advantages of the other idealized domains I've discussed in this paper, such as Copycat's letter-string analogies, Bongard problems, and Ravens Progressive Matrices. Chollet manually designed 1,000 ARC tasks, with the motivation of enabling a fair comparison of 'general intelligence' between AI systems and humans-a comparison that does not involve language or other acquired human knowledge. Instead, ARC tasks are meant to rely only on the innate core knowledge systems proposed by Spelke, 101 which include intuitive knowledge about objects, agents and their goals, numerosity, and basic spatial-temporal concepts. An important aspect of ARC is its relatively small size. Chollet made 400 tasks publicly available and reserved 600 tasks as a 'hidden' evaluation set. In 2020 the Kaggle website hosted a threemonth 'Abstraction and Reasoning Challenge' in which researchers were encouraged to submit programs to be evaluated on the hidden evaluation set. The best-performing submissions had about 20% accuracy on a top-3 metric (three answers were allowed per task, and if one or more was correct, the task was considered to be solved). However, none of the submissions used an approach that was likely to be generalizable. 16 Thus the ARC challenge remains wide open. ## Discussion: How to Make Progress in AI on Abstraction and Analogy In the sections above I have described several diverse AI approaches to abstraction and analogy, including symbolic methods, deep learning, and probabilistic program induction. These approaches each have their own advantages and limitations. Symbolic systems such as SME can be explicitly programmed with prior knowledge (in symbolic form) as well as with important heuristics of abstraction and analogy making, such Gentner's systematicity principle. Symbolic representation methods such as predicate logic or semantic networks offer unambiguous variables and variable binding, clear type-token distinctions, and explicit measures of conceptual similarity and conceptual abstraction, among other facilities associated with human reasoning. These systems also have the advantage of interpretability, since their 'reasoning' on a given problem is readable in symbolic form. However, representations that focus on the syntax of logic-like representations can suffer from brittleness; moreover symbolic approaches often require humans to create and structure substantial prior knowledge, and these systems often rely on semi-exhaustive search. There have been several interesting neuro-symbolic approaches that implement symbolic-like behavior in neural networks (e.g., see Refs. 20,75,99), but these remain limited in their generality and can suffer from some of the limitations of symbolic systems discussed above. Active symbol architectures, such as the Copycat program, were claimed to address some of the limitations of purely symbolic methods 12,53 by enabling the system to actively build its representation of a situation in a workspace, via a continual interaction between bottom-up and top-down information processing, avoiding any kind of exhaustive search. However, Copycat and other examples of active symbol architectures remain dependent on prior knowledge provided and structured by humans, and these systems as yet have no mechanism for learning new permanent concepts. Deep learning systems do not require the pre-programmed structured knowledge of symbolic systems; they are able to learn from essentially raw data. However, they require large training corpora, as well as significant re-training (and often re-tuning of hyperparameters) for each new task. In addition, they are susceptible to learning statistical shortcuts rather than the actual concepts that humans intend, and their lack of intepretability often makes it difficult to ascertain the extent to which they are actually performing abstraction or analogy. Moreover, if the goal is to imbue machines with general humanlike abstraction abilities, it doesn't make sense to have to train them on tens of thousands of examples, since the essence of abstraction and analogy is few-shot learning. While some meta-learning systems have shown interesting performance on certain few-shot-learning tasks, their overall generality and robustness still needs to be demonstrated. Probabilistic program induction, by framing concept learning as the task of generating a program , enables many of the advantages of programming in general, including flexible abstraction, reusability, modularity, and interpretability. Moreover, Bayesian inference methods enable the combination of prior knowledge and preferences with likelihoods, and a probabilistic approach enables powerful sampling methods. However, like symbolic approaches, current program induction approaches require significant human-engineered prior knowledge in the form of a domain-specific language. And the more expressive the language, the more daunting the combinatorial search problem these methods face. Stepping back from any individual approach, it is difficult to assess how much general progress has been made on AI methods for abstraction and analogy, since each AI system is developed for and evaluated on a particular domain. Moreover, as I described in the sections above, these evaluations have largely relied on the system's accuracy on a particular set of test problems. What's missing are assessments based on generality across diverse domains, as well as the robustness of a given system to factors such as noise, variations on an abstract concept, or scaling of task complexity. In order to make further progress, we need to rethink how we choose or design domains and what evaluation criteria and evaluation processes we use. The following are my own recommendations for research and evaluation methods to make quantifiable and generalizable progress in developing AI systems for abstraction and analogy. Focus on idealized domains. AI researchers have used diverse idealized domains for developing and evaluating systems to perform abstraction and analogy, including Ravens Progressive Matrices, letter-string analogy problems, the Omniglot challenge, and Bongard problems, among others (e.g., more recently, new idealized visual domains have been proposed in Refs. 70,83.) While some approaches have used natural image- or language-based domains, 2,8,73,79,86,92 there are advantages to using idealized non-linguistic domains. In idealized domains, it is possible to be explicit about what prior knowledge is assumed, as opposed to the open-ended knowledge requirements of human language and imagery. Real-world image- or language-based tasks have much richer meanings to humans than to the machines processing this data; by avoiding such tasks we can avoid anthropomorphizing and overestimating what an AI system has actually achieved. There are risks with idealized domains, since it is not always clear that these domains capture the kind of real-world phenomena we want to model, but as in other sciences, the first step to progress is to isolate the phenomenon we are studying, in as idealized a form as possible. While I believe that important general cognitive abilities can be developed using these idealized challenges, others have argued that such abilities can be enabled only by exposing systems to much richer data or experience. 48 Focus on 'core knowledge.' Chollet has suggested that challenge domains for assessing AI's 'intelligence' should rely only on human 'core knowledge' rather than acquired knowledge such as language. 15 Spelke and colleagues 101 have proposed that human cognition is founded on of a set of four core knowledge systems, which include objects and intuitive physics; agents and goaldirectedness; numbers and elementary arithmetic; and spatial geometry of the environment (which includes relational concepts such as 'in front of' or 'contains). The idealized domains I have discussed above-especially the ARC domain-mostly rely only on such core knowledge. (Note that in the letter-string analogy domain, letters of the alphabet are used as idealized representatives of objects that can be related to other objects in specific ways; knowledge of language, such as how letters are used in words, or the visual characteristics of letters, are outside of the idealized domain. 55 ) Restricting a domain's required prior knowledge to such concepts makes it possible to fairly compare performance among AI systems as well as between AI systems and humans. Indeed, I would argue that unless we create AI systems that can master such core, non-linguistic knowledge, we have little hope of creating anything like human-level AI. Evaluate systems across multiple domains. All of the domains I have discussed here capture interesting facets of abstraction and analogy-making, including the recognition of abstract similarity, conceptual extrapolation, and adaptation of knowledge to novel situations. However, the common practice of focusing on a single domain limits progress. AI research focusing exclusively on any one domain has the risk of 'overfitting' to that domain. I believe that the research community needs to adopt a diverse suite of challenge domains on which systems can be evaluated for generality; such a strategy has a better chance to develop truly general and robust approaches. This is the strategy taken by the natural language processing community, for example, with the GLUE and SuperGLUE benchmarks, 107,108 though these benchmarks focus on tasks associated with large training sets and fixed test sets, which have allowed for successful solutions based on 'shortcuts.' 39,71,78 For that reason I endorse a focus on tasks that do not allow large amounts of specific training data, as I detail below. Focus on tasks that require little or no training. Since abstraction and analogy are at core concerned with flexibly mapping one's knowledge to new situations, it makes little sense to have to extensively train a system to deal with each new abstraction or analogy problem. Thus I believe the research community should focus on tasks that do not require extensive training on examples from the domain itself. This echoes Chollet's criteria for such tasks: 'It should not be possible to 'buy' performance on the benchmark by sampling unlimited training data. The benchmark should avoid tasks for which new data can be generated at will. It should be, in effect, a game for which it is not possible to practice in advance of the evaluation session.' 15 When humans make abstractions or analogies, it can be argued that the 'training' for such abilities is the process of developing core concepts, much of which takes place in early childhood. 10,74,101 Similarly, AI systems that can solve problems in idealized domains should require 'training' only on the core concepts required in each domain. Rather than training and testing on the same task, enabling machines to have general abstraction abilities will require that they learn core concepts and then adapt that knowledge to a multitude of different tasks, without being trained specifically for any one of them. Include generative tasks. Some idealized domains offer discriminative tasks-e.g., Raven's Progressive Matrices, in which the solver chooses from a set of candidate answers. Others, such as the letter-string analogies or ARC, are generative : the solver has to generate their own answer. Generative tasks are likely more resistant to shortcut learning than discriminative tasks, and systems that generate answers are in many cases more interpretable. Most importantly, if the space of problems is diverse enough, having to generate answers forces a deeper understanding of the task. Evaluate systems on 'hidden,' human-curated, changing sets of problems. It is important that problems to be used for evaluation are not be made available to the developers of AI systems that will be evaluated on these problems. Furthermore, these evaluation sets should not remain fixed for a long period of time, since systems can indirectly 'overfit' to a fixed set of evaluation problems, even when they are hidden. In addition, the evaluation problems should be created and curated by humans, rather than relying on automatic generation by algorithms; I described above how procedurally generated problems can unintentionally allow shortcut solutions; they can also allow a system to reverse-engineer the generating algorithm instead of being able to solve the problems in a general way. 15 Of course, human-generated problems can also allow for shortcut solutions; thus evaluation sets need to be carefully curated to avoid shortcut solutions as much as possible, for example via adversarial filtering methods. 94 Evaluate systems on their robustness, not simply their accuracy. Like other research in AI, methods for abstraction and analogy are often evaluated on their accuracy on a set of test problems from a given domain. However, as I discussed above, measuring accuracy on a fixed set of test problems does not reveal possible shortcuts-strategies a system takes to solve problems that don't reflect the actual general abilities that the evaluation is meant to test. In order to make progress on general abstraction and analogy abilities, in addition to evaluating systems across multiple domains, we need to evaluate them along multiple dimensions of robustness. For example, the evaluation benchmarks should feature various kinds of challenges in order to measure a system's robustness to 'noise' and other irrelevant distractions, and to variations in a given concept (e.g., if one example tests recognition of the abstract concept 'monotonically increasing,' other examples should test variations of this concept with different degrees of abstraction). Finally, the evaluation problems should also test a system's ability to scale to more complex examples of a given concept (e.g., if a system is able to recognize 'monotonically increasing' with small number of elements, it should also be tested on the same concept with a larger number of elements, or with more complex elements). ## Conclusion In this paper I have argued that humanlike abstraction and analogy-making abilities will be key to constructing more general and trustworthy AI systems, ones that can learn from a small number of examples, robustly generalize, and reliably adapt their knowledge to diverse domains and modalities. I have reviewed several approaches to building systems with these abilities, including symbolic and 'active symbol' approaches, deep learning, and probabilistic program induction. I have discussed advantages and current limitations of each of these approaches, and argued that it remains difficult to assess progress in this area due to the lack of evaluation methods addressing generality, robustness, and scaling abilities. Finally, I proposed several steps towards making quantifiable and generalizable progress, by designing appropriate challenge suites and evaluation methods. The quest for machines that can make abstractions and analogies is as old as the AI field itself, but the problem remains almost completely open. I hope that this paper will help spur renewed interest and attention in the AI community to understanding these core abilities which form the foundations of general intelligence. ## Acknowledgments This material is based upon work supported by the National Science Foundation under Grant No. 2020103. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation. This work was also supported by the Santa Fe Institute. I am grateful to Marianna Bolognesi, Andrew Burt, Richard Evans, Ross Gayler, Brenden Lake, Adam Santoro, Wai Keen Vong, and two anonymous reviewers for comments on an earlier version of the manuscript. ## References 1. M. A. Alcorn, Q. Li, Z. Gong, C. Wang, L. Mai, W. S. Ku, and A. Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR , pages 4840-4849, 2019. 2. A. Andonian, C. Fosco, M. Monfort, A. Lee, R. Feris, C. Vondrick, and A. Oliva. We have so much in common: Modeling semantic relational set abstractions in videos. arXiv:2008.05596 , 2020. 3. B. J. Baars and S. Franklin. How conscious experience and working memory interact. Trends in Cognitive Sciences , 7(4):166-172, 2003. 4. J. A. Barnden and K. J. Holyoak. Advances in Connectionist and Neural Computation Theory Vol. 2: Volume Two: Analogical Connections . Ablex Publishing Corporation, 1994. 5. D. G. T. Barrett, F. Hill, A. Santoro, A. S. Morcos, and T. Lillicrap. Measuring abstract reasoning in neural networks. In Proceedings of the International Conference on Machine Learning, ICML , pages 4477-4486, 2018. 6. L. W. Barsalou. Simulation, situated conceptualization, and prediction. Philosophical Transactions of the Royal Society B: Biological Sciences , 364(1521):1281-1289, 2009. 7. P. Bartha. Analogy and analogical reasoning. In E. N. Zalta, editor, The Stanford Encyclopedia of Philosophy (Spring 2019 Edition) . 2019. URL https://plato.stanford.edu/archives/ spr2019/entries/reasoning-analogy . 8. J. A. Blass and K. D. Forbus. Modeling commonsense reasoning via analogical chaining: A preliminary report. In Proceedings of the 38th Annual Meeting of the Cognitive Science Society , 2016. 9. M. M. Bongard. Pattern Recognition . Spartan Books, 1970. 10. S. Carey. Pr´ ecis of The Origin of Concepts . Behavioral and Brain Sciences , 34(3):113, 2011. 11. P. A. Carpenter, M. A. Just, and P. Shell. What one intelligence test measures: A theoretical account of the processing in the Raven progressive matrices test. Psychological Review , 97(3): 404-431, 1990. 12. D. J. Chalmers, R. M. French, and D. R. Hofstadter. High-level perception, representation, and analogy: A critique of artificial intelligence methodology. Journal of Experimental and Theoretical Artificial Intelligence , 4(3):185-211, 1992. 13. X. Chen, C. Liu, and D. Song. Execution-guided neural program synthesis. In Proceedings of the International Conference on Learning Representations, ICLR , 2019. 14. F. Chollet. The Abstraction and Reasoning Corpus (ARC), 2019. URL https://github. com/fchollet/ARC . 15. F. Chollet. On the measure of intelligence. arXiv:1911.01547 , 2019. 16. F. Chollet. Personal Communication, 2020. 17. M. Crouse, C. Nakos, I. Abdelaziz, and K. D. Forbus. Neural analogical matching. arXiv:2004.03573 , 2020. 18. M. Dehghani, K. D. Forbus, E. Tomai, and M. Klenk. An integrated reasoning approach to moral decision making. Proceedings of the National Conference on Artificial Intelligence, AAAI , pages 1280-1286, 2008. 19. S. Depeweg, C. A. Rothkopf, and F. J¨ akel. Solving Bongard problems with a visual language and pragmatic reasoning. arXiv:1804.04452 , 2018. 20. L. A. A. Doumas, G. Puebla, A. E. Martin, and J. E. Hummel. Relation learning in a neurocomputational architecture supports cross-domain transfer. In Proceedings of the 42nd Annual Meeting of the Cognitive Science Society , pages 932-937, 2020. 21. K. Ellis, A. Solar-Lezama, D. Ritchie, and J. B. Tenenbaum. Learning to infer graphics programs from hand-drawn images. In Advances in Neural Information Processing Systems , 2018. 22. K. Ellis, C. Wong, M. Nye, M. Sabl´ e-Meyer, L. Cary, L. Morales, L. Hewitt, A. Solar-Lezama, and J. B. Tenenbaum. DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning. arXiv:2006.08381 , 2020. 23. L. D. Erman, F. Hayes-Roth, V. R. Lesser, and D. R. Reddy. The Hearsay-II speechunderstanding system: Integrating knowledge to resolve uncertainty. ACM Computing Surveys (CSUR) , 12(2), 1980. 24. R. Evans, J. Hern´ andez-Orallo, J. Welbl, P. Kohli, and M. Sergot. Making sense of sensory input. Artificial Intelligence , 293:103438, 2021. 25. T. G. Evans. A program for the solution of a class of geometric-analogy intelligence-test questions. In M Minsky, editor, Semantic Information Processing . MIT Press, 1968. 26. K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR , pages 1625-1634, 2018. 27. B. Falkenhainer, K. D. Forbus, and D. Gentner. The structure-mapping engine. Proceedings of the National Conference on Artificial Intelligence, AAAI , pages 272-277, 1986. 28. G. Fauconnier. Mappings in Thought and Language . Cambridge University Press, 1997. 29. R. Feinman and B. M. Lake. Learning task-general representations with generative neurosymbolic modeling. In Proceedings of the International Conference on Learning Representations (ICLR) , 2021. 30. C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning, ICML , pages 1126-1135, 2017. 31. K. D. Forbus, D. Gentner, A. B. Markman, and R. W. Ferguson. Analogy just looks like high level perception: Why a domain-general approach to analogical mapping is right. Journal of Experimental and Theoretical Artificial Intelligence , 10(2):231-257, 1998. 32. K. D. Forbus, J. Usher, and E. Tomai. Analogical learning of visual/conceptual relationships in sketches. Proceedings of the National Conference on Artificial Intelligence, AAAI , page 202, 2005. 33. K. D. Forbus, A. Lovett, K. Lockwood, J. Wetzel, C. Matuk, B. Jee, and J. Usher. CogSketch. Proceedings of the National Conference on Artificial Intelligence, AAAI , pages 1878-1879, 2008. 34. H. E. Foundalis. Index of Bongard Problems. URL http://www.foundalis.com/res/bps/ bpidx.htm . 35. H. E. Foundalis. PHAEACO: A Cognitive Architecture Inspired by Bongard's Problems . PhD thesis, Indiana University, Bloomington, 2006. 36. R. M. French. The computational modeling of analogy-making. Trends in Cognitive Sciences , 6(5):200-205, 2002. 37. C. M. Funke, J. Borowski, K. Stosio, W. Brendel, T. S.A. Wallis, and M. Bethge. Five points to check when comparing visual perception in humans and machines. arXiv:2004.09406 , 2020. 38. Y. Ganin, T. Kulkarni, I. Babuschkin, S. M. A. Eslami, and O. Vinyals. Synthesizing programs for images using reinforced adversarial learning. In Proceedings of the 35th International Conference on Machine Learning, ICML , volume 4, 2018. 39. R. Geirhos, J. H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence , 2(11): 665-673, 2020. 40. D. Gentner. Structure-mapping: A theoretical framework for analogy. Cognitive Science , 7 (2):155-170, 1983. 41. D. Gentner and C. Hoyos. Analogy and abstraction. Topics in Cognitive Science , 9(3):672-693, 2017. 42. C. D. Gilbert and M. Sigman. Brain states: Top-down influences in sensory processing. Neuron , 54(5):677-696, 2007. 43. N. Goodman, J. B. Tenenbaum, and T. Gerstenberg. Concepts in a probabilistic language of thought. In E. Margolis and S. Laurence, editors, The Conceptual Mind: New Directions in the Study of Concepts , pages 623-654. The MIT Press, 2015. 44. A. Gopnik. The theory theory as an alternative to the innateness hypothesis. In L. M. Antony and N. Hornstein, editors, Chomsky and His Critics , pages 238-254. Wiley-Blackwell, 2003. 45. L. Hahne, T. L¨ uddecke, F. W¨ org¨ otter, and D. Kappel. Attention on abstract visual reasoning. arXiv:1911.05990 , 2019. 46. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, CVPR , pages 770-778, 2016. 47. F. Hill, A. Santoro, D. G. T. Barrett, A. Morcos, and T. Lillicrap. Learning to make analogies by contrasting abstract relational structure. In Proceedings of the International Conference on Learning Representations, ICLR , 2019. 48. F. Hill, A. Lampinen, R. Schneider, S. Clark, M. Botvinick, J. L. McClelland, and A. Santoro. Environmental drivers of systematicity and generalization in a situated agent. In Proceedings of the International Conference on Learning Representations, ICLR , 2020. 49. F. Hill, O. Tieleman, T. von Glehn, N. Wong, H. Merzic, and S. Clark. Grounded language learning fast and slow. In Proceedings of the International Conference on Learning Representations, ICLR , 2021. 50. D. Hofstadter and E. Sander. Surfaces and Essences . Basic Books, 2013, p.3. 51. D. R. Hofstadter. G¨ odel, Escher, Bach: an Eternal Golden Braid . Basic Books, New York, 1979. 52. D. R. Hofstadter. Analogies and roles in human and machine thinking. In Metamagical Themas , chapter 24. Basic Books, 1985. 53. D. R. Hofstadter. Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought . Basic Books, New York, 1995. 54. D. R. Hofstadter. Analogy as the core of cognition. In D. Gentner, K. J. Holyoak, and B. N. Kokinov, editors, The Analogical Mind: Perspectives from Cognitive Science , pages 499-538. MIT Press, 2001. 55. D. R. Hofstadter and M. Mitchell. The Copycat project: A model of mental fluidity and analogy-making. In K. J. Holyoak and J. A. Barnden, editors, Advances in Connectionist and Neural Computation Theory , volume 2, pages 31-112. Ablex Publishing Corporation, 1994. 56. K. J. Holyoak. Analogical thinking and human intelligence. In Advances in the Psychology of Human Intelligence , volume 2, pages 199-230. Erllbaum, 1984. 57. K. J. Holyoak and P. Thagard. Analogical mapping by constraint satisfaction. Cognitive Science , 13(3):295-355, 1989. 58. S. Hu, Y. Ma, X. Liu, Y. Wei, and S. Bai. Stratified rule-aware network for ab stract visual reasoning. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021) , 2021. 59. J. E. Hummel and K. J. Holyoak. LISA: A computational model of analogical inference and schema induction. In Proceedings of the Eighteenth Annual Meeting of the Cognitive Science Society , pages 352-357, 1996. 60. M. Jahrens and T. Martinetz. Solving Raven's Progressive Matrices with Multi-Layer Relation Networks. In Proceedings of the World Congress on Computational Intelligence, WCCI , 2020. 61. M. Johnson. Embodied Mind, Meaning, and Reason: How Our Bodies Give Rise to Understanding . University of Chicago Press, Chicago, IL, 2017. 62. D. Kahneman, A. Treisman, and B. J. Gibbs. The reviewing of object files: Object-specific integration of information. Cognitive Psychology , 24(2):175-219, 1992. 63. B. N. Kokinov and A. Petrov. Integrating memory and reasoning in analogy-making: The AMBR model. In The Analogical Mind: Perspectives from Cognitive Science . MIT Press, 2001. 64. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science , 350(6266):1332-1338, 2015. 65. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. Supplementary Material for Human-level concept learning through probabilistic program induction. Science , 350(1332), 2015. 66. B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences , 40:E253, 2017. 67. G. Lakoff. The contemporary theory of metaphor. In Metaphor and Thought , pages 202-251. Cambridge University Press, 2008. 68. G. Lakoff. The neural theory of metaphor. In The Cambridge Handbook of Metaphor and Thought , pages 17-38. Cambridge University Press, 2012. 69. A. K. Lampinen and J. L. McClelland. Transforming task representations to perform novel tasks. Proceedings of the National Academy of Sciences , 117(52), 2020. 70. M. L´ azaro-Gredilla, D. Lin, J. S. Guntupalli, and D. George. Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs. Science Robotics , 4(26), 2019. 71. T. Linzen. How can we accelerate progress towards human-like linguistic generalization? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5210-5217, 2020. 72. A. Lovett and K. D. Forbus. Modeling visual problem solving as analogical reasoning. Psychological review , 124(1), 2017. 73. H. Lu, Q. Liu, N. Ichien, A. L. Yuille, and K. J. Holyoak. Seeing the meaning : Vision meets semantics in solving pictorial analogy problems. In Proceedings of the 41st Annual Meeting of the Cognitive Science Society , pages 2201-2207, 2019. 74. J. M. Mandler. How to build a baby: II. Conceptual primitives. Psychological Review , 99(4): 587-604, 1992. 75. J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In 7th International Conference on Learning Representations, ICLR 2019 , 2019. 76. Ana Marasovi´ c. NLP's generalization problem, and how researchers are tackling it. The Gradient , 2018. 77. J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon. A Proposal for the Dartmouth Summer Research Project in Artificial Intelligence, 1955. Reprinted in AI Magazine , 27, no. 4, 2006, 12-14. 78. T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2020. 79. T. Mikolov, W. T. Yih, and G. Zweig. Linguistic regularities in continuous space word representations. In Proceedings of the North American Chapter of the Association for Computational Linguistics , pages 746-751. Association for Computational Linguistics, 2013. 80. M. Mitchell. Analogy-Making as Perception: A Computer Model . MIT Press, 1993. 81. M. Mitchell. Artificial Intelligence: A Guide for Thinking Humans, Chapter 6 . Farrar, Straus, and Giroux, 2019. 82. E. Nichols and D. Hofstadter. Musicat: A model of music perception and expectation. In Proceedings of the 31st Annual Meeting of the Cognitive Science Society , 2009. 83. W. Nie, Z. Yu, A. B. Mao, L.and Patel, Y. Zhu, and A. Anandkumar. Bongard-LOGO: A new benchmark for human-level concept learning and reasoning. arXiv:2010.00763 , 2020. 84. R. M Nosofsky. Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General , 115(1):39, 1986. 85. T. Perkins. Joe Biden positions himself as 'bridge' to next generation at Michigan rally. The Guardian , March 10, 2020. 86. J. Peyre, J. Sivic, I. Laptev, and C. Schmid. Detecting unseen visual relations using analogies. In Proceedings of the IEEE International Conference on Computer Vision, ICCV , pages 19811990, 2019. 87. M. H. Quinn, E. Conser, J. M. Witte, and M. Mitchell. Semantic image retrieval via active grounding of visual situations. In Proceedings of the International Conference on Semantic Computing , pages 172-179, 2018. 88. M. Ren, E. Triantafillou, S. Ravi, J. Snell, K. Swersky, J. B. Tenenbaum, H. Larochelle, and R. S. Zemel. Meta-learning for semi-supervised few-shot classification. In Proceedings of the International Conference on Learning Representations, ICLR , 2018. 89. P. R. Roelfsema and F. P. de Lange. Early visual cortex as a multiscale cognitive blackboard. Annual Review of Vision Science , 2, 2016. 90. E. Rosch. Principles of categorization. In E. Margolis and S. Laurence, editors, Concepts: Core Readings , pages 189-206. MIT Press, 1999. 91. J. S. Rule, J. B. Tenenbaum, and S. T. Piantadosi. The child as hacker. Trends in Cognitive Sciences , 2020. 92. F. Sadeghi, C. L. Zitnick, and A. Farhadi. VISALOGY: Answering visual analogy questions. In Advances in Neural Information Processing Systems , pages 1882-1890, 2015. 93. K. Saito and R. Nakano. A concept learning algorithm with adaptive search. In Machine Intelligence 14: Applied Machine Intelligence , pages 347-363. Clarendon Press, 1996. 94. K. Sakaguchi, R. Le Bras, C. Bhagavatula, and Y. Choi. Winogrande: An adversarial Winograd schema challenge at scale. In Proceedings of the National Conference on Artificial Intelligence, AAAI , pages 8732-8740, 2020. 95. A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. Meta-learning with memory-augmented neural networks. In International Conference on Machine Learning (ICML 2016) , pages 1842-1850, 2016. 96. A. Santoro, D. Raposo, D. G. T. Barrett, M. Malinowski, R. Pascanu, P. Battaglia, and T. Lillicrap. A simple neural network module for relational reasoning. In Advances in Neural Information Processing Systems , pages 4968-4977, 2017. 97. S. S. Schweber. Darwin and the political economists: Divergence of character. Journal of the History of Biology , 13(2):195-289, 1980. 98. M. Shanahan. A cognitive architecture that combines internal simulation with a global workspace. Consciousness and Cognition , 15(2):433-449, 2006. 99. P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence , 46(1-2), 1990. 100. J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems , pages 4078-4088, 2017. 101. E. S. Spelke and K. D. Kinzler. Core knowledge. Developmental Science , 10(1):89-96, 2007. 102. X. Steenbrugge, S. Leroux, T. Verbelen, and B. Dhoedt. Improving generalization for abstract reasoning tasks using disentangled feature representations. arXiv:1811.04784 , 2018. 103. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In Proceedings of the International Conference on Learning Representations, ICLR , 2014. 104. A. Treisman. Feature binding, attention and object perception. Proceedings of the Royal Society, Series B , 6:1295-1306, 1998. 105. M. Turner and G. Fauconnier. Conceptual integration in counterfactuals. In Discourse and Cognition . Sage Publications, 1997. 106. J. von Neumann. The Computer and the Brain . Yale University Press, New Haven, CT, 1958. 107. A. Wang, A. Singh, F. Michael, J.and Hill, O. Levy, and S. R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop on Blackbox NLP: Analyzing and Interpreting Neural Networks for NLP , pages 353-355. Association for Computational Linguistics, 2018. 108. A. Wang, Y. Pruksachatkun, N. Nangia, J. Singh, A.and Michael, F. Hill, O. Levy, and S. R. Bowman. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems , volume 32, pages 32663280, 2019. 109. J. X. Wang. Meta-learning in natural and artificial intelligence. arXiv:2011.13464 , 2020. 110. K. Wang and Z. Su. Automatic generation of Raven's progressive Matrices. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI , pages 903-909, 2015. 111. T. W. Webb, Z. Dulberg, S. M. Frankland, A. A. Petrov, R. C. O'Reilly, and J. D. Cohen. Learning representations that support extrapolation. arXiv:2007.05059 , 2020. 112. T. W. Webb, I. Sinha, and J. D. Cohen. Emergent symbols through binding in external memory. In Proceedings of the International Conference on Learning Representations, ICLR , 2021. 113. P. H. Winston. Learning and reasoning by analogy. Communications of the ACM , 23(12): 689-703, 1980. 114. Y. Wu, H. Dong, R. Grosse, and J. Ba. The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning. arXiv:2007.04212 , 2020. 115. C. Zhang, F. Gao, B. Jia, Y. Zhu, and S.-C. Zhu. RAVEN: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR , volume 2019-June, pages 5312-5322, 2019. 116. C. Zhang, B. Jia, F. Gao, Y. Zhu, H. Lu, and S.-C. Zhu. Learning perceptual inference by contrasting. In Advances in Neural Information Processing Systems 32 , pages 1075-1087, 2019. 117. C. Zhang, B. Jia, S. Zhu, and Y. Zhu. Abstract spatial-temporal reasoning via probabilistic abduction and execution. arxiv:2103.14230 , 2021. 118. K. Zheng, Z.-J. Zha, and W. Wei. Abstract reasoning with distracting features. In Advances in Neural Information Processing Systems 32 , pages 5842-5853, 2019. 119. T. Zhuo and . Kankanhalli. Solving Raven's progressive matrices with neural networks. arXiv:2002.01646 , 2020.