2405.18073

# Towards Dialogues for Joint Human-AI Reasoning and Value Alignment **Authors**: Elfia Bezou-Vrakatseli, Oana Cocarascu ## Abstract We argue that enabling human-AI dialogue, purposed to support joint reasoning (i.e., ‘inquiry’), is important for ensuring that AI decision making is aligned with human values and preferences. In particular, we point to logic-based models of argumentation and dialogue, and suggest that the traditional focus on persuasion dialogues be replaced by a focus on inquiry dialogues, and the distinct challenges that joint inquiry raises. Given recent dramatic advances in the performance of large language models (LLMs), and the anticipated increase in their use for decision making, we provide a roadmap for research into inquiry dialogues for supporting joint human-LLM reasoning tasks that are ethically salient, and that thereby require that decisions are value aligned. ## 1 Introduction As AI becomes more integrated into our lives, one needs to ensure both explainability of AI decisions, and that AI decisions accommodate inputs from human users. This requirement is especially salient when decisions have ethical impact, and decisions need to be aligned with the preferences and values of human stakeholders. Indeed, addressing the so called ‘value alignment’ problem (?; ?) – ensuring that AI (in particular machine learning) systems achieve their goals in ways that are not anti-ethical to human preferences/values – has acquired renewed urgency in light of recent dramatic advances in large language models (LLMs), and anticipated uses of LLMs in supporting decision making (?; ?; ?; ?; ?). In this paper we make the following two claims so as to then advocate for a programme of research that: 1) advances work on argumentation-based dialogues; 2) enhances the reasoning and conversational capabilities of LLMs; 3) enables joint human-LLM reasoning. Firstly, we claim that human-AI dialogue is required to support joint reasoning, enabling human input into decision making processes. In particular, these joint reasoning or ‘inquiry’ dialogues should accommodate input and reasoning about human values and preferences. This will not only ensure that individual decisions are value-aligned, but that humans can be supported in refining their ethical positions, and the values/preferences informing decisions can then be learnt by AI systems so as to inform their future value-aligned decisions. Secondly, we claim that dialogues generalising argumentative formalisations of non-monotonic reasoning (?; ?) present the most suitable candidates for enabling joint human-LLM inquiry. In Section 2 we further develop these claims by reference to motivating examples, and then enumerate a non-exhaustive list of research challenges if one is to realise the long term goal of joint human-LLM inquiry for value alignment. In Section 3 we review current progress towards realising these research challenges, and key next steps in the research programme. We conclude in Section 4. ## 2 Motivating Research into Joint Inquiry for Value Alignment The last two decades have witnessed significant advances in machine learning, and accompanying concerns that, as AI systems become more autonomous and sophisticated on a trajectory to AGI and beyond (?), the overriding imperative of maximising reward functions may result in unforeseen decisions and actions that are harmful to humans (?; ?). One of the most prominent emerging solutions to this problem of ‘value alignment’ is Russell et.al’s Cooperative Inverse Reinforcement Learning (CIRL) framework (?; ?). CIRL proposes that AI systems remain in a constant state of uncertainty about what the human value function is, and so are perpetually engaged in the primary goal of learning the value function through observation, query, and instruction. The challenge is to then ensure that the AI simultaneously acts so as to realise other goals for which it has been tasked CIRL can be interpreted as advocating ‘enculturation of’ AI, analogous to how young children acquire an understanding of values through observation, questioning and instruction. Of course, the analogy breaks down when one considers that AI systems are expected to perform complex tasks, often surpassing human capabilities, while simultaneously being ‘ethically enculturated’.. As emphasised in (?), this entails that delegation of decision making tasks to AI systems requires simultaneously eliciting human input, and in particular ethically salient information that reveals stakeholder preferences and values; information that can then be learnt in order to inform future decisions. Consider the following simple example of an LLM specialised to serve as a personal assistant (PA) for a human user (H): PA: “Given your budget and preferences for landscape and weather, I propose two options: (a) 7 days at the Dubai Beach Resort; (b) 7 days at the Miami Beach Resort.” H: “I prefer (b).” PA: “Why?” H: “Because according to Amnesty International, Dubai has unethical labour practices.” PA: “Ok. I will prepare an itinerary for Miami.” This short dialogue illustrates how, based on H ’s provision of factual information (i.e., budget constraints) and aesthetic preferences (weather and landscape), PA proposes two options and then elicits an ethically salient preference from H, effectively leading to a joint value aligned decision in favour of option (b), and ideally, PA leveraging said ethical preference in later decisions. Any such reasoning process should ideally be prescriptively governed by rational principles for non-monotonic reasoning, wherein, as new information comes to light or there are conflicting reasons for alternative claims (inferences), preferences and values can be used to arbitrate in favour of a claim over its contradictory. We thus turn to argumentative formalisations of non-monotonic (nm) reasoning, wherein arguments constructed (via chaining premises to a claim through application of deductive and/or default inference rules) from a given individual’s belief base $B$ , are related by a defeat relation. For example, if argument $A$ ’s claim contradicts argument $B$ ’s claim, then $A$ and $B$ attack each other. $A$ successfully attacks (i.e., ‘defeats’) $B$ , only if $A\nprec B$ (where $\prec$ is a preference relation over the constructed arguments). The argument framework ( $AF$ ) of arguments related by defeats can then be evaluated under various Dung semantics (?), so as to identify the justified (winning) arguments. The claims of these arguments can then be shown to equate with various nm logical consequence relations defined directly over $B$ (e.g., see (?; ?; ?; ?)). Argument game proof theories (e.g., (?)) can then be deployed to establish whether a given argument in an $AF$ is justified under a given semantics, and hence whether its claim is an nm consequence from the underlying $B$ . These argument game proof theories can be adapted (?) to yield dialogical formalisations of nm reasoning (e.g., (?; ?)) in which agents can exchange locutions – not just arguments but claims, questions etc. – that conform to protocols generalising rules in the aforementioned argument games. The protocols and evaluation of the resultant graph of locutions are defined so that under certain conditions: a communicated claim $α$ is established by dialogue $d$ iff $α$ is the claim of a justified argument in $AF_B$ where instead of $AF_B$ being defined by a static given belief base $B$ , as in the paradigmatic case of single agent reasoning, $AF_B$ is incrementally defined either by the publicly asserted beliefs $B$ asserted in the dialogue $d$ , or the union of the belief bases of the agents participating in $d$ . One thus obtains (given the aforementioned soundness and completeness results for argumentative characterisations of nm consequence relations) communicative accounts of distributed, i.e., ‘joint’, nm reasoning ( $\mid∼$ below denotes the consequence relation of some nm logic): a communicated claim $α$ is established by dialogue $d$ iff $B \mid∼ α$ Consider our example dialogue. The budget and aesthetic information provided by H, combined with other travel related information, is used as premises in two mutually attacking arguments $A$ and $B$ , respectively claiming the Dubai and Miami options. PA claims a preference for $B$ , and upon questioning, argues against (i.e., defeats) $B$ with an argument $C$ claiming unethical labour practices in Dubai. Hence $A$ is defended by $C$ ’s defeat on $B$ and the dialogue establishes option (b). Key to enabling such dialogues is the use of argument schemes (AS) and critical questions (CQs) (?) that have been developed within the field of informal logic. AS are templates capturing stereotypical patterns of intra-argument reasoning, and can be instantiated by natural and/or formal logic languages. For example, one can suppose both $A$ and $B$ to be instances of the Argument for Action (AA) scheme (?): ‘In circumstances $S$ , doing action $Act$ will achieve goal $G$ and so promote value $V$ ’ (where $S$ includes, amongst other things, budget and aesthetic information, $G$ is the goal of ‘going on holiday’, $V$ the value of ‘pleasure’ and $Act$ the respective decision options). Associated with AA are sixteen critical questions which identify possible criteria for attacking, or indeed supporting, any concrete instance of AA. For example, the CQ ‘Is it the case that $S$ ?’ can be addressed as a question by agent $Ag$ to an agent $Ag^\prime$ who has submitted an argument $X$ instantiating AA. In response, $Ag^\prime$ is dialectically obligated to provide an argument claiming $S$ . E.g., an argument claiming $Act$ = ‘lockdown’ may cite $S$ = ‘there is a pandemic’, and is not justified until such point that in response to questioning whether it is the case that there is a pandemic, relevant evidence is cited to support this claim. CQs can also be used to challenge arguments. In our running example, H uses the CQ ‘Does action $A$ demote some other value’ to submit the argument claiming unethical labour practices. $C$ is itself an instance of the argument from expert opinion (EO) scheme which cites an expert having asserted a claim as support for the claim. EO has its own CQ which can similarly be used to respond to $C$ . Thus, AS and CQ s help map out the space of possible reasoning, and a key requirement for realising the use of AS and CQs in dialogues is the following: Strategic considerations should inform the choice of AS or CQs at any given point in a dialogue ( $R_strat$ ) Suppose that instead of $C$ , H submitted the locution $C^\prime$ : “Dubai has unethical labour practices”. Now note that $C^\prime$ is an incomplete argument – an ‘ enthymeme ’ (?) – consisting only of a claim. Use of enthymemes are characteristic of real world dialogues, and ideally, the dialogue would evolve so that either the PA or H is prompted to supply evidence in support of the claim, to, as it were, ‘flesh out’ the claim $C^\prime$ into a fully fledged argument $C$ . Thus, PA might question ‘why $C^\prime$ ’, so that H is prompted to submit the expert opinion as a premise. Indeed, H may herself submit $C^\prime$ as a query; she believes $C^\prime$ to be the case, but is inviting PA (or perhaps some other agent) to provide the missing premise and so complete $C^\prime$ into an argument. In either case, until the dialectical obligation is met, $C^\prime$ does not constitute a valid defeat on $A$ , and $B$ ’s claim is not established. Hence the following requirement for realising inquiry dialogues: Dialogues should be guided so that declarative locutions can be combined to define complete arguments ( $R_enth$ ) Consider the following example dialogue Admittedly, this does not represent a typical decision making scenario; rather it is used to illustrate requirements for ethical reasoning. between a human H and a ‘Socratic LLM’ SocrAPP: 1. H: “I propose that we should make abortion illegal.” 2. SocrAPP: “Why?” 3. H: “Because it amounts to the early termination of life.” 4. SocrAPP: “So?” 5. H: “All humans have a right to life.” 6. SocrAPP: “What about the right of a woman to choose?” 7. H: “What do you mean?” 8. SocrAPP: “Your argument appeals to the right to life, but making abortion illegal violates the right of a woman to choose.” 9. H: “But the right to life takes precedence over the right to choose.” 10. SocrAPP: “Why?” : n ( $>$ 11). SocrAPP: “Here are links to studies showing that in developing countries, giving women the right to choose reduces poverty . Here are some articles that discuss and analyse these findings.” In 1–5, SocrAPP elicits (in 2 and 4) information from H. This information completes the initial claim (the enthymeme in 1) into an argument $A$ , instantiating a variant of scheme AA – which we call AA(R +) – that appeals to a right (to life) rather than a value. In 6, SocrAPP uses a CQ for AA(R +) – Does the action violate some other right? – to submit an enthymeme for an argument $B$ instantiating a scheme (AA(R -)) prohibiting an action if it violates an established right. We note that schemes such as AA(R +) and AA(R -) and the CQ we posit for the former, are novel, and specialised for ethical reasoning. Indeed, experience with the use of AS $\&$ CQs for guiding real-world deliberation in the medical domain (?) highlights the limiting generality of AS $\&$ CQs (?), and the need for AS $\&$ CQs specialised for medical reasoning. Hence we posit the following requirement for joint inquiry dialogues that facilitate value alignment: Schemes and Critical Questions specialised for ethical reasoning ( $R_ScCQ_{Ethical}$ ) In response to H’s query in 7, SocrAPP fulfills an explanatory remit (in 8), completing 6 into the argument $B$ , and additionally making explicit that $B$ and $A$ attack each other. In line 9, H declares a preference for the right to life over the right to choose (i.e., the preference $B\prec A$ ) so that only $A$ defeats $B$ (the dialogue has therefore thus far established H’s initial claim). In line 10, SocrAPP queries H’s rationale for this preference. H’s initial claim is now ‘dis-established’, and H is now required to provide a ‘metalevel’ argument for why one argument is preferred to another in order to re-stablish her initial claim. Now, recall that we have thus far proposed sound and complete inquiry dialogues, in the sense that evaluation of a graph of locutions establishes a claim $α$ iff $α$ is the claim of a justified argument in the $AF$ constructed from $B$ (the contents of declarative locutions exchanged, or of the participating agents’ belief bases), which in turn equates with $B \mid∼ α$ in some nm logic. However, in real-world (especially ethically salient) dialogues, one does not always expect consensus as to the preferences used to determine the success of attacks and defeats; rather one expects reasoning about possibly contradictory preferences, and argumentation based resolution of these conflicts. Hence, one requires: Metalevel Schemes and Critical Questions specialised for metalevel reasoning ( $R_ScCQ_{Meta}$ ) (?) also posit meta-schemes tailored to reasoning about other object-level features of $AF$ , including e.g., the rationale for why arguments attack each other, when said rationale is not based on negation, e.g., when arguments for two actions conflict because they have mutually exclusive pre-conditions. and Soundness and Completeness of dialogues that accommodate argumentation based metalevel reasoning about preferences and values ( $R_SC-Pref$ ) Let us suppose the dialogue continues, but (as is typically the case when appealing to rights), there is no way to resolve the conflict between arguments expressing contradictory preferences over rights. The claim is then subject to consequentialist evaluation, and in line $n$ , SocrAPP is responding to a prompt requesting evidence for the positive and negative consequences of legalising/prohibiting abortion (said evidence instantiating further variants of the $AA$ scheme that justify actions on consequentialist grounds). An important feature of our proposed inquiry dialogues for value alignment is that they facilitate rational exploration of the space of possible ethical evaluations one might take with respect to a decision. As illustrated above, the implication is that although a user might be committed to a rights-based ethical perspective, they would ideally be exposed to alternative viewpoints, such as consequentialist criteria, when evaluating decisions. Indeed, recent philosophical approaches eschew a ‘one size fits all’ approach, and instead advocate judgements that aggregate evaluations of decision options under different ethical theories (?; ?). Hence, the requirement for object-level domain specific ethical AS and CQs, and metalevel AS and CQs that may be deployed to aggregate the evaluations yielded by object-level arguments. Indeed, the implication is that a user may be encouraged to refine or even revise their ethical preferences, with the proviso that the AI interlocutor’s role is not to persuade the user to commit to a particular ethical judgement, but rather to explore the space of ethical reasoning. This then implies use of other dialectical moves, such as the well known Socratic move demonstrating that an interlocutor has contradicted herself (?). For example, suppose that after line 5, SocrAPP queries whether H is for or against the death penalty, and H replies to say she is in favour. SocrAPP can then reveal that H has contradicted herself (given her appeal to the right to life), prompting H to refine her ethical stance: “All humans, except those that take the life of another, have the right to life”. Hence the requirement that: Dialogues accommodate dialectical moves other than the moving of attacking/supporting arguments, and results in changes to the beliefs of interlocutors ( $R_Dia$ ) In this section we have, through the use of illustrative examples, enumerated a non-exhaustive list of requirements for inquiry dialogues that formalise joint non-monotonic reasoning; dialogues purposed to accommodate human ethical preferences and values that may be refined and subsequently learnt by AI interlocutors that are simultaneously tasked with supporting human users in their decision making. In the next section we briefly review progress towards realising these requirements, and then focus on challenges that need to be addressed if LLMs are to successfully engage in such dialogues. ## 3 Progress Towards Realising Joint Inquiry for Value Alignment ### 3.1 Towards Argumentation-based Dialogues for Value Aligned Inquiry There is a sparsity of work on formalising Inquiry Dialogues such that the aforementioned soundness and completeness results hold. A notable exception is the work of (?) in which agents can challenge arguments and collaboratively construct arguments in the language of Defeasible Logic Programming (?). Rather, the majority of research focuses on argumentation-based ‘zero sum’ persuasion dialogues (?) that are inherently adversarial, and thus clearly unsuitable given the collaborative goal of joint reasoning that we are interested in. There are then key requirements specific to inquiry that do not arise in the case of persuasion. Notably, arguments are constructed by individual agents in the latter, whereas inquiry deploys the epistemic resources of multiple agents in constructing individual arguments. Moreover, in persuasion dialogues, an agent cannot target their own argument with defeats, whereas inquiry should licence targeting one’s own argument, given that the aim is to get to ‘truth of the matter’ in respect of factual information, and to support decision options that are rationally justified as being optimal for users. Indeed, given these collaborative aims, the assumptions under which one expects the previous section’s soundness and completeness results for dialogues to hold – namely that interlocutors are honest and are exhaustive in submitting locutions licensed by the protocol – are reasonable. In light of what we perceive to be a dearth of formal models for sound and complete inquiry dialogues, we briefly point to progress made towards realising the requirements enumerated in Section 2. Firstly, the ASPIC + framework (?) for formalising argumentative characterisation of non-monotonic reasoning, has been extended to accommodate metalevel reasoning about preferences and values (?; ?). Dialogical formulations of this extended framework have been developed (?), but it remains to accommodate moves distinctive of inquiry rather than persuasion (e.g. allowing multiple interlocutors to collaboratively construct individual arguments), and to show soundness and completeness results ( $R_SC-Pref$ ). Moreover, enthymemes are not accommodated, but sound and complete inquiry dialogues that accommodate enthymemes and are based on the standard ASPIC + framework have been developed (?; ?) (addressing $R_enth$ ), and integration of this work with (?) represents a significant next research objective. Finally, recent argumentative formalisations of non-monotonic reasoning have been developed to accommodate attacks that dialectically demonstrate the inconsistency of premises of challenged arguments (?; ?), partly addressing $R_Dia$ (it remains to formalise changes to beliefs in response to such moves). Advances have also been made in addressing $R_ScCQ_{Ethical}$ , as detailed in (?), in which prevalent AS used in ethical reasoning on the debate platform Kialo (https://www.kialo.com) are identified and have been used to guide formulation of a new, specialized taxonomy for ethical reasoning. The (ongoing) creation of this taxonomy categorises common AS used in ethical debates and introduces new AS specifically tailored for such discussions. Furthermore, this taxonomy proposes meta-schemes ( $R_ScCQ_{Meta}$ ) that facilitate reasoning about and comparing ethical evaluations yielded by object level arguments instantiating ethical schemes. ### 3.2 Towards Large Language Model Interlocutors Large Language Models (LLMs) (?; ?; ?; ?) have significantly advanced the state-of-the-art on a wide range of language understanding tasks, by harnessing vast amounts of data, and excel in generating coherent and relevant text based on minimal prompts. This technological leap has ushered in a new era of AI, where machines can engage in more human-like conversations and provide insights with unprecedented accuracy (?; ?; ?; ?; ?; ?; ?; ?; ?), with multiple studies investigating the limits of their abilities, including the ability to learn (?), reason (?; ?; ?; ?), and comprehend intricate language structures such as metaphors (?). The remarkable capabilities of LLMs have led to their widespread adoption across various fields, including finance (?; ?), robotics (?; ?), medicine (?; ?), and even crisis counselling (?). However, despite their success, LLMs face challenges when faced with complex reasoning tasks, especially involving knowledge and symbolic reasoning (?), and exhibit deficiencies in both planning and reasoning abilities (?). Dialogue, specifically, remains a challenging task for LLMs (?; ?; ?; ?; ?). In order for LLMs to engage in joint inquiry as described in the previous sections, LLMs must be able to instantiate AS and CQs effectively. We discuss below a non-exhaustive selection of challenges for realising this goal. Firstly, LLMs need to be able to classify arguments using AS and to identify how one argument relates to another, through the use of their CQs. This is an open challenge but, given their impressive capabilities, there is now the potential to overcome previous hurdles in AI natural language processing, particularly regarding the instantiation of AS and CQs. Indeed, our preliminary investigations suggest that not only are LLMs capable of classifying arguments according to the AS they instantiate, but are also able to articulate why said AS have been identified. However, as with human annotators of argumentative corpus (?; ?; ?), LLMs struggle to differentiate between similar AS. Building on these investigations suggests a number of open challenges. Firstly, harnessing the abilities of LLMs to identify arguments that support or challenge a given claim, the schemes these arguments instantiate, and hence the ability to flesh out enthymemes (recall $R_enth$ ). Secondly, while LLMs can be promted to generate example dialogues on a given topic, the ability to identify the critical questions that arguments are responding to is still limited. Moreover, effective instantiation of AS and CQs requires LLMs to not only recognise, but also apply these reasoning structures, in a contextually appropriate manner. The challenges discussed above go hand in hand with the LLMs’ need to retain conversational histories over prolonged interactions. Maintaining conversational histories in a short-term memory module is currently an active research area (?; ?), which could be leveraged to maintain coherence across extended dialogues and multiple interactions. Finally, one would want LLMs to develop reasoning capacities enabling more complex dialectical moves ( $R_Dia$ ), and to strategise ( $R_strat$ ). Besides identifying points of support and attack, there is a need for subsequent instantiation of AS to construct arguments to support/attack, supported by a dialogue manager, with evaluation supported by a protocol manager. To effectively mimic human-like discourse management, and address $R_strat$ , LLMs should additionally be able to navigate the flow of conversation, deciding when to probe deeper or when to guide towards conclusions. This can be viewed as a search problem in a game tree, where the choice of moves amounts to the choice of AS and CQs. One technique that has been developed to improve the performance and maximize the utility of LLMs is prompt engineering (e.g., (?; ?; ?)), which is increasingly essential for proficient interaction with LLMs (?; ?). Prompts are instructions given to an LLM by customizing it and/or enhancing its capabilities to enforce rules, automate processes, and ensure desired attributes and volumes of generated content (?; ?). Effectively, prompt engineering is the means by which LLMs are ‘programmed’ via prompts, dictating how the models should process information. Chain of thought (CoT) prompting (?), which consists in breaking down an initial task/question into intermediate reasoning steps, holds promise for addressing the challenges we have identified. Overall, intermediate reasoning has been shown to improve LLMs’ performance (?) and CoT is currently used for improving the reasoning abilities of LLMs (?; ?; ?; ?). Essentially, CoT represents a series of intermediate natural language reasoning steps; for instance, step-by-step (?), which guides the model through a series of reasoning steps, or decomposition (?), where complex problems are broken down into simpler sub-problems. Especially promising for joint inquiry is maieutic prompting (?; ?) which, building on the two aforementioned types, utilises a recursive dialogue-based prompting, emphasising logically consistent reasoning through recursive explanations at each step. This may be helpful for addressing $R_Dia$ and supporting LLMs in developing the reasoning capacities for addressing different dialectical moves. ## 4 Conclusions This paper advocates for a focus on argumentation-based inquiry dialogues so as to integrate human ethical preferences in joint human-LLM reasoning and decision making. We have argued that such dialogues are warranted by state-of-the-art proposals for value alignment, and moreover support humans in refining their ethical positions. We have enumerated a number of research challenges for realising said dialogues, in the areas of computational argument and dialogue, and LLMs, and pointed to progress made. While recent advancements in LLMs offer promising capabilities, significant challenges in dialogue management persist. Addressing these challenges will be key to developing AI systems that can engage effectively in joint inquiry for value alignment. ## Acknowledgments This work was supported by the UK Research and Innovation Centre for Doctoral Training in Safe and Trusted Artificial Intelligence [grant number EP/S023356/1]. ## References - Abbasiantaeb et al. 2024 Abbasiantaeb, Z.; Yuan, Y.; Kanoulas, E.; and Aliannejadi, M. 2024. Let the llms talk: Simulating human-to-human conversational qa via zero-shot llm-to-llm interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, WSDM ’24, 8–17. New York, NY, USA: Association for Computing Machinery. - Abu-Rasheed et al. 2024 Abu-Rasheed, H.; Abdulsalam, M. H.; Weber, C.; and Fathi, M. 2024. Supporting student decisions on learning recommendations: An llm-based chatbot with knowledge graph contextualization for conversational explainability and mentoring. arXiv preprint arXiv:2401.08517. - Araci 2019 Araci, D. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. - Atkinson and Bench-Capon 2007 Atkinson, K., and Bench-Capon, T. 2007. Practical reasoning as presumptive argumentation using action based alternating transition systems. Artificial Intelligence 171(10):855–874. - Besnard et al. 2014 Besnard, P.; Garcia, A.; Hunter, A.; Modgil, S.; Prakken, H.; Simari, G.; and Toni, F. 2014. Introduction to structured argumentation. Argument and Computation 5. - Bezou-Vrakatseli, Cocarascu, and Modgil 2022 Bezou-Vrakatseli, E.; Cocarascu, O.; and Modgil, S. 2022. Towards an argument scheme classification for ethical reasoning. In Grasso, F.; Green, N. L.; Schneider, J.; and Wells, S., eds., Proceedings of the 22nd Workshop on Computational Models of Natural Argument, CMNA@COMMA 2022, Cardiff, Wales, September 12, 2022, volume 3205 of CEUR Workshop Proceedings, 13–17. CEUR-WS.org. - Black and Hunter 2009 Black, E., and Hunter, A. 2009. An inquiry dialogue system. Autonomous Agents and Multi-agent Systems 19:173–209. - Bolton et al. 2022 Bolton, E.; Hall, D.; Yasunaga, M.; Lee, T.; Manning, C.; and Liang, P. 2022. Biomedlm: a domain-specific large language model for biomedical text. Stanford CRFM Blog. - Bondarenko et al. 1997 Bondarenko, A.; Dung, P.; Kowalski, R.; and Toni, F. 1997. An abstract, argumentation–theoretic approach to default reasoning. Artificial Intelligence 93:63–101. - Bostrom 2014 Bostrom, N. 2014. Superintelligence: Paths, Dangers, Strategies. Oxford University Press. - Brown et al. 2020 Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33:1877–1901. - Bubeck et al. 2023 Bubeck, S.; Chandrasekaran, V.; Eldan, R.; Gehrke, J.; Horvitz, E.; Kamar, E.; Lee, P.; Lee, Y. T.; Li, Y.; Lundberg, S.; et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. - Castagna et al. 2023 Castagna, F.; Garton, A.; McBurney, P.; Parsons, S.; Sassoon, I.; and Sklar, E. I. 2023. Eqrbot: A chatbot delivering eqr argument-based explanations. Frontiers in Artificial Intelligence 6:1045614. - Castagna et al. 2024 Castagna, F.; Kokciyan, N.; Sassoon, I.; Parsons, S.; and Sklar, E. 2024. Computational argumentation-based chatbots: a survey. arXiv preprint arXiv:2401.03454. - Chang 2023 Chang, E. Y. 2023. Prompting large language models with the socratic method. In 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), 0351–0360. - Chen et al. 2023 Chen, L.; Wang, L.; Dong, H.; Du, Y.; Yan, J.; Yang, F.; Li, S.; Zhao, P.; Qin, S.; Rajmohan, S.; et al. 2023. Introspective tips: Large language model for in-context decision making. arXiv preprint arXiv:2305.11598. - Crispino et al. 2023 Crispino, N.; Montgomery, K.; Zeng, F.; Song, D.; and Wang, C. 2023. Agent instructs large language models to be general zero-shot reasoners. arXiv preprint arXiv:2310.03710. - D’Agostino and Modgil 2018 D’Agostino, M., and Modgil, S. 2018. Classical logic, argument and dialectic. Artificial Intelligence 262:15–51. - D’Agostino and Modgil 2021 D’Agostino, M., and Modgil, S. 2021. A fully rational account of structured argumentation under resource bounds. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI’20. - Driess et al. 2023 Driess, D.; Xia, F.; Sajjadi, M. S.; Lynch, C.; Chowdhery, A.; Ichter, B.; Wahid, A.; Tompson, J.; Vuong, Q.; Yu, T.; et al. 2023. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378. - Dung 1995 Dung, P. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and $n$ –person games. Artificial Intelligence 77:321–357. - Fan and Toni 2014 Fan, X., and Toni, F. 2014. A general framework for sound assumption–based argumentation dialogues. Artificial. Intelligence. 216(0):20 – 54. - Garcia and Simari 2004 Garcia, A. J., and Simari, G. R. 2004. Defeasible logic programming: an argumentative approach. Theory and Practice of Logic Programming 4(1–2):95–138. - Habernal and Gurevych 2017 Habernal, I., and Gurevych, I. 2017. Argumentation mining in user-generated web discourse. Computational linguistics 43(1):125–179. - Hadfield-Menell et al. 2016 Hadfield-Menell, D.; Dragan, A.; Abbeel, P.; and Russell, S. 2016. Cooperative inverse reinforcement learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, 3916–3924. Red Hook, NY, USA: Curran Associates Inc. - Hämäläinen, Tavast, and Kunnari 2023 Hämäläinen, P.; Tavast, M.; and Kunnari, A. 2023. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–19. - Hoffmann et al. 2022 Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D. d. L.; Hendricks, L. A.; Welbl, J.; Clark, A.; et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. - Huang, Altosaar, and Ranganath 2019 Huang, K.; Altosaar, J.; and Ranganath, R. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342. - Huang et al. 2023 Huang, S.; Jiang, Z.; Dong, H.; Qiao, Y.; Gao, P.; and Li, H. 2023. Instruct2act: Mapping multi-modality instructions to robotic actions with large language model. arXiv preprint arXiv:2305.11176. - Jung et al. 2022 Jung, J.; Qin, L.; Welleck, S.; Brahman, F.; Bhagavatula, C.; Bras, R. L.; and Choi, Y. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. arXiv preprint arXiv:2205.11822. - Kıcıman et al. 2023 Kıcıman, E.; Ness, R.; Sharma, A.; and Tan, C. 2023. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050. - Kojima et al. 2022 Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35:22199–22213. - Kökciyan et al. 2021 Kökciyan, N.; Sassoon, I.; Sklar, E.; Modgil, S.; and Parsons, S. 2021. Applying metalevel argumentation frameworks to support medical decision making. IEEE Intelligent Systems 36(2):64–71. - Lee, Goldwasser, and Reese 2024 Lee, Y.; Goldwasser, D.; and Reese, L. S. 2024. Towards understanding counseling conversations: Domain knowledge and large language models. In Graham, Y., and Purver, M., eds., Findings of the Association for Computational Linguistics: EACL 2024, 2032–2047. St. Julian’s, Malta: Association for Computational Linguistics. - Lewkowycz et al. 2022 Lewkowycz, A.; Andreassen, A.; Dohan, D.; Dyer, E.; Michalewski, H.; Ramasesh, V.; Slone, A.; Anil, C.; Schlag, I.; Gutman-Solo, T.; et al. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems 35:3843–3857. - Liu and Chilton 2022 Liu, V., and Chilton, L. B. 2022. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–23. - Liu et al. 2023 Liu, P.; Yuan, W.; Fu, J.; Jiang, Z.; Hayashi, H.; and Neubig, G. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys 55(9):1–35. - Liu et al. 2024 Liu, N.; Chen, L.; Tian, X.; Zou, W.; Chen, K.; and Cui, M. 2024. From llm to conversational agent: A memory enhanced architecture with fine-tuning of large language models. arXiv preprint arXiv:2401.02777. - Lu et al. 2023 Lu, J.; An, S.; Lin, M.; Pergola, G.; He, Y.; Yin, D.; Sun, X.; and Wu, Y. 2023. Memochat: Tuning llms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.08239. - MacAskill, Bykvist, and Ord 2020 MacAskill, M.; Bykvist, K.; and Ord, T. 2020. Moral uncertainty. Oxford University Press. - Modgil and Caminada 2009 Modgil, S., and Caminada, M. 2009. Chapter 6 : Proof theories and algorithms for abstract argumentation frameworks. In Simari, G., and Rahwan, I., eds., Argumentation in AI. Springer. 105–129. - Modgil and Prakken 2010 Modgil, S., and Prakken, H. 2010. Reasoning about preferences in structured extended argumentation frameworks. In Proceedings of the 2010 Conference on Computational Models of Argument: Proceedings of COMMA 2010, 347–358. NLD: IOS Press. - Modgil and Prakken 2013 Modgil, S., and Prakken, H. 2013. A general account of argumentation with preferences. Artificial Intelligence 195:361–397. - Modgil and Prakken 2018 Modgil, S., and Prakken, H. 2018. Abstract rule–based argumentation. In P. Baroni, D. Gabbay, M. G., and van der Torre, L., eds., Handbook of Formal Argumentation. 286–361. - Modgil 2017a Modgil, S. 2017a. Dialogical scaffolding for human and artificial agent reasoning. In AIC, 58–71. - Modgil 2017b Modgil, S. 2017b. Towards a general framework for dialogues that accommodate reasoning about preferences. In International Workshop on Theorie and Applications of Formal Argumentation, 175–191. Springer. - Papachristou, Yang, and Hsu 2023 Papachristou, M.; Yang, L.; and Hsu, C.-C. 2023. Leveraging large language models for collective decision-making. arXiv preprint arXiv:2311.04928. - Prakken 2005 Prakken, H. 2005. Coherence and flexibility in dialogue games for argumentation. Journal of logic and computation 15(6):1009–1040. - Prakken 2009 Prakken, H. 2009. Models of Persuasion Dialogue. Boston, MA: Springer US. 281–300. - Rajani et al. 2019 Rajani, N. F.; McCann, B.; Xiong, C.; and Socher, R. 2019. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361. - Russell, Dewey, and Tegmark 2015 Russell, S.; Dewey, D.; and Tegmark, M. 2015. Research priorities for robust and beneficial artificial intelligence. AI magazine 36(4):105–114. - Russell 2020 Russell, S. 2020. Human Compatible: AI and the Problem of Control. Penguin Press. - Sumers et al. 2023 Sumers, T. R.; Yao, S.; Narasimhan, K.; and Griffiths, T. L. 2023. Cognitive architectures for language agents. arXiv preprint arXiv:2309.02427. - Suzgun et al. 2022 Suzgun, M.; Scales, N.; Schärli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q. V.; Chi, E. H.; Zhou, D.; et al. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261. - Szabo et al. 2024 Szabo, J.; Criado, N.; Such, J.; and Modgil, S. 2024. Moral uncertainty and the problem of fanaticism. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19948–19955. - Taylor et al. 2020 Taylor, J.; Yudkowsky, E.; LaVictoire, P.; and Critch, A. 2020. Alignment for Advanced Machine Learning Systems. 342–382. - Thang and Luong 2014 Thang, P. M., and Luong, H. T. 2014. Translating preferred subtheories into structured argumentation. Journal of Logic and Computation 74(4):831–849. - Tolchinsky et al. 2012 Tolchinsky, P.; Modgil, S.; Atkinson, K.; McBurney, P.; and Cortés, U. 2012. Deliberation dialogues for reasoning about safety critical actions. Autonomous Agents and Multi-Agent Systems 25(2):209–259. - Touvron et al. 2023 Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. - Turpin et al. 2024 Turpin, M.; Michael, J.; Perez, E.; and Bowman, S. 2024. Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems 36. - Visser et al. 2021 Visser, J.; Lawrence, J.; Reed, C.; Wagemans, J.; and Walton, D. 2021. Annotating argument schemes. Argumentation 35(1):101–139. - Vlastos 1982 Vlastos, G. 1982. The socratic elenchus. The Journal of Philosophy 79(11):711–714. - Wachowiak and Gromann 2023 Wachowiak, L., and Gromann, D. 2023. Does gpt-3 grasp metaphors? identifying metaphor mappings with generative language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1018–1032. - Walton and Macagno 2015 Walton, D., and Macagno, F. 2015. A classification system for argumentation schemes. Argument & Computation 6(3):219–245. - Walton, Reed, and Macagno 2008 Walton, D.; Reed, C.; and Macagno, F. 2008. Argumentation schemes. Cambridge University Press. - Walton 2009 Walton, D. 2009. Argumentation theory: A very short introduction. In Argumentation in artificial intelligence. Springer. 1–22. - Wang et al. 2022 Wang, X.; Wei, J.; Schuurmans, D.; Le, Q.; Chi, E.; Narang, S.; Chowdhery, A.; and Zhou, D. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. - Wang et al. 2023 Wang, L.; Ma, C.; Feng, X.; Zhang, Z.; Yang, H.; Zhang, J.; Chen, Z.; Tang, J.; Chen, X.; Lin, Y.; et al. 2023. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432. - Wei et al. 2022a Wei, J.; Tay, Y.; Bommasani, R.; Raffel, C.; Zoph, B.; Borgeaud, S.; Yogatama, D.; Bosma, M.; Zhou, D.; Metzler, D.; et al. 2022a. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682. - Wei et al. 2022b Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022b. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35:24824–24837. - White et al. 2023 White, J.; Fu, Q.; Hays, S.; Sandborn, M.; Olea, C.; Gilbert, H.; Elnashar, A.; Spencer-Smith, J.; and Schmidt, D. C. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382. - Wu et al. 2023 Wu, S.; Irsoy, O.; Lu, S.; Dabravolski, V.; Dredze, M.; Gehrmann, S.; Kambadur, P.; Rosenberg, D.; and Mann, G. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564. - Xi et al. 2023 Xi, Z.; Chen, W.; Guo, X.; He, W.; Ding, Y.; Hong, B.; Zhang, M.; Wang, J.; Jin, S.; Zhou, E.; et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864. - Xydis et al. 2020 Xydis, A.; Hampson, C.; Modgil, S.; and Black, E. 2020. Enthymemes in dialogues. In COMMA, 355–369. - Xydis et al. 2022 Xydis, A.; Hampson, C.; Modgil, S.; and Black, E. 2022. A sound and complete dialogue system for handling misunderstandings. In SAFA@COMMA, 19–32. - Yang et al. 2023 Yang, S.; Nachum, O.; Du, Y.; Wei, J.; Abbeel, P.; and Schuurmans, D. 2023. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129. - Young, Modgil, and Rodrigues 2016 Young, A.; Modgil, S.; and Rodrigues, O. 2016. Prioritised default logic as rational argumentation. In Proc. 15th Int. Conference on Autonomous Agents and Multiagent Systems (AAMAS16), 626–634. - Zelikman et al. 2022 Zelikman, E.; Wu, Y.; Mu, J.; and Goodman, N. 2022. STar: Bootstrapping reasoning with reasoning. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems. - Zhang et al. 2022a Zhang, S.; Roller, S.; Goyal, N.; Artetxe, M.; Chen, M.; Chen, S.; Dewan, C.; Diab, M.; Li, X.; Lin, X. V.; et al. 2022a. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068. - Zhang et al. 2022b Zhang, Z.; Zhang, A.; Li, M.; and Smola, A. 2022b. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493. - Zhang et al. 2024 Zhang, Z.; Bo, X.; Ma, C.; Li, R.; Chen, X.; Dai, Q.; Zhu, J.; Dong, Z.; and Wen, J.-R. 2024. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501. - Zhao et al. 2023 Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.