"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices (2403.19876v1)

Abstract: LLMs are increasingly applied in real-world scenarios, including research and education. These models, however, come with well-known ethical issues, which may manifest in unexpected ways in human-computer interaction research due to the extensive engagement with human subjects. This paper reports on research practices related to LLM use, drawing on 16 semi-structured interviews and a survey conducted with 50 HCI researchers. We discuss the ways in which LLMs are already being utilized throughout the entire HCI research pipeline, from ideation to system development and paper writing. While researchers described nuanced understandings of ethical issues, they were rarely or only partially able to identify and address those ethical concerns in their own projects. This lack of action and reliance on workarounds was explained through the perceived lack of control and distributed responsibility in the LLM supply chain, the conditional nature of engaging with ethics, and competing priorities. Finally, we reflect on the implications of our findings and present opportunities to shape emerging norms of engaging with LLMs in HCI research.

• The paper’s main contribution is its comprehensive analysis of ethical challenges when LLMs are used in various HCI research stages.
• It details how LLMs assist in ideation, data analysis, system development, and writing, while highlighting risks like bias and data misuse.
• The study underscores the need for proactive ethics, recommending IRB engagement and reformed informed consent processes to guide research accountability.

Examining Ethics of LLM Use in HCI Research Practices

Human-Computer Interaction (HCI) researchers have been integrating LLMs into various stages of their research workflows, leveraging these tools for ideation, data generation, analysis, system development, and paper writing. This paper investigates the ethical considerations surrounding the use of LLMs within HCI research practices, highlighting researchers' awareness and the strategies applied to navigate these concerns.

Integration of LLMs in HCI Research

LLMs have been utilized across different stages of the HCI research process, offering significant advantages in ideation, literature synthesis, project scoping, qualitative and quantitative data analysis, system building, and paper writing. At the ideation stage, researchers employ LLMs to generate research questions and explore new problem areas, adopting a breadth-first search strategy. During the data analysis process, LLMs assist in reducing the workload through automatic coding and visualization. For paper writing, they provide drafting assistance, enabling iterative refinement and alternative perspectives. LLMs also serve as design materials, enhancing creativity in system development.

Figure 1: LLMs are utilized throughout the HCI research process, including ideation, paper design, execution, and paper writing.

Ethical Concerns in Utilizing LLMs

Ethical considerations with LLM use in HCI research span several areas:

Harmful Outputs

LLM-generated content can harbor biases, stereotyping, and harmful language, disproportionately affecting vulnerable groups. Concerns regarding LLM hallucinations, where models fabricate authoritative yet incorrect information, raise issues of trust in research outputs.

Privacy of Participant Data

Researchers express anxiety over data breaches and the potential misuse of personally identifiable or sensitive data submitted to LLM systems. The opacity of LLM providers exacerbates fears surrounding data usage policies.

Violations of Intellectual Integrity

Ambiguities in content ownership and the reproducibility of results generated by LLMs prompt questions about intellectual integrity. Researchers struggle to discern and attribute portions of work co-created with LLM assistance.

Overtrust and Overreliance

Overtrust in LLMs by research subjects who may overestimate model capabilities is a significant concern. An overreliance on LLMs risks diminishing creativity and compromising research quality.

Environmental and Societal Impacts

The societal and environmental implications include the high energy footprints of LLMs and the inequality potentially exacerbated by widespread model adoption.

Approaches to Navigating Ethical Concerns

Despite awareness, researchers largely engage with ethical issues conditionally, demonstrating varied strategies without adequately addressing underlying challenges:

Conditional and Reactive Engagement

Research domains often dictate ethical involvement, creating a reactive attitude to concerns. Rather than preemptive action, researchers engage with ethics based on perceived stakes and domain safety.

Limited Disclosure Practices

LLMs are viewed as productivity tools, leading to minimal disclosure to stakeholders. Researchers liken LLM reporting to previous software tools, which are often not explicitly detailed.

Restricting LLM Use and Reflecting

Practitioners adopt restrictive LLM use across limited tasks, exercising caution in integrating outputs. Group reflection sessions facilitate explication and navigation of ethical considerations.

Delaying Research Accountability

Distributed responsibility across the AI supply chain is evident, with researchers deferring ethical handling due to perceived provider control or societal research benefits outweighing issues.

Implications for Ethical Norm Formation

To effectively engage with LLM ethics within HCI, several actionable suggestions are posited:

• Proactive IRB Engagement: Researchers should actively involve IRB at paper design stages, documenting implicit LLM usage and employing transparent communication with review boards.
• Informed Consent Process Redesign: Address ethical concerns with clear communication, incorporating LLM risks and tools during consent stages, and ensuring participant comprehension of technology impacts.
• Interruption of AI Supply Chains: Development of methods to allow researchers control over data privacy and bias identification could shift power dynamics from service providers.
• Education and Academic Incentive Shift: Generate interdisciplinary learning opportunities and redefine academic reward systems to prioritize and value ethical research challenges.

The implications contribute to emerging ethical norms, urging researchers towards collaborative consideration and integration of ethics within LLM-powered HCI research practices.