More than Model Documentation: Uncovering Teachers' Bespoke Information Needs for Informed Classroom Integration of ChatGPT (2309.14458v1)
Abstract: ChatGPT has entered classrooms, but not via the typical route of other educational technology, which includes comprehensive training, documentation, and vetting. Consequently, teachers are urgently tasked to assess its capabilities to determine potential effects on student learning and instruct their use of ChatGPT. However, it is unclear what support teachers have and need and whether existing documentation, such as model cards, provides adequate direction for educators in this new paradigm. By interviewing 22 middle- and high-school teachers, we connect the discourse on AI transparency and documentation with educational technology integration, highlighting the critical information needs of teachers. Our findings reveal that teachers confront significant information gaps, lacking clarity on exploring ChatGPT's capabilities for bespoke learning tasks and ensuring its fit with the needs of diverse learners. As a solution, we propose a framework for interactive model documentation that empowers teachers to navigate the interplay between pedagogical and technical knowledge.
- Software documentation: the practitioners’ perspective. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 590–601.
- Software documentation issues unveiled. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 1199–1210.
- Software Documentation Issues Unveiled. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 1199–1210. https://doi.org/10.1109/ICSE.2019.00122
- Roberto Aldunate and Miguel Nussbaum. 2013. Teacher adoption of technology. Computers in Human Behavior 29, 3 (2013), 519–524.
- Norah Almusharraf and Hind Alotaibi. 2023. An error-analysis study from an EFL writing context: Human and Automated Essay Scoring Approaches. Technology, Knowledge and Learning 28, 3 (2023), 1015–1031.
- Power to the people: The role of humans in interactive machine learning. Ai Magazine 35, 4 (2014), 105–120.
- FactSheets: Increasing trust in AI services through supplier’s declarations of conformity. IBM Journal of Research and Development 63, 4/5 (2019), 6–1.
- Ai ethics statements: analysis and lessons learnt from neurips broader impact statements. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2047–2056.
- Ryan S Baker and Aaron Hawn. 2021. Algorithmic bias in education. International Journal of Artificial Intelligence in Education (2021), 1–41.
- TPACK: An emerging research and development tool for teacher educators. Turkish Online Journal of Educational Technology-TOJET 10, 4 (2011), 370–377.
- Amy L Baylor and Donn Ritchie. 2002. What factors facilitate teacher skill, teacher morale, and perceived student learning in technology-using classrooms? Computers & education 39, 4 (2002), 395–414.
- Jaime L Beck. 2017. The weight of a heavy hour: Understanding teacher experiences of work intensification. McGill Journal of Education 52, 3 (2017), 617–636.
- EM Bender and Batya Friedman. 2019. Data statements for NLP: toward mitigating system bias and enabling better science. In Preprint at https://openreview. net/forum.
- Ethics and society review: Ethics reflection as a precondition to research funding. Proceedings of the National Academy of Sciences 118, 52 (2021), e2117261118.
- Brandford Bervell and Irfan Naufal Umar. 2017. Validation of the UTAUT model: Re-considering non-linear relationships of exogeneous variables in higher education technology acceptance research. Eurasia Journal of Mathematics, Science and Technology Education 13, 10 (2017), 6471–6490.
- Aditi Bhutoria. 2022. Personalized education and artificial intelligence in the United States, China, and India: A systematic review using a human-in-the-loop model. Computers and Education: Artificial Intelligence 3 (2022), 100068.
- Educating software and AI stakeholders about algorithmic fairness, accountability, transparency and ethics. International Journal of Artificial Intelligence in Education (2021), 1–26.
- Overcoming failures of imagination in AI infused system development and deployment. arXiv preprint arXiv:2011.13416 (2020).
- Sam Brill and Abby McCartney. 2008. Stopping the revolving door: Increasing teacher retention. Politics & Policy 36, 5 (2008), 750–774.
- Karen E Brinkley-Etzkorn. 2018. Learning to teach online: Measuring the influence of faculty development training on teaching effectiveness through a TPACK lens. The Internet and Higher Education 38 (2018), 28–35.
- Graham Butt and Ann Lance. 2005. Secondary teacher workload and job satisfaction: do successful strategies for change exist? Educational Management Administration & Leadership 33, 4 (2005), 401–422.
- Davide Castelvecchi. 2016. Can we open the black box of AI? Nature News 538, 7623 (2016), 20.
- Ismail Celik. 2023. Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior 138 (2023), 107468.
- The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends 66, 4 (2022), 616–630.
- Kathy Charmaz. 2014. Constructing grounded theory. sage.
- Two decades of artificial intelligence in education. Educational Technology & Society 25, 1 (2022), 28–47.
- Kun-Hung Cheng and Chin-Chung Tsai. 2019. A case study of immersive virtual field trips in an elementary classroom: Students’ learning experience and teacher-student interaction behaviors. Computers & Education 140 (2019), 103600.
- Ronnie Cheung and Doug Vogel. 2013. Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning. Computers & education 63 (2013), 160–175.
- Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies 49, 2 (2023), 295–313.
- A survey of artificial intelligence techniques employed for adaptive educational systems within e-learning platforms. Journal of Artificial Intelligence and Soft Computing Research 7, 1 (2017), 47–64.
- What is “technology integration” and how is it measured in K-12 education? A systematic review of survey instruments from 2010 to 2021. Computers & Education (2023), 104742.
- Grant Cooper. 2023. Examining science education in chatgpt: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology 32, 3 (2023), 444–452.
- John W Creswell and J David Creswell. 2017. Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.
- Interactive model cards: A human-centered approach to model documentation. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 427–439.
- Larry Cuban. 1986. Teachers and machines: The classroom use of technology since 1920. Teachers college press.
- Fred D Davis. 1987. User acceptance of information systems: the technology acceptance model (TAM). (1987).
- Ton De Jong and Monica GM Ferguson-Hessler. 1996. Types and qualities of knowledge. Educational psychologist 31, 2 (1996), 105–113.
- Reading Comprehension Quiz Generation using Generative Pre-trained Transformers.
- Peggy A Ertmer and Anne T Ottenbreit-Leftwich. 2010. Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of research on Technology in Education 42, 3 (2010), 255–284.
- Gregory M Francom. 2020. Barriers to technology integration: A time-series survey study. Journal of Research on Technology in Education 52, 1 (2020), 1–16.
- Datasheets for datasets. Commun. ACM 64, 12 (2021), 86–92.
- ATLAS.ti Scientific Software Development GmbH. 2023. ATLAS.ti: The Qualitative Data Analysis & Research Software. https://atlasti.com/
- Using TPACK as a framework to understand teacher candidates’ technology integration decisions. Journal of Computer Assisted Learning 28, 6 (2012), 530–546.
- Andrina Granić. 2022. Educational technology adoption: a systematic review. Education and Information Technologies 27, 7 (2022), 9725–9744.
- Alex Guilherme. 2019. AI and education: the importance of teacher and student relations. AI & society 34 (2019), 47–54.
- The substitution augmentation modification redefinition (SAMR) model: A critical review and suggestions for its use. TechTrends 60 (2016), 433–441.
- Karen Hao. 2019. China has started a grand experiment in AI education. It could reshape how the world learns. MIT Technology Review 123, 1 (2019).
- Will Douglas Heaven. 2023. ChatGPT is going to change education, not destroy it. https://www.technologyreview.com/2023/04/06/1071059/chatgpt-change-not-destroy-education-openai/
- Jessa Henderson and Michael Corry. 2021. Teacher anxiety and technology change: A review of the literature. Technology, Pedagogy and Education 30, 4 (2021), 573–587.
- Khe Foon Hew and Thomas Brush. 2007. Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational technology research and development 55 (2007), 223–252.
- Emily Hixon and Janet Buckenmeyer. 2009. Revisiting technology integration in schools: Implications for professional development. Computers in the Schools 26, 2 (2009), 130–146.
- The dataset nutrition label. Data Protection and Privacy 12, 12 (2020), 1.
- Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education (2021), 1–23.
- Designing for complementarity: Teacher and student needs for orchestration support in AI-enhanced classrooms. In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29, 2019, Proceedings, Part I 20. Springer, 157–171.
- Pi-Sui Hsu. 2016. Examining current beliefs, practices and barriers about technology integration: A case study. TechTrends 60 (2016), 30–40.
- Gwo-Jen Hwang. 2014. Definition, framework and research issues of smart learning environments-a context-aware ubiquitous learning perspective. Smart Learning Environments 1, 1 (2014), 1–14.
- Vision, challenges, roles and research issues of Artificial Intelligence in Education. , 100001 pages.
- Fethi A Inan and Deborah L Lowther. 2010. Factors affecting technology integration in K-12 classrooms: A path model. Educational technology research and development 58 (2010), 137–154.
- A systematic review of conversational AI in language education: Focusing on the collaboration with human teachers. Journal of Research on Technology in Education 55, 1 (2023), 48–63.
- All-in-one: Multi-task learning bert models for evaluating peer assessments. arXiv preprint arXiv:2110.03895 (2021).
- Factors influencing preservice teachers’ intention to use technology: TPACK, teacher self-efficacy, and technology acceptance model. Journal of Educational Technology & Society 21, 3 (2018), 48–59.
- Rachel Karchmer-Klein and Haruka Konishi. 2023. A mixed-methods study of novice teachers’ technology integration: Do they leverage their TPACK knowledge once entering the profession? Journal of Research on Technology in Education 55, 3 (2023), 490–506.
- ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences 103 (2023), 102274.
- Trynke Keuning and Marieke van Geel. 2021. Differentiated teaching with adaptive learning systems and teacher dashboards: the teacher still matters most. IEEE Transactions on Learning Technologies 14, 2 (2021), 201–210.
- Analyzing teacher competency with TPACK for K-12 AI education. KI-Künstliche Intelligenz 35, 2 (2021), 139–151.
- The PICRAT model for technology integration in teacher preparation. Contemporary Issues in Technology and Teacher Education 20, 1 (2020), 176–198.
- Alyson Klein. 2022. Artificial Intelligence Is All Around Us. So This District Designed Its Own AI Curriculum. https://www.edweek.org/technology/artificial-intelligence-is-all-around-us-so-this-district-designed-its-own-ai-curriculum/2022/08
- Six learning barriers in end-user programming systems. In 2004 IEEE Symposium on Visual Languages-Human Centric Computing. IEEE, 199–206.
- Matthew Koehler and Punya Mishra. 2009. What is technological pedagogical content knowledge (TPACK)? Contemporary issues in technology and teacher education 9, 1 (2009), 60–70.
- Florian Königstorfer and Stefan Thalmann. 2021. Software documentation is not enough! Requirements for the documentation of AI. Digital Policy, Regulation and Governance 23, 5 (2021), 475–488.
- Florian Königstorfer and Stefan Thalmann. 2022. AI Documentation: A path to accountability. Journal of Responsible Technology 11 (2022), 100043.
- Theodore J. Kopcha. 2012. Teachers’ perceptions of the barriers to technology integration and practices with technology under situated professional development. Computers & Education 59, 4 (2012), 1109–1121. https://doi.org/10.1016/j.compedu.2012.05.014
- Lauraine Langreo. 2022. What Teachers Should Consider Before Using AI-Powered Tools in the Classroom. https://www.edweek.org/technology/what-teachers-should-consider-before-using-ai-powered-tools-in-the-classroom/2022/12
- Tamar Levin and Rivka Wadmany. 2008. Teachers’ views on factors affecting effective integration of information technology in the classroom: Developmental scenery. Journal of Technology and Teacher Education 16, 2 (2008), 233–263.
- Robin Li. 2020. Artificial intelligence revolution: How AI will change our society, economy, and culture. Simon and Schuster.
- Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 424–435.
- AI for school teachers. CRC Press.
- Rose Luckin and Wayne Holmes. 2016. Intelligence unleashed: An argument for AI in education. (2016).
- Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of educational psychology 106, 4 (2014), 901.
- Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 2. 37–39.
- Stephen Mihm. 2023. What a Calculator Can Tell You About ChatGPT. https://www.washingtonpost.com/business/2023/09/05/chatgpt-is-2023-s-version-of-the-calculator-and-cliffsnotes/716af25a-4bf0-11ee-bfca-04e0ac43f9e4_story.html
- Punya Mishra. 2019. Considering contextual knowledge: The TPACK diagram gets an upgrade. , 76–78 pages.
- P Mishra. 2023. ChatGPT is a smart drunk intern: 3 examples.
- Punya Mishra and Melissa Warr. 2021. Contextualizing TPACK within systems and cultures of practice. Computers in Human Behavior 117 (2021), 106673.
- Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.
- Council of Chief State School Officers National Governors Association Center for Best Practices. 2010. Common Core State Standards: Writing. https://www.thecorestandards.org/ELA-Literacy/W/7/
- Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence 2 (2021), 100041.
- AI in Learning: Designing the Future. Springer Nature.
- White House Office of Science and Technology Policy. 2022. Blueprint for an AI Bill of Rights. https://www.whitehouse.gov/ostp/ai-bill-of-rights/
- OpenAI. 2020. GPT-3 Model Card. https://github.com/openai/gpt-3/blob/master/model-card.md
- OpenAI. 2023a. ChatGPT. https://chat.openai.com/
- OpenAI. 2023b. Educator FAQ. https://help.openai.com/en/collections/5929286-educator-faq
- OpenAI. 2023c. Privacy Policy. https://openai.com/policies/privacy-policy
- OpenAI. 2023d. Teaching with AI. https://openai.com/blog/teaching-with-ai
- Sonmez Pamuk. 2012. Understanding preservice teachers’ technology use through TPACK framework. Journal of computer assisted learning 28, 5 (2012), 425–439.
- Niels Pinkwart. 2016. Another 25 years of AIED? Challenges and opportunities for intelligent educational technologies of the future. International journal of artificial intelligence in education 26 (2016), 771–783.
- Towards evaluating and eliciting high-quality documentation for intelligent systems. arXiv preprint arXiv:2011.08774 (2020).
- Isak Potgieter. 2020. Privacy concerns in educational data mining and learning analytics. The International Review of Information Ethics 28 (2020).
- Justin Reich and Mizuko Ito. 2017. From good intentions to real outcomes: Equity by design in learning technologies. Digital Media and Learning Research Hub (2017).
- Michael J Reiss. 2021. The Use of Al in Education: Practicalities and Ethical Considerations. London Review of Education 19, 1 (2021), n1.
- Integrating educational technology into teaching. USA: Pearson, 2007.
- AI and the future of learning: Expert panel report. Technical Report. Digital Promise.
- Kalhan Rosenblatt. 2023. ChatGPT banned from New York City public schools’ devices and networks. NBC News (2023).
- Daniel Schiff. 2021. Out of the laboratory and into the classroom: the future of artificial intelligence in education. AI & society 36, 1 (2021), 331–348.
- Daniel Schiff. 2022. Education for AI, not AI for education: The role of education and ethics in national AI policy strategies. International Journal of Artificial Intelligence in Education 32, 3 (2022), 527–563.
- Technology-related knowledge, skills, and attitudes of pre-and in-service teachers: The current situation and emerging trends. Computers in Human Behavior 115 (2021), 106552.
- Natasha Singer. 2023. Despite Cheating Fears, Schools Repeal ChatGPT Bans. New York Times (2023).
- Evaluating the Social Impact of Generative AI Systems in Systems and Society. arXiv preprint arXiv:2306.05949 (2023).
- Jill Staake. 2023. 20 Ways Teachers Can Use ChatGPT To Make Their Lives Easier. https://www.weareteachers.com/chatgpt-for-teachers/
- C Stokel-Walker. 2022. AI bot ChatGPT writes smart essays—Should professors worry?[published online ahead of print December 9, 2022]. Nature News (2022).
- Anselm Strauss and Juliet Corbin. 1990. Basics of qualitative research. Sage publications.
- Anaïs Tack and Chris Piech. 2022. The AI teacher test: Measuring the pedagogical ability of blender and GPT-3 in educational dialogues. arXiv preprint arXiv:2205.07540 (2022).
- OpenAI Team. 2022. ChatGPT: Optimizing language models for dialogue.
- Timothy Teo. 2009. Modelling technology acceptance in education: A study of pre-service teachers. Computers & education 52, 2 (2009), 302–312.
- Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: a systematic review of qualitative evidence. Educational technology research and development 65 (2017), 555–575.
- Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence. Computers & Education 59, 1 (2012), 134–144.
- Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the 2018 chi conference on human factors in computing systems. 1–14.
- Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359 (2021).
- Florida’s technology integration matrix. Principal Leadership 12, 2 (2011), 69–71.
- Three teacher educators’ perspectives about the shifting responsibilities of infusing technology into the curriculum. Action in Teacher Education 24, 4 (2003), 53–63.
- ChatGPT: Challenges, Opportunities, and Implications for Teacher Education. Contemporary Issues in Technology and Teacher Education 23, 1 (2023), 1–23.
- Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education 16, 1 (2019), 1–27.
- Cost, benefits and quality of software development documentation: A systematic mapping. Journal of Systems and Software 99 (2015), 175–198.