Effective Feedback for Introductory CS Theory: A JFLAP Extension and Student Persistence

Computing theory analyzes abstract computational models to rigorously study the computational difficulty of various problems. Introductory computing theory can be challenging for undergraduate students, and the main goal of our research is to help students learn these computational models. The most common pedagogical tool for interacting with these models is the Java Formal Languages and Automata Package (JFLAP). We developed a JFLAP server extension, which accepts homework submissions from students, evaluates the submission as correct or incorrect, and provides a witness string when the submission is incorrect. Our extension currently provides witness feedback for deterministic finite automata, nondeterministic finite automata, regular expressions, context-free grammars, and pushdown automata. In Fall 2019, we ran a preliminary investigation on two sections (Control and Study) of the required undergraduate course Introduction to Computer Science Theory. The Study section used our extension for five targeted homework questions, and the Control section solved and submitted these problems using traditional means. Our results show that on these five questions, the Study section performed better on average than the Control section. Moreover, the Study section persisted in submitting attempts until correct, and from this finding, our preliminary conclusion is that minimal (not detailed or grade-based) witness feedback helps students to truly learn the concepts. We describe the results that support this conclusion as well as a related hypothesis conjecturing that with witness feedback and unlimited number of submissions, partial credit is both unnecessary and ineffective.