What is a "bug"? On subjectivity, epistemic power, and implications for software research (2402.08165v1)

Abstract: Considerable effort in software research and practice is spent on bugs. Finding, reporting, tracking, triaging, attempting to fix them automatically, detecting "bug smells" -these comprise a substantial portion of large projects' time and development cost, and are of significant interest to researchers in Software Engineering, Programming Languages, and beyond. But, what is a bug, exactly? While segmentation faults rarely spark joy, most bugs are not so clear cut. Per the Oxford English Dictionary, the word "bug" has been a colloquialism for an engineering "defect" at least since the 1870s. Most modern software-oriented definitions speak to a disconnect between what a developer intended and what a program actually does. Formal verification, from its inception, has developed means to identify deviations from a formal specification, expected to more or less fully encode desired behavior. However, software is rarely accompanied by full and formal specifications, and this intention is instead treated as implicit or partially-documented at best. The International Software Testing Qualifications board writes: "A human being can make an error (mistake), which produces a defect (fault, bug) in the program code, or in a document. If a defect in code is executed, the system may fail to do what it should do (or do something it shouldn't), causing a failure. Defects may result in failures, but not all [do]". Most sources forsake this precision. The influential paper "Finding bugs is easy" begins by saying "bug patterns are code idioms that are often errors"-with no particular elaboration. Other work relies on imperfect practical proxies for specifications. For example, in automatic program repair research, a bug corresponds to a failing test case: when the test passes, the bug is considered fixed. However, when we interrogate fairly straightforward definitions, they start to break down...

• The paper redefines software bugs as subjective constructs shaped by individual judgments and power dynamics.
• It contrasts positivist and constructivist perspectives to challenge traditional bug categorization in development processes.
• The study advocates for transparent, human-centric methodologies that incorporate ethical and socio-technical considerations in software research.

On the Epistemology of Software Bugs: An Analysis of "What is a 'bug'?"

The paper, "What is a 'bug'? On subjectivity, epistemic power, and implications for software research," presents a nuanced exploration of the concept of bugs within the software engineering domain. Authored by David Gray Widder and Claire Le Goues, this work rigorously interrogates the traditional understanding of bugs and challenges the positivist perspective by contextualizing bugs not merely as technical defects but as constructs influenced by subjectivity and epistemic power.

Critical Examination of 'Bugs':

The paper premises its argument on the observation that the term "bug" lacks a universally accepted definition despite its pervasive use across disciplines such as Software Engineering, Security, and Programming Languages. Traditional definitions often frame bugs as a deviation from developer intent or functional specifications. However, the scarcity of comprehensive formal specifications in practice leaves much room for interpretation and subjective judgment. Through this lens, the authors present bugs as phenomena enveloped in layers of subjectivity, exemplified by the contention that certain test failures could be unrelated to functional correctness yet still perceived as bugs.

Impact of Subjectivity and Epistemic Power:

The work asserts that the categorization and prioritization of bugs are deeply intertwined with subjective value judgments and power dynamics within software development processes. The authors delve into the mechanics of modern development frameworks, such as bug triage and error budgets, revealing the implicit acceptance of not addressing every identified bug due to resource limitations. They draw on epistemological theories, contrasting positivism with constructivism, to highlight that what is deemed a "bug" often reflects the social and power structures within development teams or organizations.

The concept of epistemic power is central to the paper's thesis, illustrating how individuals or groups with knowledge and influence—notably software designers and maintainers—can shape what is classified as a bug. Through examples like GenderMag and ethical dilemmas reported by software engineers, the authors demonstrate the real-world implications of this power in broadening or restricting the scope of what constitutes a bug in software considerations.

Implications for Software Research and Development:

The ramifications of this analysis extend into both practical methods and theoretical approaches in software research. The paper advocates for an increased emphasis on transparency and contestability in research methodologies, encouraging a shift from shielding research from critique to openly discussing value-laden decisions. This reconceptualization is posited to foster a more diverse and robust scientific inquiry, allowing for the inclusion of diverse user experiences and ethical considerations.

Additionally, the authors propose recalibrating research goals to center on human concerns, advocating for "human-in-the-loop" studies that appreciate the social contexts in which software is utilized. The emphasis is placed on evaluating technical outcomes not solely through performance metrics on historical data but through their real-world impacts, particularly on marginalized user groups.

Future Directions:

Looking forward, the paper invites further exploration into the nuanced nature of bugs, encouraging the integration of broader socio-technical factors into bug taxonomy and prioritization within software systems. It suggests a potential for enriching software engineering by redefining bugs to encompass ethical and inclusivity dimensions, thus driving the evolution of systems that better serve diverse populations.

In conclusion, the paper articulates a compelling discourse on the nature of bugs, urging researchers and practitioners to discern the subjective and power-laden dimensions that underpin software development and defect management. By situating bugs within a broader epistemic and social context, the authors contribute a thought-provoking perspective that is likely to inform ongoing advancements and debates in software engineering research.