A Survey on What Developers Think About Testing

Abstract: Software is infamous for its poor quality and frequent occurrence of bugs. While there is no doubt that thorough testing is an appropriate answer to ensure sufficient quality, the poor state of software generally suggests that developers may not always engage as thoroughly with testing as they should. This observation aligns with the prevailing belief that developers simply do not like writing tests. In order to determine the truth of this belief, we conducted a comprehensive survey with 21 questions aimed at (1) assessing developers' current engagement with testing and (2) identifying factors influencing their inclination toward testing; that is, whether they would actually like to test more but are inhibited by their work environment, or whether they would really prefer to test even less if given the choice. Drawing on 284 responses from professional software developers, we uncover reasons that positively and negatively impact developers' motivation to test. Notably, reasons for motivation to write more tests encompass not only a general pursuit of software quality but also personal satisfaction. However, developers nevertheless perceive testing as mundane and tend to prioritize other tasks. One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts.

• The paper presents a comprehensive survey of 284 developers to analyze testing attitudes and identify workplace constraints.
• It uses a detailed 21-question survey to pinpoint barriers such as limited time, recognition deficits, and complex testability issues.
• The findings suggest that enhancing motivation through gamification and improved testing tools could significantly boost testing engagement.

Overview of "A Survey on What Developers Think About Testing"

The paper, "A Survey on What Developers Think About Testing" (2309.01154), presents a comprehensive investigation into the attitudes and practices of software developers regarding testing. The primary goal is to discern whether developers are genuinely disinclined to engage in testing, or if other factors such as workplace environment constraints contribute to this perception. Through a detailed survey comprising 21 questions and encompassing 284 responses, the paper identifies various factors that influence developers' motivation to test.

Current Engagement and Challenges

The survey reveals that developers spend a limited proportion of their work time on testing, with many dedicating less than 40% of their time to such activities. This discrepancy might be attributed to technical challenges and organizational constraints that hinder more rigorous testing practices. Developers often report intricate tests as a formidable barrier due to inadequate testability and limited recognition for the added effort testing requires. However, several developers also highlight personal satisfaction in writing tests, though they frequently express that testing is mundane and prefer coding tasks over testing efforts.

Despite the apparent challenges, about half of the surveyed developers feel that their current level of testing is adequate. Yet, others perceive a lack of sufficient testing in their organizations, mentioning factors like undervaluation of testing, insufficient time and resources, and incomplete test processes as obstacles. Approximately 60% of developers claim that their projects are not tested sufficiently, whether due to low prioritization by management or technical hurdles like unmeasured code coverage.

Desired Engagement and Motivation Factors

The responses to questions about desired testing engagement reveal mixed views. Almost half of the participants indicate a desire to engage more with testing if certain constraints were alleviated, such as by making testing more technically feasible and by receiving greater recognition for their efforts. Interestingly, the study shows that developer motivation is a pivotal element in their testing practices, which implies that external motivation strategies like gamification could potentially enhance testing engagement.

Simultaneously, about 34% of developers are satisfied with their current level of testing, primarily due to either having reached a quality goal or because they find the milestone tasks sufficient. Several developers express a preference for coding over testing, often perceiving testing tasks as unimportant relative to other responsibilities.

Implications and Future Directions

The paper's findings highlight the significant role of motivation and recognition in developers' testing practices. This suggests that the industry should explore ways to enhance motivation for testing, potentially through more robust autocomplete tools, simplified testing frameworks, and gamification strategies to make testing more engaging.

The survey also implies that improvements in test recognition and quality measurement practices could lead to better quality assurance and testing engagement. Thus, future research needs to focus both on the development of automated testing tools and the cultivation of an environment where testing receives adequate recognition. Increasing awareness at both management and developer levels about the importance of test practices could also foster a culture where testing is valued equally alongside development.

Conclusion

The paper presents a nuanced understanding of software developers' perspectives on testing, elucidating the challenges they face and the factors that influence their engagement. It offers a foundation for future research to explore strategies that increase motivation and improve testing practices within the software industry. These insights underscore the need for better recognition systems for testing activities and the potential for research-driven interventions to elevate the role of testing in software development processes.