The Effects of Confirmation Bias and Time Pressure in Software Testing

Confirmation bias is the tendency to search for evidence that confirms a person’s preconceptions. Confirmation bias among software test engineers is their tendency to confirm the correct functioning of the software rather than testing it to reveal errors. Research in psychology suggests that time pressure may promote confirmation bias because time pressure hinders analytical assessment of the task. Time pressure is perceived negatively for its effects in software engineering (SE), therefore, its effect on confirmation bias may worsen the quality of software application. We investigate confirmation bias and the effects of time pressure on confirmation bias among software test engineers. We also investigate the effects of time pressure on the actual and self-assessed performance of software test engineers. Moreover, we examine what other antecedents to confirmation bias exist in software testing and how they lead to it. The theory generated from the interview data from software test engineers showed that time pressure is one of the reasons for confirmation bias because test engineers compromise on testing the code to reveal errors under time pressure. Confirmation bias was evident among software testing students in a laboratory experiment but the experiment did not show that time pressure promoted confirmation bias. Another laboratory experiment showed that time pressure did not affect the self-assessment of performance of software testing students, which was in contrast to their actual performance. The conclusion is, software test engineers should develop self-awareness of confirmation bias and improve in their self-assessments of performance to improve their actual software testing. In the industry, automated software testing may alleviate confirmation bias due to time pressure by rapidly performing software testing.