CHALLENGES OF CONDUCTING SOFTWARE ENGINEERING EXPERIMENTS: EVERYTHING YOU ALWAYS WANTED TO KNOW BUT WERE AFRAID TO ASK

Lecturer: 
Associate Professor Sira Vegas
Lecturer's institute: 
Universidad Politécnica de Madrid, Spain
Date: 
21.3.2019 09:15 to 22.3.2019 16:00

Date and place:

ABSTRACT

Experimentation is a key issue in science and engineering. But it is one of software engineering’s stumbling blocks. Quite a lot of experiments are run nowadays, but it is a risky business. Software engineering has some special features, leading to some experimentation issues being conceived of differently than in other disciplines. The aim of this course is to help participants to avoid common pitfalls when conducting software engineering experiments. The course is not intended as an experimentation (design and data analysis) course, because there is already plenty of literature on this subject. It reviews several key issues identified in published software engineering experiments, and addresses them based on the knowledge acquired after 20 years running experiments.

This course extends the tutorial “Analyzing Software Engineering Experiments: Everything You Always Wanted to Know but Were Afraid to Ask”, presented at ICSE 2016, 2017 and 2018.

DETAILED PROGRAM

Day 1: Planning & Design Challenges

9:00-12:00     Challenges planning and designing Software Engineering (SE) experiments:

  • Introduction (motivation and reminder of basic concepts in SE experimentation)
  • Constructs operationalization
  • Hypothesis formulation
  • Design strategies
  • Summary

12:00-13:00   Lunch Break

13:00-16:00     Laboratory. Students will be asked to design a SE experiment taking into consideration the challenges addressed. The proposed designs will be discussed.

Day 2: Analysis Challenges

9:00-12:00     How to analyse a crossover experiment:

  • Choosing the right data analysis technique
  • Analyzing tricky designs
  • Parametric vs non-parametric tests
  • The 3 musketeers: statistical significance, effect size and power
  • Summary

12:00-13:00   Lunch Break

13:00-16:00     Laboratory (with SPSS). Students will be asked to analyse a real SE experiment. They will They will practice the challenges addressed.

 

CREDIT POINTS

Doctoral students participating in the seminar can obtain 2 credit points. This requires participating on all of the days and completing the lab assignments.

 

BIOGRAPHY

Sira Vegas (http://www.grise.upm.es/htdocs/miembros/sira/) received her PhD degree from the Universidad Politécnica de Madrid in 2002. She is currently associate professor of software engineering at Universidad Politécnica de Madrid. Her main research interests are experimental software engineering and software testing. She is regular reviewer of highly ranked journals such as IEEE Transactions on Software Engineering, Empirical Software Engineering Journal, ACM Transactions on Software Engineering and Methodology and Information and Software Technology. Dr. Vegas was program chair for the International Symposium on Empirical Software Engineering and Measurement (ESEM) in 2007. She has been a regular visitor researcher at the University of Oulu during 2014-2018. She began her career as a summer student at the European Centre for Nuclear Research (CERN, Geneva) in 1995.

 

 

Last updated: 3.2.2019