Doctoral course - Overview of Empirical Methods for Computing Research
Event information
Time
-
Location
Lecturer
Paul Ralph, Professor (Dalhousie University, Halifax, Canada)
Overview of Empirical Methods for Computing Research
Schedule
Wed 10.4 at 9:00-17:30 in PR142
Thu 11.4 at 9:00-17:30 in PR145
Content
- 3 hours preparation
- 12 hours lectures
- 2 hours exercises
Assessment
Attendance and exercises (Pass/Fail)
Preparation
To prepare for this course, students should:
- Choose one or more empirical research questions relevant to their dissertations on which to focus during the course.
- Read Stol, K. J., & Fitzgerald, B. (2018). The ABC of software engineering research. ACM Transactions on Software Engineering and Methodology (TOSEM), 27(3), 1–51.
About the Instructor
Paul Ralph, PhD (British Columbia), is an award-winning scientist, author, consultant, and Professor of Software Engineering at Dalhousie University. Dr. Ralph’s has published more than 90 peer-reviewed articles on software engineering, sustainable development, human-computer interaction, and project management in premier venues including IEEE Transactions on Software Engineering and the ACM/IEEE International Conference on Software Engineering. Dr. Ralph is editor-in-chief of the SIGSOFT Empirical Standards for Software Engineering Research.
Course Abstract
This course begins with a brief analysis of the shift from mathematical proof to empirical research in computing research, followed by an overview of common empirical methods (controlled experiments, benchmarking, case studies, systematic reviews, etc.). Students will pose research questions relevant to their theses and assess the appropriateness of various empirical methods for addressing these questions. Much of the course consists of helping students determine which method(s) would be best for them and explore critical success factors for each of these methods. Philosophical implications of method choice will be described. This course is most appropriate for students in areas of CS that emphasize empirical testing, e.g., software engineering, human-computer interaction, CS education.
Schedule
Day 1
9:00 – 10:30 Introductions and topic brainstorming
10:30 – 10:45 Break
10:45 – 12:15 Seminar[1]
12:15 – 13:15 Lunch
13:15 – 14:45 Seminar
14:45 – 15:00 Break
15:00 – 16:30 Seminar
16:30 – 17:30 Exercises
Day 2
9:00 – 10:30 Seminar
10:30 – 10:45 Break
10:45 – 12:15 Seminar
12:15 – 13:15 Lunch
13:15 – 14:45 Seminar
14:45 – 15:00 Break
15:00 – 16:00 Exercises
16:00 – 17:30 Summary and Reflection
Learning Objectives
- Define and describe at least four common computing research methods.
- Differentiate between qualitative and quantitative research methods.
- Analyze the appropriateness of a research method for a research question.
- Select a research method appropriate for a given research question.
- Describe in detail the critical success factors of a research method. relevant to the student’s research question.
- Explain the difference between positivism, interpretivism and realism.
- Select an epistemological position and defend its appropriateness for a given research question.
Topic List[2]
- Introductions
- Student’s areas and research questions
- Fundamentals of empirical computing research
- Epistemology
- Engineering research
- Quantitative approaches (subject to student interests/directions)
- Controlled and Quasi-Experiments
- Longitudinal studies
- Simulations and Benchmarking
- Questionnaire survey
- Systematic Reviews
- Observational and Qualitative Approaches (subject to student interests/directions)
- Case Study
- Grounded Theory
- Ethnography
- Phenomenology
- Qualitative Simulations
- Mixed Methods and Multimethodology
- Closing and reflection
[1] here, seminar means an organic mixture of lecture, q/a, and class discussion
[2] Methods (e.g. experiments, case study) will be selected based on students’ research topics and preferences.