Doctoral course - Overview of Empirical Methods for Computing Research

This course begins with a brief analysis of the shift from mathematical proof to empirical research in computing research, followed by an overview of common empirical methods. Much of the course consists of helping students determine which method(s) would be best for them and explore critical success factors for each method.

Event information

Time

-

Location

Linnanmaa

Add event to calendar

Lecturer

Paul Ralph, Professor (Dalhousie University, Halifax, Canada)

Content

  • 3 hours preparation
  • 12 hours lectures
  • 2 hours exercises

Assessment

Attendance and exercises (Pass/Fail)

Preparation

To prepare for this course, students should:

  1. Choose one or more empirical research questions relevant to their dissertations on which to focus during the course.
  2. Read Stol, K. J., & Fitzgerald, B. (2018). The ABC of software engineering research. ACM Transactions on Software Engineering and Methodology (TOSEM), 27(3), 1–51.

About the Instructor

Paul Ralph, PhD (British Columbia), is an award-winning scientist, author, consultant, and Professor of Software Engineering at Dalhousie University. Dr. Ralph’s has published more than 90 peer-reviewed articles on software engineering, sustainable development, human-computer interaction, and project management in premier venues including IEEE Transactions on Software Engineering and the ACM/IEEE International Conference on Software Engineering. Dr. Ralph is editor-in-chief of the SIGSOFT Empirical Standards for Software Engineering Research.

Course Abstract

This course begins with a brief analysis of the shift from mathematical proof to empirical research in computing research, followed by an overview of common empirical methods (controlled experiments, benchmarking, case studies, systematic reviews, etc.). Students will pose research questions relevant to their theses and assess the appropriateness of various empirical methods for addressing these questions. Much of the course consists of helping students determine which method(s) would be best for them and explore critical success factors for each of these methods. Philosophical implications of method choice will be described. This course is most appropriate for students in areas of CS that emphasize empirical testing, e.g., software engineering, human-computer interaction, CS education.

Schedule

Day 1

9:00 – 10:30 Introductions and topic brainstorming

10:30 – 10:45 Break

10:45 – 12:15 Seminar[1]

12:15 – 13:15 Lunch

13:15 – 14:45 Seminar

14:45 – 15:00 Break

15:00 – 16:30 Seminar

16:30 – 17:30 Exercises

Day 2

9:00 – 10:30 Seminar

10:30 – 10:45 Break

10:45 – 12:15 Seminar

12:15 – 13:15 Lunch

13:15 – 14:45 Seminar

14:45 – 15:00 Break

15:00 – 16:00 Exercises

16:00 – 17:30 Summary and Reflection

Learning Objectives

  1. Define and describe at least four common computing research methods.
  2. Differentiate between qualitative and quantitative research methods.
  3. Analyze the appropriateness of a research method for a research question.
  4. Select a research method appropriate for a given research question.
  5. Describe in detail the critical success factors of a research method. relevant to the student’s research question.
  6. Explain the difference between positivism, interpretivism and realism.
  7. Select an epistemological position and defend its appropriateness for a given research question.

Topic List[2]

  • Introductions
    • Student’s areas and research questions
  • Fundamentals of empirical computing research
    • Epistemology
    • Engineering research
  • Quantitative approaches (subject to student interests/directions)
    • Controlled and Quasi-Experiments
    • Longitudinal studies
    • Simulations and Benchmarking
    • Questionnaire survey
    • Systematic Reviews
  • Observational and Qualitative Approaches (subject to student interests/directions)
    • Case Study
    • Grounded Theory
    • Ethnography
    • Phenomenology
    • Qualitative Simulations
  • Mixed Methods and Multimethodology
  • Closing and reflection

[1] here, seminar means an organic mixture of lecture, q/a, and class discussion

[2] Methods (e.g. experiments, case study) will be selected based on students’ research topics and preferences.

Last updated: 17.1.2024