Panos's Home Page  Research  Publications  Teaching  Student Projects

Module DDM140 - ID Masters Programme

Research Methods

Instructor: Panos Markopoulos

Nov 2016-Jan 2017

About the Course

This course aims to equip design students with the knowledge and skills needed to engage in research projects. Students will be exposed to a mixed methods approach, that combines quantitative and qualitative methods. The course will focus primarily on qualitative research methods and students will be exposed to making choices between methods, for collecting and analysing data. Students will be given an awareness of quality criteria for research and will acquire hands on experience in setting up a small research project and writing up its results in a scientific paper. While quantitative analysis of data is not covered in this course, we shall cover elements of experiment design and mapping research questions to testable hypotheses.

·         Quantitative, Qualitative, and Mixed Methods Research

·         Phrasing Research Questions

·         Choosing research design to fit the method

·         Relation of research to design

·         Analysing qualitative data

·         Reporting research.

The students will do individual work based on analysing existing papers, and work in teams of three on a small scale research project. Assessment will be based on presentations and reports delivered at the end of the quartile. 

Recommended reading

Depending on your orientation and interests I recommend three books. However, the course can be followed based on handouts and papers recommended in the course.

·         For those interested to refine their skills in qualitative methods: Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches. Sage.

·         For those interested to apply the methods taught in this course for design projects. Muratovski, G. (2015). Research for Designers: A Guide to Methods and Practice. Sage.

·         For those interested in a step by step guide to setting up a quantitative research project: Kumar, S., & Phrommathed, P. (2005). Research methodology (pp. 43-50). Springer US.

Schedule

Tue

Lecture

Homework

Thu

Presentations

15 Nov

Introduction /assignments / formulating Res.Ques.(L)

Reading assignment (I)

Formulating Res.Question (G)

17 Nov

Qualitative Data Collection Methods

22 Nov

Initial discussion of research paper (I)

Discussion Res. Question (G)

Research Method ‘manual’ (G)

24 Nov

Sampling (L)

29 Nov

Method Presentations (G)

Res. Plan for project (G)

1 Dec

Ethics Introduction (L)

6 Dec

Initial research plan (G)

Data Collection (G)

Ethics document and review(G)

8 Dec

Ethics Review and revision of research plan (G)

13 Dec

Qualitative Data Analysis

Data Collection (G)

Reading on QDA

15 Dec

Data Analysis Practice (G)

20 Dec

Data Analysis homework for project (G)

Critical review of paper (I)

22 Dec

Critical review of selected research paper (I)

10 Jan

Invited Lecture/free

Data Analysis (G)

12 Jan

Mixed Methods Res.

Communicating Res. Results (L)

17 Jan

homework

Writing paper (G)

Triangulation exercise (G)

19 Jan

homework

24 Jan

Final Presentation of Module

Final paper (G) 

Reflection (I)

26 Jan

homework

Individual assignment

Choose a research paper of your choice

·        Identify the research question

·        Review the work in terms of the concepts discussed in the course

·        Research question, sampling, suitability of research strategy, correct use of terminology on methods, suitability of the data collection method, ethics, reliability of the analysis followed

·        Discuss how well the conclusions are supported

·        Discuss the notion of objectivity/subjectivity in the context of this paper

Rubric for assessing  coursework

During the course, you will present your work and obtain formative feedback on the designated slots.

After the course you will deliver a report and all materials. You will be assessed only on the final deliverables (method manual, paper review, research paper)

·        Individual assignment (30%): paper review/deconstruction. Max 1000 words, review your chosen paper based on elements covered in course. (by Jan 26)

·        Group assignment 2 (70%): mini-research study and draft paper. Conduct a project amongst those proposed in the lecture, and report it in a scientific article format – use SIGCHI A4. (by Jan 26). Groups should be up to 4 people. 

 

Learning Objective

Weight

Insufficient

1-5

Sufficient

5,5-6,5

Good

7-8

Excellent

9-10

Awareness of concepts and methods

1

Uses terms erroneously, misrepresents methods

Recognizes different approaches to research, their strengths and weaknesses. Is not yet accurate or precise with terminology but is broadly aware of methods and the nature of methodological choices involved. E.g. confuses data collection methods with the research approach.

Is precise and accurate in terminology and provides sound argumentation for the use of different methods, but is limited in the repertoire of methods applying, unaware of epistemological traditions they relate to

Is familiar with advanced research methods and their link to different epistemological traditions, such as a phenomenology, grounded theory, case studies, etc. Shows ability to extend methodological repertoire with independent reading and combination of methods.

Research Question

1

Is not able to frame a concrete and researchable research question

Is able to to phrase a research question that can be answered by research design, but is not yet fully aware of the implications of phrasing for the type of research that should be pursued.

Shows awareness of how different research questions fit different research approaches. Formulates a question consistently and correctly.

Shows ability to make a sharp research question that is interesting and feasible to research efficiently.

Sampling

1

Is not aware of the importance or implications of sampling decisions. Is not aware of different options for sampling.

Is aware of sampling bias, and makes intuitive choices for sampling that are acceptable though not necessarily optimal.

Can choose an appropriate though not optimal sampling method and name it correctly.

Shows an extensive and refined repertoire of sampling choices and provides sound arguments for sampling approach

Makes efficient and creative sampling choices. Shows interest and awareness to extend discussion on sampling beyond materials covered in the lectures.

Method Congruency

2

Is not able to choose suitable method for the research problem at hand, or chooses methods unwittingly/unknowingly.

 

Is able to choose suitable methods for answering a research question  but, reasons incorrectly about choices.  

Demonstrates ability to choose methods consistent with the research question and context. Can make reasonable choices but is not flexible / creative in choices of methods.   

Like good, but shows ability to choose efficient methods, or make innovative combinations and adaptations of methods suitable for the research challenge at hand.

Execution of research

3

Does not apply methods correctly. Cannot relate claims to evidence.

Applies methods adequately but makes mistakes in data collection and analysis that compromise the soundness of the results.

Applies methods correctly and bases conclusions on suitable evidence and reasoning.

Demonstrates methodological rigor and applies method correctly to collect and analyse data. Makes research process explicit and traceable. Or even better

Demonstrates ability to carry out complex and challenging research procedures in collecting and analyzing data.

Framing of contribution

1

Is unable to place research in broader context and understand its potential contribution.

Makes imprecise or slightly erroneous claims about contribution of research. Makes only partial or unconvincing arguments to relate research to the broader field.

Make a coherent and sound argument as to the potential contribution to research.

Is able to identify the innovation and value in a research study and argument convincingly with respect to state of the art. Is able to think broader than the context of the study.

Ethics

1

Is not able to reason about ethics or is unaware of ethical issues in a design. Shows poor attitude in reporting with honesty or working according to ethics codes.

Is aware of ethical issues and procedures but makes oversights in ethics considerations. Is not able to assess fully ethical implications of research though attempts to apply ethics codes.

Applies correctly ethics codes and procedures. Treats research of self and others with openness, honesty and a constructive attitude.

Demonstrates awareness of different perspectives in ethical conduct and broader implications for society of ethical choices related to the specific research.

Communicating research and results

1

Incomplete presentation, with missing information, or unclear / unsubstantiated claims. Style is not suitable for audience, especially a scientific one, making the presentation / text difficult to understand and not engaging.

Makes a reasonably complete presentation of research, which does not include major flaws in reasoning, or major omissions. It suffers from some limitations in the framing of the research, violates some conventions on presenting research methods and results, and includes perhaps details that are not central to the main argument. Could improve in style, completeness of literature, and clarity.

Clear presentation of process, findings, and contributions. Makes clear the link between claims made and evidence presented. Suitable communication style for the intended audience, especially for design researchers. Results could make a first draft for a scientific publication, but require iteration for improving style, clarity and soundness of reasoning.

Creates a compelling and engaging presentation, that convinces of the value of the research, its soundness, and its importance. The paper could be accepted as is in a reasonable quality research venue where peer review is applied (e.g. with acceptance rate in the order of 30%)

 

Useful video material (from ID students to ID students)

·         Introduction to cultural probes video by Roland Corps and  Dirk van Erve (2011)based on Gaver, Dunne and Pacenti (1999) and related articles. 

·         Technology Probes. Video by Bastiaan Ekeler and Nik Sturkenboom (2011), based on paper by Hutchinson et al (2005).

·         Diary Studies, by Marco van Beers and Kim van Iersel (2011).

·         Repertory Grids, by Jelle Dekker, Michael Geertshuis, Robbert van Vliet (2011), based on the paper by Hassenzahl and Rainer Wessler (2000).

Papers for Individual Assignment

·         Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2: 77–101. http://doi.org/10.1191/1478088706qp063oa

·         O'Brien, J., Rodden, T., Rouncefield, M., & Hughes, J. (2000). At Home with Technology: An Ethnographic Study of a Set-Top Box Trial. ACM To-CHI, 6(3), 282-308.

·         Gaver, B., Dunne, T., & , Pacenti. E. (1999). Cultural probes. Interactions, 6(1), 21-29.

·         Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B.B., Druin, A., Plaisant, C., Beaudoin-Lafon, M., Conversy, S., Evans, H., Han-sen, H., Roussel, N., Eiderbäck, B., Lindquist, S., & Sundblad, Y. (2003). Technology Probes: Inspiring Design for and with Families. CHI 2005, (pp. 17-24). ACM Press.

·         Brown, B.A.T., Sellen, A.J. and O’Hara, K.P. (2000). A Diary Study of Information Capture in Working Life. Proceedings of ACM CHI’00 Conference on Human Factors in Computing Systems, 438-445

·         Carter, S., Mankoff, J., When Participants Do the Capturing: The Role of Media in Diary Studies, Proceedings ACM CHI 2005.

·         L. Palen and M. Salzman. Voice-mail diary studies for naturalistic data capture under mobile conditions. In ACM CSCW ’02, pages 87–95, 2002.

·         Kubey, R., Larson, R., & Csikzentmihalyi, M. (1996). Experience sampling method applications to communication research questions. Journal of Communication, 46, 99-118.

·         Voida, A., Mynatt, E. D., Erickson, T., and Kellogg, W. A. 2004. Interviewing over instant messaging. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM, New York, NY, 1344-1347.

·         Thomas J. Reynolds and Jonathan Gutman, "Laddering theory, method, analysis, and interpretation," in Understanding consumer decision making : the means-end approach to marketing and advertising strategy, ed. Thomas J. Reynolds and Jerry C. Olsen (Mahwah N.J. ;;London: Lawrence Erlbaum, 2001), 25-52.

·         Niall Bolger ,Angelina Davis , Eshkol Rafaeli (2002)  Diary Methods: Capturing Life as it is Lived. Annual Review of Psychology Vol. 54: 579-616

·         Turkle, S. (1998). An Ethnologist in Cyberspace. Scientific American, April 1998. [Online] http://www.sciam.com/1998/0498issue/0498profile.html [2000, May 8].

·         Rugg, G., McGeorge, P., The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Systems, 2005.

·         Dittmann-kohli, F., & Westerhof, G. J. (1997). The SELE sentence completion questionnaire : A new instrument for the assessment of personal meaning in research on aging. Anuario de Psicología, (73), 7-18.

 Older versions of this course

DHCM411 Research Methods Spring 2016