M3: Usability Testing

Co-ordinator:

Dr. ir. M.M. Bekker, Assistant Professor User Centred Design, IPO, Eindhoven University of Technology

Instructors:

Dr. ir. M.M. Bekker, IPO, Eindhoven University of Technology

Dr. H.P. de Greef, IPO, Eindhoven University of Technology

Ir. W.P. Brinkman, IPO, Eindhoven University of Technology

Guest Lecturers:

  • two from industry
  • two from acedemia

Benefits

After following this module students should be able to:

  • select the appropriate approach for usability testing in a specific setting
  • set up a usability test for a specific purpose, taking into account constraints of the design project they are involved in (e.g. time constraints and limited access to representative users)
  • conduct a usability test and translate the results to input that is appropriate for the design project

Features

  • discussion of appropriateness of testing approaches for specific design contexts
  • planning a usability test
  • conducting a usability test
  • analysing the results and translating them to input for the design project

Abstract

The module will start with an introduction to conducting usability tests in practice, discussing issues as selecting complementary methods and tailoring them to the constraints of design practice. Then, an overview will be given of the process of planning a usability test. This will cover topics such as the aims of usability, phrasing research questions, selecting subjects and tasks and running the actual experiment. The students will apply this knowledge on cases of design projects for which usability tests have to planned.

The main part of the module will consist of planning and conducting a usability test for a given system or product. Students have to hand in assignments and give presentations during this process to keep track of progress and the decisions that have been made. The final presentation will be given about the findings of the usability test and how these have been translated to ideas for improving the design.

Students will learn to select methods to fulfill the aims of the usability test. They will gain hands-on experience in planning and conducting a usability test. They will experience how to optimise the trade-offs inherent in conducting usability tests under real-life constraints. Dumas, J. and Redish, J. (1993) will be used to provide an overview of the issues to be considered when planning a usability test. Literature will be handed out during the module.

Presentation

Lectures, illustrated by video materials will alternate with working on an assignment. The emphasis of the module will be on planning and running a usability test. During the module, groups of students will present their work to the rest of the students. Design practitioners will be invited to present experiences with conducting usability tests in industry.

Instructor(s) background

Mathilde Bekker studied Industrial Design Engineering at Delft University of Technology. She obtained a PhD at the Delft University of Technology on the subject of support for communication within design teams. She has worked at the HCI group of the Computer Science Department of the Queen Mary and Westfield College in London from 1995-1997, where she did research on how to involve users in the design of interactive systems. In 1997, she became a member of the User Centred Design group at IPO, where she works on design techniques for user involvement in design.

Willem-Paul Brinkman studied Technology and Society at Eindhoven University of Technology. His graduation project was on the subject ‘combining manual and machine control into one user interface’. For a year and a half he also worked part time at the Rabobank usability laboratory. At the moment he is a Ph.D. student at IPO. His research is on the usability of user interface components, and their contribution to the overall usability of the total user interface.

Paul de Greef studied psychology at the University of Amsterdam. After graduation in 1984 he stayed in a department concerned with the intersection of behavioural, social and computer science. There he worked as a researcher in European projects in the information processing and knowledge acquisition area. In 1989 he was appointed assistant professor in human-computer interaction at Leyden University. In 1999 he obtained a PhD at the University of Amsterdam ("Computer support by knowledge enhancement: constraints and methodology"). In 1999 he joined the user centred design group at the IPO in Eindhoven.

Requirement list for the coordinator of facilities

  • The instructors need overhead and slide projector facilities, sometimes also a VCR
  • The students need to be able to present their work with either overhead sheets or PowerPointProjection facilities are needed for these presentations
  • Students need access to rooms where they can conduct the usability tests. These rooms should be equipped with a video camera’s each. They also need to have access to a VCR and TV set, for data analysis purposes. Since the time available for conducting these analysis is limited several of such workplaces need to be available. A supply of video tapes is needed for recording the video data.
  • Products to be tested have to be selected and obtained before the module starts.

Course materials for students and instructors

The course material will consist of copies of relevant articles and copies of the presentations given by the (guest) lecturers.

Relation to other modules

M3 will build on the information presented in M2. In M3 the students will learn to how to select and apply such methods with real-life constraints in mind.

Scenario for teaching the module

  • lectures and group assignment
  • writing short report on assignment
  • presentations of assignment
  • group discussion of literature and assignments

Monday

Introduction lecture

Group assignment handed out

Tuesday

Guest lecture

Prepare usability plan

Wednesday

Guest lectures

Presentations usability plan in the afternoon

Thursday

Guest lecture

Adapt plan and prepare studies

Friday

Pilot usability test

Make changes to set-up if necessary.

Monday

Run subjects

Tuesday

Data analysis

Wednesday

Data analysis, work on results section and presentation

Thursday

Work on results section and presentation

Hand in report

Friday

Presentation final results

Scenario for fall-back

Other M-module coordinators can act as replacements.

Student evaluation

Students will be working in groups and will be graded based on the report written and presentations given throughout the module using a 1-10 point grading scale.