This assignment will give you first hand experience at understanding how to evaluate an existing user interface. You will learn how to interact with study participants, develop usability study tasks, understand what problems users face with the interface, and understand what successes they have. You will also learn how to document your findings.
You have been asked by a company to evaluate the user interface for the Metro Vancouver transit application. You will perform a usability study with four participants and utilize the think-aloud and constructive interaction techniques. You will create a usability report detailing the findings of your study and recommendations for system redesign.
Focus your usability study on the main features of the site. Include the frequent tasks people need to do plus the infrequent yet important tasks. You should avoid going beyond this; do not test the entire user interface as this can easily make the assignment too broad in scope.
You will be evaluating the trip planning tools offered by the BC Transit Authority for this assignment, Translink website. The assignment must be done in pairs. You should study both the web page experience as well as the mobile experience. You should use the interface and become familiar with it. You are encouraged to consider scenarios that differ with the context of use.
1. Plan the Study: Plan out the usability study by creating:
2. Pilot Study: Try out your usability study with someone before you actually do it with real participants. This is important so you are sure your tasks and questionnaires make sense, you can run the study in less than one hour, and that users won't become extremely frustrated.
3. Find Participants: Find family members or friends who are willing to be participants in your usability study. Try to choose a range of users (e.g., novices, experts, young, old). Your study participants cannot be students at SFU. This is required because I really want you to explore the system from the perspective of someone who is not overly familiar with computers. Consider this as the Mom study where you try and see how somebody like your mom or the equivalent would use it. This should generate a LOT of usability issues.
4. Perform the Study: You need to run the study with 4 PEOPLE IN TOTAL (2 Think-aloud separately, 2 Constructive Interaction together).
Throughout the study, try to get an understanding of what the user's conceptual model is of the system. It is a good idea to ask users about their conceptual model when they first see the interface.
While the experimenter should not help the subject with the task, there are a few exceptions to this rule. If a subject has problems getting started, record the problems and give them a hint to get going. This is OK, because if they can't get started, they will not be able to do the tasks! If a subject cannot complete a particular task after a reasonable amount of time, tell them to stop and start them on the next task. Or, give them a hint if they cannot overcome some conceptual problem necessary to trying out other parts of the system. Again, record all problems.
Collect data using one or more of the observational methods discussed in class (e.g., paper notes, audio, video).
After completing the tasks, have the participant complete the post-test questionnaire. Conduct a short retrospective interview to understand where the participant feels they had the most problems with the interface.
5. Analysis: Analyze your findings to see what usability problems are common amongst users. You may want to use affinity diagramming.
6. Report: Create a report that details your findings. This should include detailing the core problems people had with the interface and suggestions for redesign (see below for more detail).
You will provide a detailed report (maximum 5 pages in length, single spaced). Appendices is not included in the 5 page-limit. It should include the following sections: