You will work with an industry or community partner to conduct a real world evaluation of one of their products. This will involve you meeting with the partner and exploring their needs for design evaluation. You will think carefully about the design challenge posed to you by the company or organization and decide on the best method for evaluating the design. The evaluations that you can consider are:
- heuristic evaluation
- usability study
- controlled study
- affective evaluation
- field deployment
You will be graded based on the quality of your work as well as the complexity of the design evaluation you conduct. For example, conducting a fairly straightforward heuristic evaluation of a single user application will likely not generate a strong grade. On the other hand, a field deployment of a new user application that requires multiple site visits and interviews will likely generate a stronger grade given the complex nature of the study.
You will work in a group of four people.
Steps
The following steps are presented in the course calendar based on when they should occur:
1. Industry or community partner: You will first need to find an industry or community partner that you can work with. You will have to meet with them to explore their needs and what design you can evaluate.
2. Study plan: You will draft a study plan that document how you could conduct a study. This should include all components, including the questionnaires, study tasks, interview questions (if needed), etc. You should also write one page that explains why you would not choose other methods to conduct the study.
2. Perform the Study: Conduct the study with an agreed upon number of participants, as detailed in class.
3. Analysis: Analyze your findings using known methods that are taught in class.
4. Report: Create a report that details your findings.
Deliverables
You will provide a detailed report that is 4 pages, single spaced (not including appendices and figures). It should include the following sections:
1. Introduction: describe the situation you are studying and why
2. Study Methods:
- participants: describe your participants’ demographics briefly
- method: explain the method you used and exactly what you did
- data collection: explain how you collected your data
- validity: describe any potential concerns with validity; talk about how you ensured validity
- questionnaires: briefly describe them; place your questionnaires in an appendix
- data analysis: describe how you analyzed your data
4. Discussion and Conclusions: Describe the implications from your results. What should be done moving forward based on the results of your study? How should the user interface be redesigned as a result of your study? Show mockups of potential redesign ideas.
5. Appendix 1: include all raw data from your study.
6. Appendix 2: include any pre and post-test questionnaires, etc.
Bring a printed copy of your report to class on the due date in the course calendar and hand it in at the beginning of class. Failure to hand it in at the start of class means it will be considered late.