Calendar

Week	Topics	Activities	Deadlines
Sept 15	Introduction Introduction of the course, assignments, and textbooks Explanations of why we evaluate and the goal of evaluation frameworks Analytic vs. participative evaluation methods Objective vs. subjective evaluation methods Formative vs. summative methods Quantitative vs. qualitative methods Research/study questions Material: Slides	Studio Lab Activities Review heuristic evaluation Review usability studies Introduction to Assignment 1: Ethics (5%) Review Materials Heuristic Evaluation (Audio, Slides) Usability Testing Plans and Methods (Audio, Slides) Interviews, Surveys, Questionnaires (Audio, Slides) Qualitative Data Analysis (Audio, Slides)
Sept 22	Controlled Experiments Scientific Method Hypothesis testing Independent & dependent variables Ethics of controlled experiments Materials: Audio (18 minutes) Slides Zoom Recording Readings: Doing Psychology Experiments, Chapter 2: How to Do Experiments Doing Psychology Experiments, Chapter 7: How to Decide Which Variables to Manipulate and Measure	Studio Lab Activities Introduction to Assignment 2: Controlled Study (15%) Exercise: controlled study design Exercise Sheets Interface Set 1 Mac Tray Interface Set 2 - Track pad vs. mouse Interface Set 3 - Ribbon menu vs. traditional menu Interface Set 4 - Wii remotes	Assignment 1: Ethics (5%) Due at the start of class.
Sept 29	Reliability and Validity Reliability, consistency, repeatability Confounding variables Face validity, construct validity, internal validity, external validity Materials: Audio (33 minutes) Slides Zoom Recording Readings: Doing Psychology Experiments, Chapter 8: How to Decide on a Between-Subjects versus Within-Subject Design	Studio Lab Activities Design an experiment for AimBooster
Oct 6	Statistics for Controlled Experiments Descriptive statistics: mean, standard deviation, median, range Basic Inferential tests: t-test Graphing basics Materials: Audio (33 minutes) Slides Zoom Recording Readings: Doing Psychology Experiments, Chapter 12: How to Interpret Experimental Results Doing Psychology Experiments, Chapter 13: How to Report Experimental Results	Studio Lab Activities Exercise on statistics Answers for the lab exercise on statistics
Oct 13	Evaluating Emotion and Affect Measuring affect and emotion Theories of emotion Quantifying qualitative codes Cued Debrief Recall Materials: Audio (30 minutes) Slides Pac Man game as an example cued debrief recall Inter-rater reliability example Zoom Recording Readings: Bentley, T., Johnston, L., and Baggo, K. Evaluation using cued-recall debrief to elicit information about a user's affective experiences, Proc. OzCHI, ACM Press (2005). (PDF)	Studio Lab Activities Introduction to Assignment 3: Affective Evaluation (15%) Conduct an example study of gameplay and collect IMI questionnaire data and cued debrief recall data	Assignment 2: Controlled Study (15%) Due at the start of class.
Oct 20	Usability Evaluations of Collaborative Systems Collocated collaboration systems Distributed collaboration systems Selecting participants as pairs or small groups Planning tasks for collocated and distributed collaborative systems Analyzing group data Materials: Audio (33 minutes) Slides Readings: None	Studio Lab Activities Plan a usability evaulation of a collaborative system
Oct 27	Field Deployments Planning and timeframes’ Pilot testing Managing home visits Minimum viable prototypes Group vs. individual interviews Data validity Ethics for field evaluations Materials: Audio (30 minutes) Slides Readings: Judge and Neustaedter, Studying and Designing Technology for Domestic Life, Morgan Kaufmann (2015). Chapters 9, 10	Studio Lab Activities Practice for exam	Assignment 3: Affective Evaluation (15%) Due at the start of class.
Nov 3	Midterm Exam Choosing Evaluation Strategies Usability evaluation considered harmful (some of the time) Applicability of usability evaluation for various scenarios Readings: Greenberg S., and Buxton, B. Usability Evaluation Considered Harmful (Some of the time), Proceedings of the ACM Conference on Computer Human Interaction, ACM Press (2008) (PDF) Watch this video of the paper's presentation: the video
Nov 10	Design Evaluation Presentations Come to class with an updated plan for the study you want to run		Come to class with a company and product you want to evaluate. Bring a document of up to one page that lists: Your team members and email addresses The name of the organization that you plan to work with The name and email address of a contact person at the organization Details of how you have talked with them An initial idea of the type of evaluation you could conduct and on what
Nov 17	Design Evaluation Presentations Come to class with an updated plan for the study you want to run		In this class we meet with every group and work on the study plan together
Nov 24	Initial Report of Results Come to class and talk about the initial results you are getting from participants		You should send us a complete set of draft study materials in a single PDF. This means that you should create an initial plan that includes details of: Participants Methods to be used, including all questionnaires Data collection: what you will collect and why Data analysis: how you will analyze your data
Dec 1	Feedback on the project Come to class and talk about the final results
Dec 8	Final Report Presentation Final presentation of your project		Assignment 4: Project (40%) Due by 11:59pm, Dec 9. Share a single PDF containing all materials into your group's Slack channel.