This example is similar to your assignment and will help you become familiar with affective evaluation, the intrinsic motivation inventory (IMI) questionnaire, and the cued debrief recall technique.
First, find a simple online game that you can evaluate. It should be easy to startup, learn, and get into real play. Just do a web search for free online games if you aren’t sure of what to play.
Study
One person from the team will play the game and the other team members will conduct the study as investigators.
1. The player will first familiarize him/herself with the game by playing for 5 minutes.
2. The player plays the game for one level (e.g., 5-10 minutes) and is video recorded from a first person view (behind them facing the screen). You can use a cell phone or webcam from a laptop to record the video.
3. The investigators conduct the cued recall debrief technique by showing the participant the video of them playing and asking about their play at various points in time. They should video record the participant in a third person view while they watch their own video and talk about it.
4. The player fills out the IMI questionnaire.
Analysis
Now perform the analysis. You must use the answered IMI questionnaire along with the video of the person talking about their play when they did the cued recall debrief.
1. Analyze the cued recall debrief video and fill out the coding sheet, one sheet per investigator.
2. Compare the coding sheets of the two investigators by computing the inter-rater reliability. Are they consistent? Is there a correlation? You should have a R value >= 0.90. If you don’t, discuss the codes that each evaluator chose and understand why their were discrepancies.
3. Score the IMI questionnaire using the scoring sheet and scoring instructions.
Use the results to assess the overall affective enjoyment of the game play, problem areas with the design, and areas of success.