Pizza Testing

Sara discovered our secret competitive advantage. Starting a few months ago we’ve been running usability tests to try to smooth out as many rough edges in the game as possible before launch. Even though our testing is extremely low budget, we have been getting fantastic feedback on the game. The usability person we hired came on board after the first round of tests and has introduced a more scientific process to the testing. That has definately increased the value of the tests.
If you don’t have access to an expert of your own, you can still do this kind of testing. The basic approach this:

  1. Invite some local gamers to the office
  2. Order some pizza
  3. Let them loose on the game
  4. Observe

We have an FLSer with a notepad watch what they’re doing, prompt them to describe what they’re doing, and take notes on what they say and do. That’s it. No fancy cameras and no real science, but plenty of gigantic pointers at things to fix in the game. The usability person we hired has been helping out with those test, and we’re bringing her aboard to increase the pace of testing. The lab itself is just a dedicated space to do more or less the same level of testing.

I’m sure that spending tons of money on equipment and a bunch of trained professionals to use it would get us better results. Maybe those results would be twice as good, even. The cost would probably be ten times what we’re paying for our simple tests with a single trained professional. If we had all the money in the world, it would be something to look at, but for now the pizza testing is working just fine.

I wish we had started doing this years ago. There’s nothing like observing a real user to tell you where your UI sucks. :)


6 Responses to “Pizza Testing”

  1. Rich Bryant commented on :

    That is an incredibly good idea. If we borrowed it, woud you want copyright payments? Does it fall under the DMCA?

  2. robusticus replied on :

    Beer too? Or was that implied?

    If so, sign me up. You guys are in Ballard?

  3. Joe commented on :

    @Rich: As long as you spell pizza with a double s and an extra ‘u’ stuck in there somewhere, I’m sure you’ll be safe. :)

    @Robusticus: Sorry, no beer. We wouldn’t want to color the results, would we? Or do most people drink and play?

    We’re up on Queen Anne, actually. Near as I can tell we’re keeping the entire Queen Anne lunch-time restaurant economy afloat with our 60ish people.

  4. justinian said on :

    I think she’d disagree with the “no real science” part.

  5. Joe replied on :

    Hey, stop reading my blog and get back to work!

    Yeah, I worded that poorly. The simple approach above is what we did for that first test before we brought in the expert. There’s definately more science to it now, and the testing has become more valuable as a result.

    Not having an expert on hand shouldn’t stop you from doing this kind of testing, however. Even with no science at all, the testing is still valuable. I edited the post to state that a little more clearly.

  6. Zubon replied on :

    That is how I have seen various places do web site testing. You have an observer or two and bring people in who talk as they try to do various things. It is a sort of guided interview. With pizza, in this case.