Stage Gates are Wrong for Games

The Playdough report from Project Horseshoe suggested that using the Stage-Gate Model would allow the game industry to improve its hit rate. According to the report, 20% of the games released actually make a profit. This rate is much higher than the 5% achieved by the cheap-to-produce product out of the music and book publishing industries. This rate is only a quarter of the 80% profitability rate that the movie industry manages.

The stage gate system was developed in the 1980s by Dr. Robert G. Cooper, and is detailed in his book on new product development. It is used in a diverse set of industries by “almost 70% of leading U.S. product developers”, according to the Product Development Institute website. It involves taking a new product idea through a well defined set of stages, each of which is followed by a gate that kills off the weak products. The idea is that if you can kill off an unprofitable product earlier in the development process, you will save your company money.

The stages used in the stage gate model are as follows:

  • Scoping — The rough idea is assessed for it’s technical merits and market prospects. This is an inexpensive check that can be performed on a given idea quickly.
  • Build Business Case — This is where all the planning for the product happens. The product is defined and scheduled. An estimate of return on investment is made based on estimated revues and the product plan.
  • Development — The product is developed, operations plans are written, and the staff required for launch are hired and trained. Everything required to launch the product is developed in this stage.
  • Testing and Validation – All of the products of development are evaluated. The development process is evaluated against the plans. The economic fitness of the product is evaluated.
  • Launch — Full commercial launch of the product.

So the question is, how can this model be applied to game development? Here is a pass at how that might work:

  • Scoping — The game idea is evaluated for feasibility. The basics of setting and gameplay are described.
  • Build Business Case — A budget and schedule for the development of the game are developed. The entire game is designed, top to bottom. Concept art is developed for the assets that the game will require. A technical plan for the game is assembled, including which middle-ware to use, what platforms to launch on, etc.
  • Development — The programmers, artists, and level designers go to work building the game. They build the entire game top to bottom. The game is 100% playable by the end of this stage. The game box is designed, the marketing campaign is planned, and the customer support people are trained. If appropriate operations centers are staffed and trained. The server hardware required for testing is installed and configured.
  • Testing and Validation – The game is put through its paces by the QA department and focus tested like crazy by the marketing department. After making whatever minor adjustments are required as an outcome of the testing, the game moves to the next stage.
  • Launch — The game launches.

Here’s the problem: This won’t work. The Stage-Gate process is the classic waterfall model zoomed out to look at product development from the business perspective. The waterfall model looks something like this:

Waterfall Model

This model works really well for a defined process, which is any process that you can define well enough to make it repeatable. This is highly desirable for manufacturing processes, and is probably applicable to almost all of the Fortune 500 companies using the Stage-Gate model. Unfortunately this is not the situation with software development in general, and may be even less true of games.

For game development to be a defined process, we would need to have the same goals for a game that we had for the game immediately before it. We would have to staff the teams exactly the same way, with at least the same levels of experience and training if not exactly the same people. We would have to want the same game to come out the other end of the process.

That’s not what we want out of game development. What we want are fresh games, and fresh ideas. We almost never start a game with an equivalent staff to the game we developed before, and given how hard it is to find good people, that’s not realistic anyway. What we need is an empirical process; one that lets us take measurements along the way and apply them to our ongoing development effort. An example of an empirical process is the Iterative or Evolutionary model:
Iterative model

This is exactly the model being pushed by the big-A Agile community. They go to extreme measures to drive the cost of this iterative cycle as low as possible. Some of them even go so far as to flip the whole model and perform the tests before they actually do the development.

While I’m not really an adherent of the Agile school of thought it does seem pretty clear to me that the Waterfall method is broken for our industry. How many times have you seen a feature take many times its original estimate because once the work was underway it was clear that it was actually more difficult that it appeared from the outside? How many times have you seen part of a design be thrown out of changed substantially based on how it actually played? These happen all the time and aren’t the fault of the programmer who made the estimate or the designer whose work is being revamped. Each game explores new territory using the experience of its developers as its foundation.

So if Stage-Gate isn’t going to help us, what can? Well that’s a big topic that belongs in a post all its own. What do you think? Are there any business-level models from other industries that actually can help ours? How can we develop through hundreds or thousands of iterative loops and still be able to budget and schedule in any kind of reasonable way? I’m very interested to hear your thoughts on the subject.

~Joe


7 Responses to “Stage Gates are Wrong for Games”

  1. Danc wrote on :

    You’ve made a leap here and fundamentally mischaracterized a stage gate process as just another name for waterfall development methodologies. One has absolutely nothing to do with the other. You can use stage gate with waterfall. You can use it with agile. I personally feel it fits agile development much better and you’ll see that more evolved stage gate processes have a striking similarity to many agile techniques even though you find them in non-software industries.

    As you may know from my other essays, I’m a huge fan of various agile process and put a big emphasis on both prototyping and rapid iteration. The stage gate process maps quite nicely onto practices like Scrum or XP and brings a lot of common lessons to the table

    Start small: Do the simplest thing possible. You may learn something.

    Don’t put all your eggs in one basket: Try out multiple ideas early on. Since you can’t know which ideas will pan out, it makes a lot of sense to try several and then invest further in the promising ones.

    Test regularly: The whole point of the stage gate process is to have formal testing *throughout* the development process, not at the end.

    Understand what success looks like: Even the most rapid iterative process is useless if you are making random changes that do no improve the product. By understanding what success looks like, you can iteratively guide your products towards that that goal.

    Invest late, not early: Product development is a learning process. You might as well invest money as you discover success mechanics, settings, etc instead of dumping a wad on hand waving specs early on.

    It is also worth noting that just because there are big formal stages that are used for large scale portfolio management, there is nothing in the process that rejects the hundreds or thousands of iterative loops within each development stage. In fact, I’d recommend it since it helps teams meet the gate criteria more successfully.

    Stage gate has some serious potential for helping us create innovative product. It brings portfolio management out of the darkness of Skull&Bones green lighting committees and encourages that smart companies set aside money to build new-to-the world, high payoff products.

    It is worth taking another look at the concept and imagining how it might fit into the iterative work flow you so rightly target as critical to success in our industry. I’m guessing you’ll be pleasantly surprised at how the two compliment each other.

    Appreciate the article and the opportunity to start a conversation. :-)

    take care
    Danc.

  2. Joe said on :

    The thing is, I agree with everything you’re saying except “stage gate”. I’m just not finding any reference to anyone using the Stage-Gate Process iteratively. I found this case study of a couple of Agile teams in a company that used stage gates. They paid lip service to the gate criteria, but in the end, they didn’t deliver everything and were still allowed to advance.

    This article on the Agile Journal refers offhand to both Stage-Gate and waterfall as though they were synonyms. In fact, everything (online) I can find to read on Stage-Gate shows it as being somewhere between exactly the same and almost the same as waterfall.

    What are you reading that points out Stage-Gate and agile being used together, or stages being used iteratively? I’m just not finding it.

  3. Danc wrote on :

    Try picking up one of Cooper’s books on the topic. Stage gate is more of a framing portfolio management technique than a development process in the way you are describing.

    Winning at New Products: http://www.amazon.com/Winning-New-Products-Accelerating-Process/dp/0738204633

    I can see where stage gate ideas can be easily mixed up with waterfall ideas. The original concept came about during the 80′s before even Lean Manufacturing had really matured in American companies. So you’ll find some of the typical stages mentioned in the literature matching onto waterfall names. However, the important idea is that there are stages and gates, not that you follow the exact model used 20 years ago by Proctor and Gamble for developing new types of cookies. In later writings, the 3rd generation processes emphasize flexibility as a key attribute.

    Because stage gate is a framework that works across multiple industries, there are lots of possible implementations. I’ll be the first to admit that there are bad business decisions made in the name of the stage gate process, just as is the case when any good idea is adopted by large numbers of people. I mentioned a few common mistakes in the paper. The trick is to take the potent ideas like options-based portfolio management or success metrics and adapt them to your business.

    I like the paper you mentioned. It is good to see this researching happening.

    take care
    Danc.

  4. Joe commented on :

    Just ordered his book. I’ll get back to you on the term “Stage-Gate” in a couple weeks. :)

    In anycase, I think we could stand to be a lot more brutal about killing our own games. Much better to kill a game at the prototype stage on your own terms than wait for the publisher to kill it late in development and let it take the whole studio with it.

  5. Danc replied on :

    Aye, I think some of the practical lessons of the Stage Gate process are quite useful even if you strip the name off. Completely agree on the need to kill projects.
    - Prototype lots of little projects/features
    - Test them
    - Quickly kill the ones that don’t work

    Looking forward to hearing your thoughts about that particular book. It has some great information in it despite the sometimes repetative writing style. :-)

  6. Jeff Freeman said on :

    What are movies doing? That 80% looks pretty good.

1 Pingback

  1. Pingback from Audio Article #117(A+B): Improving Dev. Efficiency – Why Stages gates are Wrong | IndustryBroadcast on :

    [...] B: “Why Stage-Gates are Wrong for Games“ – Original [...]

Leave a Reply