Archive for March, 2007

The Abstract Factory pattern in C++

I’m a big fan of data driven design (aka meta-programming.) It is the way to give artists and designers the maximum amount of power short of turning them into programmers (by giving them a scripting language.) One of the biggest obstacles to a fully data driven design in C++ is the complete lack of reflection features in the language. The most important of these is the lack of an ability to instantiate an object at runtime given the name of a class.

The way to get around this is to use the Abstract Factory pattern, but without reflection, there’s no direct way for your abstract factory to create the objects either. You could solve this with a code generator. We are using code generators extensively on Pirates for just this purpose. Unfortunately the biggest of those is part of a proprietary piece of middleware, and comes with tons of extra baggage. The other approach is to use templates and macros to make it easy enough to populate the abstract factory entirely inside the code. This is the approach I’m taking here.

Step by Step

There are three steps involved in getting an abstract factory up and running:

  1. The first step is registering classes with the abstract factory. This is relatively straightforward to do with static data objects and a singleton.
  2. The second step is figuring out how to create the objects. If you are willing to ignore type safety, this is pretty easy. If you are willing to use a different abstract factory for each base class, type safety isn’t so difficult to add.
  3. The third, and most annoying, step is making sure the optimizer doesn’t throw away all your static registration objects, and the classes that go with them. In most cases you won’t be referencing these classes and objects directly, so you have to force the linker to include them by adding references.

Registering classes with the factory

The first part of registering a class with the factory is to build a function that instantiates the class. An example of such a function would look like this:

void *SomeClassFactory()
{
return new SomeClass;
}

The return type is void here, because the abstract factory I’ve built uses a single factory signature. It’s not type safe, but as long as your registrations are all in order it works just fine.
Writing a function like this for every class you want to instantiate by name would be a real drag. Fortunate, templates can help:

template <typename ClassName> void * FactoryFunction()
{
return new ClassName;
}

Now we just need to get a bunch of these factory functions registered by name in the abstract factory. The simplest way to do this is to have a small object that registers itself in its constructor, but allocating memory on the heap at static initialization time is probably not a good idea, so we’ll tweak that approach slightly. Instead of registering itself with the factory directly, the registration objects will build a linked list of themselves, and then the factory will walk that list and perform the registration when it is initialized.

class utFactoryReg {
const char *_name;
utFactoryFn *_function;
utFactoryReg *_next;
static utFactoryReg * s_head;
utFactoryReg(const char *sName,
utFactoryFn *pFactoryFn)
: _name(sName), _function(pFactoryFn)
{
_next = s_head;
s_head = this;
}
};

Now we just need to create one of these objects for each of the classes we want to instantiate by name from the abstract factory. This is as simple as creating a global for the class:

utFactoryReg g_RegisterSomeClass("SomeClass", FactoryFunction);

When the factory itself is initialized it has only to walk the linked list of utFactoryReg objects, and organize them however it needs to for quick lookup.

Creating the objects

Once the factory knows about the classes, instantiating them is relatively straightforward. Just look up the utFactoryReg object and call its factory function:

void *callFactory(const char *sClassName)
{
utFactoryReg *pReg = findRegistration(sClassName);
return pReg;
}

Returning a void * to your callers is a little annoying. Even without type safety we can do better with templates:

template
ClassName create(const char *sClassName)
{
return callFactory(sClassName);
}

If you’re concerned about the type safety of all these casts through void*, you’re right to be. If a caller of create() uses the wrong type the static cast will not catch it and things will go badly. You could mitigate this risk and add more type safety to the factory by making utFactoryReg and FactoryFunction templates. If they return an accurate type you are going to need to templatize your abstract factory class too (whatever callFactory() and findRegistration() live on) and have an instance of that class per base class.

Forcing references

Most of the time it’s a good thing that the linker drops out unreferenced symbols. When you’re trying to initialize a system with static data, though, it just gets in the way. If any of the classes you want to instantiate from the factory are in static libs, you are going to need to jump through some hoops to force them to be referenced. If you don’t want your higher level code to include every possible class, one way to do this is to define the utFactoryReg objects in an array and force a reference to that array. That looks something like this:

// somewhere in your static lib
utFactoryReg g_rGlobalReference [] = {
utFactoryReg("Class1", FactoryFunction),
utFactoryReg("Class2", FactoryFunction),
utFactoryReg("Class3", FactoryFunction)};
int GlobalClassCount() {
return sizeof(g_rGlobalReference)/sizeof(utFactoryReg); }

// somewhere in the code that links to the static lib
GlobalClassCount();

You probably don’t actually care about the class count, so there’s no real reason to catch the result. Just call this at startup sometime before the factory is initialized and it will force the needed reference.

Putting it all together

You can find source for the utFactory and utFactoryReg classes here. The code should be portable, but I’ve only tried it with the included VS2005 projects. This implementation differs slightly from what is described above in that it collects the classes into categories. I’m using the categories as base classes, so I don’t have to worry about accidentally casting an object to an invalid base class. I’ve also added a number of macros to make declaring registrations and reference forcing functions more easily. Use it for whatever you like… this is part of a larger project that I intend to release under an open source license.

If you end up using the code for anything, I would love to hear what you think. I’m also interested to hear other solutions to the problem of instantiation by name in C++, or how the abstract factory approach could be less of a hack. Just leave a comment below or shoot me an email at joe <at-sign> programmerjoe.com.

Hacking the Conference

Psychochild asks:

What goes into a good conference, and what would make a conference useful for game developers (particularly online developers)?

I answer:

  1. Small size — This is the biggest thing that DICE and AGC have going for them over GDC. You can actually run into people you want to talk to.
  2. No students allowed — It’s rude to say this, I’m sure, but I don’t give a rat’s ass what some 20 year old going to Digipen or Full Sail has to say. If I’m going to sit down with a bunch of random people at lunch I want most of them to have shipped a game.
  3. Constrained subject matter — “Games” is too big for one conference as GDC shows us over and over. I don’t care to dig through piles of talks on BREW or the cell processor (or even graphics in general) to get to the talk on object databases or asset streaming over slow networks.
  4. Shorter is ok — AGC is up to three days now, which is fine… I don’t mind a three day conference. A one day conference with a laser focus on talks I really care about would be better, however.
  5. Talks about things people have actually tried in a real, shipping game — Half the talks at GDC are about pie in the sky new algorithms or high level design or art philosophies. Some of this is fine, but most talks should be about experiences, not theories.
  6. Advanced talks on topics other than graphics and physics — I care just enough about physics and graphics (and let’s put sound in there while we’re at it) to hire somebody else to deal with them. Unfortunately those are the only topics that have advanced topics, probably because they’re also the most popular topics. I want advanced talks on databases, data-driven design, software engineering, networking, object persistence models, and load distribution.
  7. More focus on failures — I want to know what happened with Horizons, Shadowbane, Asheron’s Call 2, Auto Assault, Matrix Online and D&D Online that caused these games to not meet expectations. I mean what REALLY happened.
  8. Mandatory speaker preparation — Maybe there should be a required dress rehearsal of each lecture a week before the conference. I’m sick of going to lectures where a word document full of typos is what passes for slides.

The Online Game Development Conference is trying to do some of this. They require a peer review of every paper submitted to the conference (8). They are aiming at a more experienced audience (2, 6). They are in their first year so there are a couple of freebies on the list (1, 4). The conference is about internet games: (From their website)

The Online Game Development Conference is the first conference with a razor-sharp focus on the technology, art, design, production, and business of games delivered over the internet.

Will it work out? Will OGDC turn into the new AGC? Do we even need a new AGC? It’s not clear that the CMP purchase of that show is actually going to hurt it, after all. Well the jury hasn’t even begun deliberations on OGDC yet. The first one takes place on May 10-11 here in Seattle.

Hopefully we can get some of items 5 and 7 from talks like these: (I’m particularly interested in the second two since I hear that first speaker doesn’t know what he’s talking about. :) )

Here’s hoping it works out of OGDC. It would be nice to have a local conference.

What were the best sessions at GDC?

I didn’t go to GDC this year. I stayed home to fight fires and keep the team from partying the whole week. Most of the sessions are available as audio downloads, though and I would like to hear some of the best ones.  Which talks did you think were the best? Which of those are worth $8 to hear?

How will a YouTube for Games work?

A while back there was a flurry of new “Youtube for games” announcements. These are basically all shockwave.com with no editorial filter and some social network features. I wish them all well, but I doubt that most of them will amount to much. The problem with a YouTube for games is that it’s far more difficult to make a game than a movie. The kind of game you find on these sites takes

Let’s see how they’re doing so far:

  • Pijo.com: down for “rebranding”. They launched on November 6th.
  • GameGum.com: Of the 10 “favorite” games on the front page, only one has any comments.
  • GreatGamesExperiment: They have lots of content, but don’t seem to actually host any of it. They are more like a standard gaming site than a YouTube for games.
  • Kongregate.com: This one has tons of content, and they’re all flash games, so it’s hard to tell exactly where they’re hosted. It seems pretty active from the play counts on the games. And it gives you live chat with everyone who is playing the same game you are. This one is doing pretty well!

There are three big differences between “YouTube for Games” and YouTube. I think these will keep such a portal from becoming a massively popular site that Google buys for eleventy billion dollars:

  1. Old games weren’t written in Flash and can’t be uploaded to such a site. YouTube is filled with clips of William Shatner singing… everything. Such nostalgic content isn’t going to show up on Kongregate. You won’t be able to go there and play M.U.L.E. or Mail Order Monsters.
  2. There is no game equivalent of shooting video of yourself on a web-cam playing guitar or whining about your day (only click on that second one if you enjoy whiny teenage boys.) Making games is a lot of work, and even the minimum bar for a game to upload is probably 40-60 hours of work after you know how to do it. Learning how to do make a game in flash is several months worth of all your spare time.
  3. Most of the videos on YouTube are less than five minutes long.  Any halfway decent flash game will hold your attention for longer than that.  If I end up playing as many half hour flash games as I watch 5 minute YouTube videos, my productivity is really going to suffer.

There is one site that is trying something a little different. Sploder lets visitors build levels for a simple shooter on the site and then embed them anywhere. The game itself is pretty limited, but it points in an interesting direction. If you can build a game engine with tools that reduce game creation to clicking and dragging, many more people will be able to do it. A version of Sploder with uploadable sprites, custom sounds, and designer control over unit parameters (for things like physics, damage, and health) wouldn’t be that hard to write with a small team, and would let users make a wide variety of simple shooters.

Similar engines could be written for a lot of styles of puzzle games, RTS games, and RPGs. The resulting games would obviously be limited (in the same way that web-cam videos are limited) but there’s still plenty of creative room in such a no-coding-required engine. They would form a nice stepping-stone toward full-blown game development too.

What do you think? In a few years am I going to get 3-5 emails a day with a link to some game that somebody made and posted up on GameTube?

Book Report: Winning at New Products

I give up. I’m not going to finish this book, and I suggest that you don’t even start it. I made it through Chapter 9, which is about 2/3 of the way through the book, and that’s enough. My time is worth more to me than the remaining 140 pages.

Winning at New Products: Accelerating the Process from Idea to Launch is an incredibly dense and poorly written examination of the new product process in a wide variety of industries over the past twenty years. It has a few good points, to be sure, but it bombards you with those same few good points over and over. Most of the book is mind-numbingly obvious observations about what makes a product successful. I’ll just tell you those few good points here and save you the trouble.

The Good Parts Version

Robert Cooper has studied the new product process at hundreds of companies over the years and has gathered some useful data. Here are the few points I think it’s worth taking from the book:

  1. Do your homework up front — Research your market, focus test your concepts, and get buy-in from your own management at the start of the process. If these things, don’t look right, figure out how to adjust your concept or cancel the project. Do all of this before you go into development and spend a bunch of money.
  2. Dedicate people to new products — The people working on a new product team should report to the project leader of that team. Matrix Management just leads to less-dedicated teams and makes it much harder for the project lead to do his or her job.
  3. Involve the customer every step of the way — Test your concept with them. Talk to them to get an idea what their actual needs are. Let them test the final product in the field before you sell it to everybody so you can work out the kinks. Your customer is your strongest asset in developing a new product.
  4. Have a new product strategy — Your company is hopefully going to have more than one new product. Develop a plan for what markets you want to go after, and then develop products to fit your strategy. Adapt your new product strategy to meet changing market conditions once you see how your products are doing.
  5. Have strong go/kill gates — Have a process by which projects are evaluated based on profitability, feasibility, and fit with the strategy. Actually use these gates to kill projects based on reasonable criteria and not just the internal political power of the project lead.

All of these are very good things to do. They also aren’t completely obvious, and it’s valuable to read about them and how they have worked (or not worked) in companies in the past. But a book with these five points is about 150-200 pages long, not the 400 that Winning at New Products weighs in at. This book has so much extra junk in it that it’s hard to find the good stuff.

Incredibly obvious observations

On page 101 of the book we find the two dimensions of market attractiveness:

  1. Market potential: positive market environments, namely, large and growing markets, markets where a strong customer need exists for products, and where the purchase is an important one for the customer. Products aimed at such markets are more successful.
  2. Competitive situation: negative markets characterized by intense competition, competition on the basis of price, high quality, and strong competitive products and by competitors whose sales force, channel system, and support service are strongly rated. Products aim at such negative markets are less successful, according to both NewProd and the Stanford Innovation Project.

So what he seems to be saying here is that a product that the customer needs in a wide open and growing market will do better than a new product that is going up against an established market leader that is already doing a good job of meeting the customer’s needs? Brilliant! That must be why he spent half a chapter explaining this concept. (What you see above is a summary of that longer version.)

Completely clueless about software

I didn’t expect the book to have any examples from game companies, but there also aren’t any examples from software companies. In fact, the following quote, from page 261, shows that the author doesn’t understand software in the slightest:

For example, in the development of a new software product, the proposal “to have most of the code written and partially debugged” is a very poor milestone. Words such as “most of”, and “partially” are not measurable, and further, there is no time frame. Rather, the milestone should be quantifiable; “to have 30,000 lines of code written and fully debugged by day 95 of the project” is more appropriate.

Judging the completeness of a software project by the number of lines of code it contains is ridiculous. Unroll your loops, boys! We’ve got a milestone to hit!

Not only can you not reasonably predict the number of lines a project will contain in advance, you are also usually better off if you have fewer lines rather than more. You really don’t want the criteria by which your programmers are judged to be number of lines of code they produce. You would be driving them to copy-paste-modify more, and pay no attention to re-use or reducing coupling in their code.

Most of the examples in the book come from manufacturing. Software development is much more like the “fundamental research projects” described on page 151:

Although a fundamental research project or science project may ultimately yield a new product, often the new product cannot be well defined at the beginning. Indeed, it may take months of technical research before it’s even clear what might be technically possible. [...] the Stage-Gate process described above may be inappropriate [...]

Obviously the average software project starts with a much clearer picture of the project’s goals than the average research project, but the “we don’t know what we can do technically” aspect of them is very similar.

Credit where credit isn’t due

The book, and the author’s website, state that 60% of the businesses surveyed use the Stage-Gateâ„¢ process. The reader is left to infer from this that these companies read Dr. Cooper’s books (or paid him to consult) and implemented their new product processes as a result. While I’m sure that has happened in numerous cases, it certainly isn’t the case in most of them. In many cases, the new product process in these companies predates the coining of the term “Stage-Gate”.

The author has attempted to apply Stage-Gateâ„¢ to any new product process in any company that has a series of steps with go-kill decision points between them. It’s sort of the reverse of what happened with Scotch Tape and Kleenex. He’s trying to associate his trademark with the successful new product processes in the industry even though no causality exists.

But what about Stage-Gate itself?

The Stage-Gateâ„¢ process, as described in the book and at Stage-Gate.com is not a software development process, it’s a new product development process. It operates at a higher level than project management processes like Scrum. It emphasizes cross-functional teamwork, but in the business-level context of Stage-Gate, cross-functional means Marketing, Manufacturing, Development, and Sales, not what we usually take it to mean: Art, Design, and Code. I didn’t really understand that when I wrote my previous stage-gate post.

Most of the people on a software project work on that project within a single stage (Stage 3 – Development) of the overall project. For those people, stage-gate doesn’t affect them except to define the criteria they must meet for development to be considered finished. For those of us who are in a position to influence the “what project should we start” decisions in our companies, we should consider something like stage-gate to make informed go/kill decisions on the projects. The criteria for our “want” driven entertainment products is not going to be very similar to the criteria outlined in this book for “need” driven products, but the idea of having defined stages and gates is a good one. We just need to figure out what it is we can measure in our industry to use as a basis for our own gate criteria.