Programmer Joe

Another in my ongoing series of ideas I’ll probably never act on. Do whatever you like with this. If you actually make the game and it runs on Android, let me know. I’d love to try it out.

The High Concept

Play through a role-playing game adventure on your mobile phone while moving around your local park, hiking through the woods, or wandering the streets of your home town. Participate in simple quests solo, or play with friends. Or if you prefer, develop your own adventures and share them online for other people to play.

The Inspiration

In the 1980s and 90s there were a series of books called Fighting Fantasy that let players play through an RPG-like adventure without a gamemaster. Combine a mobile implementation of those books with geocaching and then let the users create all the actual content and you have GPS Quest.

The Technology

Building GPS Quest would be straightforward:

1. Develop a simple RPG engine with monsters, loot, stats, and leveling up. Leave as many hooks as possible for user-generated content to modify things. Combat will probably want to be turn-based. Write a mobile app to resolve combats in the system.  This is by far the hardest step. Probably do this with just one player for the first version.
2. Build a back end that can track a player’s RPG stats over time. Include a quest system that can unroll an adventure in front of the player as they move around the world. This would probably be waypoint-based so the user can look at the next place to go on a map on their phone.
3. Build quest creation tools that let a user (you) write new quests in the actual physical world.
4. Publish all this to the world.
5. Iterate until massively popular

There are many hundreds of features that could be added once the basic system is up and running. Some ideas include:

• Multiplayer. First for small parties of 2-5 people and them maybe for raids of 20-40 people.
• Audio for monsters, navigation,  and quests. You could download it all before starting the quest so it could play quickly.
• Custom art for quests. Might want to download this ahead of time too.
• A builder-maintained bestiary
• A builder-maintained loot catalog
• Tools to remap existing quests onto new locations
• Matchmaking tools to help players find each other
• The sci-fi, zombie, pirate, superhero, spy, vampire, giant robot, caveman, and not-at-all-fantasy-medieval versions of the same game. That last one is so some SCA people can feel comfortable playing your game.
• Leaderboards for quest builders, quest remappers, and players to give all of those people bragging rights.
• More simple geocaching features like buried treasure and traps that players can leave for each other.
• Ports to whichever platform you didn’t launch for in the first place.
• Trading systems, auction systems, crafting systems

What do you think? Would you play?

The Problem

I am bad with faces. I mean really bad with faces. My brain just doesn’t seem to be very good at mapping what someone looks like with their name.  This often makes things difficult for me at networking events and conferences.

The Solution

A passive mobile application that scans the environment around the user for faces. When it detects a face that it recognizes the application speaks the person’s name to the user via their bluetooth earpiece. Ideally this solution would also involve a discrete camera that could operate without being obvious to the people it is operating on. The point is to serve as a passive aid to memory while not changing the behavior of the people you are interacting with.

The Competition

There are a couple concept applications out there along this line including Recognizr and Comverse Social AR. Both of these applications have the same problem, which is that you have to hold up your phone to take a photo of the person you want to identify, then wait for the result to come back.  That is intrusive enough that a simple “I’m sorry, remind me what your name is…” would be a better option.

The Pieces

1. Facial recognition. There are many providers of facial detection and recognition APIs, so it should be possible to license this piece. Unfortunately most of the providers don’t seem to be very good at licensing their SDK to people. I get the idea that these are all very small companies that spun out of someone’s PhD research.
   1. PittPatt never replied to my email.
   2. Luxand put me on their marketing email list, but never sent me an evaluation key.
   3. Betaface actually gave me a chance to evaluate their SDK. It works quite well. I wasn’t a fan of their licensing terms, but you might have different needs than I did.
   4. Ayonix got back to me right away but never provided the promised evaluation link.
   5. I don’t remember if I contacted Seeing Machines or not.
2. Bluetooth camera – I bought an OptiEye. It works pretty well. If you ask them nicely they will send you the protocol documentation. The specs claim a four hour battery life, which is plenty for most networking events.
3. Text to speech – I haven’t done any research here.  Many applications do it, though, so I would imagine SDKs are available.  If nothing else the user could record the names and the software could just play back the recordings.
4. Mobile computer – Both Android and iPhone allow communication over RFCOMM, which is what the OptiEye uses. Existing devices are also too weak in the CPU department to do much visual processing on the phone, but they could stream video or individual frames up to a server for further processing.

What do you think?  Dream product?  Interesting project? Terrible idea?

At TED 2010 Blaise Aguera y Arcas from Microsoft demoed live integration of video into the existing structure-from-motion dataset in Photosynth. Though his demo showed a video feed moving around a scene the same data could just as easily be turned around to find the precise position of the camera in real-time. That capability is a key part of building a head-mounted augmented reality system.

Two weeks later Google announced that they are incorporating user photos into Google Street View. This requires essentially the same data as Photosynth. Google has the added advantage that they can combine it with the Street View images and LIDAR data they are already collecting. Though they haven’t demonstrated real-time capability with this data they certainly have all the pieces they need to make this happen.

Access to the data required to perform pose recognition with cameras is a novelty at the moment, but if mobile augmented reality takes off in a big way it will become a key component of that system.  In my opinion this component is too important to be left in the hands of one company. A much more desirable situation would be to have an OpenStreetMap-type project to accumulate and curate a freely available dataset to provide structure from motion and pose recognition for use in mobile augmented reality and whatever other uses someone can dream up.

OpenStreetMap is a project that sprung up to provide access to data that was free from the costs and restrictions that come with commercial data. It uses a Creative Commons license to make the data free for use by anyone for most any purpose. Although OpenStreetMap came about in response to the restrictions on commercial data sources, the same approach could be taken for 3D structure and image data even though commercial sources for that data do not yet exist. If OpenStreetMap had existed when car navigation systems became feasible in the late nineties it is likely that many commercial products could have been developed on open data at far lower cost and in much more variety.

All such a project needs is a small number of dedicated people to get it started. Download a copy of Bundler (an open source structure from motion library based on the same research that spawned Photosynth) and seek out publicly available photograph libraries. Then talk a cloud computing provider into sponsoring the project by hosting the data and build things up from there. The project won’t have many users for a few years, but as the accuracy and coverage of the dataset grows the set of applications based on this open data will grow too. Somebody just has to get the ball rolling.