Obvious Idea #1: OpenStreetMap for AR Tracking Images

At TED 2010 Blaise Aguera y Arcas from Microsoft demoed live integration of video into the existing structure-from-motion dataset in Photosynth. Though his demo showed a video feed moving around a scene the same data could just as easily be turned around to find the precise position of the camera in real-time. That capability is a key part of building a head-mounted augmented reality system.

Two weeks later Google announced that they are incorporating user photos into Google Street View. This requires essentially the same data as Photosynth. Google has the added advantage that they can combine it with the Street View images and LIDAR data they are already collecting. Though they haven’t demonstrated real-time capability with this data they certainly have all the pieces they need to make this happen.

Access to the data required to perform pose recognition with cameras is a novelty at the moment, but if mobile augmented reality takes off in a big way it will become a key component of that system.  In my opinion this component is too important to be left in the hands of one company. A much more desirable situation would be to have an OpenStreetMap-type project to accumulate and curate a freely available dataset to provide structure from motion and pose recognition for use in mobile augmented reality and whatever other uses someone can dream up.

OpenStreetMap is a project that sprung up to provide access to data that was free from the costs and restrictions that come with commercial data. It uses a Creative Commons license to make the data free for use by anyone for most any purpose. Although OpenStreetMap came about in response to the restrictions on commercial data sources, the same approach could be taken for 3D structure and image data even though commercial sources for that data do not yet exist. If OpenStreetMap had existed when car navigation systems became feasible in the late nineties it is likely that many commercial products could have been developed on open data at far lower cost and in much more variety.

All such a project needs is a small number of dedicated people to get it started. Download a copy of Bundler (an open source structure from motion library based on the same research that spawned Photosynth) and seek out publicly available photograph libraries. Then talk a cloud computing provider into sponsoring the project by hosting the data and build things up from there. The project won’t have many users for a few years, but as the accuracy and coverage of the dataset grows the set of applications based on this open data will grow too. Somebody just has to get the ball rolling.


I have a bunch of ideas like this one rattling around in my head. Some of them could be products or businesses, and some are just cool projects. I have looked into them all to some degree but probably never start real work on them. I’m going to post them here in an attempt to spawn a discussion and encourage you to share your thoughts in the comments. Feel free to do whatever you like with these ideas.

~Joe


3 Responses to “Obvious Idea #1: OpenStreetMap for AR Tracking Images”

  1. Robert wrote on :

    None of these ‘cluster providers’ will have anything like the IO and memory needs mass bundling will need.

    Their clusters are not set up for this sort of workload.

  2. Joe thought on :

    When I talk about cloud computing providers I was thinking specifically of Amazon Web Services which already hosts many large datasets. Other options would be Google App Engine or Windows Azure. These providers certainly have enough IO and memory, or would if there were a market for it.

    You bring up a good point though. This will require far more processing on the back end than OpenStreetMap requires. That may mean commercial providers between the data and the end user’s device even if the data itself is actually free.

  3. Robert said on :

    Look a bit further into AWS and the like, and you discover that IO into and between individual machine instances (equivalent of disk IO) is shocking.

    This is due to the technologies (virtualization) these services are built on. Consumer demand won’t just fix that.

    I’m also not sure if they provide the gigs of memory that these huge feature indexes require. You would essentially be trying to build a distributed database, which is a notoriously difficult area of computer science.

    I’m fairly convinced a cluster for this sort of work would have to be custom designed.

Leave a Reply