Augmented Reality should be open

Over the past year I’ve spent a lot of time thinking about what piece of the augmented reality ecosystem would be the best to start a business around. I’m still not ready to take that jump so, in my case at least, the answer is still “none yet”.  However, in my exploring I keep coming up against a problem:

  1. The absolute most profitable place to be in augmented reality is the platform provider at the center of everything.
  2. The profit motives of that platform provider could set the development of AR back by about ten years.

A brief history of the web

Whether by design or happy accident the protocols (HTML and HTTP) behind the web are easy to implement and completely open. This meant that by the time Netscape came along, there were already browsers on the Macintosh (CERN’s and Mosaic), Windows (Mosaic), and X (CERN, Mosaic, Viola, etc.) There were also 200 active web servers and port 80 accounted for more than 1% of the traffic on the NSF backbone.

That ecosystem meant that Netscape remain compatible with what already existed in order to succeed.  Sure, they were selling licenses to their own software, which let them cash in on the shocking growth of the web, but the Netscape browser had to work just as well against pages served by HTTPD, IIS, Apache, and any other random web server anyone decided to write. The same thing was true from the other side.  Netscape Now! buttons aside, website operators soon had to deal with at least two and possibly more different browser, as well as various versions of each browser.

This made life interesting for web designers, but it was good for the web as an platform. The nature of the web meant that nobody had to convince somebody else to say “Yes” to get involved.  There is no way that any one company (or any ten companies for that matter) could have even authorized, let alone managed, all of the initiatives that went on with the web between 1994 and 2000. There was just too much stuff happening.

The open nature of the web allowed the cost of innovation to be spread around to thousands of organizations around the world.  It also let anyone with enough cash to buy some hosting try out their big idea. Most of those ideas failed, of course, but when taken as a whole they succeeded beyond anyone’s wildest expectations.

I think that augmented reality has the potential to follow a growth curve with the same shape as the one the web followed. The web had very few institutional barriers standing in the way of its growth, and the AR ecosystem would do well to learn from that.

Open Augmented Reality

If the emerging augmented reality ecosystem wants to grow as quickly as the web it cannot include anyone who must say “Yes” to allow existing users to get a new capability. That implies a few things:

  1. Anyone can publish content into the system. There are no controls for quality or appropriateness of content on this ability to publish.
  2. Clients from multiple vendors are able to view that content. Anyone who choses to can write a new client that works with existing content.
  3. Servers from multiple vendors are able to respond to requests for data. Choosing server technology is primarily a decision for content providers to make and their choice is invisible to end users.
  4. The network itself is neutral to the data being transmitted across it. This means the mobile internet providers must not white-list content from publishers that it has partnerships with.
  5. There is no single central directory that all content (or every content provider) must be listed in to be available.

Note, that this does not require that the software in question be open source. Open source software (in the form of Linux, HTTPD, Apache, Perl, PHP, and others) was instrumental in spreading the web far and wide. However, the personal computer revolution happened with little in the way of open source software and was just as rapid as the spread of the internet.

Open Standards

As VRML and many other standards over the years have taught us, developing a new standard from whole cloth is fraught with peril. It is even more difficult (as in the case of VRML) when there is not an existing standard that the new standard is intended to supplant. The AR community must avoid repeating the history of VRML. Fortunately there are existing standards that lend themselves well to the problems augmented reality developers are trying to solve.

The first of these is good old HTTP. As a transport protocol, HTTP fits the list above very well. The protocol is well understood, decentralized, and available in server or client library form for every platform. Minor new standards for querying location-specific data are already emerging.

The second current standard that the augmented reality developers can adopt and bend to their will is KML. KML is the file format that Google Earth uses to represent geocoded information. It has support for points, lines, and shapes. KML is an open standard and is supported by many GIS packages in addition to Google Maps and Google Earth.  Google has open-sourced its own KML parsing library so there is a place to start there too.

Any augmented reality client that supports attaching web browsers (including URLs) to locations can also take advantage of most other existing web standards for whatever happens to be in those browsers.

Is this how things are actually going?

So far, I have seen very little discussion of how different augmented reality systems will work together.  In large part that is  the point of this post. But then there are also very few AR systems that exist outside of laboratories, so we could just be in the bad old proprietary hypertext system days of the late 80s.

So far the AR systems that seem to be designed for lots of different kinds of data (Layar and Seer) have not announce any way for third parties to publish data for their clients. My twitter exchanges with Raimo at SPRXMobile make me think that Layar is at least thinking about it.  Hopefully they will turn out to be as open as I’ve outlined above.

How important do you think open AR standards are? Can an AR solution succeed without them?


17 Responses to “Augmented Reality should be open”

  1. Alex Kasper replied on :

    I think you’re right. I think it would help a lot if there was an open standard for AR content. This would make it a lot easier for hardware vendors to create gadgets. The same goes for content creators.

    What I’m not sure about is, if an open standard could help solve the apparent problems AR still has, which I think are good hardware solutions and robust software.

  2. Darkflame replied on :

    I think your dead right here, KML is what Id go for too.
    But id like to prepose another existing standard into the mix;


    For a means of comminicating dynamic or tempory AR data i think it would be perfect.

    Its open, anyone can create a channel, anyone can access it, and different permisions can be set too.
    Better yet, you can be logged into many channels at once.

    Naturaly, IRC has a string limit of about 500, so youd have to exchange links rather then the mesh data itself.
    But this too is an avantage, as those links can be just an IP address and port number leading back to the client that posted it.
    In this way IRC could be the protocal used to play AR games (which -anyone- can login to your #channel and view), or it could be used to communicate with friends and family. (chatlogs, after all, are normaly just saved on the clients, you dont want this data perminatly hosted).
    Theres lots of AR use’s that dont demand perminate hosting, and for that I think a solution could be made very quickly based around IRC :)

  3. Thomas K Carpenter commented on :

    I think you’re right as well, but those discussions need to start happening (@ ISMAR09?). I know Ori brought up this topic in an interview at UgoTrade, but I’m not sure how far along it is.

  4. adam wrote on :

    I would rather [insert unpleasant thing] than see IRC be used for anything ever again.

    That is the single worst, most mind-blowingly stupid, total crap protocol it has ever been my misfortune to implement.

    You can do better than IRC in about 3 weeks, from scratch, as an eager but naive CS undergraduate who grew up with IM. IRC *needs* to die, be buried, remembered fondly – and never be used again.

    I think you are nuts, my friend. Why the heck didn’t you suggest something like RSS or Twitter instead? Both are surely far better than IRC.

  5. adam said on :

    “I think it would help a lot if there was an open standard for AR content. This would make it a lot easier for hardware vendors to create gadgets.”

    What do you actually *mean* here?

    I don’t see how there is meaning to AR apps inter-operating, or having shared semantic domains.

    For the Web this did make sense – because everything was documents, with references, and any document can sensibly reference any other document – even if the media is different.

    But AR apps don’t need to “reference” anything – that’s a “feature” of documents and data systems, and AR is not a document and is not a data system.

    Where an AR app needs to ref, it’s because it’s doing something with docs, and then … it will use HTTP, because “not integrating with the web” is insane for *any* doc-system these days, and it’s hard to find any that do that.

    I think you start with a good point, Joe, but I think at the end you go a bit off-track – until/unless you add-in some more exposition about why *anyone* would even care about interoperation in a web-like way.

    (not saying it’s not possible that we would, just that I think it’s very hard for most people to see *why* they should care?)

  6. Joe wrote on :

    I am planning to go to ISMAR in October. Hopefully I’ll spend the whole week on this and many other topics. :)

    Adam, individual AR apps probably won’t need to interact with each other (much), but they many of them will need to interact with the user at the same time. I don’t want to have to close my navigation app to open my “where is my bus” app or my “show me the time when I look at my wrist” app. Multiple applications need to run at the same time and provide their output to the user via a single channel that they all share. The same is true for input from the user whether that input is on a touchscreen, gestural, or driven by some other sort of sensor.

    I don’t believe that the standards should come FIRST, just that people who are building these systems should build in a direction that allows cross-vendor standards to emerge instead of building in a direction that promotes vendor lock-in and ultimately stagnation.

  7. Darkflame wrote on :

    hu? I posted a long, detailed reply here :-/

    “Why the heck didn’t you suggest something like RSS or Twitter instead? Both are surely far better than IRC.”

    In short;
    No, they are completely different and wrong to the point I suspect you dont understand the requirements.

  8. mj replied on :

    The advantage with the ‘standardisation’ of AR is that anyone can play. The disadvantage with it is that anyone can play.

    AR may suffer from becoming the web – adverts and pop-ups everywhere – unless there are multiple platforms. I don’t want AR to be pervasive, I want it to be immersive. And nothing will annoy my walk-to-work-playing-a-spy-game more than being targetted by WalMart or Sony for in-game adverts which are visible because I’m using a standardised platform and interpreter.

    I see the need for commonalities. I just don’t trust the folk who have the power to make it common.

  9. Joe replied on :

    You see the ads you see on the web because you opted into them by using sites that are free to use and supported by advertising. It’s not like any of those advertisers can just push ads at you no matter what you’re doing. I see no reason to believe that AR will be any different. If your spy game is ad supported you will see ads. If you’re paying $X per month for the premium no-ad version of the game you won’t see ads from the “spy game” layer regardless of what WalMart and Sony would like you to see.

    The only way that would happen would be if the AR platform client itself were ad supported (like many free IM clients.) In that case open standards are actually in your favor because anyone is free to develop an open source alternative to the ad-heavy commercial client you’re using.

  10. Maarten Lens-FitzGerald thought on :

    Layar just opened up its API. The Augmented Reality Web is yours!

  11. francesco commented on :

    totally agree, but it look like things are progressing differently, which is rather appalling or maybe just inevitable…

    will prior art prevent this patent to be granted?

  12. Joe commented on :

    I would think that Wikitude, Sekai Camera, and even SnapTouch would cover the same ground. Hard to tell with the PTO though.

  13. OkinKun said on :

    I agree with the IRC idea. But not in actually Using IRC.. Just the concept which IRC is based on.

    It’s too old and too limited a standard for AR… But the concept of channels/layers/visions is still a VERY good idea for AR. It allows for privacy, sharing, public elements, etc, and it allows you to turn off things you DON’T want to see.
    I just finished reading Rainbows End, and I hope someday the AR standards will be something like that, where you can select which layers/visions/channels will be visible, any time you want.

    In this way, we can keep problems like spam, and over-intrusive advertisements, at bay. There would be public channels for various publicly-accessible necessities, and that could include virtual billboards… But remember, property ownership STILL holds valid in an Augmented Reality world. Advertisers can’t put ads where property owners don’t want them, obviously.

    Along with the public stuff, you could have private channels, with various sharing access levels. So you can share stuff with your friends, or not, and only those people could see it, or just you.

    Also, if you’ve seen Dennou Coil, in it certain real-world property/locations, which are owned by private groups, are not accessible by public AR applications/things. Which works well for the advertisement thing.

  14. Joe said on :

    I’m not sure how a property owner would have any ability (or right) to keep other people from posting AR ads on their property. It’s not as though the ad in question actually exists outside of a database somewhere.

    This is a bigger question for content other than advertising. Think about how unhappy Scientology would be if Anonymous could post stuff all over their buildings.

  15. OkinKun wrote on :

    Well, that’s why I’m just guessing.. lol
    But I’d have to think some system could be devised, to deal with the issue of private-properties and unwanted AR graffiti/ads.
    Maybe a system with data from map/zoning records, in order to set initial AR zonings, or an opt-in/or/opt-out system of some sort would be better.
    I don’t really know, and I don’t think anyone really can at this time, these are problems we’ll have to deal with as we go along.

    On a side note, I was looking at info on Layar (the cell phone app) the other day, and I couldn’t help but wish that it allowed people to post text messages, in a “layar” designed for that purpose, as a fun real-world online-community feature, or maybe even location-based forum threads!
    Which would allow for things like: Posting reviews on the restaurants or other places you’ve been, leaving comments in GPS-tied locations almost anywhere you please, or even posting follow-up comments to other people’s messages.

    I’m sure such a thing would be easily possible with what we’ve currently got, but someone just has to design some cool software to do it, first. =/ Wish I had enough ability to do it myself. lol

  16. Joe said on :

    Layar may get there eventually, but it seems like they have a little way to go before community-based features like virtual graffiti will be supported.

  17. Franklin Lyons replied on :

    After ISMAR09 is there a follow up to this post? Is this not the most important discussion happening in the community? It should be. Layar opening its API is not the answer – we need an open platform before the creation can begin.

Leave a Reply