If Augmented Reality is the solution, what is the problem?

Although augmented vision is where I started with my interest in the field, I have really moved a bit beyond that now.  These days when I say Augmented Reality I really mean wearable mobile computing with an interface that actually works.

With my new job at Valve came a new commute. I spend on the order of three hours a day riding on the bus, waiting for the bus, or walking to or from the bus stop. So far I have occupied my time with podcasts and paperbacks, but I am going to try to spend more time on my AR work starting today.  Thus, I am writing this while on my morning commute to Bellevue.

The mobile computing experience is poor for many reasons but they can be summed up as two things: Input and Output. Yes, that’s all. :)

On the input side the problem is that so much of what I want to do is driven by entering text. My phone (a Treo 650) has a great keyboard… for a phone. Even so, it really doesn’t compare to the experience of typing on a laptop or desktop keyboard. On my laptop the act of entering the text into the computer is not hindered by the act of typing. I’m far from the fastest typist, but I can still type much faster than I can generate the words I want to type. Until a mobile platform offers the same level of comfort and speed it will never be suitable for writing anything longer than text messages and tweets. The idea of writing code via a phone keyboard is just absurd.

My laptop only works while I’m riding the bus or waiting someplace I can sit down. It is s not an option at any stop where I wait while standing. That means I can’t use my laptop at full half of the places where I wait for the bus. I’m not sure I have ability to write while I’m actually walking and not knock over my fellow pedestrians (or get hit by a car) but I certainly have a lot of downtime-while-standing when waiting for my transfer on the way home.

So the first problem I want to solve that I keep in my mental file labelled “Augmented Reality” is the ability enter thousands of words of text comfortably while out and about.

The second problem is getting video output from the computer while out and about.  My laptop does an good job for the part of the time when I can have it open. It’s hard to see the screen on a bright day, but we fortunately don’t get too many of those here in Seattle. The trouble is that I could use output from my computer in many more places than I can break out the laptop.

The screen on my Treo (or my iPod Touch) is a little better in some ways. It’s much more feasible to bring it out when I need to check a bus arrival time. Thanks to onebusaway.org and my phone I can get this information on demand. All I have to do is wake up my phone, open the web browser, hit one of the bookmarks I’ve saved for the bus stops I frequent, and wait for 10-15 seconds while the page loads. If I’ve recently looked up that information for a stop I just need to wake up the phone and hit refresh to get updated information. It’s definitely better than looking at a clock and the never-very-accurate schedules on the stop itself.

What I really want, however, is to just know this information. On my way home from work there are two questions I often ask:

  1. How far is the #550 from my stop? Do I need to run? – By the time I can see the bus it’s only about 40 feet from the stop, so having some notice that I’m going to miss it would really help.
  2. How long do I have before the #1, #2, #4, and #13 arrive at this stop? – Any of these busses will get me within walking distance of home, the only real difference is how far I have to walk. The #2 stops a half block from my house, so I prefer it, but if it is lagging behind the others by a wide enough margin it isn’t worth waiting.

This kind of ambient awareness is where the augmented vision comes in. If it is within an hour of one of my usual riding time at one of my usual bus stops I want to see the current data from onebusaway.org for that stop.  Big obvious columns of light that let me see the bus approaching from blocks away would be cool, but they probably don’t solve the problem as well as a 2D display on my personal HUD that I can glance at occasionally. “When will my bus arrive” is just the most obvious question I want a constant answer to. Once that one is solved I imagine that many more will present themselves. (I also imagine that many of those will actually require some level of registration with the world.)

Once I am wearing a head-mounted display I will probably use it for one more purpose. I would like to be able to block out the world in front of me once I am actually sitting on the bus. I am prone to visual distractions, and have a hard time focussing on much of anything when a bunch of people are around me. If I could occlude most of my field of view with whatever I’m working on the distraction would be greatly reduced.

The problem that I am interested in solving is “Mobile computing sucks.” Location and temporally aware wearable computers with first person displays are the solution to that problem.

~Joe


4 Responses to “If Augmented Reality is the solution, what is the problem?”

  1. Jacob Chapel thought on :

    I have been intrigued by what is possible with AR and am enthralled by your post. You really go over many of the hurdles we face today and what AR would provide (or will hopefully provide) once it’s available.

    The visuals are being developed, the location awareness is being developed, but what really needs some thought and work are input methods like you detailed and a sort of decision awareness. I don’t know how to solve the input problems while keeping what you are doing private. The decision awareness would be more predictive based on actions and could be programmed.

    I cannot wait for the day that I can walk around unobtrusively computing like I would at home on my own computer with little to nothing hindering my enjoyment or work.

  2. rouli wrote on :

    When I think about ambient knowledge, I’m reminded of a certain accessory that was somewhat abandoned in the last 10 years due to the prevalence of mobile phones.
    I’m talking about the wrist watch. Though it’s far from AR, I can imagine it could be the perfect display for ambient data, such as when the next bus is coming. Let it talk with the cellphone in your pocket (or with your laptop) via bluetooth, and program it display, as a function of a certain location/time the information that interests you. Checking for the next bus will be as easy as checking the time, and it’s more feasible than going around with an HUD these days.

    One thing that HUDs are good for is the input problem, since they can provide you with virtual keyboards. However, I don’t think anything can beat up the plain old keyboard, especially for our generation that grew accustomed to it.

  3. Joe thought on :

    A wrist watch is pretty much exactly the kind of thing I’m talking about. Back when I still wore one I had constant access to the time and date. Now I have to haul a phone out of my pocket to even get those. It might take more space than a wristwatch occupied to display all the different kinds of location aware data that I’d like to see, but it’s a start.

    One input option that comes up a lot are those laser-projected keyboards. The problem with them is that you often don’t have a flat surface to project a keyboard onto when you’re out and about. Flexible keyboards have the same problem.

  4. Darkflame wrote on :

    I actualy think one possibility is we might go back to a more pen-like interface, using either special stylus’s or any ordinary pen that the camera can pick up with some object recognition.
    -

    As for AR in general, I see it as the next evolution of the multi-function tool. (and wrist watchs were very much a step along that journey).

    A computer, after all, is a meta-tool. A tool that can become dozens of other tools as and when we choose.
    AR is a massive extension of this. AR can do everything a computer can do…only it can do it everywhere, and give put it in a real-world context too.

    I suspect we haven’t fully grasped the implications of this.
    But given the inevitable AR-versions of websites like “Instructables”, I think the abilitys of the average human to do stuff is again suddenly going to increase’s hugely.

    Just like when the first caveman learnt to tie a sharp rock to a stick.

Leave a Reply