It’s obvious: we want to enhance the way we interact with the world. Everything from virtual reality to smart watches show this trend. This isn’t a new desire, of course, but we are now on the verge of making new ways of interaction possible.
Today I’ll focus on what I’m most excited about in this new world that is soon to come: the ever growing internet of things and our interaction with it through wearable technology. Both seek to expand our human umwelt, allowing for new and better interactions with the world that surrounds us.
Many of you may know her already, but let me introduce Jane, an artificial sentient being from Orson Scott Card’s Ender series1. Essentially, Jane is what Siri dreams of being — a computer program capable of doing millions of things at once with a near perfect ability to interact with humans like humans interact with each other through devices. Although I love the character (all of my devices are named Jane partially because of a hope that one day they will be connected through a similar system23), she’s not my topic for today.
Instead, I will be focusing on the primary devices that Ender uses to interact with Jane — a device called the jewel. The jewel is a very small device placed in Ender’s inner ear which lets Ender audibly hear Jane speak and which Ender can use to subvocalize thoughts and commands back to her. This enables him to be connected to the internet and any attached technology without disrupting his regular senses and physical activities.
Regardless of the technology, we want to focus on the content at hand, not the medium used to access it. For example, that means that we want to focus on the music, the argument presented, or the story being told instead of the audio speaker, the presenter, or the book, respectively. The less we notice the medium we interact with, the better.
Devices and interfaces today are too invasive. The most obvious example of this are phones. Now that we have the ability to connect to the internet and social networks at any point in time, most people are constantly consumed by them. The majority of students on my college campus walk around looking at their phones instead of at other people, the world around them, or simply thinking their own thoughts when by them self. Wouldn’t it be nice to have the ability to use the internet in addition to our regular, physical lives instead of replacing one with the other?
This is where things like smart watches and virtual/augmented reality are headed in the right direction, but are still pretty far from perfect.
Smart watches allow us to no longer pull out our phone to check something; instead it’s already out in a sense, but that’s only one step saved and comes at heavy costs — the lack of screen space, terrible battery life, and missing features that our phones have. At least these innovations arguably decrease some of the friction of our interactions. And admittedly, some of these limitations will no longer be a problem as technology advances, but that’s still holding onto (as Bret Victor puts it so well in his fantastic talk) an inhumane way of interacting which restricts us to only one of our five modes of understanding.
Similarly, virtual reality, including virtual projection, is awesome and I greatly look forward to it becoming more mainstream, but the current constraints of headgear and the need for a particular environment make it unusable for the majority of uses I imagine. On a related note, while large displays and holograms will be helpful, if they require that we interact with only our hands then I don’t think they’d be especially successful, because constantly moving our bodies around is exhausting.
Augmented reality, while similar to virtual reality in a technical sense, is actually based on a very different paradigm — instead of exploring a new world, you’re enhancing your experience of the current one. This is much more promising for our purposes, since it allows for more meaningful personal and digital interactions as well as seamless integration of social life and useful information. Well-made augmented reality instills a far broader sense of exploration and awareness than any opaque, rectangular screen can. Arguably, Jane and the jewel fall in this tag as an exceptional example.
The ultimate wearable, then, is something as unobtrusive and natural as Ender’s jewel, with which we interact almost subconsciously, either with thought or with our bodies. Something like this is a long way away from reality4, and there are many stepping stones to get there, but it is absolutely a goal worth chasing after.
Wouldn’t it be sensational if we could turn off the lights or put on music just by thinking it? What about having a virtual helper like Link’s fairy companions (side note: I made a visualization of one) that both you and your friends can interact with? What if items in a grocery store light up when they are almost out of stock in your refrigerator and you have a party coming up? Or a virtual plaque with information about the monument you’re looking at appears for your group to read?
Pushing our world towards that better future is a priority for me. That leads me to build designs with these principles in mind which in turn create pleasant and useful interactions. Your path to making the world a better place in this way may be completely different, but I encourage you to contribute wherever you can.
P.S. If you like this article, you might also like my article about The AI Stanley Parable.
- If you haven’t read the Ender series, I’d highly recommend it; it’s the one fictional series I’ve liked enough to reread multiple times. I recommend that you don’t read the Wikipedia article on Jane or anything similar if you plan to read the books because that’d spoil some of the story!↩
- Fun fact, Apple actually prevents you from changing Siri’s name to Jane.↩
- The fictional futuristic movie Her did an okay job of showing some possibilities with virtual assistants like this, I think it’s worth a watch.↩
- There are some really cool things happening in the wearables world, like David Eagleman’s work.↩