Inventing Interactive » tablet

Seamless Interaction

David — Fri, 08 Nov 2013 18:25:20 +0000

(Image source: The American Resolution)

A couple weeks ago I was at dinner with some friends. At some point, talking about recent trips, people wanted to see pictures. As iPhones ware passed around (always a little worried that swiping to the next photo might reveal something not intended to be shared), we realized that we could exchange photos with Apple’s new AirDrop feature. While this was pretty cool, the technology wasn’t seamless, and didn’t work for everyone. So the conversation took a detour into settings and configurations screens. The joy of sharing devolved into an IT lesson where some people got confused and frustrated. We eventually put our phones away and switched topics.

The problem

We know we use technology constantly our daily lives, both individually and socially. But the event also illustrated how far we still have to go to invent something that’s both powerful and effortless.

This is especially true with our mobile devices. We rely on them as memory aids and as ways to enhance conversations by introducing new content. Talking about a movie we look up the director to see what else they did; we take a note of a book that a friend recommends; we check the reviews to find a restaurant when we’re in an unfamiliar neighborhood. The devices give us access to virtually infinite content; but as we use them, our attention to our surroundings, and the people we’re with, diminishes. Conversations take a pause. We get distracted.

With human factors and design research we know to design interfaces that are based on user needs and take into account the contexts in which they’re used. For example, a running app has larger buttons so it’s easier to hit when running; and a navigation app reduces information clutter to prevent driver distraction.

But these are instances of interfaces designed for specialized contexts. It’s when we use general-purpose apps in unintended environments that we get into trouble. And it ranges from real danger, such as when using your phone while driving, to minor distraction, such as looking up some information while at dinner.

For the “danger” category, there’s lots of work being done trying to find individual design solutions that are safer. But what about the “distraction” category? Is there a better way to weave general purpose connectivity into our lives? I’m not claiming to have any solutions, but bear with me as I do a bit of a ramble (and link in some previous posts)…

Device solutions

One approach is through augmentation. There’s a lot of talk of how augmented reality can aid manufacturing, and there are plenty of one-off advertising toys. But I can’t think of many examples of general purpose apps that use augmentation. Perhaps Yelp’s Monacle or Nokia’s City Lens? But these examples are still fairly specialized (they help you explore a city) and don’t especially facilitate social interaction.

Google Glass takes augmentation in the opposite direction, skipping social entirely, and focusing on just the wearer. While there are some interesting design opportunities, the technology feels like a backwards step — it’s inward looking, increases isolation, and discourages sharing. Taking this to the extreme, Domestic Robocop (and Augmented City) projects an absurd possibility of this future.

Domestic Robocop

The introduction of the iPad was a interesting advance for social interfaces for it freed us from the desktop. Because it was portable, it could be used almost anywhere. And, because it had a larger display, multiple people could look at it simultaneously.

There have been a couple specialized apps that treat the iPad like a multi-user device. Cotracks, for example, lets multiple people make music together. And Scrabble connects the iPad to each player’s iPhone for multi-user and multi-device play.

Scrabble

But, we’re not yet all carrying around iPads and placing them out in the middle of the table. And while it’s larger and more social than a mobile phone, it doesn’t scale well for more than a couple of people.

Environmental solutions

What if we stepped away from the idea of individual & personal devices? What about larger-scaled technology — stuff that’s built into our environment?

Well, technology that’s built into the environment tends to date quickly and be expensive to update. And there are plenty of challenges to designing for sharable technology which would also know your preferences and link to your networks. But there are some interaction examples that are noteworthy and from which maybe we can learn.

Microsoft’s Surface Table (now called PixelSense) is a touch-sensitive display, usually mounted as a table , around which people can gather, and that recognizes objects placed on it. It’s a nice, and relatively seamless, way for the display to connect to the real world. It aims to support natural user interface (NUI) design principles so that the interface effectively becomes invisible to users.

But the Surface Table never really took off. It was expensive, had too specific a form factor, and there were few applications that really took advantage of the new design principles that the technology offered. Samsung now offers a larger, more flexible and powerful version.

Microsoft Surface Table

Other real-world examples of technology built into environments tends to be in support of fairly constrained use-cases — not the general purpose functionality hinted at with Surface. For example, high-end video conferencing systems such as Cisco, Halo, and Oblong, build technology into meeting spaces. (Media:scape is in somewhat more natural environments.) They let you bring your content and provide ways to structure and work with it. While they’re built around meeting taxonomies, they might be expanded and generalized for other ways in which we share.

Halo conference room

Media:scape installation

For non-business examples, I think back to more specialized things like the dining tables at Inamo and Barneys. Inamo’s is quite narrow in its use, really just being a way to order food. The Barneys table also gives you food and fashion content, appropriate to when you’re in their store shopping. But both are still solitary interfaces, without means to share with your fellow diners.

Barneys genes@CO-OP table

Future technologies

Looking at future technology, stuff still in the concept and research phases, we see the emergence of some pretty powerful ideas that could support more general-purpose activities.

“Projection-lights,” such as Berg Lamps and LuminAR (among others), look at how we might be able to build projection and gesture recognition into our everyday world. It nice thinking about how we could get away from screens and have information displayed seamlessly anywhere we may need it. Berg’s explorations of how projection exists as a design medium, and the general rules for “smart light”, are especially interesting. This, combined with all of our devices being connected as part of an internet of things, seems full of exciting possibilities.

LuminAR projection bulbs

Science fiction interfaces, as well as vision films, portray future scenarios and how new technologies could be incorporated into our everyday world. They actually are closest to portraying a future where information seamlessly is part of the everyday context. But, as Scott Smith pointed out, these films often “show our lives as simplified and passive, with technologies that magically anticipate our needs.” They show solutions to specific problems — not overall new interface approaches.

Corning’s A Day Made of Glass

Back to where I started

I think my last post, about food and dining, was partly to blame for all this talk. I wanted a restaurant where I could share with my fellow diners, and also get content that would enhance my dining experience. But now I want something more generalized. I want new ways for us to get and share information. Ways that are transparent and seamless. Ways that don’t distract or cause us to loose the human connections we have with others.

This post isn’t about solutions… technology, and our expectations of it, change too fast. Instead it’s about challenges and opportunities. This post also isn’t intended to cover all the research that’s happening in this field — as there’s lots, with a great and deep history. Instead, hopefully it’s also about how the future can be much more than what we currently have. And it’s a reminder that we need to continue designing new interaction paradigms.

Who do you think you really are?

David — Fri, 17 Dec 2010 05:00:50 +0000

London’s Natural History Museum recently opened the interactive film “Who do you think you really are?” It’s the inaugural experience in their David Attenborough Studio lecture theater — a 64-seat theater where each seat has its own tablet computer.

The 50-minute film teaches visitors about evolution using a mix of techniques. It looks like an interesting experience, using narrative storytelling, heads-down individual interactive segments, and heads-up/social augmented reality and image sharing.

The project was a design collaboration between Pentagram (read here for more about the overall concept and space), BBC Research and Development (read here for more about the technology behind the augmented reality), and the Natural History Museum (read here for more about the film).

Although I’ve always loved science museum shows where you sit back and watch in wonder, this looks like an intriguing way to make shows more interactive. The challenge, over the long term, is preventing the show from feeling out-of-date — a real concern since technology changes so quickly.

An insightful comment on this Guardian article about the film points out a recurring problem with these sorts of experiences — that short durations (less than an hour) force authors to keep the content fairly light. It’s a problem I hadn’t really considered before — that science museums tend to tell the same stories over-and-over again. This may be fine for young audiences who are being introduced to the subjects, but its not very interesting for people who have already learned the basics (both for parents and the kids who’ve been to other science museums).

Augmented reality, dynamic books, and tablet computers are booming right now and Open Exhibits is creating an open source library of tools for museums to build “interactives.” How can museums embrace these developments and engage with people in an ongoing manner? Well, it’s more than just technology. Museums should think about:

How can the depth of content be controlled by each individual visitor to match their interests and previous understanding?
Can visitors bring content to the museum, and have it put into the context of the exhibition?
Can visitors prepare for their visit to the museum?
How can people continue their experience after they leave the museum? Can they explore the subject in greater depth, see how it applies to their daily lives, or …?

It’s a tough but exciting challenge.

All images: Copyright Natural History Museum- Rights Reserved.

Star Trek: PADDs

David — Tue, 26 Jan 2010 20:39:49 +0000

On the eve of the rumored announcement of an Apple Tablet, I started thinking about similar devices in Star Trek. In Trek lingo they are referred to as PADDs. For a show made in the 60s and 80s it’s interesting to see how their conception changed over time. In the original show the earliest padds were pretty basic, like clip-boards that the crew could carry around — making notations with a stylus. In The Next Generation and later shows the padds had touch screens, hard buttons, and an occasional stylus. They were designed in the early days of the internet, and before wireless networking — so there was little consistency of whether a pad was the keeper of the information, or if it was networked. Padds often seemed to be special purpose devices, so users had various versions of them.

They were said to run an operating system called LCARS — which was used on all interfaces in the show. The look and feel of LCARS was designed by Michael Okuda. Michael’s brief, from show creator Gene Roddenberry, was “that the instrument panels not have a great deal of activity on them. This minimalized look was designed to give a sense that the technology was much more advanced than in the original Star Trek.” (source.) It’s this abstraction that helped the padds fit into the show without needing too much detail or explanation.

Over time some of the interfaces got more complex and literal. It was geeky fun to try to look for UI details. At the time, they felt like a very cohesive system — like a real part of the Star Trek universe.

The tablets definitely caught people’s attention. There are screensavers, games, mockups, a theme to turn your Nokia phone into a padd — all sorts of ways for people to pretend they’re real. Unfortunately, because the original design brief was for an abstract UI, these interfaces loose much of the magic due to the details they’re forced to include.

Pretty abstract, but you can carry it around, too.

Smaller pad with hard buttons.

Image-based work on a larger padd.

Some padds were pretty technical.

Using a stylus.

All images from: Memory-alpha.org. (c) Paramount Pictures and/or CBS Studios.

Are Phones Leading OS Innovation?

David — Fri, 22 Jan 2010 01:26:02 +0000

Here’s a great piece on Engadget by Paul Miller: Editorial: 10 outdated elements of desktop operating systems. It’s a very interesting read, not just about the problems of existing desktop operating systems, but pointing out that some of the possible solutions are right in front of us.

What’s cool is the degree to which phones are leading the development of new ways for us to use computers. Why? Phones are not encumbered by legacy software, so it’s easier for them try something new. And they’re built around mobility, the internet, and data that’s in the cloud — so their approaches are very different. Plus, phones are not in research labs, but in the hands of millions of users, so designers had to make solutions usable and scalable. (That probably sounds hostile to design research — which I don’t mean at all!)

Paul acknowledges that his editorial is inspired by all the rumors of what an Apple “Tablet” computer may be, “..a new sort of consumer ‘computer,’ a second coming of the Macintosh that rethinks what a personal computer should be.” But perhaps part of what’s driving all these rumors is our collective desire for a new way to use computers. As computers and software more fully embrace the internet and the data stored there, we’ll certainly be seeing a lot more innovation in interfaces and interaction. It’s a future that will definitely be very diverse and exciting.