Designing the UIfor the Internet of Things Tim Lynch Design lead, mobile and consumer products Nuance Communications @clampants | @NuanceInc Voice interfaces for connected experiences #IoTUI #SXSW
A little bitabout me… (Yes, we beat the record.) I live in Boston, where I am the lead UX Designer for web/mobile devices at Nuance. I love Austin, have family here, and almost moved here, yet somehow I ended up living in a city with 800 ft. of snow in a month and below 0° F winters. So thank you for having me and letting me thaw.
A little bitabout Nuance… We innovate technology to reduce the distance between want and get. “Tweet this and say not a bad way to start the day” “Bring up the record for patient Nicole Redman”
A little bitabout our design team… We are a diverse team of interaction and visual designers, user researchers, dialog designers across the country, redefining speech experiences across handsets and tablets, automotive, television, desktop, IoT, wearables, augmented reality, gaming… The list goes on. Did I mention it’s fun?
Narrative design iscore to what we do as designers. We have the ability to pull from general design experience to create meaningful voice experiences for users.
We – designersand consumers – face challenges each day when designing for the Internet of Things: • Smaller (or varying) screen sizes • Designing for transmodal experiences • Disparate device branding and cues • User expectations of immediacy • And so on.
When we talkabout devices that make up the Internet of Things, what comes to mind? Light bulbs, thermostats, smart hubs, speakers, Crock Pots, refrigerators…
These devices arethe poster children of connected devices – the IoT we know and love today. They are generally “smart devices” that… a) Are loaded with sensors b) Can “talk” (convey data and information) to us and each other
But when weconsider all the things that are taking in and communicating data, we often overlook the things we already know –things that were smart and sensor-laden well before the phrase “Internet of Things.”
Sometimes, these thingsact as the interface for other connected devices. Other times, these things unto themselves have become connected devices.
Then there are connectedthings that feel like science fiction… By 2020, there could be 200-billion connected things, from smart dust to entire cities.* How will we interact with it all? * http://www.intel.com/content/www/us/en/internet-of-things/infographics/guide-to-iot.html
This shift inecosystem presents a fundamental challenge. As people living in this world of connected experiences, we need to interact with these things in personal, meaningful ways… and our devices need to interact with us similarly.
Small screens Pebble Time Considerdifferent form factors (screens). Screens Honeywell WiFi Smart Thermostat No screens Kohler Moxie Showerhead & Speaker
As these devicespervade our everyday lives – and become smaller, pushed into the background, and more personal – our interactions with them become more enmeshed in the day-to-day.
Different inputs andoutputs. From Nicolas Nova’s “Curious Rituals” - https://curiousrituals.wordpress.com/
We’re in thewild-west with these devices. Each device seems to put a stake in the ground with some novel way to interact with it. Touches, gestures, and swipes on one device trigger one response, while on another they trigger something completely different.
Method’s Henri (viaFastCoDesign) - http://method.com/work/ixda15 “With the rise of smart objects and the connected home, we’ve found that products increasingly need to communicate even without a screen, through things like light and sound patterns.” –Daniel Nacamuli
Speech is the simplestand most human communication method. Using speech, the user interface becomes almost invisible and the experience is as natural as part of the day-to-day.
The goal isto communicate with devices as we would each other – as humans. How Voice Activates and Advances the Human-Computer Relationship - by Clifford Nass - http://goo.gl/YYozKs Dialog evokes meaning, identity, emotion, and trust.
Let’s pretend youhave a connected mattress. The Chattress Hey friend :)
Foundations of the‘voice interface’ The Chattress You seem down today…what’s wrong? ASR Recognize the words NLU Meaning behind the words Dialog Appropriate response Nothing, mattress :(
Intelligence is thesecret sauce that permeates the entire experience, influencing the conversational partnership we have and creating a much more personalized experience.
Intelligence & personalization Theintelligent Chattress Intelligence Context Knowledge Memory ‘Smarts’ can manifest itself through context, memory, and knowledge.
A speech experienceitself, though – without a holistic and thoughtful design – won’t make for a good user experience. Speech experiences will fail (or fail to be adopted) when they: • Fail to meet people’s expectations • Don’t take into account context or other modalities • Are hidden or unclear in purpose • Ignore conversational norms • Treat errors as dead-ends
Understand expectations Leverage the strengths of speech Partnerwith other modalities Introduce yourself Frame the scope Support what is natural Provide conversational feedback Identify errors as opportunities Deliver a consistent point of view To remedy this, consider these speech design factors.
1. Understand expectations. Peopledon’t want to speak to their devices simply to have a conversation… they want to get things done. Design with those things in mind. Police Dog, Tess (via the State Library of New South Wales, NZ - https://flic.kr/p/5TJoyH)
2. Leverage thestrengths of speech. Speech can empower goal-oriented tasks, streamline existing flows, and improve the experience in certain contexts.
3. Partner withother modalities. Rarely should speech be thought about as the only modality. Use it to support and amplify other modalities.
4. Introduce yourself. Makespeech obvious and well-integrated into the full experience. Once discovered, people will experiment.
5. Frame thescope with guidance. The promise of natural language is you can say anything. A challenge is you think you can say anything…
6. Support whatis natural. The “natural” in “natural language” can mean lengthy phrases… but it can also mean simple fragments. Natural language should encompass structured commands, but be able to extend out to full, grammatically correct sentence structure.
7. Provide conversationalfeedback. Speech systems should follow our own conversational norms, conveying they are listening and understanding.
8. Identify “errors”as opportunities. “Sorry, I’m not hearing anything. Try checking your mic settings.” Audio “Sorry, what time was that?” Recognition “Did you mean two people or two o’clock?” Interpretation “Ah…I can’t help you with that yet, but try this…” Dialog It’s OK if something goes wrong – as long as people understand what happened (and what to do to fix it). (Tweet this)
9. Deliver aconsistent point of view. Dialog, TTS, visual, audio, interaction, scope, content, form factor… should all work in concert across devices.
So, if youtake away one thing, it should be… Thoughtfully-designed speech systems allow us to meaningfully interact with our connected devices.
Thank you! Tim Lynch Designlead, mobile and consumer products Nuance Communications @clampants | @NuanceInc #IoTUI #SXSW