The run time is 33 minutes.
Additionally the full transcript is below. To distinguish between us I've indented Karrie.Transcript
Good afternoon, Karrie, how are you, today?
Good afternoon. I'm well, how are you?
I'm fairly well. I was up a little late, today, I must admit. I got up at 4:00 p.m. I've only been out of bed for three hours, now, so I feel pretty refreshed. I'm sure by 4:00 a.m. this morning, I will be back to feeling dog tired again [laughs].
You are going to be a speaker at the forthcoming Emerging Communications Conference. I see you are Assistant Professor in computer science, at the University of Illinois. I want to ask; what do you do in that role? I know you could answer back, "I'm professor of computer science," but can you tell me what it is you are looking into, there?
I guess my role there is three-fold. Firstly, I'm a researcher. My main area of research falls into the category of social computing, but my primary interest is actually communication. That has been my passion for some time.
For some history, I actually started out in electrical engineering, as a graduate student, looking at building network switches for tele-media networks. I guess, one thing that happened to me, there, is I was lucky enough to see the first movie playing in Mosaic browser. After that, I realized that someone else was going to make the networks faster, the boxes smaller, and I wanted to focus on the ends.
That sort of led me into human computer interaction, which led me to a branch of it, called social computing. My main passion is actually trying to see how people communicate in our new network media environments. Those are the areas that I research in. That entails several different aspects.
One, we build new interfaces to actually allow people have different styles of back channels they may not have had before. Two, we analyze existing tools and social networks. Three, I named my group Social Spaces, so we're not tied to any specific thing, but we do a lot of different work with social visualization and different styles of visualizing patterns of communication and behavior.
That is sort of my role as a researcher at the university. My second role there is as an educator. I teach undergraduate and graduate classes in multi-media networking, human computer interaction, social spaces, and social visualization.
Third, I feel like my role at the university is also kind of like a community broadcaster to let people know that this is a valuable area of research. We haven't had HCI in its current instantiation at UIUC in a CS department, for a while. It's only been like this for about six years. I feel as if I have to go around and make it clear what we do and why it's important, to the whole community, not just to the university.
It's interesting; at the start there, you mentioned Mosaic. I remember Mosaic, too, but we probably shouldn't get all nostalgic. Do you remember Gopher?
Yeah, yeah, Gopher - I used to read news using TIN and RN. That's what really got me interested. I love the idea that there were these amazing public spaces, some of them that were quite anarchical. You could just go in and talk to people. You kind of felt this bizarre freedom.
Did you say, "networked media environment"?
Yes
Can you explain? I have a good clue as to what you mean by that, but could I have you describe what you mean by networked media environment?
Early on, that would entail Usenet. It would entail email, even mailing lists. Today, it goes further. Today, you can take it so far as to be games, like World of Warcraft, Second Life, but even things we build ourselves.
One example of something that we've built are the sculpture chairs; we put them in cafes. Someone can go to a café with a friend and talk to their friend, or they could talk to someone else who possesses this chair, kind of like being in the movie Being John Malkovich, in this case, were actually logging into a chair and having discussions with people in another space. It includes media environments that people are familiar with, on typical computers, but also things we build ourselves to try and see what cues are necessary to sustain conversation.
What cues are necessary to sustain conversation? Can you describe that? It sounds very interesting.
One example with our chairs - we had these chairs that had "Mr. Potato Head" faces. We were inspired by Tony Oursler, in his projection art pieces. What happens is you have this remote interface. You can log into the chair. You choose what your face looks like. It was very abstracted. One of the points of this was to talk to people when you felt isolated and had no one else to talk to, so you didn't show your exact face. You got to choose one of three styles, whether it be hand drawn, a cartoon-like face, or claymation.
Then, you are connected to the people physically present at that café. We're trying to think of different ways to actually include back channels. Traditional back channels might be something like the way somebody says, "Uh", "Uh-huh," or maybe with their gestures and what they're doing.
For example, if I were talking to you and doing my nails at the same time, which might imply that maybe I wasn't as interested, but that back channel is something we would like to transmit across a media'd space so people get some idea of what is going on.
One of the things we were looking at with the chairs was based on some type of behavior, could the color of the face change? Could something else indicate, abstractly, your level of attention or agreement, if you didn't have the "uh-huh's".
Another example of that is with another piece we have called "telemurals". We use abstracted video instead of photo-realistic video. A lot of video studies have shown that there are two main reasons people like teleconferencing with video. One is to see somebody's posture, just to get their general state. Are they happy? Are they sad?
Two, is the level of attention. By having just silhouettes of people, we can get at your posture, your mood, and you can get a general idea of somebody's level of attention, but they don't have to give up their privacy. The idea is how can we show things that are different from the way you see them in face-to-face, but still provide some understanding to people?
Probably, the closest idea of a back channel in media, as we know it today, is just the indicator that you're typing on Skype or MSN Instant Messenger. It's a back channel that someone's doing something or thinking, and that you're about to get a message, before you've gotten it, so you don't freak out.
It's interesting because video telephony has never taken off. It was pushed in the 1960's, the 1970's, 1980's, and 1990's. People have said because the cognitive load is too high. You end up spending more time worrying, "Am I in frame; Do I look okay," rather than concentrating on the conversation. Yet, there are important things; there are beneficial things to seeing each other. It depends on the context.
If it's your child, you want to see them. If it's business-like conversation, you may just wish to know the state or the posture, or in particular, how much attention is being given here. It sounds like a way of reducing out the parts we don't like, often in video telephony, the "Am I in frame; am I looking okay," and we don't really care what each other's color of shirt is. That sounds like very interesting work to signal that across.
Yes, and just to emphasize your point, you're probably not going to use a system like this to do tele-medicine, or to get instructions about how to run a space shuttle. The interface I described is more for social banter, for somebody that you know, or if it's somebody you're meeting for the first time; the nice thing about seeing an outline is that person chooses their own level of disclosure. They get to choose how much of themselves they'd like to display. They could start with something more abstract, something less stressful.
Are you aware of the work of Distance Lab, who I was interviewing the other week?
Yes, I actually went to graduate school with Stephan Agamanolis.
Are you aware he is a speaker, as well?
Yes, I am. I was very excited to see that. I love their shadowboxing piece.
When it comes to yourself, there is one project. I don't know if you led it or were part of it, but certainly, from the same department - it was Visaphone.
Yes
Could you describe Visaphone, and your involvement with that?
That was actually part of my PhD thesis. I'm going to be speaking briefly about it at the conference because it's been inspiring for a lot of our other work. In general, one of the points of Visaphone - let me start with the motivations.
One of the main inspirations was that somewhere we were doing a lot of teleconferencing back and forth. It was quite fascinating that people would stare at this big black box of a speakerphone, even though it provided no additional information.
One of the reasons for that, we think, is that people just are not used to speaking to disembodied voices. You are used to associating a voice to a body, to a being. We figured if people were going to stare at something, we might as well give them something useful, or something that might help them in another way to stare at.
A second inspiration for that was that sometimes when you're speaking to a distant location, because you can't see them, you don't know what's going on there, you can't tell if your voice is being heard. We wanted an interface that lets you gauge your voice with respect to somebody else's voice, as well. You won't have the scenario where you are yelling and somebody is being deafened at the other end because you can actually see the volume of your voice in relation to the other side's volume.
I just wanted to say that with Visaphone, what we were looking at was conversational dynamics between spaces, things like conversational dominance, turn taking, and also how people would use such a thing. If you had an object that was aesthetic, and let's say one day in your homes you could have objects that were reconfigurable instead of collecting porcelain china plates, if you had an object that was dynamic, what types of objects would you collect and what information would you put on them?
That sounds quite interesting. If you were asked what is exciting you, at the moment, in the sphere of communications, what would you say?
I would say, first, don't fear communication. Two, what I am getting really excited about, lately, is our work with sentiment analysis. We're doing a lot of work to look at levels of anxiety in communication. I guess that ties into fear, to some degree, like you said.
Looking at general moods, there is a lot of work, recently, in natural language processing. We can actually look at and predict, not with perfect accuracy, but just the level of mood in different communities. We are doing this, right now, with peoples' blogs. We're doing sentiment analysis in different Facebook scenarios.
One, from the research standpoint, I'm excited about the possibilities of doing this. Two, I'm excited just by this idea of a social mirror. For example, if you could show this to people, will they change their behavior?
Sorry, you said "field of communications" not "fear of communications" [laughs].
Yeah, I just popped you an instant message to say I said [laughter]... I said what gets you excited. I thought you might say, "I'm getting married in six months," so I tried to say, "What's getting you excited in the field of communications". Some people find my accent strange. You thought I said "fear of communications" [laughs]. You're thinking of all this sentiment-type stuff.
Yeah, your message to me is also another example of a back channel to the conversation, so that was fitting. Looking at a lot of the sentiment analysis for happiness, as well as anxiety and fear, I think, is fascinating. I think there is a lot to work on there. I think one of the reasons I'm excited about it is because if we do it right, in the beginning, it won't be misused, later on. Depending on how you build these interfaces, you don't want to scare people.
One of the things I get very sad about is interfaces that try to do a lie detection. If you do anything like that, you can seriously destroy conversation if you incorrectly assume something is a lie. I like my interfaces to be abstract enough where you let people make their own assumptions about what is going on. You don't say this is anxiety or you don't say this is happiness, but you just show different metrics and let people make up their own minds.
You have plug-ins for Skype, which say somebody is lying. You don't like them.
I don't like those, no. We are working on some of our own plug-ins for Skype, where we look at patterns of communications. Basically, after this message, what it would have shown was our turn taking, our interruptions, if we speak over each other, if we argue.
We have some other versions, now; they're not working perfectly, with speech recognition. It also makes these visualizations of text that was spoken, so you can see words that were spoken a lot as bigger, words that weren't spoken as much as smaller.
A tag cloud
Exactly, so it's kind of like a varied version of a tag cloud, where people can annotate onto, later.
When do you think the Skype plug-in will be available?
One of my students if finishing up her masters in May. I'm assuming she'll graduate, so I'm going to say May.
Instead of saying this person is lying, or anxiety, how do you display or indicate something, without being explicit, to let people make their own make up their own mind? I was imagining the color red showing if somebody is lying. What do you mean? How do you obscure this?
Brainstorming, the first thing you would do is have different parameters, make them different colors. Sometimes we try to avoid red just because it signals alarm or anger. You map each of these to different colors.
The simplest example - you have a bar graph with several different parameters. You just let these bars rise and fall, but you don't label them. As people experience interface, they'll make up their own minds and their own decisions about what it means to them.
It's okay if the green bar means something different to different people. People should adapt the interfaces as to how they want to use them. We don't say that this is how you have to talk. There is no right or wrong. It's basically up to you and we provide the information.
That sounds kind of interesting. At the moment, you can record your calls. You can have CDR's (Call Detail Records), start time, and end time. It kind of begins to point to the possibility of applying metrics to people. If you call "X" person, you get talked over X% of time. Although you might not like it being explicit, this person tends to show anxiety 20% of the time. Maybe you could group your buddy list according to certain conditions.
I would rather not label something as anxiety, but maybe have everyone with a color stamp. Over time, let's say you enjoy talking to people who have a lot of green and blue. That is what you'll remember. You're not going to say, "This person is anxious". You'll say, "I like this fingerprint of the person and I'm going to talk to more people like that," or as an imprint, as an alternative to an avatar.
That sounds like an interesting means of discovering other people, by fingerprinting people with these conversational metrics, if it's okay to call them that. Then, maybe being able to say, like in Skype, you can set it to "Skype Me" mode. I'll speak to anybody, but it would be nice if it were maybe granular. I could speak to anybody with this type of - click a contact you already know and say, "I'm willing to go on 'Skype Me' mode for people with the similar fingerprint".
Yes, and one of the other things about this is it also teaches you things about yourself. It also gives you some level of self-awareness as you see your own color imprint change with respect to different conversations. You might start wondering, "Do I really talk that much?" Maybe I talk more than I thought I spoke. "Why do I interrupt everybody?" Maybe I didn't realize I was being so intrusive. "Why don't I talk at all? Why am I so silent in conversations?" Sometimes, one of the ways our interfaces have been described has been I showing things you know, but don't realize you know.
It sounds as if it could be used to help a few couples out. [Laugh]
It's hilarious that you say that. When we showed Visaphone at SIGRAPH, a couples' counselor actually came up to me and asked if we could set this up in his office. That was not our intent, but we've had a lot of interest from that domain.
Yeah, "You always interrupt me". "No, I don't". "You always do, you never listen to me," and so on.
It was really a happy surprise to see that by making something physical and visible, the level of self-awareness and level of self-fear, almost, was surprising and exciting, at the same time. When you see something like this, people are a bit more cautious about how they behave. It probably does affect interaction, to some degree.
Something very similar is in the field of speech and hearing. Mirror therapy is one common way to get children to speak with speech delays. In many ways, this is an alternate form of a mirror.
That sounds really nice. I must admit, earlier in the conversation you mentioned an art piece, possibly which Visaphone came out of. Which art piece did you mention, or was it a figment of my imagination.
It really wasn't, but because of a figment of my Attention Deficit Disorder, that ...
[Laughter] I thought you mentioned an art piece, and I thought, "Did I hear an art piece?" Which one, because although it might be a bit strange because I'm meant to be a telecommunications engineer and have an engineering background, I've actually been very interested in people representing emerging communications in art form, just because I often see paths to commercialization, people so loosely experimenting and that's kind of what we need.
Talking of which, I'm not sure you're exactly tasked with monetization, commercialization, and so forth, but what kind of opportunities do you see being spun out of the work that you have been doing, what you call commercial nature, if any?
Some things that we're looking at are actually building applications for cell phones, which actually combine conversations and games, things to break the ice, things to introduce people. Some of the more obvious things are applications for Facebook.
Also, this doesn't have exactly to do with direct communication, but we're building a lot of interfaces for showing people how they use energy in their homes. We're tying these interfaces to the community, anonymously, so that people can converse with each other about how their visualizations look and what they're doing to curtail their energy usage.
In some ways, we're using them in what I call social catalysts, as imprints to encourage further conversation. By no means are we trying to replace face-to-face conversation. We're trying to provide different channels and back channels to get things across, but also to encourage more discussion. A lot of our interfaces are meant to be interrogative, to sometimes even ask more questions.
You asked earlier about an art piece. Visaphone has been considered an art piece, in the past, and so has Chit Chat Club, which is a telesculpture that I mentioned. A lot of work kind of blurs the boundaries, although the work isn't originally set up as an art piece, sometimes, the pieces to end up in museums. I like this idea of free flowing between them because you get different people to see the work and you get a lot of perspective. In going back to my point about social catalysts, you get more people involved in the conversation.
It reminds me of David Troy, who is a speaker. He developed FlickRVision and TwitterVision. He ended up displayed in the New York Museum of Modern Art. It was actually another speaker, I can't remember at this point, who ended up with their piece displayed in an art museum. That is fairly interesting.
Can you expand a touch more, on the cell phone work you were meaning, like give a concrete example of what can be done?
One example might be for people that might want to archive a conversation and what it might look like. I gave those archival examples earlier; taking that into the game domain might have each person represented as a little centipede. As they move about the screen, the longer they talk to this particular person, the longer their centipede gets. They tend to cluster around people they talk to a lot. All of a sudden, you start seeing things like group patterns. You start seeing behavioral patterns, conversational patterns.
For me, one of the reasons that we're doing this work with phones - if I could have my way, these would be on interactive t-shirts. The way people leave their computers on 24/7, the way people leave cameras on 24/7, it's not unfeasible to think that you might leave a microphone and speaker on 24/7 in your home, and what way would you want to show different patterns, obscure different patterns for privacy, and how might you want to see behaviors like this, over time.
I'm really thinking about the idea of when you do have microphones on 24/7, whether they are represented on cell phones or on your clothes, to highlight your social network as you walk down the street, much in the same way kids wear t-shirts to show their favorite bands or their favorite products; how might these visualizations of conversation behavior be used to show your identity?
That sounds very fascinating, just because it ties into the whole digital identity field. I'm not sure if that is a field of yours, at all.
Identity is a big field in psychology and sociology, and social psychology. The idea of identity and the differences between your fragmented identities online, offline, even in your different face-to-face communities is a big research piece that we're only slow chipping at the moment. It is something we discuss a lot.
The idea of leaving a passive microphone on doesn't seem unreasonable. Actually, what I often wish for - it was an idea that came from Martin Geddes; we need push-to-hear.
Often, we make a call and another person is already in a conversation with people in the room. Wouldn't it be good if you could push a button and just hear for two seconds? Obviously, the other person at the other end gets a blip or warning, but push-to-hear, because what we're trying to do is close time and space between people. That is a nice form of presence, is to hear what's happening at another location.
One interesting take on that, just to flip it over a bit, is with Visaphone people oftentimes use the visualization with the sound off. In that case, it would be push-to-see what you hear, without actually hearing it. You would see abstracted circles of volume of sound. You would get some idea of presence, some idea of activity, almost the equivalent of a baby monitor for adults, which actually took privacy into account.
Like a baby monitor for adults. Presence was a hot topic about two to four years ago, mostly about three, actually somewhere in between. We had the likes of USB-powered plants that when your loved one came online it grew to show they were online, or lit up. I'm not sure - the work you've been doing ties into presence. Do you see evolution in presence? Maybe the word has been misused because many people, by presence, mean availability management rather than sense of other at distance.
I think that what has happened is that other people have altered the word because people became frustrated with not that much progress in the area. I think people have looked at it from different perspectives. There has been a lot of work, recently, in disruption; when is the ideal time to disrupt someone, if there is one. Other areas are, like you said, availability with instant messenger. Other things are just bringing people together, what people might want to meet each other. Facebook does this by trying to match people up on its own, trying to introduce people.
One thing that I would like to bring into the discourse about this idea of disruption is that disruption has this connotation of being bad. What if we think of disruption as a good thing? We just start by thinking of it in a different way. There are great things that happen from disruption. When I get disrupted from work, I get to have a cup of coffee and talk to somebody. I would say that's a success.
[Laughs] Disruption is good.
Yeah, so maybe we've been thinking about it the wrong way. I think it's good that we don't hear the word presence as much, anymore. It just means we've started looking at different parts of it and started using different words to describe the ideas of presence.
Does it amaze you that fifteen years later we still have online and offline, busy and don't disturb? It seems rather as if we are still living fifteen years ago in that department.
True, it is interesting but maybe that's the best solution. We haven't shown that yet. Sometimes with people - doors in my office, if my door is closed, people aren't as inclined to knock. If my door is open, they're more likely to walk in. We do those cues face-to-face. Whenever we move to different mediums, we tend to appropriate what we know and use those before we can move on and find different social cues to use.
It doesn't surprise me. I think we'll also come up with different things. Probably ten years ago, if you sent somebody a message an instant messaging and they didn't reply for two hours, you would probably get more upset than you would, today, knowing that people walk around, get cups of coffee, and go to the bathroom.
In the very beginning, we expected it to be so close to face-to-face, that if somebody had replied to you, within a second each time, and they didn't once, you immediately thought that something was wrong, didn't like you.
I agree with you. It's kind of happened with email, now. You're kind of expected to reply the same day. Now, if you reply the same week, it's still okay. It's even getting to the point where if you don't reply, it's still okay.
Email is actually stopping to work, for me. In many ways, I could use more of a pull medium than a push medium, in that respect. With all these spam filters out there, they're catching some important mail. My email has turned into my newspaper. I glance at it as opposed to reading it the way I used to.
I agree with you and I think it's getting like that for all of us, just because of the sheer rise in communications volume across growing numbers of channels. We seem to be seeing the long tail of communications happening, more means of connecting with each other.
Speaking of which, I don't suppose you're aware of Tony, with his Sense Networks?
No, I'm not familiar with his work.
He's from Columbia University.
Oh, Tony Jebara - yes, we went to school together.
Did you know he was a speaker?
No, I didn't know. He's amazing!
Exactly, and I was chatting with him, yesterday, and it really overlapped with many areas that got me excited. When you mentioned discovery of others, Tony is very much focused in that area. It's some amazing work going on, there.
The reason I mention it was just to begin closing loops between you, Stefan at Distance Lab, Tony at Columbia stroke Sense Networks. I just see you three - I was going to say you should sit at the same table together, at lunch [laughs], but maybe it's better since you know each other, to spread out. I just see commonalities there. If conversations are not happening, they should be.
I should begin finishing off by asking you - I've asked you what gets you excited. Do you want to share any last thoughts on where you see opportunities in the communications space? At the beginning, you said your passion was the communications field. Where do you see communications, for example, going over the next two, three, or five years? Where is the excitement? Where do you see development or experimentation, or where would you like to look, personally, over the coming years?
What I'm intrigued about is the creation of new types of interfaces that might even create new social etiquette or new social behavior in communication. Right now, we have thousands of years of experience speaking face-to-face, and we tend to adopt that in different mediums.
The telegraph came along. We took some of that, but we created some of our own mediums on top of that. Then, we had radio, telephones, email. We had little emoticons with an email. I'm curious; if you include visualization with communication, how might you influence communication, encourage different types of communication, and what types of channels that we're not familiar with or even aware that we use, can we show using visualization? I would love to see some sort of visual feedback with communication. I think we're slowly on the road to that.
That's fantastic and I look forward to the Skype plug-ins. I hope they're good. I very much appreciate your time.
Thank you very much. I look forward to seeing you in San Francisco.
See you in March. Thanks, bye.
Bye
