Tuesday, May 8, 2012

Remote Research in Practice, Pt. 1

Why Remote Research?
Rightpoint is located in Chicago. I work out of Portland, OR. Our clients are spread across the United States. Many of our clients are multi-national corporations and have employees sprinkled all over the globe. We firmly believe in a user-centered approach to researching, designing and validating our projects’ user experiences. While we love contextual inquiry and face-to-face user interviews, for example, the vast majority of project budgets prevent travel to conduct that research extensively in client offices in the US, much less client offices located in Hong Kong, Buenos Aires or Frankfurt.

How can we conduct insightful moderated user research programs, usability studies and prototype validation we know is invaluable to a successful user experience project? None of this would be possible without our embrace of remote research.

In fact, we deliberately do not have a lab because we feel it’s an unnatural environment that fails to provide information about a user’s context. When we aren’t able conduct user research in someone’s work environment, we’re working remotely. For our remote research studies, we use a web-based, screen-sharing conferencing service.

With remote research, there are clear trade-offs.
1. You are going to lose fidelity in terms of the information you are gathering. For example, it’s hard to tell visually and spatially what kind of environment someone is working in (although you can tell to some degree if it’s a loud space or one where someone is constantly interrupted).
2. Engaging with someone over web conferencing software makes it more difficult to develop a strong emotional connection. You can mitigate this to some degree by overemphasizing the empathy in your voice and listening more carefully to her or him.
3. At this point in time, you are limited primarily to testing web-based experiences and not mobile devices or other kinds of user experiences. However, tools like Revelation and dScout are finding ways of incorporating qualitative research via mobile devices.

If you are willing to embrace the limitations of the medium, it offers several advantages.
1. You can more easily and inexpensively test and interview a more geographically dispersed audience, including international participants.
2. Recruiting and scheduling participants is generally easier because they don’t have to come to a specific lab location and you can meet them at a time and place that suits them. While not as desirable as contextual inquiry in terms of offering the best portrait of a user, you can get some sense of a users’ environment that you wouldn’t be able to gather in a lab setting.
3. It’s easy for clients and stakeholders to observe your work and understand it’s value. I always encourage clients to observe our research activities whether remote or not. Not only does it provide them with perspectives from actual users they may not be aware of, but those perspectives and the subsequent insights they witness reinforce the value of user research in their minds. In my experience, it’s much easier for busy stakeholders to jump on an hour-long shared screen session than it is for them to get to a lab or another physical location.

As digital user experiences will continue to evolve, remote testing eliminates some of the challenges posed by geography, time and budget. As workforces become more geographically, linguistically and culturally diverse, we now have a window into understanding and developing empathy with that diversity. Right now, that window is a little hazy and hard to see through, but it is translucent enough to allow us to make out the shapes on the other side.

Over a series of four blog posts, I’d like to share the approaches we’ve developed at Rightpoint for specific kinds of user research activities and the lessons we’ve learned from our experience.

Part 1. Tools of the trade 
Part 2. Recruiting and preparing participants
Part 3. Conducting specific research activities
Part 4. Working with international participants

Part 1. Tools of the trade

Before we discuss technologies, here are the fundamentals you’ll need to get started.

Make sure you and your participant(s) have a high-speed connection. Without a fast enough connection, the technologies required simply won’t work. However, in 2012, I would assume that most of the professional clients you are working with will have adequate connection speeds in their office environments. If you are using a recruiter to recruit non-professional participants, include a question for a high-speed connection a pass-only requirement in your screener.

Use headphones or a headset. This leaves your hands free to type or write notes.

Try to work with a colleague. Preferably with your colleague taking notes, while you lead the interview or test. This is no different than non-remote interviews or testing.

Our primary tool for conducting remote research is a web-based conference and screen-sharing tool. There are many services available (such as WebEx, Go2Meeting, etc.) and we do not endorse any particular service as most of them all essentially have similar feature sets and pricing.

Regardless of the service you use, the technologies that are essential for conducting any remote research activity are the following:
A reliable conference calling service. Voice communication is your primary means of conducting remote research. Research facilitators need to pose tasks and questions. Participants need to be able to answer questions and describe their thoughts and reactions. The verbal conversation is from where our research insights emerge. When conducting simple user or stakeholder interviews, the phone is often the only tool you’ll need. 
The conferencing part of this is useful because it allows clients and others to observe the sessions. When conducting remote sessions, I like to work with a colleague who is taking notes (and also might be in a different location) while I guide the session. In terms of additional observers, I always advocate that clients observe research sessions. Almost nothing shows clients the value of user research and the insights it provides besides observing a research session. Remote testing makes this easier. Just be sure to tell your clients up front that they are there as observers and not participants or session leads. No talking!
It’s helpful to have VOIP options with your service provider. In addition to regular telephony, it eliminates a barrier for international callers because they can call in through their computer. 

A screen-sharing tool. This will allow you to show and observe users using a current website, browser or mobile prototypes, online card-sorting tools, and collaborative sketching tools. Most screen-sharing tools are integrated into conference calling services. They are much speedier and more reliable than they were in the recent past and I have rarely encountered significant lags that would cause either user frustration or have a significant impact on quantitative data gathered from usability testing tasks.
Screen-sharing tools also offer you an interesting perspective. Unlike sitting with a participant in a contextual inquiry, when you are observing a user via screen-sharing, you are seeing them interact with an interface or prototype through their eyes. Their screen is your screen and you are watching them click on links and retrieve information as they see it. In spite of the technological mediation, I like to think of screen-sharing as a way to redevelop a specific kind of empathy with users.

Recording. All of the major online conference services provide synced video and audio recording services to record your screen-sharing and conference call. Recordings are essential for reviewing when conducting analysis on your tests as well as for evidence to display to clients of specific user behaviors and insights. Make sure you hit record when starting your session! 

Can be important, but I’ve personally used it rarely:
Video conferencing. One of the things that we lose when conducting remote research as opposed to contextual inquiry or interviews in person is the extra layers of information you get from directly seeing a user’s environment (if not in a lab setting) or their facial expressions. Video conference can be a tool to get at least some of this information back.
However, video conferencing can be tricky for two reasons:
Not everyone has a built-in camera on their computer, and
In spite of our past dreams of the future videophone, not everyone is comfortable engaging via video conference (David Foster Wallace discussed this extensively in Infinite Jest way back in 1996, http://kottke.org/10/06/david-foster-wallace-on-iphone-4s-facetime).
That said, I believe at least the first of these will increasingly become less of an obstacle. Video conferencing as a component in remote user research does offer us the ability to regain the stream of facial expressions as a qualitative input. Further, it allows us a small window (well, smaller than sitting with them) into users’ working environment.

Next time we’ll talk about preparation. Most of your participants have probably never been involved in a user research study. I would venture of the small subset who have, a tinier percentage have been involved in a remote research study. I’ll explain how we prepare participants as clearly and thoroughly as possible before they get anywhere near their computers for a remote research activity.

(Thanks to Gemma Petrie for her comments on this series!)