More Thoughts on Rethinking User Research

Dana Chisnell recently posted her thoughts on the ineffectiveness of today’s typical research methods when studying the social web. Her perspective on our current methods is spot on and it’s an issue that’s been in the back of my mind as well.

There are four challenges I believe we should address in order to evolve the way we conduct our research:

1. Natural vs artificial environments
We need to get out of the mentality that our users need to come to us and enter our shiny laboratories to encounter our products and provide feedback. Users visit and use our products everyday in their natural environments. They interact with us from their offices, homes and on the go. Speaking to them in these circumstances provides more detail about how our products fit into their lives than we would ever be able to get in a conversation in our labs.

Tools like Ethnio and GoToMeeting have been around for awhile now and many people are taking advantage of them, but too many (including me) are not using them enough.

We need to get out of the lab more and take our research into the homes and places where our users are. This can be done with remote recruiting, online meeting tools and other emerging remote usability tools, but it can also be done through home visits, participant diaries and more.

2. Multi-person vs. single person conversations
More and more of the products we develop are used in a social way. Either the user requires the input of someone close to them to use the product or they use the product with others in social settings.

For example, many families watch TV together and share TV sets and DVRs. To build a tool that helps them to manage this entertainment requires that we understand the tool will often not be used in isolation of other household members. The activities a user would perform to record a program or delete a setting effects more than just them, but the household that shares that device as a whole.

In Dana’s example of researching a tool based on financial and retirement planning, it became vary apparent that more than the participant in the room was involved in this activity. Without those other members present, the participant has to pretend thus making the validity of what we learn less reliable and useful.

We should look for ways we can gather data from the people that live in the influence circle of our participant and the area we are studying in addition to the participant themselves.

3. More Active and Generative Participation
Most research studies involve a user’s reaction to a manifestation of the product in question. Few studies actually engage participants as active members of the design process. I’m not lobbying that we eliminate evaluating our creations with users, but rather that we find ways to increase how active our participants are in our studies.

Liz Sanders has been dedicated much of her career to finding ways to increase the participation of our users in the design process. Her site is full of interesting ideas on how to help participants contribute in a way that makes their latent feelings, dreams and imagination more tangible. This kind of information could really change the direction of our products in ways we never could have imagined with traditional research.

4. Efficiency and Cost Savings
Stepping out of years of tried and true traditional methods can be time-consuming and risky at first. The more qualitative and rich data we collection can really add significant time to our analysis. While we evolve our methods and get closer to our participants and their influence circles we also need to keep an eye on being efficient and cost-effective. Whatever processes we test should be evaluated on their quality output as well as their time to delivery. The best research we do can still be a failure if it can’t get to the products teams in time to absorb it.

I’m really excited about Dana’s thoughts on this subject and helping to rethink how we do user research for the ever evolving social and connected web.

Cranky Talk Workshop

I had the absolute privilege of attending the Cranky Talk Workshop for New Speakers on Friday, October 22. The workshop exceeded my expectations and brought in all the elements they preached about good presentations.

I know the faculty — Dan Willis, Adam Polansky, Russ Unger, and Dan Brown — they are a great bunch of guys that are really knowledgeable, approachable and fun. They complemented each other well and made all my fellow workshoppers feel at ease.

The day started with a “stretch” exercise to get the creative and performance juices going. We were all asked to introduce ourselves in a silly and ridiculous way as indicated by a note in a lunchbox tin. The exercise was amazing for getting everyone in a safe and comfortable zone with the room.

We then went on to the learning portion of the program. Each faculty member presented their thoughts and ideas on how to create a great presentation. Following their talk, we were allowed to opportunity to share our feedback with them on how effective we found the material and the performance of the material. This was one of the most helpful portions of the workshop. Each faculty member had a wealth of information for us, they each had their own style of presenting, and they were very candid about their ticks and tendencies and how they overcome them.

My two biggest takeaways from the event were how important it is to 1) have a singular message and that the message can be as simple as “How does [x] work for designers?” and 2) make that singular message speak to your passions. Passion captivates attention and focus aids retention. The difference between a good presentation and a great presentation are these two things — focus and passion!

Learn more about the Cranky Talk Workshop at

Experience Frameworks Talk at PhillyCHI

This week I gave my first talk at PhillyCHI about Experience Frameworks. The purpose of my talk was to inspire and encourage other user experience professionals to put experience frameworks to the test.

I defined experience frameworks as models that can help designers frame and evaluate an experience. In this category I do not include frameworks that attempt to define and communicate design process, roles or responsibilities.

With this definition, there’s a lot of frameworks out there in the user experience community (and other communities) that can be really great tools. Some of them have been around for awhile, and some of them have emerged recently.

Here are a few that I’ve reviewed, including those that I shared in the presentation:

And be on the look out for more experience frameworks, like Cindy Chastain’s Story(Thinking). There’s a lot of great ideas in the community. Let’s put some of them to the test to see how they can be used as tools for designing great experiences.

Review my full presentation on slideshare:

An experience does not come from Design alone…

I’m currently working on a presentation to help introduce tools to my colleagues that may help improve the work that we do towards enabling great user experiences with our online and mobile properties. One particular point I am trying to make is that the web sites we may create do not stand alone in a user’s experience with our company. There are many other external factors at work that contribute to the experience they ultimately have. These external factors are important to recognize and understand in order to effectively design and build the web components we are responsible for.

While taking a break from my work I noticed Dan Willis’s post at about the role of Design and Web sites in user experience.

A Web site can affect emotion, but only fleetingly. A customer’s experience can involve a Web site, but even in the case of an online retailer, the Web site is only ever a part of the overall experience. So by itself, Web design is an impotent tool for affecting an organization’s brand.

That’s the thing though, Web design isn’t a solo act. It’s not an end unto itself. Design is one of the side effects of a solid problem-solving process that defines these key elements:

1. Who are your Web site’s most important users?
2. What are the goals of your Web site’s most important users?
3. Which of the goals of your Web site’s most important users can your organization address most effectively?
4. Is what you gain from satisfying those user goals worth what it costs?

I couldn’t have said it any better myself, and his great words will find a way into my work 🙂 Thanks, Dan!

Communicating Issues with Processes

Livia Labate and I have been trying out new ways to help communicate issues we observe with flows. Here’s a high-level example.

Documenting Issues with Process

Use this tool to document the experiences you observe directly from users or from other metrics. This can reflect the experience of an individual user or a composite of similar user behaviors. The diagram can help communicate troublesome areas of a particular process in the user experience.

The idea is…
The more unexpected activities we add to a process the more room there is for complication.
The more a user slides the more likely they may fall and not achieve their goal.

To address this…
We must remember to better manage expectations and assist users in completing required tasks so they don’t fall.

Preparing a Portfolio

It’s been almost two years since I sat down to really evaluate and articulate all that I’ve accomplished professionally. I worked on so many different projects and at different capacities it’s challenging to identify what I should concentrate on next.

With almost 10 years experience in UX it’s time to evaluate and expand. I’m beginning this journey by preparing an online portfolio.

Preparing an online portfolio is new for me, but will be an incredible exercise in identifying my best work, succinctly articulating my accomplishments, and broadcasting this to the world.

One thing is true with all of my endeavors… I plan them out. So, here’s the plan:

1. Identify strengths and skills to include
I’m starting with an assessment of the strengths and skills I wish to highlight from my work. Here’s what I’m looking to demonstrate:

  • design and user research
  • information architecture
  • interaction design
  • user experience strategy
  • usability evaluation
  • models and frameworks
  • collaboration and brainstorming
  • process development

2. Review past work for examples
Next I dig into my past work looking for examples of all of these strengths and skills. I’m not looking to categorize all of my work, but I am looking for good examples where I had a significant effort that I can speak to.

3. Pick the 5 best
Why only 5 examples? More is not better. I want to really shine without being overwhelming or too repetitive.

4. Write mini case studies
Once I narrow the examples, each will be described through a mini case study that covers:

  • purpose and goals
  • process and challenges
  • effect and result

In the end I should have a good idea of all that I’ve accomplished in the past 10 years and where I want to improve.

2009 Year in Review

All this review is a little delayed it was well worth the time. Despite the overall gloomy haze that covered most of 2009 for the nation, there were many things to be proud be proud of.

small checkmarkChaired for Pre-Conference at the IA Summit in Memphis, TN
With the help of fantastic folks like Samantha Bailey, Keith Instone, Mags Hanley, Jennifer Bohmbach, Donna Spencer, Stacy Surla, and Dick Hill, I took on the responsibility to organize and manage the efforts of the pre-conference workshops for the 2009 IA Summit. In addition to workshops, we also were able to hold the first-ever consortium in the history of the Summit. An awesome accomplishment with great success for the content strategy community.

small checkmarkRan Second Marathon in Baltimore
In 2008 I ran my first marathon in Philadelphia with my good friend, Regan. It was such a major accomplishment for the both of us in my hometown that we also decided to give it a go in hers. Read about the fantastic experience we had in October.

small checkmarkCompleted Full Year as a UX Researcher
In my quest to be the best User Experience Designer I can I chose to spend the past 18 months concentrating on understanding our users and how to best use that information to affect the products we provide. I’m very proud of the work I’ve done so far and the progress I’ve made as a new researcher and a designer.

small checkmarkMentored a Transitioning UX Designer
I had the privilege this year of providing guidance to a very smart and enthusiastic woman interested in transitioning to UX. This is really fantastic and rewarding opportunity. I’m very proud of the progress and hope to continue playing a role in her transition.

small checkmarkTraveled to London, Edinburgh, Zagreb and Hawaii
Like every year, we try our best to travel to new places while also making a stop in Zagreb to see family. This year was no exception. The UK is always a treat, but Edinburgh was amazing and I really hope we get the opportunity to live there one day. Enjoy the pictures from my trip to London and Edinburgh.

I can only hope that 2010 continues to be a year of great experience and accomplishment. Looking forward to whatever lies ahead.