97 UX Things
Plan User Research with the Customer Question Board (feat. Julia Cowing)
Julia Cowing discusses her chapter about a method to choose the right UX research method for your questions about end-users
Sponsored by Watch City ResearchWatch City Research is your trusted UX research partner
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Julia Cowing discusses her chapter about a method to choose the right UX research method for your questions about end-users
Sponsored by Watch City ResearchWatch City Research is your trusted UX research partner
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Hi, everyone, and welcome to another edition of the 97 UX Things podcast. Dan Berlin here, your host and book editor. I'm joined this week by Julia Cowing, who wrote the chapter 'Plan User Research with the Customer Question Board'. Welcome, Julia.
Julia Cowing:Hi, Dan. Nice to be here.
Dan Berlin:Thanks for joining me, can you tell us a little bit about yourself?
Julia Cowing:Love to. My name is Julia Cowing. I am a user researcher, user experience researcher and I live in Manhattan. Currently, I work at InVision, which is in the visual collaboration space, including design prototyping and virtual whiteboards. Before InVision I was at Google, MailChimp, and a couple of companies in the financial industry, including Bloomberg and Citibank.
Dan Berlin:That's quite the career trajectory, that's interesting! Can you tell us about that? How did you discover UX and how did you wind up where you are today?
Julia Cowing:I definitely took the scenic route. I got my bachelor's degree in Chemistry, and a Master's degree in Digital Media. It all makes sense from a bird's eye view. In undergrad with Chemistry, I was really interested in the composition of things, and how things get broken down and consumed by people. But, I also had an interest in Visual Communication, which led me to a graduate degree in Digital Media. I didn't know it at the time but I was basically creating the perfect combo of science and art. So what I do now as a User Researcher is, I break visual things down to see how people consume design and how they understand things.
Dan Berlin:Yep, break visual things down. I like that. Great! You said you're a researcher, where do you tend to focus your research, if at all?
Julia Cowing:Qualitative, user research methodology- either foundational or evaluative.
Dan Berlin:Yep. Are there particular research techniques or methodologies that you like?
Julia Cowing:Yeah, I definitely like the Jobs to Be Done method for foundational research. For evaluative research, there's nothing like usability testing. I think we need to know it like the back of our hands.
Dan Berlin:Agreed. Why Jobs to Be Done? What do you like about that?
Julia Cowing:I think Jobs to Be Done helps people to zoom out and think about customer problems, divorced of solutions- so being solution-agnostic. When we do this type of foundational work, we don't have to do it too often, because people's problems don't change very much. Their core purposes really don't change much over time. So rigorous foundational research- we don't have to do that all the time, but we do need to do evaluative research.
Dan Berlin:Great. Well, thanks for that. Your chapter, 'Plan User Research with the Customer Question Board'- can you tell us about that please?
Julia Cowing:Love to. It provides a step-by-step method for figuring out what to do next in a way that's inclusive of everyone on your team. Alignment, prioritization, and transparency are so key in healthy, high-performing teams. This method is all about like rowing in the same direction, as well as setting up research insights to be more impactful. The question board is a two-by-two whiteboard, physical or virtual, with questions that a team has related to the customer, or the customer experience. The questions are plotted into four different quadrants, and these quadrants each have suitable research methods for the questions based on opposing characteristics, qual versus quant, and attitudinal versus behavioral. In seven simple steps, you can do this- the first step being to send out a calendar invite to a group of people with different functional expertise. A cross functional group is always best to get a more you know, well-rounded, collective brain. That calendar invite would be explaining what the workshop will be about (for example)-'This will be a collaborative session to figure out what research we need to do next so we could prioritize our backlog. Bring your top five questions about the user or user experience'. This invite is already setting up the workshop and starting the prioritization muscle. Yes, we have a lot of questions, but what are your top five? I think that forced prioritization really does work.
Dan Berlin:Those questions that people bring- do you give them any guidance on the types of questions they should be bringing, or do you see any patterns there?
Julia Cowing:I think very similar to the Crazy 8s brainstorming in collaborative design methods, it is more like what are YOUR top five questions and forcing team colleagues to really understand what's really blocking them and not being worried about grammar or spelling, or whatever. It's just plotting those five questions down so that everyone is able to figure out what they really need to answer so that they could do a great job at their job.
Dan Berlin:Yep, yep. Okay, so you have the calendar invite, telling them about the questions- what comes next?
Julia Cowing:For step two and three, I'll talk about them together on the day of the collaboration, have everyone put their five questions on five different sticky notes, then have one person start the discussion. That first person places their five notes in a central area, and explains why these questions are important to answer. Then, grabbing from the K-J method, everyone who feels they have similar questions, puts their sticky notes next to those first five. Then, the next person talks about their questions not yet discussed. They put their note in a central area, and everyone with similar questions, places their notes next to that, and so on, and so forth, round-robin until everyone has voiced their questions. At the end, there should be a few question clusters that you can name if you want with short titles, or you don't have to. By doing this real-time affinity mapping, the team sees how they are all asking the same things, but in different ways, and the team starts seeing the mutual data that's needed for everyone to do their job well.
Dan Berlin:Yep. It sounds like you are putting similar questions together, but you're not categorizing them quite yet.
Julia Cowing:Correct. That's right.
Dan Berlin:Why that first step? What helps with that first step?
Julia Cowing:It really helps the team to align and see what's on each other's minds. It's that collective brain- the collective process of clustering together, so a lot of questions being asked, are very similar with each other.
Dan Berlin:It is always eye-opening, when you realize that your co-workers have the same questions as you, that would be really helpful for design, yeah. You mentioned the K-J method. Can you fill us in on that?
Julia Cowing:Yeah. The K-J method- Christine Perfetti and Jared Spool talk about it. A Japanese researcher discovered it and it's all about a quiet synthesis of data that a group can do. So first doing it quietly, it's the diverge method, and then converge, discussing why people put different concepts together in different clusters. Then, with that cadence of diverging and converging, you get to better alignment behind the affinity mapping.
Dan Berlin:Yep, interesting, thanks for that. What comes next? You have your affinity map of questions- what's the next up there?
Julia Cowing:Next in preparing for step four and five, you want to create a two-by-two grid with an X axis and a Y axis to get four quadrants. For the X axis, label it 'attitudinal' on the left and 'behavioral' on the right. On the Y axis, label it'qualitative' on the top and'quantitative' on the bottom. The idea behind step four, is that you want to take one question cluster at a time and ask the team - is this a'Saying' or a 'Doing' question? Do we want people to answer this question by asking them- either verbally, or in written format? Or, do we want to see what their actions are about, you know, what they are doing? For asking same questions, put the clusters anywhere in the middle of the left two quadrants. These are attitudinal questions. For seeing action doing questions- put those clusters anywhere on the right two quadrants. These are behavioral questions.
Dan Berlin:Attitudinal versus behavioral- something that comes up a lot in UX. I liked the way that you just described it about saying or doing, but can you tell us a little bit more about the difference between these two for folks who may not have run into this before?
Julia Cowing:Yeah. People don't always do what they say and they may not know this. A good example is, I can tell you that I'm on a diet. I want to lose weight, I want to exercise. But if you watch me, you'll see that, you know, I'm hand to mouth, munching, I'm not jogging, I'm not doing exercise. And so it is this attitude that might not always be aligned with the actions. And this is data, you know- we want to know both.
Dan Berlin:Yep. Yep. Understanding what they're saying, but not doing and understanding exactly what they're doing.
Julia Cowing:Right.
Dan Berlin:Yep. All right, we got we have the affinity diagram, and we're starting to move them out into behavioral versus attitudinal, or qualitative versus quantitative. Can you tell us a bit more about that and the next step here?
Julia Cowing:Right. At this point, you should have clusters on the left- which is attitudinal and on the right, which is behavioral. In Step five, in the same manner, asked the team-is this question cluster about insights? Because we're in the beginning of the software development lifecycle, and we just need to start somewhere? Or, is it about validating the insights? We need, you know, a larger sample size? These are questions, usually around how often is something happening? So for insights, those are clusters that should belong in the top two quadrants, depending where the cluster was left to right. So these are qualitative questions. And for how-often-is-something-happening type questions, put those clusters on the bottom two quadrants- and these are quant questions. Now you have those four groups of questions in the four different quadrants. And step six, is voting. Have everyone vote for their top cluster they want answered-what answers will help them get unblocked the most? And then finally, step seven is choosing the best quadrant method. The top left quadrant is qualitative-attitudinal methods, where you only need small sample sizes. So interviews, field studies, contextual inquiries, jobs to be done are methods or types of methods. And the top right quadrant is also qualitative with small sample sizes. But these methods are to understand user behaviors, their actions, what they're doing, because what people say is not necessarily what they do. And then the bottom-left quadrant is attitudinal but with larger sample sizes, so surveys would fit in this bucket. If we partner with market research- this would be a good time to partner with them to do these studies. The bottom-right quadrant is behavioral, but with larger sample sizes, so analyzing usage stats, or A/B testing would be methods here and we want to partner with Product Analytics.
Dan Berlin:Yep. Do people have trouble with the qual versus the quant, or the insights versus the validation when you're describing that, at all?
Julia Cowing:I think that to check where people are in the Software Development Life Cycl(SDLC), it helps to make sure t at you're doing the right me hods. Usually in the beginn ng phases, it is more qual, and towards the end of the S LC, it is more quantitative.
Dan Berlin:Yep. What are some of those quantitative methods that are on your mind, you mentioned surveying, but anything else quantitative that people can use?
Julia Cowing:Well, you can also do usability testing in a quantitative manner. Doing large scale usability testing, doing experimentation- like fake door experimentation, A/B testing, or multivariate testing helps, or beta testing.
Dan Berlin:How about alignment? We started off this conversation with the thought that this type of activity will help get people in alignment. Can you tell us about that? How does this activity get people on the same page in terms of the design trajectory?
Julia Cowing:I love that question because alignment is so important in order to make sure that everyone is understanding the prioritization. And there's transparency in the research roadmap. Alignment is figuring out the type of data that's needed, and how representative you want the data to be. Then they're systematically finding out what the research method is. So like design, you know, research never ends. So the team shouldn't think about this being a one time thing. Ideally, you want to do all four quadrants. It's always just a matter of how much time and resources that the team has available to collect the data, and how confident they want to be. We always want to seek congruence and data collected from different means. The more everyone is able to see the data, and to collect the data and plan together, the more aligned everyone will be. It's setting up research so that there's more impact in the insights.
Dan Berlin:Absolutely. Mixed methods are a big conversation these days in terms of doing the right type of method at the right time, as you said, insights in the beginning and validation later on. This is a wonderful way of really giving people a way to make mixed methods actionable.
Julia Cowing:Definitely, I like the way you said it. Being actionable, I think, is very important so that everyone can pull together and know what to be done.
Dan Berlin:You mentioned something that came up earlier in my mind about time. One of the first things that came to mind when you started talking about which method to choose is a big factor- that is not only where we are in the project timeline, but how much time and budget do we have? How does that fit into your conversation?
Julia Cowing:I think it's a big part of figuring out which method in that quadrant you can do. Is there a one-person researcher on your team that could do it? Or, can the whole team be able to collect the data together and synthesize together? How much time do you need? Do you need it scrappy or do you need it more rigorous? So the collaborative planning part- I think it's very culture-changing, because you just want to do it in a cadence. Early and frequent is always better than having these huge studies that takes up too much time.
Dan Berlin:Absolutely. And then you realize that your design trajectory is way off, and then you're in trouble. It's all about early and often. Outcomes. Let's talk about outcomes here. So we've run this workshop with co-workers and we're looking to get alignment, we have all these questions out on the table... how do we come to consensus and have an outcome from this workshop?
Julia Cowing:I think a big part of the customer question board is that the outcome is to get everyone changing into a learning mode. The more we get into learning by doing, and understanding that we need to triangulate the data, that is strategic in itself. It's just steering the ship to change the culture in that way. Because the more we're able to triangulate the data and see congruence in the data, the more confident we are in product direction. A great outcome is the confidence. Taking confidence of how the team feels about the roadmap is an outcome, if it does improve because of the research- because of the workshop and the research planning.
Dan Berlin:Are there typical outputs from the workshop- you have mentioned the roadmap in terms of what the design roadmap will be- but are there other outputs that you have from the workshop?
Julia Cowing:Each of the quadrants has different research methodology and each of those research methodologies do have outputs. So let's just say Jobs to be Done- if that's something that came out of the workshop, let's say more of this foundational understanding is needed for jobs to be done. You get segmentation of your audience, and so the output is a clear persona development or segmentation. In other quadrants there are other things like- for usability testing, you could get top five most important things to redesign so that it's easier to use, or easier to understand.
Dan Berlin:Yep. So it's really less about the output from the workshop itself and more about the output of the activities that you decide upon in the workshop.
Julia Cowing:Yes, yes.
Dan Berlin:Okay. Was there anything else about the customer question board that you were hoping to convey here today?
Julia Cowing:Well, you know, I feel a little bit guilty for not doing anything special here. I just connected existing frameworks and principles- like you could Google 'research methodologies', and you could find different things. But in order to change culture, and increase the company's velocity towards customer-centricity, to be a learning culture- they all just made sense to be put together. It is about helping the company become more customer-centric.
Dan Berlin:Absolutely. Anything else about the customer question board that you're hoping to convey here?
Julia Cowing:I would love to hear how it's used and if it's helpful for people and for teams. Whoever out there is trying it, let me know, send me a LinkedIn message or whatever! It's all about helping to encourage a user research mindset and culture, and to have more focused conversations on customer problems versus methodology. Just focus on the questions and by default, you're gonna understand what methods to use.
Dan Berlin:That's great, because a lot of folks get into a cadence with their methods, and they just continue doing that. But I think what you've described here allows people to break out of that, and to allow them to ask themselves what other methodologies should we be thinking about based on the questions we have?
Julia Cowing:Yes, yes, thank you.
Dan Berlin:How about a tip? Do you have a career tip for folks either breaking into UX, or who have been around for a while?
Julia Cowing:I would say, just like Lean UX prototyping, we could treat our own careers the same way. Lean UX is about learn by doing- making small, incremental changes and iterating. In that same vein, for our own careers we want to keep learning, iterating, and improving incrementally. If you're in a job where you just don't think you're growing, then take action. Do something, and be okay with not knowing, because every move is good. It's giving you more data to recalculate. So don't be afraid of getting it wrong, because wrong is also data.
Dan Berlin:Right! That's a great point. When we have all these new experiences, we get all these new data points. We're also learning about what we're passionate about. You may start one thing and realize you're not exactly passionate about that, but start something else in UX and realize that's your gig- whether you're moving from design to content, for example, or content to research. There's so much opportunity in UX.
Julia Cowing:Yeah, and definitely, what you just said about passion- it is about following your passion. At the end, it will all make sense- very similar to like how my passion at the time with Chemistry. But then I also liked visual communication. I didn't know what was going to happen. But in the long run, I created my own curriculum, because I followed my passions.
Dan Berlin:The wonderful thing about UX is that people have different passions, but there's a place for so many different types of people in UX, whether you are a designer, or a researcher, or UX writer, or an IA- there's just so many opportunities.
Julia Cowing:Yes, definitely.
Dan Berlin:Alright. So my guest today has been Julia Cowing, who wrote the chapter 'Plan User Research with a Customer Question Board'. Thank you so much for joining me today, Julia.
Julia Cowing:Thank you, Dan. Thank you for giving me the opportunity!
Dan Berlin:Happy to- this has been a wonderful conversation! And everyone- you've been listening to the 97 UX Things podcast. This is your host, Dan Berlin. Thank you for listening.
