Online Quali Quant gets real at Qual360 North America
Invoke Solutions’ Wayne Goodreau and Dialsmith’s David Paull preview the interactive, live research event they’re hosting at Qual 360 NA and chat about how technologies are converging to make real-time, online quali-quant studies a reality.
How many market research conferences give their attendees the opportunity to observe a live research study in action? At the upcoming Qual 360 North America, delegates will get to do just that. With the help of a new, innovative approach that integrates online technology from Invoke Solutions and Dialsmith, Qual 360 delegates will get a front row seat in a virtual viewing room to observe and guide a live online focus group whose respondents will go through a series of qual (verbatims, 1-on-1 chats, on-the-fly follow ups, etc.) and quant (moment-to-moment dial testing) exercises. The topic of the study is to determine the DNA of a successful summer movie trailer but the study’s design, approach and collaborative use of technology will be of as much, if not more, interest to delegates as its topic.
Meshing online qual and quant in a live environment, like what will be on display at Qual 360, is state-of-the-art stuff. So, to preview the Qual 360 session and give some insights into the approach and the technology driving it, we chatted with the two co-hosts of the session: Wayne Goodreau, Vice President, Research & Insights at Invoke Solutions and David Paull, Founder & CEO at Dialsmith.
Q: For the upcoming Qual360 Atlanta workshop, you will be leading a live research session that evaluates two different summer movie trailer formats. Can you briefly preview what event attendees can expect to get out of their participation in the workshop?
Wayne: Honestly, I am pretty excited about this event. Attendees will be able to watch a real, live session with participants and view data as it comes in in real-time. I hope attendees will get out of this session what our clients do. Doing a qualitative session with 100 attendees and being able to analyze not only closed- but also open-ended data may seem like an impossible task to those unfamiliar with this methodology. I have gone into sessions before with stakeholders skeptical that we can even do it. But I hope attendees walk out knowing this is possible. That we can get closed-end data, real honest verbatims and dial testing data over the course of an hour-long session. And I want them to not only walk out thinking this could be possible or that this could be valuable but also that this represents a real next step in evolution of market research.
Also, I hope attendees enjoy it. I really do enjoy doing these sessions and I hope that comes through when attendees come to this workshop. At the end, I hope they can say they learned something and they had fun. That’s typically what I want whenever I do any of these sessions.
Q: Why would a researcher want to use Invoke’s Large Scale Focus Group solution as opposed to a traditional in-person focus group or an online survey?
Wayne: While traditional focus groups give researchers and clients the deep, emotive verbatims needed to get behind numbers – the “why” behind the “what” – they also carry with them certain limitations. Traditional focus groups, for example, rely on a small number of participants. Relying on such a small number often has stakeholders “behind the mirror” basing their takeaways on one or two people. Not only does this bring the veracity of the findings into question, but it also can create a lack of consensus as you may be focusing on one research participant while I may be looking at another. Also, focus groups bring their own sets of biases into play. Phenomena such as groupthink and dominant voice are common to focus groups. Finally, the cost of doing a set of focus groups properly and the time needed to properly run them are high. Running a single, anonymous, large-scale focus group in real-time with hundreds of participants helps to overcome these (and other) obstacles.
Q: How does the live online focus group experience compare to an in-person focus group?
Wayne: Well, I have always regarded these large-scale focus groups as combining the best of qual with the best of quant. You get the deep-emotive verbatims and the ability to react to shifts in conversations through on-the-fly questions and 1-on-1 chats, and you get to combine this with a group of up to 350 people at once, which gives you a feeling of confidence in the results. You also get real numbers. Because we can ask both open and closed-end questions, you not only get valuable verbatims but you also get percentages and the ability to sub-segment data which you don’t get with traditional qual.
In terms of the actual experience, it’s a pretty dynamic environment. Our researchers (me included) travel to client sites and sit with them as the data unfolds in real-time. They typically invite pretty large teams of key stakeholders to view the session. Because travel isn’t necessary, getting all these stakeholders in one room is more doable. And then as we are watching the data come through, something pretty amazing happens… We all get a chance to weigh-in and bring our own perspectives to the table, which creates a very collaborative atmosphere. Once the session ends, stakeholders often walk away with some insights already in-pocket.
Q: What advantages are there for researchers conducting dial testing online versus offline?
David: Online dial testing allows you to reach many more respondents in a much shorter timeframe and generally at lower costs. While there are certainly times when offline focus groups and dial testing are best, more and more, we’re finding that we can get what’s needed online with quicker turnaround and a much more diverse sample set. As opposed to a survey where you have to sacrifice most of the qualitative feedback, online qual through Invoke allows us to blend qual and quant online just as we have for years in an offline environment.
Q: Are there specific types of research that lend themselves to this type of methodology/solution?
Wayne: In terms of session focus, we deal with a pretty wide variety… We run a good amount of sessions focused on specific pieces of stimuli such as advertisements, packages, concepts, websites, programming, etc. These work well because we can get our stakeholders their key metrics while also uncovering areas of optimization through the verbatim responses. However, the ability to get deep with consumers and react to shifts also enables us to successfully run more exploratory types of sessions (brand positioning, co-creation, etc.) This methodology is pretty flexible. On top of all that, we can also incorporate add-on tools such as dial testing or mobile to round-out an offering and get to what our stakeholders need to know.
Q: How closely do results from the online dial testing (using tools like Perception Analyzer Online) track to results seen from in-person (or offline) dial testing?
David: In A/B tests of the same content tested with our offline Perception Analyzer dials and online with our Perception Analyzer Online slider, results have been remarkably similar. With the physical offline dials, we find respondents tend to dial more to the extremes whereas with the online slider they don’t go quite as far toward the end points. But, the overall sentiment is very, very similar and when we plot the results as a line chart we see similar shapes to the lines.
Q: How have solutions like Invoke’s Large Scale Focus Group changed the game for clients who are targeting markets in Europe, Asia and other locales around the world?
Wayne: International is an interesting beast, especially when it comes to qual. Running International qualitative studies has always been challenging. From travel to finding translators and moderators in-country, it’s a pretty big endeavor for US-based companies. We can apply our methodology and tools to International work so that our stakeholders can talk to people from all over the globe from the comfort of their own office. We work with trusted translators that often join us for these sessions. And we can bring together internal teams as well, no matter where they are. Also, if we are running multi-country studies, all this data can be consolidated on the back-end to give a holistic picture while still enabling us to filter the findings so we can understand country-by-country differences. This makes International qual work more efficient and often less costly. Sessions that might have taken weeks (or even months) to conduct can be conducted in one day in many cases.
Q: How has integrating with Dialsmith’s Perception Analyzer Online changed Invoke’s offering?
Wayne: Like I said, we have done sessions in the past that focus on stimuli such as ads or programming. And these sessions have always provided very valuable qual/quant information for our stakeholders. Consumers have traditionally helped us to uncover concepts that work and don’t work for our clients and we can also help them understand tweaks that can make their ads, programs, etc. better. Perception Analyzer gives us another layer of data to provide for our clients. Now, not only can we have discussions with our clients after they watch, for example, an advertisement, but we can almost watch along with them. Perception Analyzer enables us to uncover specific moments that work and don’t work. Combining that type of pinpointing with our approach to understanding “why” helps us give our clients even more specific insights and recommendations.
Q: What has Dialsmith learned through working with technology partners like Invoke?
David: While Dialsmith offers our own end-to-end survey solution, we find that in many cases, clients already have a preferred platform. That led us to focus in on the core value we can provide, which is to be able to easily integrate our online dial testing element with those online survey and community platforms that our clients are using. Invoke’s online qual solution was a natural extension of that.
Once things get set up the first time, integration is pretty seamless and we find that both respondents and clients have a great experience. Partners like Invoke also help ensure these methods are available on the widest scale. It’s a matter of bringing together the best in breed for what each partner does best and no one trying to be all things to all clients.
Q: Why should market and media research pros be sure not to miss the Invoke/Dialsmith workshop at Qual360 Atlanta?
David: Dial testing of media is a tried and true method that’s here to stay. For those mainly familiar with it through traditional focus groups, this will be a really interesting look at emerging trends in the blending of online quant and qual. We’re committed to making this the kind of session we’d want to attend and ensuring everyone learns something new and has a good time.
Thanks to both David and Wayne for their time and for giving us a look at what’s in store for the delegates at Qual 360 North America. To learn more about this session or see the full agenda for the conference, visit the Qual 360 conference site.