Tragon’s new closures report, transparency, and the marketing vs. science clash

Tragon, a consumer sensory testing firm based in San Francisco, just released the fourth installment of a study into how consumers feel about natural cork versus screw caps. Tragon conducted surveys in 2004, 2007, 2011, and 2013. The 2013 report shows that consumers are more accepting of screw caps than they’ve been in the past. Still, the bottom line is the same now as it was in 2004: people prefer natural cork, perceive cork-topped wines as higher quality, and think they’re more appropriate for fancy occasions. Those conclusions held true across the US, Germany, and Australia, though the Aussies see screw caps as being very nearly equal to cork in nearly all settings.

I have no trouble believing that the average wine buyer prefers cork and thinks that it’s classier. Aesthetics, tradition, and familiarity are important. Cork wins on all three accounts. Given the exact same wine under different closures, my experience as a human who interacts with other humans tells me that most wine geeks will choose the screw-capped option, most non-wine geeks the cork.

Here’s where I have a serious problem accepting Tragon’s report. At first glance, their study seems to show that the average-Jessica wine consumer cares more about closure type than where the wine came from or what variety it is. Really?

Maybe lots of people (non-locavore people) don’t look at country-of-origin because they just see the brand and label design without reading the fine print. But closure is more important than whether the wine is white or red? This is hard to believe. Do people go to the store saying “I’m looking for a wine with a cork” or “I’m looking for a wine with a screw cap” more often than “I’m looking for a red?”

It took me a few reads to realize that the problem is probably with how the survey questions were worded** The summary report says that closure beat out country of origin and color in “character importance.” That tells us nothing about what question consumers were actually asked, but let’s imagine that the survey item was something along the lines of “How important are the following characteristics in terms of telling  you about a wine’s quality? Rank in order of importance.”

Whether a wine is red or white isn’t at all important in telling me about its quality. Red wines aren’t always better than whites or vice-versa. Duh. Similarly, for country of origin, a wine from California or Italy or New Zealand can be either very high quality or very low quality; not helpful. But if you show average-Jessica wine consumer two identical bottles, one corked and one screw capped, she’ll identify the corked one as higher-quality.

Nomacorc, makers of the leading plastic cork-like closure, sponsored a study in 2012 that found that American consumers only care about closures when they cause a problem: when a wine is corked, when they have trouble opening the bottle, etc. Those results might appear to be at odds with the Tragon study, but I’m not sure they are.  The Nomacorc report says that consumers don’t think much about closures. The Tragon study asks people to think about closures, then asks them whether they think of corks or screw caps as higher-quality. 

I don’t watch basketball. If someone asks me whether I think about basketball, I’m going to say that I only think about basketball when I’m annoyed and inconvenienced by traffic created by a basketball game (anyone who’s been on the Washington State University Pullman campus on a Thursday game night can probably relate). But if someone gives me a list of basketball teams and tells me to rank them in quality, I’ll come up with some kind of list based on what I’ve overheard from friends and news reports.

Something still doesn’t quite compute here. Tragon’s report shows that price was the most important factor in “character importance” — sensibly enough — but that $10-15 wines were ranked higher in “character importance” than $15-20 or wines over $20. I’d expect perception of quality to increase steadily with price, but that’s not what the graph shows. If Tragon shared their methods and their data — if they published the survey itself and graphs documenting actual results instead of just the slick summary — we’d know how to understand their results.

But instead, since this is a private company doing research on behalf of Wine Vision — an industry conference in which Amorim, the world’s largest manufacturer of natural cork, is a major sponsor — we see only the highly polished conclusions. So instead of research that adds to global understanding, we have research that supports Amorim’s market position, just as the Nomacorc study supported Nomacorc’s market position. A shame.

My conclusions? One: surveys are always more complicated than they appear at first glance. Two: how we ask questions has an enormous effect on the answers we get. Three: when private companies don’t share the details of their study methods or data, misunderstanding follows. Four: I’m not sure that science and marketing must always be at odds (though, frankly, I think they probably are) but, when marketing means no transparency, science loses.

**Neither the public summary reports nor the Tragon “research methods” web pages (frustratingly rich in graphics and poor in information) give any details, so all of this is speculative.

8 thoughts on “Tragon’s new closures report, transparency, and the marketing vs. science clash

  1. Pingback: Afternoon Brief, March 17 : WIN Advisor

  2. The results that puzzle you are actually the product of a conjoint analysis, wherein consumers are presented with a series of choices in different combinations of closure, price, country of origin and color; then asked which they prefer. The level of importance measure is based on he timpact the variance in that measure has on the choice. So in some respects the results are not going to resemble a decision hierarchy for a particular purchase occasion.

    Increasing importance within the variable “price range” probably means “positive impact on purchase decision” in this case. This would explain why the lowest price and the highest price fall in the bottom or negative impact zone (the slide doesn’t give the scale) – because a very low price may be a proxy for low quality and a very high price is, well, too high a price and an impediment to purchase.

    I believe the reason that red/white has a very low importance level is because most wine consumers drink both. Hence in abstract (as opposed to a particular purchase occasion) the fact that a wine is red or white does not rule it out as a choice.

  3. Erika,

    I am surprised and extremely disappointed with your attack on the integrity of Tragon’s research. You contacted me on Thursday of last week (March 13th, 2014) and I quickly volunteered to have a phone conversation with you to clarify our research methodology. You were unavailable. I agree that no research is perfect nor answers all of the questions. These studies are designed to understand consumer perception and we have a variety of measures and methodologies that we use. In the early years of this tracking study (2004, 2007, and 2011), we provided more detail on our methodology, including conjoint, and it has changed little since we first began.

    Based on the questions in your e-mail message last week, it was clear that you are unfamiliar with discrete choice conjoint research. This methodology is used to help understand consumer choice behavior and when presented with a set of options, we can derive the key drivers that most influence purchase interest. The unique combination of choices is based on an experimental design. In our research, we used closure, wine type, price, and region/origin. There are many good and easily available scholarly articles on conjoint methods.

    You also state that Wine Vision and Amorin paid for this research.
    This statement is simply not true and I would expect any writer to check facts before publication! These groups have been aware of Tragon’s research for a number of years and they invited me to participate because thought it would add a “facts-based” consumer perspective to the discussions, something that is sorely lacking in the wine industry. As background for you, Tragon fully funded the 2004, 2007, and 2013 research. We first designed this research because we observed several high-end wineries quickly moving to screw cap closures without checking in with their key target consumer on how this might change their perception.

    Our original hypothesis, like yours, was that the “wine knowledgeable consumer/wine geek” (higher purchase price, higher frequency usage, etc.) would be more accepting of screw caps. However, what we discovered was just the opposite and we have multiple data points to support this position. The wine knowledgeable consumer was even more interested in natural cork closures than consumers who purchase primarily on price or are less involved with wine (non-geek). We included red and white wine in the conjoint as our hypotheses was that screw caps may be more appropriate for white wines. We found that also was not the supported by the data.

    The issue of natural cork versus screw cap closures is an interesting one and it will continue to be hotly debated. Different types of closures (natural cork, synthetic, and screw-cap among others) can co-exist but wineries would be wise to understand the different perceptions that they convey at the point of purchase. Consumers can see a cork-style closure versus screw cap at the shelf and yes, it does impact purchase behavior. Screw caps continue to be polarizing, especially for the USA consumer and in other markets that we are currently studying. No one piece of research provides the full answer but each study can provide a window into the mind of the consumer. I am hopeful that as you continue in your career, that you seek out scientifically derived data to support your conclusions and your writing.

    Rebecca N. Bleibaum
    Tragon Corporation

    • Christian and Rebecca, apologies for my slow reply while I’ve been traveling. Yes, Rebecca, I did contact you — a phone call from NZ to the States on such short notice wouldn’t have been practical for me even had I been available, though I appreciate your willingness to speak with me. The extra materials you forwarded to me and your response to the follow-up questions I emailed, unfortunately, didn’t provide me with any information beyond what was available in the public summary, which did make it seem as though you were being deliberately oblique.

      I don’t see myself as attacking the Tragon report. Pointing out a problem with privately funded research, yes; claiming improper methodologies of the Tragon report, absolutely not: I have neither the information nor the expertise to make that judgment.

      My objective — which perhaps I failed to make clear — was to try to make sense out of how we (the public, on the outside) can see two different reports in the news, one saying that American consumers really care deeply about closures, the other saying that American consumers don’t care about closures at all.

      Based on Christian’s comment, it sounds as though I was misled by the term “character importance.” What is “character importance?” The report never says. I thought that “character importance” meant “how much character does a wine have?” It sounds as though Christian is saying that “character importance” refers to “would I personally buy this wine?” The term “character importance” may have a defined meaning among wine conjoint analysts, but the term just doesn’t have meaning for me.

      In terms of funding, if Wine Vision/Amorim wasn’t involved, then why is their name prominently on the front of the report? I guess I found that confusing.

      For me, the bottom line here remains: based on the summary report, it is REALLY hard to make sense of these data in the context of other, conflicting reports. I’d love to see more transparency in communicating with the public.

  4. Very interesting post. I would actually have to disagree on the number of points. First, I’m a wine geek, and I would take cork over screw top at any moment. Second, I observed people putting down the bottle of wine because it had a screw top. And I heard it from the number of people “ahh, this wine has a screw top, it must be a bad/cheap/plonk wine”. Lastly, if you are talking about the average consumer, the quality is associated with the price of wine only up to the certain level – once the price exceeds what people think is reasonable, they stop distinguishing between the wines. If someone doesn’t understand how you can spend $50 on the bottle of wine, $100 bottle will not have the higher perceived quality than the $50 bottle of wine.

    One thing I do agree on is that the quality of the answer is directly related to the quality of the question – if you ask bad questions, you will get the bad answers…

    • Talkavino, I’m fascinated by the incredible variability in folks’ feelings on cork vs. screw cap. An informal poll among my friends told me that the bulk of people I know in Washington state and Oregon — all casual wine consumers, admittedly better educated than average — either don’t think about closures or prefer screw cap. But friends from the midwest told me that they won’t bring a screw-capped wine to a party because they’ll be told that they brought cheap wine.

      In my ideal little fantasy world, we’d trust winemakers/businesses to make choices that made sense for their wines and stop using closures as a purchase decision-making factor…

      • well, this works with many other products, but there are a two important factors we need to take into account when it comes to wine. For the casual wine consumers, the factor is perception. To this group, there is nothing simple about wine. The label, the review, the rating, shape and weight of the bottle, “rich looking” versus “cheap looking”, and, of course, the closure are all parts of the perceived image and all should match.
        For the wine geeks and oenophiles, it is the emotional connection to the wine. As a wine geek/oenophile, I don’t have any perception issues (at least I hope I don’t), so I will accept the winemaker’s choice of closure no matter what wine is that (even the Screaming Eagle), but due to the emotional connection, my preference is the cork – it just feels different when you open the wine enclosed with cork.

  5. Pingback: Wednesday’s Meritage – #MWWC8 Last Reminder, Water into Wine – A Hoax, Corks and Twist Caps, and More Wine in Numbers | Talk-A-Vino

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s