Tragon’s new closures report, transparency, and the marketing vs. science clash

Tragon, a consumer sensory testing firm based in San Francisco, just released the fourth installment of a study into how consumers feel about natural cork versus screw caps. Tragon conducted surveys in 2004, 2007, 2011, and 2013. The 2013 report shows that consumers are more accepting of screw caps than they’ve been in the past. Still, the bottom line is the same now as it was in 2004: people prefer natural cork, perceive cork-topped wines as higher quality, and think they’re more appropriate for fancy occasions. Those conclusions held true across the US, Germany, and Australia, though the Aussies see screw caps as being very nearly equal to cork in nearly all settings.

I have no trouble believing that the average wine buyer prefers cork and thinks that it’s classier. Aesthetics, tradition, and familiarity are important. Cork wins on all three accounts. Given the exact same wine under different closures, my experience as a human who interacts with other humans tells me that most wine geeks will choose the screw-capped option, most non-wine geeks the cork.

Here’s where I have a serious problem accepting Tragon’s report. At first glance, their study seems to show that the average-Jessica wine consumer cares more about closure type than where the wine came from or what variety it is. Really?

Maybe lots of people (non-locavore people) don’t look at country-of-origin because they just see the brand and label design without reading the fine print. But closure is more important than whether the wine is white or red? This is hard to believe. Do people go to the store saying “I’m looking for a wine with a cork” or “I’m looking for a wine with a screw cap” more often than “I’m looking for a red?”

It took me a few reads to realize that the problem is probably with how the survey questions were worded** The summary report says that closure beat out country of origin and color in “character importance.” That tells us nothing about what question consumers were actually asked, but let’s imagine that the survey item was something along the lines of “How important are the following characteristics in terms of telling  you about a wine’s quality? Rank in order of importance.”

Whether a wine is red or white isn’t at all important in telling me about its quality. Red wines aren’t always better than whites or vice-versa. Duh. Similarly, for country of origin, a wine from California or Italy or New Zealand can be either very high quality or very low quality; not helpful. But if you show average-Jessica wine consumer two identical bottles, one corked and one screw capped, she’ll identify the corked one as higher-quality.

Nomacorc, makers of the leading plastic cork-like closure, sponsored a study in 2012 that found that American consumers only care about closures when they cause a problem: when a wine is corked, when they have trouble opening the bottle, etc. Those results might appear to be at odds with the Tragon study, but I’m not sure they are.  The Nomacorc report says that consumers don’t think much about closures. The Tragon study asks people to think about closures, then asks them whether they think of corks or screw caps as higher-quality. 

I don’t watch basketball. If someone asks me whether I think about basketball, I’m going to say that I only think about basketball when I’m annoyed and inconvenienced by traffic created by a basketball game (anyone who’s been on the Washington State University Pullman campus on a Thursday game night can probably relate). But if someone gives me a list of basketball teams and tells me to rank them in quality, I’ll come up with some kind of list based on what I’ve overheard from friends and news reports.

Something still doesn’t quite compute here. Tragon’s report shows that price was the most important factor in “character importance” — sensibly enough — but that $10-15 wines were ranked higher in “character importance” than $15-20 or wines over $20. I’d expect perception of quality to increase steadily with price, but that’s not what the graph shows. If Tragon shared their methods and their data — if they published the survey itself and graphs documenting actual results instead of just the slick summary — we’d know how to understand their results.

But instead, since this is a private company doing research on behalf of Wine Vision — an industry conference in which Amorim, the world’s largest manufacturer of natural cork, is a major sponsor — we see only the highly polished conclusions. So instead of research that adds to global understanding, we have research that supports Amorim’s market position, just as the Nomacorc study supported Nomacorc’s market position. A shame.

My conclusions? One: surveys are always more complicated than they appear at first glance. Two: how we ask questions has an enormous effect on the answers we get. Three: when private companies don’t share the details of their study methods or data, misunderstanding follows. Four: I’m not sure that science and marketing must always be at odds (though, frankly, I think they probably are) but, when marketing means no transparency, science loses.

**Neither the public summary reports nor the Tragon “research methods” web pages (frustratingly rich in graphics and poor in information) give any details, so all of this is speculative.