How do you ensure the consistency of blind wine reviews from one flight of wines to the next?

Ask Dr Vinny

Hello there! I'm Dr. Vinifera, but you can call me Vinny. Ask me your toughest wine questions, from the fine points of etiquette to the science of winemaking. And don't worry, I'm no wine snob—you can also ask me those "dumb questions" you're too embarrased to ask your wine geek friends! I hope you find my answers educational, empowering and even amusing. And don't forget to check out my most asked questions and my full archives for all my Q&A classics.

Dear Dr. Vinny,

You’ve written that for Wine Spectator blind tastings, the wines are organized into flights by varietal, appellation and/or region, and that the ratings reflect “How highly our taster regards each wine relative to other wines.” My question is, does the rating reflect each wine relative to other wines for each flight? If yes, how do you compare wines from different flights related to the same appellation or region?

—Sergei

Dear Sergei,

Good question. Let me take a step back and make sure you can picture the tasting process in its entirety.

Each reviewer has his or her own "tasting beats," or wine categories, like Bordeaux, or New Zealand, or Port. That gives our tasters a chance to develop expertise in their regions. When they sit down to a flight of wines that the tasting coordinators have arranged, they're bringing all of their experience with that category to the table.

The wines are tasted in what is called a single-blind tasting, in which the reviewer knows the varietal, vintage and/or appellation of the wine, but not the producer or the price. We stand behind this process because we believe that the very best wines reflect where they’re from. The wines are disguised in bags, to keep this process as fair and objective as possible. I like to say it’s a very democratic way of reviewing wine, where everyone has an equal chance at a great score.

But we go a step further. Two, actually. First, each flight begins with a benchmark wine, which has already been reviewed. That’s the only wine that’s not bagged, and the score the wine had been given is marked clearly on the bottle. That’s a really helpful reference tool for the taster to use throughout the flight as a way to calibrate.

Finally, each flight also has at least one ringer somewhere in there. A previously rated wine is hidden among all the others, and the taster doesn’t know which wine it is until all of the scores are locked in and the bags are removed. They can check and make sure they are on track with how they previously rated something. It’s a helpful tool to make sure we are being consistent with our scores from flight to flight.

—Dr. Vinny

Blind Tasting Ask Dr. Vinny

More In Dr. Vinny

What’s the difference between Tuscany and “super Tuscan” wines?

Wine Spectator's expert Dr. Vinny shares a short primer on Tuscany's key wine designations.

Aug 8, 2022

What’s the minimum amount of alcohol in wine?

Wine Spectator's expert Dr. Vinny explains how "wine" is legally defined in terms of …

Aug 2, 2022

Why are some of Wine Spectator’s reviews "web only"? What does that mean?

Wine Spectator's resident wine expert Dr. Vinny explains why some of Wine Spectator's …

Jul 26, 2022

What’s the deal with wine “legs”?

Wine Spectator's expert Dr. Vinny explains the science behind wine "legs," or "tears," and …

Jul 18, 2022

What is “dry” wine? Aren’t all wines wet …?

Wine Spectator's resident wine expert Dr. Vinny explains what makes a wine "dry" vs. …

Jul 12, 2022

What does "père et fils" indicate on a wine label?

Wine Spectator's expert Dr. Vinny explains a few of the French terms you might see on a …

Jul 5, 2022