The Process of Q Calibration
Author | Jeff Chean Date | October 09, 2017
I am a Q Grader. On SCAA founder and former executive director Ted Lingle’s Excel spreadsheet of Q Grader test-takers, I was #35. So I was licensed shortly after the program started.
A Q Grader is a person “licensed” by the Coffee Quality Institute (CQI), after taking a number of classes and passing an array of sensory tests, to grade coffee and (as part of a panel of three cuppers) assign a score/grade to that coffee that will impact the market’s perception of its value. The higher the Q score, the higher the price that the coffee will demand in the market. While this was the plan behind the advent of the Q designation, what it has come to mean, as discussed below, is something different.
Every three years, a Q Grader has to go in for a recalibration.
It sounds like going in for routine maintenance on your car and, in a way, it is. With the introduction of the Q system, a lingua franca for specialty coffee was introduced, allowing people from disparate countries and cultures to communicate what they had found in and thought about a particular coffee.
So, why a recalibration?
Over time, people change. These changes may be based on their experiences or alterations in their bodies. Maybe some licensed Q Graders cup coffee once a week, while others cup multiple times a day, and still others cup very infrequently. All of these people, regardless of how often they cup, carry the Q License designation. So, having a recalibration makes sense if what Q holders say is to be taken seriously. Passing the recalibration demonstrates, theoretically, that the Q holder hasn't lost "it" — that they are still aligned with other Q holders; that everyone who holds a Q License is still on the same page of the script as their peers.
If you're not on the same page as the others, you don't pass; but you can get another chance at it. If you can't make it happen then you get to go through the entire testing process again, which is not fun and, so I have heard, gets harder as you get older.
That adds some pressure to an already high-pressure process. But for me, the issue is having one’s pass/fail score relying so heavily on consensus.
Achieving that consensus among a group of cuppers can be elusive.
This problem is exacerbated if the group you're calibrating with has varying post-licensing or overall experience.
Some of you may have read an earlier blog of mine in which I discuss how an international jury magically aligns over the course of a competition. But a competition lasts about a week, not a day. The first day of competition is usually spent cupping and discussing coffees in order to facilitate the alignment of the cuppers. This sort of “magic” is a bit tough to pull off in one day.
As an example, using the industry 100-point scale where any coffee scoring higher than 80 is considered Specialty, one sample in the first cupping of the recalibration (meant to be our alignment cupping) had scores ranging from 79.5-89.75, all for the same coffee. I virtually slapped my forehead and knew this was going to be a challenging process. CQI gives a bit of a nod to this issue by allowing a +/- 0.75-point variance from the group average to count and still be considered calibrated. Additionally, a cupper need pass only two of three cuppings in order to be successful. Given the difficulty of the situation, I think CQI’s solution is about as fair as one can reasonably expect.
But why get a Q Grader license and go through this process every three years?
Being a Q Grader used to mean that someone or some entity would submit a sample of green coffee for Q Grading. The call went out and the first three Q Graders to respond received the samples to roast and grade. I don’t think that’s happening too much anymore. What I am hearing from the other people taking this recalibration and observing as I travel around is that not many of them are using their Q to Q Grade anymore.
The license for many, it seems, has turned into more of a professional accreditation than a license for evaluating and assigning a specialty score to a coffee.
I think I am OK with that, so long as those carrying the license are practicing industry professionals and cupping is part of their job. The Specialty Coffee industry needs a designation that speaks to a person's expertise and experience. I think that may have to be an article of its own, and it’s probably best left for another day.
In the meantime, I am just going to take this one cup at a time and focus on passing this recalibration.