13 replies to this topic

[DaBearSox]

My last score sheets that came back made me wonder....How common is it that beer judging is just all over the place. I mean its easy to discern that yea people have different tastes and some can taste better than others but you would think that beers would be within 6 points at competition to competition. This was only my third comp so I really don't have much to go on but wondering what others experiences have been.What brings this question on is that I had a porter that scored a 41 (three judges) then the same porter at a comp a week or 2 later scores a 30 (three judges) with all three of these judges picking out off flavors and totally contradicting what the other 3 score sheets say. This also happened with an Old Ale I had a 40 at the first and a 27 at the next...and a Cali Common 37 at the first 24 at the next...All three of them at the 2nd comp were loaded with off flavors (one judge checked like 5 boxes) and problems with the beer but at the first comp there are notes like very clean fermentation....I don't see how it is possible if these beers were loaded with off flavors that at the first comp any of the judges wouldn't have noticed them...At the most there were 2 weeks between judging and the beers sit in a 60 degree dark room...I am just perplexed....All I can think is that the judges at the first were just common joes and the next grand master dragons (not many of the people check any of those boxes to let me know)

[BarefootBrews]

Something to keep in mind is that once you drop your homebrew off at the shipping company, you have no control over how the beer is handled/stored. That includes from the moment of shipping until the bottle arrives at the judge's table. Lots of unknowns along the way once the homebrew leaves your possession.

[Thirsty]

All I can think is that the judges at the first were just common joes and the next grand master dragons (not many of the people check any of those boxes to let me know)

Take a look at the rank of the judges on the scoresheets. Every pairing should always have at least one BJCP accredited judge in the group, but may also contain an apprentice or novice that is non-BJCP but experienced. I have seen this before, and I feel that sometimes when a more experienced judge is in a group, it can intimidate another judge into changing their score to come much closer. We always discuss opinions and make sure we are within a few points, but some who question there ability may have been judging right, and caved to meet the more experienced. Position in a flight may have something to do with it too, if it is 7th out of 8, and there were a few great beers early, sometimes judges are searching for perfection to score well, and really search for off flavors that the first comp may not have picked up on. The last thing is I would retry your beer (if you have some left) and be completely neutral and nonbiased, and see if the off flavors they mentioned are present.

[DaBearSox]

Something to keep in mind is that once you drop your homebrew off at the shipping company, you have no control over how the beer is handled/stored. That includes from the moment of shipping until the bottle arrives at the judge's table. Lots of unknowns along the way once the homebrew leaves your possession.

Good point, I did drop them off at the same homebrew store, but i guess that would be the only constant.

Take a look at the rank of the judges on the scoresheets. Every pairing should always have at least one BJCP accredited judge in the group, but may also contain an apprentice or novice that is non-BJCP but experienced. I have seen this before, and I feel that sometimes when a more experienced judge is in a group, it can intimidate another judge into changing their score to come much closer. We always discuss opinions and make sure we are within a few points, but some who question there ability may have been judging right, and caved to meet the more experienced. Position in a flight may have something to do with it too, if it is 7th out of 8, and there were a few great beers early, sometimes judges are searching for perfection to score well, and really search for off flavors that the first comp may not have picked up on. The last thing is I would retry your beer (if you have some left) and be completely neutral and nonbiased, and see if the off flavors they mentioned are present.

Good tips....I have retried the Porter and the Cali Common...the score sheets on the common basically said that the yeast made it taste like a Belgian with all the phenolics in there...my palate may not be as refined but I could not pick up anything that would resemble something I made with a Belgian/German yeast...I am going to submit the porter to a comp on the 27th...we'll see what that score sheet says...

[djinkc]

I agree with the above suggestions. Also entries can get mixed up. I've had score sheets that simply could not have been the beer I entered.

[DaBearSox]

I agree with the above suggestions. Also entries can get mixed up. I've had score sheets that simply could not have been the beer I entered.

I was totally thinking this could have been a possibility...unless my taste buds are just useless.

[Thirsty]

I agree with the above suggestions. Also entries can get mixed up. I've had score sheets that simply could not have been the beer I entered.

This has happened to me before as well. I entered a maibock into last year's SA lomgshot and it scored a 13. I thought it was about a 35 beer. After reading the commemts of the off flavors and color (they said it wsas amber and not to style- while mine was really lightgold-gold) I knew it definitely was not mine. I had my 4 reserve bottles to taste from- and it was still killer a 35-40 beer easily, someone got a scoresheet that read "great beer! wrong category" I am sure. The irony was i judged that comp myself, (obviously not that cat) and it was extremely well organized, so I was very surprised when I saw the results. The largest part of the irony was we set the floor at 17 points at the start of the comp for the minimum score to be, and mine came back with a 13! 4 points below worst possible! Oh well, isolated incident- but it can happen with 670 entries.

[Howie]

It's common for beers to do well in one comp and not another. I had a porter place in Best of Show one week and then get a 28 the following week.Also, I've had beers ribbon at multiple comps and then get panned in another. In the competition where it got panned, all three judges said they tasted astringency, when it wasn't mentioned at all in other comps. This was simply a case of one judge saying they tasted it, and the others jumped on board.Judging is completely subjective, and can be swayed by a number of factors. Many people have made good points thus far. In addition to those, some judges just have very different palettes and different ideas of what a beer style "should be."I once judged with a certified judge who insisted that any trace of diacetyl was a flaw - in the ESB category!! It was a LONG day of judging, and we didn't agree on much of anything.I think you need to enter the same beer in a LOT of competitions to get a feel for an "average" score.

[dmtaylor]

I always say that you need to enter your beers into at least 3 competitions before you know which judges are good and which are full of it. You're very often going to find some judges that score way too high or too low. But if you enter into 3 or 4 competitions, then you get a better idea of which competitions might have judges who are outliers and don't know what the hell they're talking about. It happens. It's reality. I'm a judge and I know how it is. We ain't perfect. Except, maybe I am.

[Stout_fan]

... we set the floor at 17 points at the start of the comp for the minimum score to be, ...

So the organizers are telling the judges how to score beers?Accepted min is 13. Why be different?and FWIW I've gotten some pretty bad score sheets back that were out in left field. The guy's qualification: He's a professional brewer. Now most pro's know what they're doing. But not all are good judges. So take the proverbial grain of salt.

Edited by Stout_fan, 24 March 2010 - 06:34 AM.

[Thirsty]

So the organizers are telling the judges how to score beers?Accepted min is 13. Why be different?and FWIW I've gotten some pretty bad score sheets back that were out in left field. The guy's qualification: He's a professional brewer. Now most pro's know what they're doing. But not all are good judges. So take the proverbial grain of salt.

Yeah I think this is perfectly acceptable though. I know the BJCP says all scores should be within 7 points of eachother as well, but most comps usually set a house rule of a maximum 5 point differential. Which I think is fine. Anything more than 3, and there is a disagreement at the quality of the decision. As far as the decision to set the floor at 17, there really is not too much difference as long as the beer is under 20. The organizer is a pretty great guy, and the idea is to not hurt anyone's feelings giving them the world's worst possible score. However with that being said, I judged a specialty made with only molasses as fermentables. It literally clumped out of the bottle, and looked like a cup of separated mud. I would have given that one a 2 if I could have. I like to use the score of 30 as my baseline. If the beer has any off flavors, it is just unfair to any others to score it over 30. Unless it is a minor hint of off flavor, and the rest of it is outstanding, then it should be below 30, how much is determined by the rest of the factors. Conversely, if a beer demonstartes no off flavors, then it should score a minimum of 30, now the factors of how close to stylistic perfection come in to play at how much over 30.

[MakeMeHoppy]

We have our competition this weekend so this thread is interesting. I only entered one comp before. I really don't care so much about the actual score but more the comments. I do like the idea of entering the same beer in different comps to get more opinions.I was a steward the past two years and am far from having a trained pallet. However, I did notice significant taste fatigue set in quickly espectially since I was pouring barleywines and belgian strongs. I take the comments and scores as constructive and if I disagree with the judge and like the beer the way it is, then so be it. However, I did get some good comments and did change two of the beers I entered last year.

[denny]

and FWIW I've gotten some pretty bad score sheets back that were out in left field. The guy's qualification: He's a professional brewer. Now most pro's know what they're doing. But not all are good judges. So take the proverbial grain of salt.

I agree. Commercial brewers may know a lot about beer, but in general I find them to be the poorest BJCP judges. They often have an idea of a style that's at odds with the BJCP definition. Whether the commercial brewer is right or wrong, we judge to BJCP standards, not theirs.

[Stout_fan]

I agree. Commercial brewers may know a lot about beer, but in general I find them to be the poorest BJCP judges. They often have an idea of a style that's at odds with the BJCP definition. Whether the commercial brewer is right or wrong, we judge to BJCP standards, not theirs.

Amen brother!