Questioning Results

Candy Sue said:

I have issues with mandating comment cards at a certain score too (like a 5 or below).

Click to expand...

Yeah, It's so freakin' tough to get a pencil out and write "Presentation was great, but the ribs were tough/falling apart/spicy/bland"....

Almost as tough as working 45 hours in a week, preparing for a contest in the evenings, driving for 2-1/2 hours on a Friday after working til lunch on Friday, setting up your cooksite, prepping meat, doing the cooks meeting, prepping more, figuring out WTH you're gonna eat for supper, getting your smoker(s) started, seasoning, putting meat on, forgetting that you haven't eaten, then try to sleep for a few hours in the same seat you drove in earlier in the day.

Yep, writing a sentence or two while eating free food can't be expected...

Candy Sue said:

The table-level results are coming soon with KCBScore. It's not going to help the 3 point scoring problem though. I have issues with mandating comment cards at a certain score too (like a 5 or below). All that does is gently direct judges to a 6-7-8-9 score.

My opinion only...

Click to expand...

I wonder if a system like Kcbscore could be used to assign a standard deviation to judges based on past contests? You could then even out the tables by assigning a similar number of new, high tendency, and low tendency judges to each table.

The best thing that could happen would be for some sort of underground website to be formed for contest organizers.

Not only could they trade information about how better to run one, but talk amongst themselves about good or bad contest reps, troublesome teams and those judges that give out poor scores and/or refuse to fill out comment cards.

Kind of like the 'star rating' system that nearly every online review-type site uses. If a judge gets blackballed more than a few times, the organizer can throw that judge's application in the circular file. If that judge all of the sudden realizes he's not getting selected to eat free BBQ on Saturdays, he might wise up or move on to bothering someone else...


KCBS need not spend gobs of money trying to develop some sort of convoluted scoring system to analyze judges...let the organizers compile information from experience.

Pigs on Fire said:

Yeah, It's so freakin' tough to get a pencil out and write "Presentation was great, but the ribs were tough/falling apart/spicy/bland"....

Almost as tough as working 45 hours in a week, preparing for a contest in the evenings, driving for 2-1/2 hours on a Friday after working til lunch on Friday, setting up your cooksite, prepping meat, doing the cooks meeting, prepping more, figuring out WTH you're gonna eat for supper, getting your smoker(s) started, seasoning, putting meat on, forgetting that you haven't eaten, then try to sleep for a few hours in the same seat you drove in earlier in the day.

Yep, writing a sentence or two while eating free food can't be expected...

Click to expand...

As having been on both sides of the table, I completely agree. I am actually of the opinion that they could be required for ALL scores, and I'm also of the opinion that they could be check lists (examples given in many other threads) so they go much faster than having someone spell out every detail an slow the process....

After a couple of hundred or so contests you tend to build an immunity or at least a tolerance of that one judge on the table...albeit that judge has no idea just how much that ill placed score just costed that team...probably don't care noways. All the sanctioning bodies have appeal proceedures in line....but about all that can truthfully be appealed is the calculations....the judges ate all of the actual evidence. There's no way that anybody can honestly or truethfully say that their leftover ribs or chicken or whatever is EXACTLY the same as what they put in the box. They way I see it...I have told myself as well as others this before....at the end of a contest if you don't like the way the scoring went you only have two choices....be at the next one or stay at home.

By the way...it was real nice meeting you Dave. Please keep doing what you do cause I really enjoy reading your reports.

Pigs on Fire said:

Yeah, It's so freakin' tough to get a pencil out and write "Presentation was great, but the ribs were tough/falling apart/spicy/bland"....

Almost as tough as working 45 hours in a week, preparing for a contest in the evenings, driving for 2-1/2 hours on a Friday after working til lunch on Friday, setting up your cooksite, prepping meat, doing the cooks meeting, prepping more, figuring out WTH you're gonna eat for supper, getting your smoker(s) started, seasoning, putting meat on, forgetting that you haven't eaten, then try to sleep for a few hours in the same seat you drove in earlier in the day.

Yep, writing a sentence or two while eating free food can't be expected...

Click to expand...

If I were you I would hang up the old smoker because it doesn't sound like you enjoy competing anymore. Nobody is holding a gun to your head, forcing you to compete. It's easy to attack the judges, but the fact is they are not required to fill out anything. You should be ranting at the KCBS, not the judges for not requiring the comment cards. Maybe it's a good thing that not a lot of judges read the forums. If I were a judge and read some of the crap spoken about them, I would never judge again.

gettinbasted said:

I wonder if a system like Kcbscore could be used to assign a standard deviation to judges based on past contests? You could then even out the tables by assigning a similar number of new, high tendency, and low tendency judges to each table.

Click to expand...

I like this idea....Like a Golf Handicap for BBQ Judges!

markG said:

And there is the winning answer. I am a newbie this year and there were some things that I turned in that should have been a solid 3 and got 7's. The scoring scale is wide for a reason, I thought. I would rather get an honest 3 than a (lowball in the mind of a nice judge) 7, so I can know that it was REALLY bad and give me something to work on.

Click to expand...

My husband judged a competition and gave some 5's and 6's for bad BBQ, (pork so tough you couldn't chew through it, brisket so dry you had to drink water to get it down, ribs just thrown in upside down and every which way). He actually got pulled aside 1/2 way through and told he was scoring too harsh by the rep. I agree that the scoring is getting too narrow to not make the competition a lot about luck, instead of actual skills.

Eggspert BBQ

hogzgonewild said:

I like this idea....Like a Golf Handicap for BBQ Judges!

Click to expand...

Behold the power of statistics!

Candy Sue said:

The table-level results are coming soon with KCBScore. It's not going to help the 3 point scoring problem though. I have issues with mandating comment cards at a certain score too (like a 5 or below). All that does is gently direct judges to a 6-7-8-9 score.

My opinion only...

Click to expand...

The same kcbscore that has been in the hopper for years? By the time this thing sees the light of day a lot of us will be retired from competition!

Sawdustguy said:

If I were you I would hang up the old smoker because it doesn't sound like you enjoy competing anymore. Nobody is holding a gun to your head, forcing you to compete. It's easy to attack the judges, but the fact is they are not required to fill out anything. You should be ranting at the KCBS, not the judges for not requiring the comment cards. Maybe it's a good thing that not a lot of judges read the forums. If I were a judge and read some of the crap spoken about them, I would never judge again.

Click to expand...

Is it really attacking the judges to suggest they take it seriously and put a little effort into it when the cooks spend so much effort and money to make them the food in the first place?

I dont know exactly what would work best, but there seems like there could be some improvement with the judging. I think providing information about judges to everyone could ease a lot of worries about the judges. sunshine and transparency and all.

I've cooked, judged, table captained, and worked the turn in table. I have experienced the low scores on turn ins that I felt were great, and have got calls when I felt my stuff wasn't fit to eat. I have done non sanctioned events where I feel the process was flawed and an error (or hanky panky) could easily have occurred. For there to be a mixed up box in a KCBS event would require numerous oversights, and from my experience is highly unlikely.

Now, to the issue at hand. When all six scores are close, we have to accept that our opinion is obviously different from the judges. The problem is when we have a couple of "Judge #6's" at a particular table. There is no process to evaluate our judges. We call someone a "Master Judge" when they judge 30 contest, regardless of how well they judge. We need to develop a method for evaluation!!! The mandating of the use of comment cards may be a good start. The table captains, reps and organizers should see these cards and see if the explanations for the given score is a valid point. From this forum, and others like it, I would like to see a push for our judges to lead discussion on comment cards prior to the start of judging and encourage other judges to fill them out on as many entries as they can. Personally, I think the BOD should mandate it. I realize that probably won't happen.

Judges also need more education than a four or five hour class. I have been amazed by the questions I am asked by judges when they find out that I also cook. There are things I take for granted that everyone knows about "q" that many judges don't know. The initial training probably needs to be done by video with ample time for discussion and interaction. This would assure that everyone is trained to the exact same standard. Cooking with a team needs to be mandated early in the process. I can't count the number of Master Judges that have said that it made a huge difference in their understanding of what they see in the box. One more thing and I'll shut up. More cooks need to judge. Go to the class, set aside a few week-ends a year to judge, and make a difference. We can help educate the judges, and most are very receptive to it.

Candy Sue said:

It was bound to happen when the "start at 6" judging point went away.

My opinion only!

Click to expand...

Just curious, when was that implanted.

Thanks.

Mike

I found this thread very informative. I judged at an event this fall in the ancillary categories and enjoyed the experience. I found it quite intimidating. I was thinking - "who am I to judge the work of these folks". I didn't get all that much instruction and was fortunate to have an experienced KCBS judge sitting beside who clued me in on a few things.

After reading this thread I'm glad to say following my intuition was the "right" choice. I judged using the whole numbering system. To me it seemed only the fair way. In my head everyone started at a 6 and went either up or down from there.

I plan to go to a KCBS judges class in the new year and start judging in the spring. The goal is to use this is as one tool to help educate me on the road to cooking in competitions. It will be interesting to learn what is taught in this area. I plan to bring this topic up in the class to see what the "official" line is.

I get the reality that whenever you have humans involved in anything that requires personal opinions/judgement you will get at best a wide range of thoughts and at worst thinking that approaches the irrational. However, that said, I think it would take me a while to build up a thick skin to this and I wouldn't be as calm about it as the OP appears to be.

just a way left field thought....the OP's topic seems to have arisen because there is no clear check and balance to be sure an errant scoring was correct.

what if....7 pieces were required, and if one judge at a table is more than 3 points off from the rest, the table captain samples the seventh piece to "verify". sort of a comment card work around.

dunno if that makes any sense though.

DawgPhan said:

Is it really attacking the judges to suggest they take it seriously and put a little effort into it when the cooks spend so much effort and money to make them the food in the first place?

I dont know exactly what would work best, but there seems like there could be some improvement with the judging. I think providing information about judges to everyone could ease a lot of worries about the judges. sunshine and transparency and all.

Click to expand...

I don't think anyone denies that judging can be improved. I am just tired of all the threads beating up the judges. My point is that we compete knowing that the system is imperfect. If you get dealt a hand you don't like, deal with it. Everyone assumes the judges did something wrong when they don't do well. If thats true why do the Smoking Triggers, Lotta Bull, Jacks Old South etc. always seem to do well? Even if we had comment cards filled out religiously, someone is going to find something else about the judging to bitch about.

Golf handicaps have nothing to do with judging food. A handicap shows golfers potential against like and fixed courses. Judges are evaluating personal preferences of individual foods.
Unless a stricter implementation of judging criteria is mandated, any rating of judges is worthless. In other words “A brisket must be elastic to 1/2in in a pull test. No more No less”. I’ll also say that if that is done and the good cooks/teams have a fixed target to shoot at ... the rest of us won’t have a chance.

Sawdustguy said:

I don't think anyone denies that judging can be improved. I am just tired of all the threads beating up the judges.

Click to expand...

Agreed, the "problem' is not the judges .. its the criteria we use to score.