Kant touch that!!!

[airedale]

I never intended to replace KCBS "suggestions" on judging criteria ...

Yeah. Underline "suggested." The red OP text is based on a false premise, that judging classes specify methods. At least the training book does not and I don't recall any verbal instructions to use or not use a particular method.

If some group at KCBS wants to get into describing approved methods and compiling a reference book I guess that is within their purview but as of today AFIK there are NO prescribed methods for any of the scoring categories.

For tenderness, maybe the reps could carry around a Warner-Bratzler machine. (https://www.testresources.net/appli...ratzler-method-for-measuring-meat-tenderness/)

For taste, maybe they could require judges to come a couple of hours early to sample calibration standards for salty, sweet, bitter, mushy, tough, etc. This is the way it is done at Pilllsbury's food labs. Before a panel samples actual product their palates are first calibrated using standards.

Appearance? I would be fascinated to see the required method(s) for determining an appearance score. Items parallel to within one degree? Sauce thickness no more than 30 mils? Item weights within 1/4 oz. of each other? Certain kinds of psyhcologically disturbed people could have a lot of fun with this.​

 

[rob g]

In my opinion you will never get consistency in scoring until KCBS defines "average". An average piece of meat rates a 6 by KCBS standards. By that definition shouldn't most of the meat coming across the judging table get a 6 if average was defined by meats found at competition?

 

[airedale]

In my opinion you will never get consistency in scoring until KCBS defines "average". An average piece of meat rates a 6 by KCBS standards. By that definition shouldn't most of the meat coming across the judging table get a 6 if average was defined by meats found at competition?

IMO that is an impossible challenge because no matter what pool is averaged no individual judge can be familiar with the pool.

For example, if the pool is entries at the current comp, no judge will experience all the entries and, worse, the first box gets judged before the judges have experienced any entries.

If the pool is "meats found at competition," no judge can possibly have experience with more than a tiny fraction of that pool and a novice judge will have none at all.

Your goal is good, but the whole concept of using "average" in judging is IMO fundamentally flawed.

 

[pharp]

IMO that is an impossible challenge because no matter what pool is averaged no individual judge can be familiar with the pool.

For example, if the pool is entries at the current comp, no judge will experience all the entries and, worse, the first box gets judged before the judges have experienced any entries.

If the pool is "meats found at competition," no judge can possibly have experience with more than a tiny fraction of that pool and a novice judge will have none at all.

Your goal is good, but the whole concept of using "average" in judging is IMO fundamentally flawed.

I genuinely appreciate you being a judge and trying to highlight things in the process that could be better.

That being said, I think you are trying to make a mountain out of a molehill in terms of the word average. The word can (and I think pretty obviously in this context) mean "of the usual or ordinary standard."

 

[tduffy]

In my opinion you will never get consistency in scoring until KCBS defines "average". An average piece of meat rates a 6 by KCBS standards. By that definition shouldn't most of the meat coming across the judging table get a 6 if average was defined by meats found at competition?

I'm just not sure how it could be done. One judge's 9 is another judge's 7. Every judge is going to base "average" off of something they find as average. Depending on the quality of bbq they've experienced in the past, it will be very different.

 

[rob g]

I'm just not sure how it could be done. One judge's 9 is another judge's 7. Every judge is going to base "average" off of something they find as average. Depending on the quality of bbq they've experienced in the past, it will be very different.

I believe KCBS needs to define "average" as those meats found on the judging table. This will narrow the wide variation in exposure to bbq to a smaller range. As an example my exposure to bbq was burgers, dogs and the occasional steak growing up as a kid here in Canada. Compare that to a kid growing up in the southern US that was eating low and slow bbq from the time they were old enough to lick the sauce off the bones. It will take new judges 3-4 competitions to get a handle on competition meat but their scores will soon come into line. If most of the meats are getting a 6 then the truly stellar sample actually has the ability, and room to move, to separate itself from the pack. In my opinion most of the samples I get tend to taste the same. It's usually how well the meat is cooked that makes it stand out for me. At the same time I think judges need to be a little more critical in their judging while using the complete scoring range. Giving only 7-9 scores makes judging easy but isn't particularly fair to the teams that actually have turned in a superior product.

 

[airedale]

... The word can (and I think pretty obviously in this context) mean "of the usual or ordinary standard."

I don't think that using a different word helps unless you can describe to me what this "standard" is and how all judges will understand it and be able to refer to it in some kind of consistent fashion.

I'm just not sure how it could be done. One judge's 9 is another judge's 7. Every judge is going to base "average" off of something they find as average. Depending on the quality of bbq they've experienced in the past, it will be very different.

Exactly.

I will probably get flamed for this but the scoring I have seen looks to me like it is pretty much on a scale of six to nine with nine being "wonderful" and six being "not very good" or "flawed in some way." IOW, no reference to "average."

(I was typing when @rob g's post popped up but I don't think I disagree with him. "Typical competition meat" is probably what "average" has to end up being but as he says, that means judges have to gain experience over several comps before they really know what "average" is. And, as he also says, scores will have to include more sixes. Good luck with that. )

 

[Serious Hog]

Personally wouldn't bother me to scrap 2-6 entirely and let 7,8,9 be Bad, Good, and Excellent respectively. Leave the 1 in there for DQs and such. I just think it's a more attainable goal to get consistency on 3 numbers than on 9.

 

[Jorge]

Personally wouldn't bother me to scrap 2-6 entirely and let 7,8,9 be Bad, Good, and Excellent respectively. Leave the 1 in there for DQs and such. I just think it's a more attainable goal to get consistency on 3 numbers than on 9.

I'm I life member, so KCBS can't raise my dues to pay for the new Rep laptops annually as they melt from the strain of breaking all of the ties.:mrgreen:

 

[tduffy]

Personally wouldn't bother me to scrap 2-6 entirely and let 7,8,9 be Bad, Good, and Excellent respectively. Leave the 1 in there for DQs and such. I just think it's a more attainable goal to get consistency on 3 numbers than on 9.

I completely disagree. Turning this into a 3 point scoring system anymore than it already is will make more ties. There is garbage entries that absolutely deserve the 5 and below that they get.

 

[Serious Hog]

I'm I life member, so KCBS can't raise my dues to pay for the new Rep laptops annually as they melt from the strain of breaking all of the ties.:mrgreen:

Haha point taken, but maybe it's worth having a tie breaking issue to solve if scoring gets more consistent. And how many meaningful ties are currently being broken by scores in the 2-6 range anyway?

 

[airedale]

Well, if we can rip up the scoring system how about adding my favorite, a category for creativity.

Maybe that would reduce or eliminate all those identical chicken pucks.

 

[Serious Hog]

Well, if we can rip up the scoring system how about adding my favorite, a category for creativity.

Maybe that would reduce or eliminate all those identical chicken pucks.

Not sure if you're being serious or not, but I agree that a good score shouldn't rely on putting uniform samples in a box. In my opinion, the brisket pictured below looks awesome but most cooks (myself included) would never try to build a box that looked anything like that for fear of being dinged on Appearance.

 

[airedale]

Not sure if you're being serious or not, but I agree that a good score shouldn't rely on putting uniform samples in a box. ...

Well, I guess I'm both serious and not.

As a relatively new judge (2017) I have been extremely disappointed by the same-ness of the entries I see. (I got shot up pretty bad here a few months ago for saying this, so I guess I'll go looking for my bulletproof vest now.) I have come to understand the reasons for the sameness (cooking with Travis last summer helped) but that doesn't make me like it. So I am serious from that point of view.

But I am not serious in that I think there's zero chance of this kind of radical change being accepted by KCBS. That's not a shot at KCBS, incidentally. It's just an observation based on how organizations behave. And it has some problems, the biggest of which would be how to maintain anonymity (the reason "meat sculpting" is now prohibited). So from that angle, it's just forum BS.

... In my opinion, the brisket pictured below looks awesome but most cooks (myself included) would never try to build a box that looked anything like that for fear of being dinged on Appearance.

I think it looks quite good too, because I like bark, moist, and fat. If I was lucky enough to be in the first few judges for that box I'd be looking for the piece that looked like it had the most fat. :mrgreen: But I agree that under current practices someone would probably mark it down in appearance for not being six uniform, fatless slices like virtually all the others.

 

[Blanton]

Not sure if you're being serious or not, but I agree that a good score shouldn't rely on putting uniform samples in a box. In my opinion, the brisket pictured below looks awesome but most cooks (myself included) would never try to build a box that looked anything like that for fear of being dinged on Appearance.

Sorry, but I do not see anything awesome about that from an appearance standpoint. Bark does look tasty, but from a competition POV, looks dry, crumbling, and hacked with a butcher knife. Now take the same brisket, get some even cuts that the bark holds too, and moisten it up with some au jus, then think it would score well

 

[Jorge]

Haha point taken, but maybe it's worth having a tie breaking issue to solve if scoring gets more consistent. And how many meaningful ties are currently being broken by scores in the 2-6 range anyway?

We already have a system that has effectively become 7-9 in practice with more 8s and 9s than 7s. Cooks have gotten better, but not that mulch better. If you take the top teams I have no doubt that they are deserving of the 8s and 9s they earn. When we get to the next tier and below there is no doubt in my mind that more of the available range needs to be used to accurately reflect the quality of the entries.

I'd argue that more of the range needs to be used rather than narrowing it. There are cooks that, based on current scoring, believe that they are closer to the top than they actually are. That does nothing to help them understand where they truly stand or to improve.

 

[Serious Hog]

We already have a system that has effectively become 7-9 in practice with more 8s and 9s than 7s. Cooks have gotten better, but not that mulch better. If you take the top teams I have no doubt that they are deserving of the 8s and 9s they earn. When we get to the next tier and below there is no doubt in my mind that more of the available range needs to be used to accurately reflect the quality of the entries.

I'd argue that more of the range needs to be used rather than narrowing it. There are cooks that, based on current scoring, believe that they are closer to the top than they actually are. That does nothing to help them understand where they truly stand or to improve.

I agree with most everything you said. We aren't certified judges, but we had the opportunity to be volunteers in the judging tent at a KCBS comp back in April. We got to try a lot of samples and quality was definitely all over the place.

I just think it's a harder goal to try to get consensus about what 9 different scoring points mean than it is with 3. My understanding is some judges have been taught to start at 9 and work their way down, others start at 6 and move up/down. And probably other variations as well. Given this situation, it just seems like it would be easier to get more reliable agreement on what Bad, Good, and Excellent mean than it is for Excellent, Very Good, Above Average, Average, Below Average, Poor, Bad.

 

[Jorge]

I understand your reasoning. I just don't think that it is in the best interest of KCBS or the teams.

Back in the day, judges were instructed to start at 9 and work down as you mentioned. The computer was deciding who won categories when 8 or 9 teams had 180s in some cases.

KCBS needs the longer term fix for the current issue, not the one that is the easiest to implement.

 

[tduffy]

A 3 point scoring system means that on my worst day, my entries are going to score the same as the team that turned in excessively smoked, ash tray tasting garbage. That or they get a middle score that they didn't deserve. That's not how we fix judging. But on that token I don't feel the judging system we have is broken. I feel that we have a lack of consistency in training issue and not enough continued training.

One thing is for certain, every time KCBS steps in and tries to do something it makes it worse. Look at what seating judges by scoring averages did this year. I cooked a comp in Iowa last year that a buddy of mine judged. After judging he stopped over and mentioned that the scoring average formula seated a table of judges consisting of all women and another of all old people(60+). Tell me how that's a good idea.

 

[airedale]

... I feel that we have a lack of consistency in training issue and not enough continued training. ...

We have a block of dead time between judges getting registered and the seating coming back from the mother ship. I wonder if it would be possible to put little snippets of training in at that point.

I would go to refresher classes maybe once a year or once every year or two but I have no idea how many judges would be receptive to that. In flying, non-airline pilots are required to have a flight review with an instructor every two years. The idea of this is to confirm or to re-teach skills. Without a current flight review we can't fly. Probably that kind of thing would be too tough for KCBS judging however.