Is the KCBS BOD schizophrenic?

[Podge]

I keep trying to get Dr. BBQ to help me as well. Or at least treat me to a Boston's Italian Beef the next time he is in town... :becky:

I like for people to not know me. Keeps them from asking for samples, shigging, etc..

 

[Bigdog]

I like for people to not know me. Keeps them from asking for samples, shigging, etc..

Podge is changing his team name to "The Pube."

This is really not a bad idea, esp. at the Royal where many people don't know BBQ etiquite.

 

[The Pickled Pig]

The tracking and "penalization" of outlier judges will skew the judging pool. No judge will want to be "labeled" an outlier and will worry about how his fellow judges are going to score and may hide his/her true opinion.

To some degree, that is the desired outcome. If a judge consistently gives outlier scores based upon their "true" opinion, then I'd hope that they would conform to the norm with some positive mentoring.

I don't get the sense that anyone believes there is a rampant problem of rogue judges. I do believe there are a small number of judges that consistently score entries using a different scale than their peers use. Left on their own, those judges might as well stay home since their scores are likely being thrown out anyway.

As cooks we get feedback at every contest about things that could use improvement. Don't we owe it to our judges to give them the same type of feedback about their craft? How cool would it be for our judges to get a score sheet after every contest they judge that showed them how their scores compared with others at their same table. I think judges would get as much benefit out of that that I do getting cook score sheets.

The very strong fact remains in all of this: There are very familiar faces at the top of every contest.

The judging at contests of 40 teams or less is consistent. While the judging may be flawed at times, it is still fair. We all have the same probability of hitting a "bad" table. And tossing the low score out goes a long way to overcoming an outlier judge. Just because the system is fair doesn't mean it can't be improved.

 

[LGHT]

I'd like to know how many judges score high or low, often enough to stand out. I'd like to see how many judges routinely score everything virtually the same, etc....

I don't think it's possible mainly because I doubt that the software KCBS is using now has a feature to keep track of all the CBJ's scores. Although it would be beneficial entering that much data would also be a huge task.

At this point I think all we have is suspicion, anecdotal evidence, etc... If we have factual evidence of the problem, we can then determine the cause and proper corrective action.

This is a very valid point. It may be 1-2 judges out of every comp that isn't on point with the other judges or may not realize how the judging has changed over the years. If a team gets 9-9-8-9-9-5 in appearance does the team have a way to protest the odd score? If not that may be the quickest and easiest way to single out the odd scores. If a judge is that off it will show up on ALL the scoring and then the KCBS rep can review and handle as needed. I mean it will be very hard to review ALL judges scores, but it shouldn't be that hard to review a few "questionable" scores if pointed out by a team.

 

[Scottie]

I like for people to not know me. Keeps them from asking for samples, shigging, etc..

Me too. Except if they are passing out bottles of Jack... :becky:

 

[ThomEmery]

As cooks we get feedback at every contest about things that could use improvement. Don't we owe it to our judges to give them the same type of feedback about their craft? How cool would it be for our judges to get a score sheet after every contest they judge that showed them how their scores compared with others at their same table. I think judges would get as much benefit out of that that I do getting cook score sheets.

Great idea.... Paul

 

[CBQ]

I don't think it's possible mainly because I doubt that the software KCBS is using now has a feature to keep track of all the CBJ's scores. Although it would be beneficial entering that much data would also be a huge task.

I think the data is already there. They capture the scores, and when you are a judge, they ask you for your CBJ number. I think there would be an issue of privacy though. KCBS itself would probably want to review the data itself for action, and not make it public. Whether or not they are staffed to do that kind of analysis is uncertain.

Certainly others are - like what The Pickled Pig is able to do with published results data - but if KCBS does do this kind of evaluation, I don't see it being made widely available.

 

[goodsmokebbq]

To some degree, that is the desired outcome. If a judge consistently gives outlier scores based upon their "true" opinion, then I'd hope that they would conform to the norm with some positive mentoring.

I don't get the sense that anyone believes there is a rampant problem of rogue judges. I do believe there are a small number of judges that consistently score entries using a different scale than their peers use. Left on their own, those judges might as well stay home since their scores are likely being thrown out anyway.

As cooks we get feedback at every contest about things that could use improvement. Don't we owe it to our judges to give them the same type of feedback about their craft? How cool would it be for our judges to get a score sheet after every contest they judge that showed them how their scores compared with others at their same table. I think judges would get as much benefit out of that that I do getting cook score sheets.




The judging at contests of 40 teams or less is consistent. While the judging may be flawed at times, it is still fair. We all have the same probability of hitting a "bad" table. And tossing the low score out goes a long way to overcoming an outlier judge. Just because the system is fair doesn't mean it can't be improved.

Judging feedback in the form of a score sheet could be productive in allowing judges a way to "hone" in there scoring. This sounds like "learning" and thats what we all want.

I do however think the judges are going to be very wary of being "put on a list" and "identified for further training". I am not sure if I understand how this would bias a judges decision. Not to mention my fleeting confidence that KCBS can execute a program like this.

I really don't think the judging is flawed, I just think you have different individuals making decisions based on their own opinion and distinct experience. There is a level of variance that will never go away when humans are involved (Computer judging Mod :-D).

 

[LGHT]

I agree I don't think the judging is flawed I would chalk it up to the lack of proper and in-depth training of the judges, but of course you can't re-train all current CBJ's so you still have to do something.

I used to do a bit of programming back in the 80's and the one quote that always made sense is Trash in = Trash out.