KCBS BoD's Meeting Re: Judges Scores

F

Funtimebbq

Full Fledged Farker
Joined
Nov 19, 2009
Messages
470
Reaction score
254
Points
0
Location
Whittier...
There is a meeting scheduled for 6-28 to discuss changing guidelines regarding organizers, reps. or others from gaining access to judges average scores for seating purposes. There are 7 reasons listed, all to protect judges from the disclosure of their numbers for fear it will increase scoring averages. No reasons (or methods) are given to help organizers balance tables with both low and high scoring judges.
As some of you know, I've been advocating the use of these averages to seat judges in a balanced manner. I still believe it can be done but it is clear the BoD has no intention of helping the teams with this problem.

Here is the link http://www.kcbs.us/news.php?id=1034
 
I've written to the BoD with a method to use the numbers w/o disclosing them to anyone. I've shared that here as well.
In short, organizers can use the KCBS judge sign up system or send a list of judges membership numbers to a designated person at the KCBS office. That person can then use the system to build tables. The judges membership numbers and their table assignments can them be sent back to the organizer (or forwarded to the KCBS Rep). There would be no average scores published or known to either the organizer or Rep, simply a list of tables and membership numbers. It can be done without the actual averages being published.
 
As a judge, I don't care who sees my scores. I am confident is my abilities, and my scoring won't change. However I do see a potential for abuse by organizers.

What needs to be addressed is scoring inflation. This hurts the really good cooks, and gives a false sense of competence to mediocre cooks. I know, tilting at windmills.
 
rorkin is correct about Slamdunkpro being correct about Dennis being correct.

Seating judges by scoring average is lipstick on a pig. It attempts to mask the problem instead of fixing the problem. The last thing we need is anything that might drive scores higher.
 
gettinbasted said:
rorkin is correct about Slamdunkpro being correct about Dennis being correct.

Seating judges by scoring average is lipstick on a pig. It attempts to mask the problem instead of fixing the problem. The last thing we need is anything that might drive scores higher.
Click to expand...

It is hard to define what the "problem" is. I will say, starting when judge's scores were first recorded, the threat to send high/low scoring judges for retraining resulted in higher scoring (at least in CA). When table numbers began showing up on team score sheets, scores took another jump. I know of one instance when a judge encouraged others to "let's not be the TOD".

What is the purpose of the judge score tracking system if KCBS is not going to use it to make tables more equal. Why give teams all that information in the first place? All it does is cause us to bitch and moan about low scoring judges/tables.

I've proposed a fix, call it a band aid or lipstick, to use the information readily available to help with a situation many think is broken. If the system is not going to be used to make tables more level, the simple thing is to do away with collecting the information. Then we can all go back to being in the dark about whether we hit a high/low scoring table.

I'm not long for competing, maybe 10 more contests through the end of 2018. But I hope to see something done to produce a more balanced table assignment. It worked perfectly at the "King of the Smokers" where those of us who judged submitted our averages to help create balanced tables. I have a feeling the teams who were there appreciated that as well. But that was only 24 people. Too bad it will not happen again if this proposed rule/advisory gets approved. KCBS can do it and keep all that information anonymous. All they have to do is want to.

I'll leave it at that and go crawl back in my hole.
 
Funtimebbq said:
It is hard to define what the "problem" is. I will say, starting when judge's scores were first recorded, the threat to send high/low scoring judges for retraining resulted in higher scoring (at least in CA). When table numbers began showing up on team score sheets, scores took another jump. I know of one instance when a judge encouraged others to "let's not be the TOD".

What is the purpose of the judge score tracking system if KCBS is not going to use it to make tables more equal. Why give teams all that information in the first place? All it does is cause us to bitch and moan about low scoring judges/tables.

I've proposed a fix, call it a band aid or lipstick, to use the information readily available to help with a situation many think is broken. If the system is not going to be used to make tables more level, the simple thing is to do away with collecting the information. Then we can all go back to being in the dark about whether we hit a high/low scoring table.

I'm not long for competing, maybe 10 more contests through the end of 2018. But I hope to see something done to produce a more balanced table assignment. It worked perfectly at the "King of the Smokers" where those of us who judged submitted our averages to help create balanced tables. I have a feeling the teams who were there appreciated that as well. But that was only 24 people. Too bad it will not happen again if this proposed rule/advisory gets approved. KCBS can do it and keep all that information anonymous. All they have to do is want to.

I'll leave it at that and go crawl back in my hole.
Click to expand...

The problem is easy to define. It's a lack of consistency in the CBJ program and judges using different criteria and starting points to score the meat. Focusing on getting every judge on the same page will go a long ways.

I hear what your saying, I just think the end goal should be fairly scored BBQ, not equal scoring tables. The worst outcome of seating judges by scoring average is that it works. We are missing a huge variable in the process and that is the quality of meat judged.

I understand the proposed fix, and as a cook I think it's a huge mistake (if there's enough data to make it work, which is another discussion entirely). It's nothing personal towards you at all. I'm all for ideas and discussions to make competition BBQ better.

I personally love all the data KCBS gives us and I'm not sure why we would want it taken away?

I was one of the 24 at King of the Smoker.
 
Trying to seat judges per past scoring is folly! Todays contest scoring has nothing to do with past scoring. Why? The cooks offering is judged on its' own merits in this moment not compared to an entry from 6 months ago.
Granted I'm using the same line of thinking but your entry has nothing to do with the past.
Different day/different cook.
Ed
 
gettinbasted said:
The problem is easy to define. It's a lack of consistency in the CBJ program and judges using different criteria and starting points to score the meat. Focusing on getting every judge on the same page will go a long ways.
Click to expand...

BINGO!

While were are at it... can we get a solid definition of "average" (All experiences, comp only, restaurant, etc.). This has always blown my mind.
 
Funtimebbq said:
I've written to the BoD with a method to use the numbers w/o disclosing them to anyone. I've shared that here as well.
In short, organizers can use the KCBS judge sign up system or send a list of judges membership numbers to a designated person at the KCBS office. That person can then use the system to build tables. The judges membership numbers and their table assignments can them be sent back to the organizer (or forwarded to the KCBS Rep). There would be no average scores published or known to either the organizer or Rep, simply a list of tables and membership numbers. It can be done without the actual averages being published.
Click to expand...

Just a couple problems with this idea:

1. There are so many contests each weekend (especially May-Sept) that it will take more than one individual at KCBS devoted to nothing but seating judges based on their score history. Who would be paying for these extra KCBS employees?

2. As withdrawals come in, each contest has to be re-seated based on the new situation. - I'm a judge coordinator for a couple contests and recently I had 30 withdrawals for a 70+ team contest. I had pre-seated judges based on experience (the only criteria that I had available), but threw it out the window when I had seven withdraw in the last week (3 during the last 36 hours).

3. No-shows blow the advance seating completely out of the water. Almost EVERY contest that I've been to this year (12 to date) has had no-show judges. Anywhere from 1 to 10 judges didn't show up without any communication. Replacements have to be found or drop a table. You can't expect KCBS to reseat multiple contests at 11:00 Saturday morning.

I like your idea, but just don't think that it would work based on these problems.
 
Funtimebbq said:
It is hard to define what the "problem" is. I will say, starting when judge's scores were first recorded, the threat to send high/low scoring judges for retraining resulted in higher scoring (at least in CA). When table numbers began showing up on team score sheets, scores took another jump. I know of one instance when a judge encouraged others to "let's not be the TOD".

What is the purpose of the judge score tracking system if KCBS is not going to use it to make tables more equal. Why give teams all that information in the first place? All it does is cause us to bitch and moan about low scoring judges/tables.

I've proposed a fix, call it a band aid or lipstick, to use the information readily available to help with a situation many think is broken. If the system is not going to be used to make tables more level, the simple thing is to do away with collecting the information. Then we can all go back to being in the dark about whether we hit a high/low scoring table.

I'm not long for competing, maybe 10 more contests through the end of 2018. But I hope to see something done to produce a more balanced table assignment. It worked perfectly at the "King of the Smokers" where those of us who judged submitted our averages to help create balanced tables. I have a feeling the teams who were there appreciated that as well. But that was only 24 people. Too bad it will not happen again if this proposed rule/advisory gets approved. KCBS can do it and keep all that information anonymous. All they have to do is want to.

I'll leave it at that and go crawl back in my hole.
Click to expand...

gettinbasted said:
The problem is easy to define. It's a lack of consistency in the CBJ program and judges using different criteria and starting points to score the meat. Focusing on getting every judge on the same page will go a long ways.

I hear what your saying, I just think the end goal should be fairly scored BBQ, not equal scoring tables. The worst outcome of seating judges by scoring average is that it works. We are missing a huge variable in the process and that is the quality of meat judged.

I understand the proposed fix, and as a cook I think it's a huge mistake (if there's enough data to make it work, which is another discussion entirely). It's nothing personal towards you at all. I'm all for ideas and discussions to make competition BBQ better.

I personally love all the data KCBS gives us and I'm not sure why we would want it taken away?

I was one of the 24 at King of the Smoker.
Click to expand...

You cannot average random events and produce meaningful data. Seating judges by average score is as effective as seating them by shoe size. Brad hit it right on the head. It's a standards of education problem coupled with a lack of enforced policies. Everyone on the same page should be the goal; not 8's across the board.

KCBS needs to get on top of this quickly and exert some authority as another more serious issue is morphing out of this problem. Some Reps and volunteers now seem to be taking it upon themselves to try and fix what they consider inequities in the judging tent by ignoring standard KCBS procedure and implementing their own personal "fix". This type of freelancing at contests is a huge concern. Even if a KCBS procedure is flawed it's a KNOWN process. Contests are not laboratories for people to experiment using the cook teams as lab rats because they feel something is unfair. All contests should be run the same - or as close as humanly possible. If a cook team adds red tipped lettuce or apple slices in their box because they think it "looks better" they get DQ's because it's a hard and fast rules violation (black letter law). Judges judge samples in the order they are placed on their judge's mat. (black letter law). Why are people being allowed to modify the turn in procedure? This needs to be addressed ASAP. KCBS needs to issue an edict that judging procedures will be followed to the letter and, if they aren't, there will be consequences.
 
I quit competing in KCBS contests in 2012. That was the same year I received a 5 and 9....on the same meat, at the same table and not a single comment card was issued.

With that kind of inconsistency with scoring, it was not much more than a crap shoot, so why bother?
 
Funtimebbq said:
It is hard to define what the "problem" is. I will say, starting when judge's scores were first recorded, the threat to send high/low scoring judges for retraining resulted in higher scoring (at least in CA). When table numbers began showing up on team score sheets, scores took another jump. I know of one instance when a judge encouraged others to "let's not be the TOD".

Retraining will never work. Until KCBS begins testing judges on rules and actual tasting (similar to what CA State does the wine judges), you will have judges all over the place. Worse yet, you have judges afraid to give the scores deserved, with 7 being their bottom number.

What is the purpose of the judge score tracking system if KCBS is not going to use it to make tables more equal. Why give teams all that information in the first place? All it does is cause us to bitch and moan about low scoring judges/tables.

I've looked at my scores on KCBS, and how the table voted, and I am usually less than 1 point difference. Example: My scores from March of this year, Brisket 33.25, table average 33.17, Chicken 33.48, table 33.62, Pork 31.88, table 31.86, Ribs 34.05, table 33.44. This doesn't seem to be of much use to KCBS for trying to place judges at various tables.

I've proposed a fix, call it a band aid or lipstick, to use the information readily available to help with a situation many think is broken. If the system is not going to be used to make tables more level, the simple thing is to do away with collecting the information. Then we can all go back to being in the dark about whether we hit a high/low scoring table.

I know cooks are convinced there are TODs, but honestly I have never seen it. I have been at a table where the rep shook his head and said, "The scores are pretty low here folks," but what are you to do when the entries are really poor? Table scores were 26-28 for ribs.

I'm not long for competing, maybe 10 more contests through the end of 2018. But I hope to see something done to produce a more balanced table assignment. It worked perfectly at the "King of the Smokers" where those of us who judged submitted our averages to help create balanced tables. I have a feeling the teams who were there appreciated that as well. But that was only 24 people. Too bad it will not happen again if this proposed rule/advisory gets approved. KCBS can do it and keep all that information anonymous. All they have to do is want to.

I'll leave it at that and go crawl back in my hole.
Click to expand...

:thumb:
 
SirPorkaLot said:
I quit competing in KCBS contests in 2012. That was the same year I received a 5 and 9....on the same meat, at the same table and not a single comment card was issued.

With that kind of inconsistency with scoring, it was not much more than a crap shoot, so why bother?
Click to expand...

I can see your frustration. I firmly believe that testing should be required of all judges, not just sitting in a class and then raising your hand. Cooks spend a loot of money; they should get a fair shake. But nobody died and made me God, so I deal the cards I'm dealt.
 
Start a new CBJ training VIDEO. All existing CBJs have to watch it and can watch it for free on kcbs.us when they login with their member number. Standardize the training, don't allow human instructor drift.
 
JeffR said:
No show/No Call judges should loose their CBJ. A simple call if an emergency comes up is not too much to ask for.
Click to expand...

Believe me, I agree!!!

My understanding is that the KCBS BOD is working on a 3-strike program that I believe will de-CBJ any judge who is a no-show three times. I might be off-base on this, but time will tell.

Meanwhile, there is a rumor floating around that a list might exist that has no-show and trouble-making judges. If it exists, I just hope that I'm not on it!!!
 
You must log in or register to reply here.

Similar threads

GrillBillie_D
KCBS judge seating changes
2 3
Replies
46
Views
5K
rorkin
rorkin
DUBBAGA
KCBS Score - Judges' Scorecard is live
3 4 5
Replies
90
Views
6K
rorkin
rorkin
P
Consistent Taste
2
Replies
31
Views
5K
pharp
P
Rich Parker
KCBS Table of Death and Angels - Idea
2 3
Replies
59
Views
3K
peterz
P
Rich Parker
KCBS ToY Bonus Points Change
2 3
Replies
49
Views
4K
gettinbasted
G
Back
Top