KCBS Table of Death and Angels - Idea

[Rich Parker]

So I have been thinking about this KCBS Table of Death and Table of Angels and have an idea I am thinking about sending to the board of directors but would like your input. I look forward to reading your comments.

I have heard on numerous occasions on how the “Table of Death” (TOD) and “Table of Angels” (TOA) either helped me or hurt me and 99% of the time it is how the TOD hurt them. The TOD and TOA are basically created by pure circumstance of where judges sit after the judges meeting. This purely by chance circumstance can be greatly reduced or eliminated by placing judges at the appropriate table based on the averages of the other judges at the event.

Example of a 12 team event this situation could occur with table #1 having an average of 194 and #2 an average of 156. No need to drop the lowest when calculating averages. In this extreme example, you could have a TOD and TOA at the same event.

Table #1 average 194 - Table #2 average 156
Judge #1 = avg 32 - Judge #7 = avg 27
Judge #2 = avg 30 - Judge #8 = avg 28
Judge #3 = avg 33 - Judge #9 = avg 26
Judge #4 = avg 32 - Judge #10 = avg 23
Judge #5 = avg 34 - Judge #11 = avg 25
Judge #6 = avg 33 - Judge #12 = avg 27

If the SCORE program was storing a judge’s individual or average score based on the member number, a screen could be developed that takes in a list of member id numbers supplied by the organizer and SCORE could be setup to take these numbers and calculate the average scores among the judges present and can put them at the appropriate table based on the average score from past events.

Example after adding the 12 member ids in to SCORE it would even out the average to table #1 having an average score of 176 and #2 an average score of 174 which would most likely eliminate the TOD and TOA.

Table #1 average 176 - Table #2 average 174
Judge #1 = avg 32 - Judge #5 = avg 34
Judge #2 = avg 30 - Judge #10 = avg 23
Judge #4 = avg 32 - Judge #3 = avg 33
Judge #8 = avg 28 - Judge #11 = avg 25
Judge #7 = avg 27 - Judge #6 = avg 33
Judge #12 = avg 27 - Judge #9 = avg 26

Some exceptions:
1. Non CBJ can be given a default average and SCORE could make sure they are distributed evenly to the different tables.
2. CBJ judging their first event (or could easily make it first three events) get a default score.

 

[Lake Dogs]

This is a very good idea and I hope it's something they're seriously considering implementing down the line.

Otherwise, they should probably do what other sanctioning bodies do, and that is that you (as a judge) never sit at the same table twice, and not with the same people... It doesn't statistically account and balance like the approach above does, but it does mix things up so that no 1 table is always high or low...

 

[Podge]

This is a very good idea and I hope it's something they're seriously considering implementing down the line.

Otherwise, they should probably do what other sanctioning bodies do, and that is that you (as a judge) never sit at the same table twice, and not with the same people... It doesn't statistically account and balance like the approach above does, but it does mix things up so that no 1 table is always high or low...

Judges Never sit at the same table twice. I like it. It can be mathematically programmed easy to do so. A judge gets a sheet, he/she sits at tables #1, then #2, then #3 then #4. Reps don't even have to mess with it. Then, My entry then always hits Table #1... It will make alternate #'s easy to do too.. Moonswiners #24 is now "165 table #1"... No need to judge tables and entries that way.. judges juggle themselves.

 

[Garrett]

Now we're on to something.

 

[Lake Dogs]

MiM did this, MBN does this, GBA does this...

Mind you, it's not perfect either. You can still end up with TOD's and TOA's, but is REALLY unlikely to happen more than occasionally here or there...

 

[cpw]

So I have been thinking about this KCBS Table of Death and Table of Angels and have an idea I am thinking about sending to the board of directors but would like your input. I look forward to reading your comments.

I have heard on numerous occasions on how the “Table of Death” (TOD) and “Table of Angels” (TOA) either helped me or hurt me and 99% of the time it is how the TOD hurt them. The TOD and TOA are basically created by pure circumstance of where judges sit after the judges meeting. This purely by chance circumstance can be greatly reduced or eliminated by placing judges at the appropriate table based on the averages of the other judges at the event.

Example of a 12 team event this situation could occur with table #1 having an average of 194 and #2 an average of 156. No need to drop the lowest when calculating averages. In this extreme example, you could have a TOD and TOA at the same event.

Table #1 average 194 - Table #2 average 156
Judge #1 = avg 32 - Judge #7 = avg 27
Judge #2 = avg 30 - Judge #8 = avg 28
Judge #3 = avg 33 - Judge #9 = avg 26
Judge #4 = avg 32 - Judge #10 = avg 23
Judge #5 = avg 34 - Judge #11 = avg 25
Judge #6 = avg 33 - Judge #12 = avg 27

If the SCORE program was storing a judge’s individual or average score based on the member number, a screen could be developed that takes in a list of member id numbers supplied by the organizer and SCORE could be setup to take these numbers and calculate the average scores among the judges present and can put them at the appropriate table based on the average score from past events.

Example after adding the 12 member ids in to SCORE it would even out the average to table #1 having an average score of 176 and #2 an average score of 174 which would most likely eliminate the TOD and TOA.

Table #1 average 176 - Table #2 average 174
Judge #1 = avg 32 - Judge #5 = avg 34
Judge #2 = avg 30 - Judge #10 = avg 23
Judge #4 = avg 32 - Judge #3 = avg 33
Judge #8 = avg 28 - Judge #11 = avg 25
Judge #7 = avg 27 - Judge #6 = avg 33
Judge #12 = avg 27 - Judge #9 = avg 26

Some exceptions:
1. Non CBJ can be given a default average and SCORE could make sure they are distributed evenly to the different tables.
2. CBJ judging their first event (or could easily make it first three events) get a default score.

I'm all for it. I would imagine that every competitor out there would support it as well.

 

[drbbq]

I like the original idea a lot but I think the judges should be ranked on how close they are when compared to the other judges at their table in previous contests instead of average score. It's a little more complicated but surely a program could do it.

 

[New Pal Frank]

If threre are Reps or Board members out there, please correct me if I am wrong.
It is my understanding that if an entry is DQ'd at the table, and given all 1's, it figures into the judges average score for that event.
Aside from that, it sounds good. KCBScore should be able to sort the judges as they check in.

 

[Candy Sue]

That is true above. Board has had discussions regarding cutting DQ out of averaging. A new release of KCBScore is eminent to fix some stuff. Hopefully this can be put in next revision, if it passes the board. Also "handicapping" judges has been discussed as well, but the general opinion is that more data is needed. Next month is the first anniversary of KCBScore!

 

[Rich Parker]

That is true above. Board has had discussions regarding cutting DQ out of averaging. A new release of KCBScore is eminent to fix some stuff. Hopefully this can be put in next revision, if it passes the board. Also "handicapping" judges has been discussed as well, but the general opinion is that more data is needed. Next month is the first anniversary of KCBScore!

I submitted it to the BoD along with a formatted document. Thanks for the comment.

 

[DawgPhan]

I like the original idea a lot but I think the judges should be ranked on how close they are when compared to the other judges at their table in previous contests instead of average score. It's a little more complicated but surely a program could do it.

Yeah, you need to be a little better than just using averages.

There is no context to an average.

And this is still just trying to mitigate the damage done by judges that are not judging according to the current KCBS standards. Both high and low scoring.

 

[Kave Dweller]

I like the original idea a lot but I think the judges should be ranked on how close they are when compared to the other judges at their table in previous contests instead of average score. It's a little more complicated but surely a program could do it.

Or you could just say you need to have judged 10 comps to have a "weighted" average. Less then 10 comps you're working on an average like the rest of the newbie's?

 

[rob g]

And this is still just trying to mitigate the damage done by judges that are not judging according to the current KCBS standards. Both high and low scoring.

Having recently taken the KCBS judging course there don't seem to be any hard and fast "standards". They are very reluctant to tell you what constitutes good BBQ but leave it up to you to make your own decision. Having recently sat through another CSBBQA judging course, as an administrator, it seems there is a lot more that can be done to show and explain what good BBQ is and likewise what bad BBQ is. While taste is a very personal decision a lot more can be taught about texture/tenderness and appearance. Just saying a box looks good doesn't mean a lot. Explaining symmetry, even saucing etc. would go a long way to giving judges a better understanding of what to look for. I'm sure the judging classes also vary depending on who the instructor is.

 

[Rich Parker]

Yeah, you need to be a little better than just using averages.

There is no context to an average.

And this is still just trying to mitigate the damage done by judges that are not judging according to the current KCBS standards. Both high and low scoring.

This idea came to mind because of all the talk about judges not judging up to a standard. We can't force someone to think a box looks like a 9 or tastes like a 9 as that is going to be different on anyone including you and I. The only criteria a judge can be trained on is tenderness and that is because there are examples such as biting a rib in the middle and not falling completely apart, pushing the pork to the roof of your mouth and seeing if it sticks, or pulling a slice of brisket apart to see how tender it is.

We have to move past thinking we can teach people on taste and appearance before we can find a solution.

 

[DawgPhan]

This idea came to mind because of all the talk about judges not judging up to a standard. We can't force someone to think a box looks like a 9 or tastes like a 9 as that is going to be different on anyone including you and I. The only criteria a judge can be trained on is tenderness and that is because there are examples such as biting a rib in the middle and not falling completely apart, pushing the pork to the roof of your mouth and seeing if it sticks, or pulling a slice of brisket apart to see how tender it is.

We have to move past thinking we can teach people on taste and appearance before we can find a solution.

You can have a standard for taste and appearance and judge towards that standard. KCBS doesnt, but that doesnt mean you can't and it doesnt have to limit the creativity of a cook.

Just like tenderness can mean different things to different people and is judged differently in different sanctioning bodies because they set different standards.

A KCBS 9 in tenderness is not an MBN 9 in tenderness for ribs because the standards are set differently.

 

[Rich Parker]

You can have a standard for taste and appearance and judge towards that standard. KCBS doesnt, but that doesnt mean you can't and it doesnt have to limit the creativity of a cook.

Just like tenderness can mean different things to different people and is judged differently in different sanctioning bodies because they set different standards.

A KCBS 9 in tenderness is not an MBN 9 in tenderness for ribs because the standards are set differently.

Does other sanctioning bodies have standards for taste? I have never heard such a thing and can't see how what I think is a 9 can match what someone else will think as a 9 every time.

 

[Smoke'n Ice]

As long as the reps and BOD (same difference) do not have to do any extra work or think, then it would work. If the reps were required to supervise the movement then the Clint Eastwood movie "Heart Break Ridge" and the 'Cluster F%$3' line comes to mind. Shoot, they are even against the comment cards because it would require extra work.

 

[TooSaucedToPork]

A few things need to be done. The sorting judges according to average is one that has been talked about since KCBS score came out.

The next, and biggest thing is mandatory judge testing. Every year judges have to log in and take a test with randomly generated questions. There should also be a portion that reviews the basic judgment rules.

Pass...you are allowed to judge for a year...until the next test
Fail and you are taken to a continuing education course.Complete it Then you can take the test again.

You want across the board fairness and improved scores...focus on a part of the KCBS Mission Statement...EDUCATION.

 

[Burnt at Both Endz]

Maybe I'm over thinking this averaging stuff....But it seems to me that judges that set at a table and get 3 or 4 top cooks is gonna end up with a higher average then those that judge a small town community comp. Not sure they are better or worse judges, just the luck of the entries per say.

 

[Rich Parker]

Maybe I'm over thinking this averaging stuff....But it seems to me that judges that set at a table and get 3 or 4 top cooks is gonna end up with a higher average then those that judge a small town community comp. Not sure they are better or worse judges, just the luck of the entries per say.

The judges average would be over all 4 entries at multiple contests such as 3 which then would even out if they got all good or all bad entries. I have talked to multiple judges who say the newer judges score lower so the BoD could put average of 3 events so the newer judges with less than 3 events would get a default average set by BoD when seating the judge. No system is perfect just need to limit the chance or risk of a TOA or the dreaded TOD.