Tie breaking

[DawgPhan]

With the group of fine cooks in this contest, yes a good team was gonna be at the top...with that said, it's pretty apparent to me that if you didn't hit a hot table, you had no chance at all in being in the top 10. That is just a fact any way you want to read the statistics.

But the assumption in all of this is that regardless of what boxes hit that table, that was going to be a hot table.

You have to make that assumption because you dont have any data to support the conclusion you are making otherwise.

 

[musicmanryann]

after all that though, doesn't that seem to be a pretty standard distribution? Some tables are going to score high, some score low, and most score right in the middle.

Why is one table scoring higher or lower than another table a problem?

You want to and do see a standard distribution of scores within each table at a contest, but across tables, ultimately you want to see those curves align/overlap as close as possible. The overall quality of the food or present trends in scoring patterns may skew those curves one way or another (throwing out outliers such as DQ's), but each table should present the same curve with the same mean if each is judging equally. If they do not, that suggests to me, tables are not consistently scoring and there is an advantage if you hit the tables with a higher mean and disadvantage if you hit a table with a low mean. This is not an issue if you have a 24 team contest every team hits each table once, but the larger the contest the more important it is to "hit the right tables". This is a notion that was present amongst teams long before KCBScore came out. The KCBScore has merely confirmed it.

To add, the scoring data of the almost 40 contests we have competed in since KCBScore came out and through conversations we have had with many other teams regarding countless other contests does show generally the same distribution that Vince has presented here. It was observed at the very first contest we saw KCBScore (which we won), and all the way to the last contest we competed in this season, The Jack. You can call them good/bad tables, hot/cold tables, TOD/TOH or whatever you want. But the pattern I have seen, regardless of whether we win or lose is that in order to win you have to Hit one of the good tables and miss the bad ones, or hit the good table(s) and survive the bad, or hit all four average scoring tables (preferably on the higher end of average). If you hit a bad one and miss the good, you aren't taking home a big check that day. To be honest, I didn't even think this was a question anymore--the question now is what to do about it, if anything.

While late last season and early this season I was getting pretty down about it, at this point I am in the same boat as brother Steph. While I am still aware of the issue, I try to do the best I can do and enjoy myself and this hobby I love as much as possible, and hope for the best results with no expectations. No sense in beating a dead horse until the issue gets adequately addressed by the board. I have my ideas, but for now my mantra is SUAC! :becky:

 

[DawgPhan]

You are still also assuming that the table scoring is set prior to the boxes hitting the table.

The table scoring is determined by the quantity of quality boxes that hit it.

 

[CBQ]

I know all three of you are d@mn good brisky cooks but out of curiosity did you all hit different tables?

I was on the same table as Little Pig Town. Quau was a different table.

 

[CBQ]

I'm astonished that with the current overscoring problem, and previous contests in different areas of the country that have had multiple 180s, this hasn't been addressed already.

It's a new problem for us, not generally an issue in the northeast. :becky:

 

[CBQ]

Regardless of the tie breaker system, it was great they they called you all up there and recognized you and allowed the decision of how to divide up the proceeds ... I think when you hit the payout calls, ties should be split as ZBQ mentioned - much like golf purse's are divided.

Yes, the organizer asked us if we wanted to accept the computer's verdict or split the prize pool before telling us the results. They also announced to the crowd that the three teams had tied with "perfect perfect" scores. I thought it was a good way to handle it.

The teams then stepped away to confer. I said "chop the pot", something that happens a lot in poker where prize winners at a final table will agree to divide a prize pool rather than gamble on random cards. (Poker is a game of skill, but when it comes down to the end game, chance plays a bigger role.) I thought it was a fair solution for everyone.

 

[CBQ]

IMO, it's just coincidence that the rank was alphabetical.

I thought it was, but the reps told us it was alphabetical.

 

[musicmanryann]

You are still also assuming that the table scoring is set prior to the boxes hitting the table.

The table scoring is determined by the quantity of quality boxes that hit it.

Of course the table scoring is determined by the quality of boxes that hit it. However, the quality, or lack thereof, of food samples available at a contest --prior to being judged--is also normally distributed. This is exactly why you would expect and do see a normal distribution of scores within a table, as I stated before. Because of this the probability of each table receiving 24 unique samples of the same quality of food--some good, most average, and some bad is equal. It is highly improbable that a table is going to get 24 unique samples food that is all below average and bad or above average and excellent, when compared to the scores given out to other tables at a contest. Yet this is what is consistently being observed from contest to contest. Of course there is that probability, so you may see a few isolated instances of it from time to time, (clustering of higher scoring due to high quality of food or vice-versa) but not at the present rate.

 

[CBQ]

IMO, the contest rep should be applauded for doing the fairest thing and this should be followed at all contest. Announce a tie, split the money, both get the points.

Announcing the tie and allocating the price money is, I believe, in the control of the organizer. Awarding the points isn't. That was done by KCBS is the order the computer mandated.

As for trophies, give the trophy to the team who is furthest from home, the organizer can have duplicates made and mailed to the other team(s), that is, if the team wants the trophy.

They let the teams decide how to handle the trophy, and I think that worked out OK. (Although we would have gotten it using the "furthest from home" suggestion. :becky: ) I haven't approached the organizer about having a duplicate made, although I have thought about doing so.

 

[musicmanryann]

Announcing the tie and allocating the price money is, I believe, in the control of the organizer. Awarding the points isn't. That was done by KCBS is the order the computer mandated.



They let the teams decide how to handle the trophy, and I think that worked out OK. (Although we would have gotten it using the "furthest from home" suggestion. :becky: ) I haven't approached the organizer about having a duplicate made, although I have thought about doing so.

In the PGA Tour, ties split the money and also split the FedEx Cup (TOY) points. This is how I feel it should be done, and get rid of the tie-breaker altogether.

 

[goodsmokebbq]

While it is very possible that a single table could be hit with all excellent, average or subpar entries in a single category, it is unlikely for that to be the case over the entire contest.

The judges are normally distributed and so is the quality of the food. That means there is the exact probability of a food TOD as there is for a judge TOD. To put this all on the judges is premature.

Our problem is that we are not taking enough samples to ensure that each tables mean approaches the population mean. 6 judges is way to few. It is a limit of random sampling. Theory guarantees TOD/TOH with only 6 samples.

We need to minimize the inter-judge variance or increase the samples to even out the tables means. The later will probably not happen, so we have to focus on the first. I think we can optimize the system we have if we can get our hands on some data (which ZBQ and others have been cheer leading for). There are many factors to sort out.

 

[CBQ]

Yhere has to be a way to minimize that from happening and allow the quality of the food to dictate the outcome as opposed to a table of judges that are all scoring everything super high or super low.

I think there is, and I have suggested that in a few threads, including here: http://www.bbq-brethren.com/forum/showthread.php?t=200251

 

[DawgPhan]

Of course the table scoring is determined by the quality of boxes that hit it. However, the quality, or lack thereof, of food samples available at a contest --prior to being judged--is also normally distributed. This is exactly why you would expect and do see a normal distribution of scores within a table, as I stated before. Because of this the probability of each table receiving 24 unique samples of the same quality of food--some good, most average, and some bad is equal. It is highly improbable that a table is going to get 24 unique samples food that is all below average and bad or above average and excellent, when compared to the scores given out to other tables at a contest. Yet this is what is consistently being observed from contest to contest. Of course there is that probability, so you may see a few isolated instances of it from time to time, (clustering of higher scoring due to high quality of food or vice-versa) but not at the present rate.

You dont need 24 samples to be above or below average. You just need to get more than the next table to be the "bad" table. So if you happen to be at a table that gets 2 burnt overseasoned rib boxes and then you follow that up with 3 really really awful brisket boxes and the rest of the boxes are just average, cooks are going to get on the internet and complain about your table of death when the reality is that multiple cooks turned in food that was not competition quality and likely screwed over their fellow competitors by turning in garbage.

I know the cooks want to wave these scoresheets and wag their fingers at judges but the data simply doesnt support it. You dont know enough about what happened to make a claim about the quality of judging.

 

[musicmanryann]

The judges are normally distributed and so is the quality of the food. That means there is the exact probability of a food TOD as there is for a judge TOD. To put this all on the judges is premature.

Our problem is that we are not taking enough samples to ensure that each tables mean approaches the population mean. 6 judges is way to few. It is a limit of random sampling. Theory guarantees TOD/TOH with only 6 samples.

We need to minimize the inter-judge variance or increase the samples to even out the tables means. The later will probably not happen, so we have to focus on the first. I think we can optimize the system we have if we can get our hands on some data (which ZBQ and others have been cheer leading for). There are many factors to sort out.

I agree with everything you said, but because each team only hits a table once, there are ~24 samples, not 6. So yes, while it is much more probable that in one category a table receives a bad batch or really great batch of food, the chances of this occurring across all four categories is very low. In fact, your chances of receiving the opposite quality of food increase, assuming that teams are consistently cooking the same or similar quality of food across the categories. Of course this varies based on the number of teams at a contest.

 

[Burnt at Both Endz]

You dont need 24 samples to be above or below average. You just need to get more than the next table to be the "bad" table. So if you happen to be at a table that gets 2 burnt overseasoned rib boxes and then you follow that up with 3 really really awful brisket boxes and the rest of the boxes are just average, cooks are going to get on the internet and complain about your table of death when the reality is that multiple cooks turned in food that was not competition quality and likely screwed over their fellow competitors by turning in garbage.

I know the cooks want to wave these scoresheets and wag their fingers at judges but the data simply doesnt support it. You dont know enough about what happened to make a claim about the quality of judging.

The problem with your thinking is you aren't seeing the print out that us cooks are seeing. We see "who" hits the tables and how they score those boxes. The top cooks here in the Mid-West are pretty dang consistent with their cooking. In fact, good enough to tell how a table scores,IMHO!

 

[musicmanryann]

So if you happen to be at a table that gets 2 burnt overseasoned rib boxes and then you follow that up with 3 really really awful brisket boxes and the rest of the boxes are just average,

If the quality of food presented at the entire contest as a whole is normally distributed, the chances of this occurrence is very low. Not saying is doesn't actually happen, but probably not at the present rate observed.

It has been a long-standing policy for teams to only hit a table once in a contest. This is because it has been known for a long time, long before I was a part of this game, that some tables give you an advantage, whereas others a disadvantage. If this wasn't the case why would it matter if you went to the same table every time? This is nothing new, but now we have the data that is bringing the issue in to the limelight.

 

[gettinbasted]

The problem with your thinking is you aren't seeing the print out that us cooks are seeing. We see "who" hits the tables and how they score those boxes. The top cooks here in the Mid-West are pretty dang consistent with their cooking. In fact, good enough to tell how a table scores,IMHO!

I Think this is where the statistics end and common sense comes in. In osawatomie I hit a chicken table with Dirt Road, Pellet Envy, Thump & Strum and Shake'n Bake. All are very good chicken cooks. The scores were in the low 160s. I know we all had good stuff because I tasted most of it. You can then see that the same table scored low across the board so an assumption can be made. That being said, I think we are all pissing in the wind. I don't think we will ever have enough data on judges to fix it. Good and bad tables are part of the game we play.

 

[Burnt at Both Endz]

I Think this is where the statistics end and common sense comes in. In osawatomie I hit a chicken table with Dirt Road, Pellet Envy, Thump & Strum and Shake'n Bake. All are very good chicken cooks. The scores were in the low 160s. I know we all had good stuff because I tasted most of it. You can then see that the same table scored low across the board so an assumption can be made. That being said, I think we are all pissing in the wind. I don't think we will ever have enough data on judges to fix it. Good and bad tables are part of the game we play.

I thought ya'all had an off day and it was just my pork that sucked.:wacko:

 

[DawgPhan]

The problem with your thinking is you aren't seeing the print out that us cooks are seeing. We see "who" hits the tables and how they score those boxes. The top cooks here in the Mid-West are pretty dang consistent with their cooking. In fact, good enough to tell how a table scores,IMHO!

<whisper> I cook competitions </whisper>

 

[Qbert60]

We all hit both types of tables. Is there a way to control it? Probably not one that is either going piss judges off or piss cooks off.

What concerns me is how look at the stats that Vince posted, the 3rd place team didn't have 1 "Cold" table. Is he supposed to feel lucky? or does he just celebrate the top 3 finish. Is it a tainted finish because his results weren't as "balanced" as everyone else's?

Tie-breakers? Easy. Rock-paper-scissors and if there are more than 3 add lizard, spock for my Big Bang Theory friends out there.