De-constructing a Popular Preseason Poll

HawkKing81, Jon is actually making the point that they aren't being consistent. While we can agree or disagree with making a projection or ranking for today as the model, you have to be consistent with that. If you are going to flat out say this is based on where things are today, then what is your criteria for a good program? Returning Starters, Key Losses and maybe a little bit of momentum from last year. How could CFN then justify Oklahoma, Georgia, Florida or even Wisconsin being as high as they are?

You can either rank teams based on what they have shown up to this point (no predictions about how the season will go) or you project the entire season for all teams (like Deace does) with wins / losses. We can discuss the merits of either system, but whatever the logic is used it has to be consistent.

So if you are going to use a rank-based system, then you can only use past performance as an indicator. Most voters and polls don't, which is why they are so frustrating and poorly done. If Georgia loses the 2nd week to South Carolina, then they need to drop out of the rankings completely even if they are ranked 3rd in the country before that game. This is because the only criteria you have to judge the team would be a win over Louisiana-Lafayette (Sunbelt Conference) and a loss to South Carolina. This isn't what often happens. Instead CFN and others will justify keeping Georgia ranked by saying the team will win a bunch of other games in the season. They are now projecting and not being consistent with their criteria.

So if you want to use the criteria is only what has been done, then there needs to be extreme flexibility early in the season and there is absolutely no justification for what happened last year where 4-0 Iowa was ranked behind 3-1 PSU after beating them in Happy Valley by 11 points.
 
bhawk326 said:
HawkKing81, Jon is actually making the point that they aren't being consistent. While we can agree or disagree with making a projection or ranking for today as the model, you have to be consistent with that. If you are going to flat out say this is based on where things are today, then what is your criteria for a good program? Returning Starters, Key Losses and maybe a little bit of momentum from last year. How could CFN then justify Oklahoma, Georgia, Florida or even Wisconsin being as high as they are?

You can either rank teams based on what they have shown up to this point (no predictions about how the season will go) or you project the entire season for all teams (like Deace does) with wins / losses. We can discuss the merits of either system, but whatever the logic is used it has to be consistent.

So if you are going to use a rank-based system, then you can only use past performance as an indicator. Most voters and polls don't, which is why they are so frustrating and poorly done. If Georgia loses the 2nd week to South Carolina, then they need to drop out of the rankings completely even if they are ranked 3rd in the country before that game. This is because the only criteria you have to judge the team would be a win over Louisiana-Lafayette (Sunbelt Conference) and a loss to South Carolina. This isn't what often happens. Instead CFN and others will justify keeping Georgia ranked by saying the team will win a bunch of other games in the season. They are now projecting and not being consistent with their criteria.

So if you want to use the criteria is only what has been done, then there needs to be extreme flexibility early in the season and there is absolutely no justification for what happened last year where 4-0 Iowa was ranked behind 3-1 PSU after beating them in Happy Valley by 11 points.
Click to expand...

Bhawk, I think you and I are saying the same thing, but this isn't what Jon's article was saying.

Jon says "I am one that prefers to make preseason rankings based upon my projections of where I think the teams will finish." This is what I disagree with and this is what CFN is not doing. I like Deace's "predictions", but those don't factor into "rankings" and I think the two have to be separate.

CFN is ranking teams. Again, I think it's ok to disagree with what they have concluded, I for one think Iowa should be a top 5 team for what they did last year and for who they have returning. But they are right to not include what their schedules look like.
 
Last edited:
jon acknowledges that many other sources don't do their pre-season stuff like he does. you're not getting the point here at all
 
I actually have no problem with Jon's system though, as long as he is being consistent with his criteria. For Jon and Steve they have a ranking that is based on playing out the whole season, then as the year goes along and team wins or loses as expected or against expectations then that information should get added in. They would then project the rest of the season based on available data and re-rank the teams according to their new final projections. Under their system (if they are right about the year a given team will have) they will finish the year with a ranking similar to what they had at the beginning of the year.

What bothers me with the way CFN and a lot of other polls approach their rankings is that under the no projection system there should be wide swings in rankings from week to week. Also, there is absolutely no justification why a 1-1 or 2-1 team should be ranked ahead of a 2-0 or 3-0 team which happens quite commonly in these polls.

I think the biggest problem in all of this is that whatever system is used should finish the season with teams ranked appropriately which more often than not it doesn't. If you use a no projection criteria, then your preseason rankings should be meaningless after games are played, but that is not what happens. Instead of looking at a 1-1 Georgia team, CFN and many like it see a 1-1 Georgia team they ranked at #3 to start the season so now it is 15th or 18th instead of off the poll.
 
The problem with not using full-season projections is that under the current system we get ranking bias. Teams aren't moved based on the entirety of their past performance for the season, but instead on how they were ranked last week and how they did this past week. This would be fine if polling were started late enough in the season to remove the bias of preseason opinion. This isn't what happens though. The pollsters are looking at how #3 Georgia did against South Carolina. Well they lost so that drops them 12 spots. That is extremely flawed logic since the #3 ranking was based off of complete conjecture on their talent. They have proven to the pollster that they don't have the talent to beat South Carolina so there performance shows them with 1 win over Louisiana-Lafayette.
 
I personally think the projection system is the way to go since you are forced to each week re-evaluate how the past performance (data grows each week) will dictate future events. To me this removes the bias of preseason polls based purely on talent evaluation. Also, if accurate with predictions, the system has fewer drastic rankings swings that are against a voter's biased nature and are required for using only up-to-date performance to determine rankings.
 
Take a look at their evaluation of Boise State (8th). Their reason for not picking them #1 is because of the stigma of being in WAC. Talk about bias ... oh wait they do. In their reasoning for not putting Boise State #1, they actually say they would throw out the bias if they beat VT and Oregon State. What about beating TCU and Oregon each twice in the last 2 seasons and beating Oregon State and Oklahoma in 2006? How does that not remove CFN's bias now, but future results (beating VT and Oregon State) this year would remove it.

HawkKing81, there are many things we could disagree about on this board, but hypocrisy is something I think we can all agree to absolutely abhor. This is why CFN and Pete Fiutak suck, not that there is anything wrong with them using up-to-date criteria as their rankings.

I don't want to speak for Jon, but I'm guessing that's what upset him more than what they state is their criteria. If they were being consistent, that would be a different story and maybe Jon wouldn't have put so much time and effort into his critique. In that case he probably would have just stated he disagreed with their method instead of show how poorly they adhered to their own stated criteria.
 
bhawk326 said:
I actually have no problem with Jon's system though, as long as he is being consistent with his criteria. For Jon and Steve they have a ranking that is based on playing out the whole season, then as the year goes along and team wins or loses as expected or against expectations then that information should get added in. They would then project the rest of the season based on available data and re-rank the teams according to their new final projections. Under their system (if they are right about the year a given team will have) they will finish the year with a ranking similar to what they had at the beginning of the year.

What bothers me with the way CFN and a lot of other polls approach their rankings is that under the no projection system there should be wide swings in rankings from week to week. Also, there is absolutely no justification why a 1-1 or 2-1 team should be ranked ahead of a 2-0 or 3-0 team which happens quite commonly in these polls.

I think the biggest problem in all of this is that whatever system is used should finish the season with teams ranked appropriately which more often than not it doesn't. If you use a no projection criteria, then your preseason rankings should be meaningless after games are played, but that is not what happens. Instead of looking at a 1-1 Georgia team, CFN and many like it see a 1-1 Georgia team they ranked at #3 to start the season so now it is 15th or 18th instead of off the poll.
Click to expand...

Look we both agree that the polls should be re-evaluated every week and if there calls for a drastic change, then their should be one. I think it even states this in the by-laws of the AP voter instructions. And I agree with you that the pollsters don't do this. People that say "it will all work itself out in the end" are wrong. It almost never does. In polls if you start out lower than you should you almost never finish higher than where you would have if you started higher and vice versa.

But I don't see why Jon's system would do a better job of that then the other system.

I get your points about why you like Jon's system. I think for both of us, we just want to make sure that at the end of each week that the pollsters re-evaluate the games played and re do their rankings.

My point is that in Jon's system, teams with easy schedules are over-rated TODAY and teams with difficult schedules are under-rated TODAY. They might not be by the end of the year, but that's the way it is today and for me I have an issue with that.

Rankings today affect hype. Hype is huge for a program, sometimes bad, but usually good. Hype can affect recruiting, it can affect the teams play, etc. Which again, I think you have to base your rankings on how good you think the team is TODAY, not how they will finish the season.
 
Last edited:
HawkKing81 said:
First of there is also a difference between "projections" and "predictions". It is ok to project how good you think a team is going to be, based on the talent they have, but it is not ok to rank a team based on their upcoming schedule.

That's why you MUST not include upcoming schedule in rankings.
Click to expand...

I agree. I think a lot of the confusion is the difference between predictions, projections, forecasts and rankings. I liked Jon's article article, but he seemed to confuse projections and predictions while at the same time talking about rankings. These are not the same thing.

Jon, like myself, prefers a projection model that forecasts where the team will finish the season. That's not a ranking, but rather a projection.

There's nothing wrong with ranking a team based on their talent right now. Ignore the schedules, which team is best, 1-25. That's a ranking. Schedule should have nothing to do with a ranking. I'm not saying the rankings Jon is talking about is something I agree with. I didn't pay much attention to the rankings. A ranking isn't about where a team is projected to finish. It can't be about that.

I don't think there's too much disagreement here other than some confusion. Jon prefers a projection. Me too. Since we prefer those, neither of us are particularly fond of rankings. CFN does rankings.
 
To put it another way, the objective data that is used in rankings is the relative strength of each team while the objective data used in projections, forecasts (and even most predictions) includes the relative strength of each team and their schedules.

Neither method is wrong. It's just a completely different way of organizing information.
 
HawkKing81, I appreciate the discussion. I agree with you that in an ideal world, no poll would start until at least 4 weeks into the season (maybe not even until after week 6). If this were the case either system would lend itself to better determining the ranking. Either you have X weeks of data to make a ranking based on prior results or you have that data to predict the rest of the season and still have an accurate ranking.

Sadly, I don't think this is going to happen in reality. In that case you are left with the previous data v predicting the entire season model. The problem with the previous data is that pollsters have shown themselves time and time again incapable of removing their preseason bias as you have pointed out. If voters are instructed to rank the teams based on how they will finish the season then I think there would be a better chance that they would look at schedules, match-ups and have a better understanding about the team in context to all things that go into their ranking.

Regardless of system, we are assuming that voters and rankings panelists are actually going to spend more than 5 minutes filling these things out. That doesn't happen and we are left with unqualified people spending absolutely no time critically thinking about where teams should be ranked. If this was just about having a ranking to hold over other fan base's head then it wouldn't be such a big deal. But as you mentioned rankings mean everything in college football, so why are rankings allowed to be so extremely flawed?
 
I agree the Hawkeyes are a double digit team. Probably not far from the top 10. But there are some big question marks. Particularly the O'line.

My pet peeve is Boise State. They played one tough game in the reg season last year. This year they will play two. Then they will cruise to a easy conference win. And whine about how no one will play them and the BCS gives them no respect.
 
mb21, there is no problem with having a ranking as you stated, but what happens is that preseasons rankings conducted that way bias further polling. As I stated above, and we have seen numerous examples of this, is that losses early in the season only move teams from their preseason positions instead of completely change the polls as they should. This bias is what makes rankings done this way hard to defend.

The big problem is that CFN and many other rankings aren't actually using the objective data they claim. Read #3 Georgia's profile on the website and they don't provide adequate reasons why up to today the team should be ranked that high. Also, #8 Boise State's profile exposes the bias they are trying not to have with a current ranking anyway.
 
bhawk326 said:
mb21, there is no problem with having a ranking as you stated, but what happens is that preseasons rankings conducted that way bias further polling. As I stated above, and we have seen numerous examples of this, is that losses early in the season only move teams from their preseason positions instead of completely change the polls as they should. This bias is what makes rankings done this way hard to defend.

The big problem is that CFN and many other rankings aren't actually using the objective data they claim. Read #3 Georgia's profile on the website and they don't provide adequate reasons why up to today the team should be ranked that high. Also, #8 Boise State's profile exposes the bias they are trying not to have with a current ranking anyway.
Click to expand...
I'm not defending CFN's rankings. As I said, I didn't read them closely.

As for the bias issue, the only real way to avoid that kind of bias is for computerized rankings, independent of human rankings. I favor that idea and have for a long time. With all the information we have today about college football and the technology at our hands, computers could do what the humans have been messing up. There would still inevitably be a few examples where they get it wrong. But there would be fewer mistakes.
 
Actually I enjoy the human rankings because I like to know why people vote the way they do. I just think all these polls should publish each voter's ballot. Look what happened after the AP poll did this last year. Scott Wolf, the USC writer, kept making the worst pollster list College Sports Polls, College Football Rankings, BCS Rankings and when questioned on his own website he routinely exposed his bias and lack on intelligence.

I think it would be great if they did the same thing with the ESPN/Coaches Poll. Could you imagine if a coach got asked about him being last week's worst pollster at his presser? How many coaches who "vote" even know what their ballot says with the SID filling it out most of the time? That poll might get better if there was some sunshine on it.

Also, while computerized rankings are great, do you remember what happened earlier this decade? A field goal by Fresno State ended up determining who went to the national championship in 2003 (LSU over USC).
 
Correct me if I'm mistaken, but didn't that poll consider heavily the human polls too? But there are going to be errors no matter what. You're right. We can't stop that from happening no matter what system we devise to rank the teams. I just think we end up with fewer errors with a computerized system than one that includes human beings. At the very least, the computer only considers information we tell it to. Humans can consider whatever silly information they choose when they construct polls and as we know, they actually do that.

I fully support the coaches and writers still having a poll, but just for the sake of humor.

I know what I want is never going to happen though.
 
mb21 said:
I agree. I think a lot of the confusion is the difference between predictions, projections, forecasts and rankings. I liked Jon's article article, but he seemed to confuse projections and predictions while at the same time talking about rankings. These are not the same thing.

Jon, like myself, prefers a projection model that forecasts where the team will finish the season. That's not a ranking, but rather a projection.

There's nothing wrong with ranking a team based on their talent right now. Ignore the schedules, which team is best, 1-25. That's a ranking. Schedule should have nothing to do with a ranking. I'm not saying the rankings Jon is talking about is something I agree with. I didn't pay much attention to the rankings. A ranking isn't about where a team is projected to finish. It can't be about that.

I don't think there's too much disagreement here other than some confusion. Jon prefers a projection. Me too. Since we prefer those, neither of us are particularly fond of rankings. CFN does rankings.
Click to expand...

His point was that the poll specifically said it ONLY was going to consider how good a team is RIGHT NOW, then predicted the team's record. But there was an awful lot of projection/predictions going on. Florida and Texas were the two prime examples. They are NOT top 5 teams right now. You don't lose as much as those two teams did and be THAT good right now. They may very well end up that way by the end of the season. But that's projecting how good the team might be, not how good they are right now.
 
mb21, USC was actually #1 in both human polls and #3 in the computers. As you stated though the computers evaluated the results based on a set of criteria so it is sound in that least. The tough part becomes weighing all variables. That year, Oklahoma lost its conference championship game by 28 points to KSU and was still the #1 team in the country thanks to the computers, so something doesn't pass the sniff test there.

I like your idea of having computerized polls, but there needs to be more transparency on how they are calculated and simulations should be run and tested based on prior years to make sure the public and insiders can agree things are weighted in a way that generates highly ranked teams that deserve to be so.

Alas, you are right and this isn't going to happen. I think we would all be happier and better off if we could see ESPN ballots each week like the AP so there can be the proper scrutiny.
 
You must log in or register to reply here.

Latest posts

Share this page

Top