PFF Player Grades: Petras 10/224 Eligible NCAA Quarterbacks this Season

[Luftgekuehlt67]

Petras is better than he gets credit for. In my opinion, 75-ish percent of a QB's effectiveness is not making huge mistakes. Petras doesn't make tons of big mistakes, which lets our defense and punter work their magic.

That said, Pro Football Focus is a sham. They claim their people analyze every player on every snap of every game all season long. There were on average 168 offensive and defensive snaps per game last year.

168 snaps * 22 players * 272 regular season games = 1,005,312 evaluations just in the regular season not counting special teams or post season. Nope.

TJ Lang has put some good commentary out there as far as how bullshit they are in grading, especially since other teammates affect players being graded, the graders do not have football credentials, and they also have no idea what anyone’s assignments are on any given play

Lang went on a radio show talking about it, and offered to fly in one of PFF’s OL graders (all expenses paid) to sit down with him and grade a player together for a full game. They declined. He then asked if he could do the same thing, grade separately, and then compare afterwards. They also declined, and this time said that all of their OL graders reside in India and due to the rigorous schedule during the season it wouldn’t work. Dead serious, they have people in some shithole in Delhi or wherever grading their games and they even admitted it publicly.

PFF is a lot like how a professional whisky taster will never do a blind test because they know they’d be outed by picking Cutty Sark over Lagavulin. I saw a “professional” spirits taster on YouTube who agreed to taste test vodkas get completely embarrassed by grading 4 vodkas all completely differently, even so far as talking about the different flavors and “notes,” only to find out later all four were poured from the exact same bottle of $6 Popov.

PFF’s secret method, “ancient Chinese secret” is kinda bullshit but their marketing department does a bang up job getting people to drink the Kool-Aid.

If you guys want a somewhat valid ranking system, use Football Outsiders. Their rankings are 100% based on stats with no subjective analysis applied whatsoever. They even share their methods.

I had no idea. I'm glad you posted this, because I was considering a sub.

Sounds like it might be better than nothing, be perhaps not up to the marketing hype.

Their UI is very slick and professional, I kinda fell for it, I guess.

I sub to KenPom and have really enjoyed looking at their numbers. Ironically, the KenPom UI is a mess...maybe that's a tell!

Does anyone know of a good CFB analytics service? Based on how much I've had with KenPom numbers, I would be willing to pay a sub for a good CFB equivalent.

 

[JoeyLabasForPresident]

I had no idea. I'm glad you posted this, because I was considering a sub.

Sounds like it might be better than nothing, be perhaps not up to the marketing hype.

Their UI is very slick and professional, I kinda fell for it, I guess.

I sub to KenPom and have really enjoyed looking at their numbers. Ironically, the KenPom UI is a mess...maybe that's a tell!

Does anyone know of a good CFB analytics service? Based on how much I've had with KenPom numbers, I would be willing to pay a sub for a good CFB equivalent.

Overall, they don't really provide much you can't find elsewhere, other than straight up grading for all players, which as Fry pointed out isn't the highest level of grading out there, and certainly has it's flaws. Their NFL stuff is definitely more trustworthy, and I think their partnership with the NFL guarantees that. They have like six different subs for different data sets though, so keep that in mind as well. If you are only interested in college coverage I'd skip it.

 

[Fryowa]

Overall, they don't really provide much you can't find elsewhere, other than straight up grading for all players, which as Fry pointed out isn't the highest level of grading out there, and certainly has it's flaws. Their NFL stuff is definitely more trustworthy, and I think their partnership with the NFL guarantees that. They have like six different subs for different data sets though, so keep that in mind as well. If you are only interested in college coverage I'd skip it.

One issue that any rating system is going to have is non skill positions. There's no metric for OL blocking that anyone could possibly give unless you're a high level coach or former high level player. If a guy lets a DT by him and gives up a sack, how do we know that someone else didn't have that assignment or whether there was a mixup in the backfield. Even something like QB hurries is kinda bullshit. Who determines if it was a hurry and where do you draw the line? Or a pass defended. Sometimes the WR is just as much at fault.

 

[kicker22]

Petras is better than he gets credit for. In my opinion, 75-ish percent of a QB's effectiveness is not making huge mistakes. Petras doesn't make tons of big mistakes, which lets our defense and punter work their magic.

That said, Pro Football Focus is a sham. They claim their people analyze every player on every snap of every game all season long. There were on average 168 offensive and defensive snaps per game last year.

168 snaps * 22 players * 272 regular season games = 1,005,312 evaluations just in the regular season not counting special teams or post season. Nope.

TJ Lang has put some good commentary out there as far as how bullshit they are in grading, especially since other teammates affect players being graded, the graders do not have football credentials, and they also have no idea what anyone’s assignments are on any given play

Lang went on a radio show talking about it, and offered to fly in one of PFF’s OL graders (all expenses paid) to sit down with him and grade a player together for a full game. They declined. He then asked if he could do the same thing, grade separately, and then compare afterwards. They also declined, and this time said that all of their OL graders reside in India and due to the rigorous schedule during the season it wouldn’t work. Dead serious, they have people in some shithole in Delhi or wherever grading their games and they even admitted it publicly.

PFF is a lot like how a professional whisky taster will never do a blind test because they know they’d be outed by picking Cutty Sark over Lagavulin. I saw a “professional” spirits taster on YouTube who agreed to taste test vodkas get completely embarrassed by grading 4 vodkas all completely differently, even so far as talking about the different flavors and “notes,” only to find out later all four were poured from the exact same bottle of $6 Popov.

PFF’s secret method, “ancient Chinese secret” is kinda bullshit but their marketing department does a bang up job getting people to drink the Kool-Aid.

If you guys want a somewhat valid ranking system, use Football Outsiders. Their rankings are 100% based on stats with no subjective analysis applied whatsoever. They even share their methods.

Of course, because when I think of those with superior football knowledge I immediately think India. Great stuff Fry!!!

 

[cincyhawk]

Petras is better than he gets credit for. In my opinion, 75-ish percent of a QB's effectiveness is not making huge mistakes. Petras doesn't make tons of big mistakes, which lets our defense and punter work their magic.

That said, Pro Football Focus is a sham. They claim their people analyze every player on every snap of every game all season long. There were on average 168 offensive and defensive snaps per game last year.

168 snaps * 22 players * 272 regular season games = 1,005,312 evaluations just in the regular season not counting special teams or post season. Nope.

TJ Lang has put some good commentary out there as far as how bullshit they are in grading, especially since other teammates affect players being graded, the graders do not have football credentials, and they also have no idea what anyone’s assignments are on any given play

Lang went on a radio show talking about it, and offered to fly in one of PFF’s OL graders (all expenses paid) to sit down with him and grade a player together for a full game. They declined. He then asked if he could do the same thing, grade separately, and then compare afterwards. They also declined, and this time said that all of their OL graders reside in India and due to the rigorous schedule during the season it wouldn’t work. Dead serious, they have people in some shithole in Delhi or wherever grading their games and they even admitted it publicly.

PFF is a lot like how a professional whisky taster will never do a blind test because they know they’d be outed by picking Cutty Sark over Lagavulin. I saw a “professional” spirits taster on YouTube who agreed to taste test vodkas get completely embarrassed by grading 4 vodkas all completely differently, even so far as talking about the different flavors and “notes,” only to find out later all four were poured from the exact same bottle of $6 Popov.

PFF’s secret method, “ancient Chinese secret” is kinda bullshit but their marketing department does a bang up job getting people to drink the Kool-Aid.

If you guys want a somewhat valid ranking system, use Football Outsiders. Their rankings are 100% based on stats with no subjective analysis applied whatsoever. They even share their methods.

The problem with crapping all over things like this is that you don't really know their true methodology, and you have to check their results vs what has already happened. For QBS, PFF has a pretty good track record compared to where players are drafted at the position, at least in the 2019 draft.

Kyler Murray was selected as the top QB in the draft. Drew Lock was in their top 5 at 4 but was selected second. Haskins was 2 but selected 3rd. Daniel Jones was selected 4th but was #5 on PFF list. Ryan Finley was #6 on PFF but drafted 5th. Jarret Stidham was #10 on PFF but selected 6th,etc. That's pretty close, Fry. (https://www.pff.com/news/draft-top-10-quarterbacks-for-2019-nfl-draft)

Whether we like it or not, Moneyball has come to college football, and just about every other sport. People who never even played any sport, might not have even seen a game played are crunching numbers and coming up with new ways of evaluating players and strategies that win ballgames.

Numbers aren't everything, of course. But just because PFF won't grade side by side with a guy on a radio show live, doesn't mean that they can't compare their grades at the end of a game in a blind test. PFF publishes their numbers already. Lang can put his grades out and compare them side by side, and you can use the draft on the top guys to see how things turn out.

 

[okeefe4prez]

I totally agree people, in general, don't understand. Which is interesting to me for two reasons.

First, as a general football fan, there are 130 FBS program. Each one of them over the course of a season will trot out at least 1 starting QB. Every spring, you've got the NFL draft. A tiny percentage of those QBs will get drafted. Of that tiny pool, only a fraction will end up getting meaningful next level playing time. Of that fraction, an even smaller fraction will go on to prove to be *actually good* at playing the QB position. Yet, every college fanbase in America seems to expect their guy to be That Guy (or at least a close facsimile of That Guy).

Secondly, specifically as a Hawk fan that's been an enthusiastic student of The Parker Way, our entire defensive scheme is built on the premise that that overwhelming majority of college QBs just aren't that good. I believe the estimate Norm gave JDM once was 85%. Obviously, there is a lot of complexity and nuance, but at its core, our defensive philosophy is "sit back and wait for the QB to F up". And, the overwhelming majority of the time, it works.

Correct. The entire Iowa philosophy is to just be patient because the QBs can't execute for 12 play drives. Been that way for 2 decades. A few years ago I was worried because I thought the philosophy was outdated, but then not long after I got worried I started to notice a reasonable drop in QB talent. The early entrant rules take the elite out a year early and the Big Ten is not recruiting the cream of the crop anymore.

Parker is the best coordinator in football. We have talent deficiencies on defense, but by God those guys play extremely good fundamental football and understand their assignments. If every guy follows his assignment there will be soft spots that a good QB like Tua Jr. or Purdy will be able to take apart a few times a game, but they ain't gonna do it enough to score a ton of points.

I think a lot of QBs are coming into programs with a really high level of arm talent from the offenses they are running in high school. Guys might look good for a year, but once teams see them once and have tape, they fall back to Earth. Purdy is a prime example of that. So many of these guys are peaking at like 17 or 18. Again, there are definitely outliers, but playing QB with pressure in your face, a grown man who makes 7 figures playing chess against you with pre-snap reads, and back sevens that have equal or greater speed than all your receivers can turn a guy mediocre pretty quickly.

 

[okeefe4prez]

Of course, because when I think of those with superior football knowledge I immediately think India. Great stuff Fry!!!

In all honesty, if you have an objective rating system and quality control over the graders, it doesn't matter who does the gradings. I think people in India or China who know nothing about football and won't have rooting allegiances or knowledge of who is supposed to be "good" would be better suited for this than people like us. I'm sure most of us know nothing about cricket, but if given training and an objective framework we could grade cricket players.

 

[Fryowa]

The problem with crapping all over things like this is that you don't really know their true methodology, and you have to check their results vs what has already happened. For QBS, PFF has a pretty good track record compared to where players are drafted at the position, at least in the 2019 draft.

Kyler Murray was selected as the top QB in the draft. Drew Lock was in their top 5 at 4 but was selected second. Haskins was 2 but selected 3rd. Daniel Jones was selected 4th but was #5 on PFF list. Ryan Finley was #6 on PFF but drafted 5th. Jarret Stidham was #10 on PFF but selected 6th,etc. That's pretty close, Fry. (https://www.pff.com/news/draft-top-10-quarterbacks-for-2019-nfl-draft)

Whether we like it or not, Moneyball has come to college football, and just about every other sport. People who never even played any sport, might not have even seen a game played are crunching numbers and coming up with new ways of evaluating players and strategies that win ballgames.

Numbers aren't everything, of course. But just because PFF won't grade side by side with a guy on a radio show live, doesn't mean that they can't compare their grades at the end of a game in a blind test. PFF publishes their numbers already. Lang can put his grades out and compare them side by side, and you can use the draft on the top guys to see how things turn out.

Spoken like someone who pays for their services and doesn’t want to feel cheated.

 

[Fryowa]

In all honesty, if you have an objective rating system and quality control over the graders, it doesn't matter who does the gradings. I think people in India or China who know nothing about football and won't have rooting allegiances or knowledge of who is supposed to be "good" would be better suited for this than people like us. I'm sure most of us know nothing about cricket, but if given training and an objective framework we could grade cricket players.

The PFF ratings aren’t objective, though.
“Every player on every snap” (which is BS anyway, but I digress)is given a score of either -2, -1, 0, 1, or 2 each play he’s in the game. Scores given by an army of people who e never played the game.

Football Outsiders uses a purely statistical (and purely objective) model that works better, imo.

 

[Luftgekuehlt67]

Correct. The entire Iowa philosophy is to just be patient because the QBs can't execute for 12 play drives. Been that way for 2 decades. A few years ago I was worried because I thought the philosophy was outdated, but then not long after I got worried I started to notice a reasonable drop in QB talent. The early entrant rules take the elite out a year early and the Big Ten is not recruiting the cream of the crop anymore.

Parker is the best coordinator in football. We have talent deficiencies on defense, but by God those guys play extremely good fundamental football and understand their assignments. If every guy follows his assignment there will be soft spots that a good QB like Tua Jr. or Purdy will be able to take apart a few times a game, but they ain't gonna do it enough to score a ton of points.

I think a lot of QBs are coming into programs with a really high level of arm talent from the offenses they are running in high school. Guys might look good for a year, but once teams see them once and have tape, they fall back to Earth. Purdy is a prime example of that. So many of these guys are peaking at like 17 or 18. Again, there are definitely outliers, but playing QB with pressure in your face, a grown man who makes 7 figures playing chess against you with pre-snap reads, and back sevens that have equal or greater speed than all your receivers can turn a guy mediocre pretty quickly.

Well said.

The System definitely went through some growing pains as we needed to adjust and account for the preponderance of spread offenses but, credit to Phil, he took Godfather Norm's system and made the necessary adaptations. Without Phil Parker (and Norm before him), we're Iowa State.

Just the other day, I heard JDM, the man himself, recounting a conversation he had with Norm back in the day. I mentioned the "85%" number, but something else Norm said is this system dares you to try and beat is with a big play. Most programs can do it once or twice in a game, a few times there will be 3. But, beyond that, almost impossible.

That's how you go almost 30 straight games holding teams to < 25, my friends.

 

[GesterHawk]

Ok, let's roll with the idea that QB talent has been dropping overall for the last decade probably. Yes, each year pumps out 1-5 elite QBs, but the rest are on the decline quality wise. This is why Nick Foles and Andy Dalton and the like are still getting paychecks.

Does this coincide with the rise of the "everybody look at the sideline so you know how to play this play without having to think about a damn thing" offenses?

And is this why 4 of the last 5 Iowa QBs have had a cup of tea, at the least, in the league?

Could this be why a Pro ranking service appreciates Petras?

 

[okeefe4prez]

Well said.

The System definitely went through some growing pains as we needed to adjust and account for the preponderance of spread offenses but, credit to Phil, he took Godfather Norm's system and made the necessary adaptations. Without Phil Parker (and Norm before him), we're Iowa State.

Just the other day, I heard JDM, the man himself, recounting a conversation he had with Norm back in the day. I mentioned the "85%" number, but something else Norm said is this system dares you to try and beat is with a big play. Most programs can do it once or twice in a game, a few times there will be 3. But, beyond that, almost impossible.

That's how you go almost 30 straight games holding teams to < 25, my friends.

Slovis would have gotten past 25 until AJ ripped his fucking arm off. That's another guy who probably peaked as a freshman.

 

[okeefe4prez]

Does this coincide with the rise of the "everybody look at the sideline so you know how to play this play without having to think about a damn thing" offenses?

Definitely. Kids who don't have to read the defenses probably aren't as good at figuring out what they are doing after the snap.

The fact that CJ Beathard was able to competently run Greg Davis's offense makes me think he could be a slightly above average starter in the NFL. If you put him on a decent team with a solid defense, he'd get you to the playoffs or within spitting distance. He ain't gonna win the Super Bowl, but he'd make you respectable.

 

[cincyhawk]

Spoken like someone who pays for their services and doesn’t want to feel cheated.

I don't pay for the services, but I do do data analytics for a living.

 

[Luftgekuehlt67]

Definitely. Kids who don't have to read the defenses probably aren't as good at figuring out what they are doing after the snap.

The fact that CJ Beathard was able to competently run Greg Davis's offense makes me think he could be a slightly above average starter in the NFL. If you put him on a decent team with a solid defense, he'd get you to the playoffs or within spitting distance. He ain't gonna win the Super Bowl, but he'd make you respectable.

I've said the same many times, I'm convinced CJ could keep an NFL offense moving.

Elite? Nah, no way. But, crimony, look at the rejects and retreads that teams keep trotting out every weekend. I'm genuinely shocked he hasn't been given a shot.

 

[Fryowa]

I don't pay for the services, but I do do data analytics for a living.

Then you should know that a bunch of people with no football credentials arbitrarily and subjectively assigning an offensive lineman a -2, -1, 0, +1 or +2 isn't quality data.

Oh...but because they got the top 5 draft picks somewhat close means that the results confirm the data? Is that it? Sounds like confirmation bias to me.

What if Mel Kiper got all 5 of the first round correct? What would that mean?

Give us some sort of conditions that would make sense for giving a lineman on a particular block a +2 vs a +1...Because he moved the guy forward, or pancaked him, or he made the DT take 4 steps to the outside vs 3 steps to the inside, or maybe give him a -2 because he pulled to the TE and the C missed his assignment and gave up a sack?

He pancaked a guy super duper hard vs super hard means he gets a +2 instead of a +1? Keeping in mind that these guys (or girls) are outsourced overseas and have no connection to football, let alone coaching or playing experience.

If you're using a statistical model based on non-subjective interpretations you definitely have a case for using Joe Blows for analysis. But that ain't PFF. It's smoke and mirrors with good marketing to get rubes to pay the subscriptions.

 

[Luftgekuehlt67]

I think the main challenge I've found with college sports analytics is the disparity in quality of competition.

I grew up strictly a college fan (football/basketball). Only relatively late in life did I get into pro sports (NBA...I follow NFL only very casually). I need to finish it (probably this winter), but I started reading Dean Oliver's "Basketball on Paper" (wonderful book, btw have you read that one @Fryowa ?). I think lots of smart people have taking Dean's work and really run with it but, on the hoops side at least, it seems he can really lay claim to being the Godfather of advanced basketball analytics.

At any rate, one of the things that struck me as I read through that book while deep in the throes of both Nuggets and Hawks hoops fandom was how much *easier* it is to do analytics in the pro game vs the college game. Again, it's the uniformity of the competitive field. 30 teams, every plays everybody else. Someone won, someone lost, some guy put up 35 points. It's all in the books, done and dusted.

On the college side? Oh boy. FBS, FCS, P5, mid-majors, highly regionalized schedules...yikes.

Using KenPom to do some just-for-fun analysis of the Hawks, you can filter for "conference only". That's great for looking into matchups within a given conference or even NCAA matchups between P5 programs. But anything outside of those constraints? There are 358 D1 teams - holy shit, good luck.

You can still get an idea of what a given team likes to do, who their key players are (even the ones that don't necessarily make a big splash on the scoreboard) - it's great for that, but it's all very...relative.

 

[cincyhawk]

Then you should know that a bunch of people with no football credentials arbitrarily and subjectively assigning an offensive lineman a -2, -1, 0, +1 or +2 isn't quality data.

Oh...but because they got the top 5 draft picks somewhat close means that the results confirm the data? Is that it? Sounds like confirmation bias to me.

What if Mel Kiper got all 5 of the first round correct? What would that mean?

Give us some sort of conditions that would make sense for giving a lineman on a particular block a +2 vs a +1...Because he moved the guy forward, or pancaked him, or he made the DT take 4 steps to the outside vs 3 steps to the inside, or maybe give him a -2 because he pulled to the TE and the C missed his assignment and gave up a sack?

He pancaked a guy super duper hard vs super hard means he gets a +2 instead of a +1? Keeping in mind that these guys (or girls) are outsourced overseas and have no connection to football, let alone coaching or playing experience.

If you're using a statistical model based on non-subjective interpretations you definitely have a case for using Joe Blows for analysis. But that ain't PFF. It's smoke and mirrors with good marketing to get rubes to pay the subscriptions.

Lots and lots of statistical models are based off of subjective criteria. Including, but not limited to: insurance risk profiles, credit profiles, criminal offense paroles, job performance evaluations, etc. We tend to have people pick from a set of categories, or on a spectrum of choices to collect good - bad valuations. It's done constantly. As long as the evaluation is peer reviewed, and let's face it, nothing is as critical of subjective evaluations as sports, and repeatable, and amortized over lots of samples, it tends to fall out to a set of statistical inference values that are correlated with quality. What's even better, is you can correct for bias over time as well, and rank evaluators for who tends to be right more often. The model itself can be retrained over time to remove noise and outliers. It's entirely plausible.

Would I pay for PFF? No. I have eyes. But as a an input for who might be good and who might be bad, yeah, it makes sense that some would. Even if I knew more about football than the average Joe, someone who watches individual performances as their job for 8 hours a day, 5 days a week, is probably going to be better at it than me. And if they're not, they can be replaced by someone who is.

And yes, you can rate a guard on how well he did on a + or - scale. It's not terribly difficult to see if a guy runs right past a guy on a run play, or if a guy gets caught holding, or jumps offsides, or gets bull rushed into the backfield or holds his ground, or doubles a guy when a pulling blocker is coming, or makes a reach block.

I know we all try to make this crap rocket science, but it's just not when you get to the individual things that someone does during a snap. It is teachable, there are college courses on it, and books, and clinics and etc. Knowing what is right and wrong is not an issue and not something untrained people can't learn to identify. Fixing it when something is wrong is the hard part. Executing everything is difficult. Categorizing an unfinished block to a finished one is something you can teach to a freaking 8th grader.

 

[GesterHawk]

Lots and lots of statistical models are based off of subjective criteria. Including, but not limited to: insurance risk profiles, credit profiles, criminal offense paroles, job performance evaluations, etc. We tend to have people pick from a set of categories, or on a spectrum of choices to collect good - bad valuations. It's done constantly. As long as the evaluation is peer reviewed, and let's face it, nothing is as critical of subjective evaluations as sports, and repeatable, and amortized over lots of samples, it tends to fall out to a set of statistical inference values that are correlated with quality. What's even better, is you can correct for bias over time as well, and rank evaluators for who tends to be right more often. The model itself can be retrained over time to remove noise and outliers. It's entirely plausible.

Would I pay for PFF? No. I have eyes. But as a an input for who might be good and who might be bad, yeah, it makes sense that some would. Even if I knew more about football than the average Joe, someone who watches individual performances as their job for 8 hours a day, 5 days a week, is probably going to be better at it than me. And if they're not, they can be replaced by someone who is.

And yes, you can rate a guard on how well he did on a + or - scale. It's not terribly difficult to see if a guy runs right past a guy on a run play, or if a guy gets caught holding, or jumps offsides, or gets bull rushed into the backfield or holds his ground, or doubles a guy when a pulling blocker is coming, or makes a reach block.

I know we all try to make this crap rocket science, but it's just not when you get to the individual things that someone does during a snap. It is teachable, there are college courses on it, and books, and clinics and etc. Knowing what is right and wrong is not an issue and not something untrained people can't learn to identify. Fixing it when something is wrong is the hard part. Executing everything is difficult. Categorizing an unfinished block to a finished one is something you can teach to a freaking 8th grader.

I know dick about diving because I don't care to normally watch it, let alone do it. But once every 4 years I watch it in the Olympics and I can tell a plus entry from a minus entry.

 

[InGoodCo]

Iowa’s defense preys on inpatient offenses and QB’s that take risks cause like said above, they know 85% of them aren’t good enough to sustain long drives and score on a stout D.

The thing is, they know this same case is true about their QB as well. It’d take a real idiot to think that philosophy didn’t apply to themselves. They teach them to minimize the risk, don’t hurt us, don’t give the other team a short field and we’ll give you great field position with D and special teams. Over the length of the game, you’ll get your scoring opportunities. This team would rather be on D anyways. Guess what? This has won Iowa a shit ton of games