Thursday, September 29, 2005
Am I Being Offensive?
How do we measure an individual's contribution to offense? I recently posted some suggestions at Clueless Finn (hartti.com), and I want to expand on those remarks here.
Before we can define a measure, i.e. a numerical value which tries to capture what we *mean* by offensive contribution, we first have to agree on what we are looking for. We can know good offense when we see it, and we're reasonably sure that it involves things like throwing percentage, scores thrown, scores caught, and offensive success fractions. But how do we combine our statistics to produce some reasonable numbers. In baseball, there are many kinds of measures, the most basic being batting average, on-base percentage, and slugging percentage. No single one tells the whole story, but we humans like to rank players, so we search for an overall measure.
I think of a player's contribution as being how well s/he helps the offense keep or get back the disc and advance it toward (and into) the goal. In this view, a middle receiver may contribute greatly even if s/he never throws nor catches a goal. This "concept" of offensive contribution includes intangible factors like creating space for other cutters or effectively motivating a team to achieve that goal. Sadly, these intangibles will not be reflected very much in the index I define (except through osr -- see below).
We are limited in defining a measure by the kinds of statistics that are available. With complete videographic data of every game, we could get very technical in our definition, but that would be impractical. Instead, I will assume that the statistics correspond to someone on the sideline marking who is in on a given point, and keeping track of every pass, recording whether it was
1 : complete short, and who caught it,
2 : complete medium, and who caught it,
3 : complete long and who caught it,
T : throwaway,
D: drop, and who dropped it,
I : incomplete for other reasons (forced turnover)
G : goal, and who caught it
also, on defense
B : block or catch on defense.
For our purposes, we can ignore other defensive data.
Apologies to those who know how stats are taken and if these assumptions are unrealistic, but I am imagining charting offensive progress by a string of players a, b, c, d, e, f, g with the corresponding symbols. So, "a2c1f2Ge" would mean "a throws a medium-length pass to c who throws a short pass to f who throws a medium length goal to e," and "b3Df" would mean "b throws a long pass to f, who dropped it." In this way, I think it is not unreasonable to assume that ultimate teams are capable of recording statistics at this level of detail. There are probably much better ways.
Given these data, we can form a few useful quantities.
gt = goals thrown per point (less than or equal to one)
gc = goals caught per point
tlp = throwing length per possession = sum of the numbers (1, 2, or 3) of completed passes divided by the number of points played (for me, possession equals point on O).
rlp = receiving length per possession = sum of the numbers (1, 2, or 3) of received passes divided by the number of points played.
tep = throwing errors per point
dp = drops per point
bp = blocks per point (on defense)
tr = throwing ratio = number of complete throws / number of throws (in fact I won't use this below, and this might cause some controversy).
dbr = fraction of times defense causes a turnover (so (1-dbr) equals the fraction of times the defense gets scored on (without having caused any turnovers -- I won't worry about higher iterations).
osr = offensive scoring ratio, i.e. the number of times a team scores on offense divided by the total number of times receiving the pull
isr = individual scoring ratio, i.e. the percentage of times the team scores when player x is in on offense, divided the by the total percentage of times the team scores on offense.
[isr is some overall measure of a player's effectiveness, but it is very crude, since s/he may have had nothing to do with her/his team's success. We will use this as an overall factor, rather than using this number alone. Thus, when all other things are equal, a player with higher osr is more effective. One nice thing about osr is that it *may* be able to differentiate between otherwise equal players through their intangible contributions which help the team score. Those are partly reflected in isr.]
Okay, I will now consider 7 throwing lengths to be equal to one score caught.
We can now give a rough measure the effective number of "scores" that a player is responsible for. (It doubly rewards goals caught or thrown, as they count toward two categories, but that's why I used 7 throwing lengths and not 6 or 5.) For example, a block leads to a possession on offense, which has a chance (osr) of leading to a score. So (bp)(osr) is gives the number of scores per point that a player creates through his/her blocks. (I know: the offensive squads and defensive squads are different, but whaddayagonnado.) The Unweighted Individual Scoring Index will be
UISI = gt + gc + (tlp + rlp)/6 - (tep + dp)*(1 - dbr) + (bp)(osr)
Finally, using isr, we get the Individual Scoring Index:
ISI = (isr)(UISI).
What do you think?
Before we can define a measure, i.e. a numerical value which tries to capture what we *mean* by offensive contribution, we first have to agree on what we are looking for. We can know good offense when we see it, and we're reasonably sure that it involves things like throwing percentage, scores thrown, scores caught, and offensive success fractions. But how do we combine our statistics to produce some reasonable numbers. In baseball, there are many kinds of measures, the most basic being batting average, on-base percentage, and slugging percentage. No single one tells the whole story, but we humans like to rank players, so we search for an overall measure.
I think of a player's contribution as being how well s/he helps the offense keep or get back the disc and advance it toward (and into) the goal. In this view, a middle receiver may contribute greatly even if s/he never throws nor catches a goal. This "concept" of offensive contribution includes intangible factors like creating space for other cutters or effectively motivating a team to achieve that goal. Sadly, these intangibles will not be reflected very much in the index I define (except through osr -- see below).
We are limited in defining a measure by the kinds of statistics that are available. With complete videographic data of every game, we could get very technical in our definition, but that would be impractical. Instead, I will assume that the statistics correspond to someone on the sideline marking who is in on a given point, and keeping track of every pass, recording whether it was
1 : complete short, and who caught it,
2 : complete medium, and who caught it,
3 : complete long and who caught it,
T : throwaway,
D: drop, and who dropped it,
I : incomplete for other reasons (forced turnover)
G : goal, and who caught it
also, on defense
B : block or catch on defense.
For our purposes, we can ignore other defensive data.
Apologies to those who know how stats are taken and if these assumptions are unrealistic, but I am imagining charting offensive progress by a string of players a, b, c, d, e, f, g with the corresponding symbols. So, "a2c1f2Ge" would mean "a throws a medium-length pass to c who throws a short pass to f who throws a medium length goal to e," and "b3Df" would mean "b throws a long pass to f, who dropped it." In this way, I think it is not unreasonable to assume that ultimate teams are capable of recording statistics at this level of detail. There are probably much better ways.
Given these data, we can form a few useful quantities.
gt = goals thrown per point (less than or equal to one)
gc = goals caught per point
tlp = throwing length per possession = sum of the numbers (1, 2, or 3) of completed passes divided by the number of points played (for me, possession equals point on O).
rlp = receiving length per possession = sum of the numbers (1, 2, or 3) of received passes divided by the number of points played.
tep = throwing errors per point
dp = drops per point
bp = blocks per point (on defense)
tr = throwing ratio = number of complete throws / number of throws (in fact I won't use this below, and this might cause some controversy).
dbr = fraction of times defense causes a turnover (so (1-dbr) equals the fraction of times the defense gets scored on (without having caused any turnovers -- I won't worry about higher iterations).
osr = offensive scoring ratio, i.e. the number of times a team scores on offense divided by the total number of times receiving the pull
isr = individual scoring ratio, i.e. the percentage of times the team scores when player x is in on offense, divided the by the total percentage of times the team scores on offense.
[isr is some overall measure of a player's effectiveness, but it is very crude, since s/he may have had nothing to do with her/his team's success. We will use this as an overall factor, rather than using this number alone. Thus, when all other things are equal, a player with higher osr is more effective. One nice thing about osr is that it *may* be able to differentiate between otherwise equal players through their intangible contributions which help the team score. Those are partly reflected in isr.]
Okay, I will now consider 7 throwing lengths to be equal to one score caught.
We can now give a rough measure the effective number of "scores" that a player is responsible for. (It doubly rewards goals caught or thrown, as they count toward two categories, but that's why I used 7 throwing lengths and not 6 or 5.) For example, a block leads to a possession on offense, which has a chance (osr) of leading to a score. So (bp)(osr) is gives the number of scores per point that a player creates through his/her blocks. (I know: the offensive squads and defensive squads are different, but whaddayagonnado.) The Unweighted Individual Scoring Index will be
UISI = gt + gc + (tlp + rlp)/6 - (tep + dp)*(1 - dbr) + (bp)(osr)
Finally, using isr, we get the Individual Scoring Index:
ISI = (isr)(UISI).
What do you think?
# posted by Eric Zaslow @ 9/29/2005 08:31:00 PM 
Comments:
<< Home
Way ahead of you, Zaz. This is very similar to what Jim and I did for the 94 semis and finals of nationals. We divided it into short passes, medium passes, and long passes. We awarded an extra-"yardage point" to a swing pass if it broke the mark to a wide open person and it led to a fast-break up the weak side.
The ratio was like 5/12/45 (although this slightly overvalues short-passes against a zone). Since both the receiver and thrower received yardage points on a single throw, a full field at that time (from brick to goal) would be considered "120 yards," i.e.60 throwing yards, and 60 receiving yards.
We determined how many error-free yards per turnover each person generated. The record, if I remember correctly, for both Semis and Finals for DoG's first Nationals championship was by one particularly charming gentleman with 554 yards per turnover (this neglects some people with significantly less yardage gained but had no turnovers). That was a total of (using 120 yards as the length of a field) 4 1/2 fields, of perfect play, both throwing and receiving, before the turnover.
That seems pretty good -- as it was very windy and was against N.Y. and Double Happiness.
--Dennis
The ratio was like 5/12/45 (although this slightly overvalues short-passes against a zone). Since both the receiver and thrower received yardage points on a single throw, a full field at that time (from brick to goal) would be considered "120 yards," i.e.60 throwing yards, and 60 receiving yards.
We determined how many error-free yards per turnover each person generated. The record, if I remember correctly, for both Semis and Finals for DoG's first Nationals championship was by one particularly charming gentleman with 554 yards per turnover (this neglects some people with significantly less yardage gained but had no turnovers). That was a total of (using 120 yards as the length of a field) 4 1/2 fields, of perfect play, both throwing and receiving, before the turnover.
That seems pretty good -- as it was very windy and was against N.Y. and Double Happiness.
--Dennis
For some reason, though, we decided that a break pass was worth extra but getting open with the force was just the receiver doing his job.
It would have been relatively easy to treat zone points differently. My feeling is that passes _to_ handlers in the zone should just be ignored unless it's a turnover or unless the pass actually does something to make the team more likely to score (perhaps the handler crashes through the cup because they're playing off). I suspect that if you did this, the average number of "passes" in a zone point would come close to the average number of passes in a man point.
Not to rehash a 10 year old argument, but take a look at the following to see more info.
http://www.shelltown.com/~parinell/total.yp, http://www.shelltown.com/~parinell/cojo.yp, http://www.shelltown.com/~parinell/double.yp, http://www.shelltown.com/~parinell/explain.yp
Having looked a lot at advanced baseball stats, I think the proper way to measure value is contribution above some baseline, that baseline usually being either "average" or "replacement", using "opportunities" (in baseball, outs or at-bats; in ultimate, turnovers or passes). So, if player A had two turnovers and created 180 yards while the average (or replacement) player creates 50 yards per turnover, A would be created with 180 - 2*50 = 80 yards above average (or replacement). If player B has eight turnovers and created 560 yards, he would be created with 560 - 8*50 = 160 yards above average (or replacement). In terms of value, he'd be worth more than player A since he created a lot more, even if B was more efficient with the disc. (This also allows you to eliminate from MVP consideration those with 40 yards created and no turnovers without having to wave your hands.)
An alternative would be to assign a yardage penalty for turnovers (like is done on footballoutsiders). It is tricky to pick the proper value, though. And I have also suggested (and I stand by it) that the yardage of a turnover should be included in the totals (in football, a team with the ball at its own 20 is no more likely to score the next point than the other team, so a 60 yard turnover doesn't cost the team anything, while a 10 yarder does).
It would have been relatively easy to treat zone points differently. My feeling is that passes _to_ handlers in the zone should just be ignored unless it's a turnover or unless the pass actually does something to make the team more likely to score (perhaps the handler crashes through the cup because they're playing off). I suspect that if you did this, the average number of "passes" in a zone point would come close to the average number of passes in a man point.
Not to rehash a 10 year old argument, but take a look at the following to see more info.
http://www.shelltown.com/~parinell/total.yp, http://www.shelltown.com/~parinell/cojo.yp, http://www.shelltown.com/~parinell/double.yp, http://www.shelltown.com/~parinell/explain.yp
Having looked a lot at advanced baseball stats, I think the proper way to measure value is contribution above some baseline, that baseline usually being either "average" or "replacement", using "opportunities" (in baseball, outs or at-bats; in ultimate, turnovers or passes). So, if player A had two turnovers and created 180 yards while the average (or replacement) player creates 50 yards per turnover, A would be created with 180 - 2*50 = 80 yards above average (or replacement). If player B has eight turnovers and created 560 yards, he would be created with 560 - 8*50 = 160 yards above average (or replacement). In terms of value, he'd be worth more than player A since he created a lot more, even if B was more efficient with the disc. (This also allows you to eliminate from MVP consideration those with 40 yards created and no turnovers without having to wave your hands.)
An alternative would be to assign a yardage penalty for turnovers (like is done on footballoutsiders). It is tricky to pick the proper value, though. And I have also suggested (and I stand by it) that the yardage of a turnover should be included in the totals (in football, a team with the ball at its own 20 is no more likely to score the next point than the other team, so a 60 yard turnover doesn't cost the team anything, while a 10 yarder does).
All good stuff, though any complex enough system is likely to generate some debate.
I particularly like the inclusion of break-mark passes which are valuable because I believe they make yardage gains on the NEXT pass more likely (that's hypothesis. Happy to be backed up shown wrong).
Question: is there any value to including 2nd assist stats?
Also, would the baseline be set team by team, or is there some generic baseline? If it's an assumed replacement player on the same team, it's useful for that team, but does it make it harder to compare across teams or between good teams and bad ones?
I particularly like the inclusion of break-mark passes which are valuable because I believe they make yardage gains on the NEXT pass more likely (that's hypothesis. Happy to be backed up shown wrong).
Question: is there any value to including 2nd assist stats?
Also, would the baseline be set team by team, or is there some generic baseline? If it's an assumed replacement player on the same team, it's useful for that team, but does it make it harder to compare across teams or between good teams and bad ones?
Hi Jim,
Great stuff. It's a great idea, but the only problem is that your method, I believe, like Zaz's method above, conspicuously underestimates the negative value of turnovers: First your method:
"Short passes are worth 1, 10-25 yarders are 2, 25-40 are 4, and
hucks are worth 6. Add these up to get gross thrower points
(TP) or receiver points (RP). To get net points, subtract
for throwaways or drops: -6 for a short one, -5 for a
middle, and -3 for a huck, reflecting the opponents field
position and their ability to get a fast break going."
According to your method, you break even if you throw .33% complete deep, when this is an abominable statistic, particularly for a team that is scoring from brick at 58%. 58% complete deep has to be the break-even. So if you are awarding 6 points for a complete huck, then you have to award -8.3 for a huck turnover (as opposed to -3). This leads to a break-even (0 pts.) for a 58% huck-completion (i.e., scoring) percentage.
Even your short-pass turnover is undervalued. At 1 point for a short complete and -6 for a short turnover, all you need to do to break even is throw 6/7 short (85.7%) – which, as you know, is beneath the Adner line. There are perhaps 8-10 short throws in a pure-short-pass goal, and the odds of you throwing 8 complete throws in a row (when you are completing 85%) is .85^8 = .27 . That means the teams scoring percentage would be 27% with those short throws.
Short throwing percentage -- Scoring percentage on right (assuming the need for 8 complete throws in a row).
85% -- 27%
90% -- 43%
92.5% -- 53.5%
93 % -- 55.9%
93.5 -- 58.4%
94% -- 61%
95% -- 66.3%
97% -- 78.3%
This really clarifies how significantly important throwing percentage is.
Anyway, so the break even point for short throws is 93.5% – and so if you award 1 point for short throws, then short turnovers should be worth -14.38. Using the same logic for medium (and medium-long) and assuming you would need 4 complete throws in a row (and 2 complete throws in a row), then you need to complete roughly 87% (76 %) to break even and score 58% of the time. If you award 2 points for medium completions, you should subtract -13.38 for medium turnovers, and if you award 4 points for medium long then you should subtract -12.666 for medium long. So a reasonable TO subtraction-- which would not award positive points for negative play (like throwing 40% deep) -- would be:
Comp. Turnover points:
Short 1 -14.4
Medium 2 -13.38
Medium-long 4 -12.66
Very long 6 -8.3
With this method, it is likely that a significant percentage of the team will be negative – because it is determined based on the effort to score at 58% for the team from brick to goal. But what is important is that it avoids the problem of awarding positive points for unquestionably negative play (like throwing 86% on short throws or 40% deep). So given this more realistic method, the top four stats from semis and finals of '94 Nationals would be (and average player would likely have a value of 0):
GT GR GT NT NR NT
Moons: 89 78 167 51.9 78 129.9
Moi: 65 62 127 50.6 62 112.6
Cork 25 39 64 25 39 64
Lenny 34 26 60 34 26 60
This makes sense as neither Cork nor Lenny had a turnover – which is immense.
But more importantly in the very-tight, torch-passing semis game against Cajones ("greatest game ever"), the break down is:
Moi: 48 42 90 48 42 90
Alex: 65 34 99 35 34 69
Moons: 53 45 98 24 45 69
Cork: 12 17 29 12 17 29
More on Zaz's method later…
--D
Great stuff. It's a great idea, but the only problem is that your method, I believe, like Zaz's method above, conspicuously underestimates the negative value of turnovers: First your method:
"Short passes are worth 1, 10-25 yarders are 2, 25-40 are 4, and
hucks are worth 6. Add these up to get gross thrower points
(TP) or receiver points (RP). To get net points, subtract
for throwaways or drops: -6 for a short one, -5 for a
middle, and -3 for a huck, reflecting the opponents field
position and their ability to get a fast break going."
According to your method, you break even if you throw .33% complete deep, when this is an abominable statistic, particularly for a team that is scoring from brick at 58%. 58% complete deep has to be the break-even. So if you are awarding 6 points for a complete huck, then you have to award -8.3 for a huck turnover (as opposed to -3). This leads to a break-even (0 pts.) for a 58% huck-completion (i.e., scoring) percentage.
Even your short-pass turnover is undervalued. At 1 point for a short complete and -6 for a short turnover, all you need to do to break even is throw 6/7 short (85.7%) – which, as you know, is beneath the Adner line. There are perhaps 8-10 short throws in a pure-short-pass goal, and the odds of you throwing 8 complete throws in a row (when you are completing 85%) is .85^8 = .27 . That means the teams scoring percentage would be 27% with those short throws.
Short throwing percentage -- Scoring percentage on right (assuming the need for 8 complete throws in a row).
85% -- 27%
90% -- 43%
92.5% -- 53.5%
93 % -- 55.9%
93.5 -- 58.4%
94% -- 61%
95% -- 66.3%
97% -- 78.3%
This really clarifies how significantly important throwing percentage is.
Anyway, so the break even point for short throws is 93.5% – and so if you award 1 point for short throws, then short turnovers should be worth -14.38. Using the same logic for medium (and medium-long) and assuming you would need 4 complete throws in a row (and 2 complete throws in a row), then you need to complete roughly 87% (76 %) to break even and score 58% of the time. If you award 2 points for medium completions, you should subtract -13.38 for medium turnovers, and if you award 4 points for medium long then you should subtract -12.666 for medium long. So a reasonable TO subtraction-- which would not award positive points for negative play (like throwing 40% deep) -- would be:
Comp. Turnover points:
Short 1 -14.4
Medium 2 -13.38
Medium-long 4 -12.66
Very long 6 -8.3
With this method, it is likely that a significant percentage of the team will be negative – because it is determined based on the effort to score at 58% for the team from brick to goal. But what is important is that it avoids the problem of awarding positive points for unquestionably negative play (like throwing 86% on short throws or 40% deep). So given this more realistic method, the top four stats from semis and finals of '94 Nationals would be (and average player would likely have a value of 0):
GT GR GT NT NR NT
Moons: 89 78 167 51.9 78 129.9
Moi: 65 62 127 50.6 62 112.6
Cork 25 39 64 25 39 64
Lenny 34 26 60 34 26 60
This makes sense as neither Cork nor Lenny had a turnover – which is immense.
But more importantly in the very-tight, torch-passing semis game against Cajones ("greatest game ever"), the break down is:
Moi: 48 42 90 48 42 90
Alex: 65 34 99 35 34 69
Moons: 53 45 98 24 45 69
Cork: 12 17 29 12 17 29
More on Zaz's method later…
--D
I will have to take more time to digest these comments, but I can make a few responses right away.
First of all, I meant to say in my original post that almost all statistics are overapplied, as they result from too small a data set. This would be the case, for example, if you look at an individual's statistics for a given game. A team's statistics for an individual game has a bit more meaning, since a team "does" more things than any of its individual players.
Secondly, it looks as though the Dennis/Jim method relies on a stat-taker who is recording more information than I am allowing (break-mark throws and type of defense, for example). Of course, with more sophisticated data, you can make more sophisticated indices.
Third, the Dennis/Jim definition does not seem to count a block as being a contribution to offense. It should.
Fourth, the D/J stat is not a per-possession number, but an absolute. This makes it hard to compare players who were subbed in different amounts.
Also, with a per-possession count, subtracting a baseline would be a common subtraction for everyone and therefore would be unnecessary.
I agree with the ideas that certain passes do more (or less) than others, but given only the data that I was assuming, I don't think one can build a model "fine" enough to account for these differences.
First of all, I meant to say in my original post that almost all statistics are overapplied, as they result from too small a data set. This would be the case, for example, if you look at an individual's statistics for a given game. A team's statistics for an individual game has a bit more meaning, since a team "does" more things than any of its individual players.
Secondly, it looks as though the Dennis/Jim method relies on a stat-taker who is recording more information than I am allowing (break-mark throws and type of defense, for example). Of course, with more sophisticated data, you can make more sophisticated indices.
Third, the Dennis/Jim definition does not seem to count a block as being a contribution to offense. It should.
Fourth, the D/J stat is not a per-possession number, but an absolute. This makes it hard to compare players who were subbed in different amounts.
Also, with a per-possession count, subtracting a baseline would be a common subtraction for everyone and therefore would be unnecessary.
I agree with the ideas that certain passes do more (or less) than others, but given only the data that I was assuming, I don't think one can build a model "fine" enough to account for these differences.
On Zaz's method:
It's a great start but I think you are seriously underestimating the negative value of turnovers (and the positive value of defensive plays) especially relative to a goal caught or goal thrown. For many offenses, there's a standard group that essentially plays every offensive point – so for comparisons between them the "per point" thing is largely irrelevant. (We can always divide values per point later.) The only question is if you want to divide all values by 12 or 13 or whatever.
Also, even against better teams, a great D will get turnovers perhaps 50 percent of the time. Thus, (1 – dbr) will usually equal 0.5. This means, according to your formula, that every goal thrown will cancel out two of your turnovers – which is just too immense.
Imagine a game in which defense is getting it back 50% of the time, and two players, say A and D have the same amount of throwing length (yardage) in a half. The only difference is that D has thrown no goals and A has thrown for 3 goals and 5 turnovers. Who is having the better game?
"A" of course is having one of the worst games imaginable – and would likely be benched. After keeping stats for years – 5 turnovers in a game (let alone a half) is absolutely horrendous and is nearly unsurvivable. People would be screaming at him from the sidelines, "what the F!?"
But according to your system A would be having a more valuable game than B.
He would have plus 3 for the goals and only -2.5 for the turns. (If you want you may divide both values by points-played, but it doesn't matter.)
The way to determine relative value of a goal compared to a turnover is as follows:
Imagine randomly placing a player 1000 times at random spots on the field with the disc, and he's about to throw the disc into the endzone. What completion percentage should he have so that his value is essentially a wash – such that a lower percentage would be hurting a good team and a higher percentage would be helping the team.
Well, DoG in 94 was scoring 58% from a ten yard brick. So the value must be greater than 58%. If you consider the fact that many of the times he'll be very near the goal line and half the time he will have less than half the field into the end-zone, I think the value is a minimum of 75% completion percentage – which is to say, you wouldn't be happy if he's throwing a turnover for every goal or, for that matter, a turnover for every two goals, but you probably want him throwing at least three goals for every one of his turnovers. (If he's on the goal line, you want him to throw at least 9 goals for every turn, but this is an average.)
Another way to look at it. Here's an average game (assuming a 58% scoring): Offense has received the pull 15 times and scored 9 times immediately (off the pull) and 11 times eventually. Thus, they share 22 goals (throwing and receiving), and have roughly 6 or 7 turnovers. Every turnover more than that becomes a worse game – and every turnover fewer is a better game. Again, the ratio of GT + GC to turnovers is 3 to 1. And this makes intuitive sense. A player with a 3 turnover game is having a subpar game – and would have to do a lot to make up for it.
Also deep throws should be worth 5 points (1/2/5) and you should have to move more than a field (perhaps a field and a half) per turnover.
So a more realistic assessment would be
(gt + gc)/ 3 + (tlp + rlp)/9 - (turnovers) + (blocks)
You could divide this by points-played if you like.
--D
PS: An easier way is yards per turnover. And you keep goals caught and defensive plays separate.
It's a great start but I think you are seriously underestimating the negative value of turnovers (and the positive value of defensive plays) especially relative to a goal caught or goal thrown. For many offenses, there's a standard group that essentially plays every offensive point – so for comparisons between them the "per point" thing is largely irrelevant. (We can always divide values per point later.) The only question is if you want to divide all values by 12 or 13 or whatever.
Also, even against better teams, a great D will get turnovers perhaps 50 percent of the time. Thus, (1 – dbr) will usually equal 0.5. This means, according to your formula, that every goal thrown will cancel out two of your turnovers – which is just too immense.
Imagine a game in which defense is getting it back 50% of the time, and two players, say A and D have the same amount of throwing length (yardage) in a half. The only difference is that D has thrown no goals and A has thrown for 3 goals and 5 turnovers. Who is having the better game?
"A" of course is having one of the worst games imaginable – and would likely be benched. After keeping stats for years – 5 turnovers in a game (let alone a half) is absolutely horrendous and is nearly unsurvivable. People would be screaming at him from the sidelines, "what the F!?"
But according to your system A would be having a more valuable game than B.
He would have plus 3 for the goals and only -2.5 for the turns. (If you want you may divide both values by points-played, but it doesn't matter.)
The way to determine relative value of a goal compared to a turnover is as follows:
Imagine randomly placing a player 1000 times at random spots on the field with the disc, and he's about to throw the disc into the endzone. What completion percentage should he have so that his value is essentially a wash – such that a lower percentage would be hurting a good team and a higher percentage would be helping the team.
Well, DoG in 94 was scoring 58% from a ten yard brick. So the value must be greater than 58%. If you consider the fact that many of the times he'll be very near the goal line and half the time he will have less than half the field into the end-zone, I think the value is a minimum of 75% completion percentage – which is to say, you wouldn't be happy if he's throwing a turnover for every goal or, for that matter, a turnover for every two goals, but you probably want him throwing at least three goals for every one of his turnovers. (If he's on the goal line, you want him to throw at least 9 goals for every turn, but this is an average.)
Another way to look at it. Here's an average game (assuming a 58% scoring): Offense has received the pull 15 times and scored 9 times immediately (off the pull) and 11 times eventually. Thus, they share 22 goals (throwing and receiving), and have roughly 6 or 7 turnovers. Every turnover more than that becomes a worse game – and every turnover fewer is a better game. Again, the ratio of GT + GC to turnovers is 3 to 1. And this makes intuitive sense. A player with a 3 turnover game is having a subpar game – and would have to do a lot to make up for it.
Also deep throws should be worth 5 points (1/2/5) and you should have to move more than a field (perhaps a field and a half) per turnover.
So a more realistic assessment would be
(gt + gc)/ 3 + (tlp + rlp)/9 - (turnovers) + (blocks)
You could divide this by points-played if you like.
--D
PS: An easier way is yards per turnover. And you keep goals caught and defensive plays separate.
I like Zaz's (1-dbr) as a weather indicator or serious game indicator. If the other team is scoring at 20%, a turnover is not as bad and a block is not as good (as if the team were scoring at 80%).
So let's change it to:
(gt + gc)/ 6 + (tlp + rlp)/18 - (1-dbr)(turnovers) + (1-dbr)(blocks)
is the value...
--D
So let's change it to:
(gt + gc)/ 6 + (tlp + rlp)/18 - (1-dbr)(turnovers) + (1-dbr)(blocks)
is the value...
--D
I just skimmed Dennis' first set of comments on "Jim's method" (c. 1994), and I like the way he is applying the break-even criterion to determine the cost of a turnover. I have to go to practice now and will look some more this evening probably.
Okay, now I've read through D & J's posts carefully: great stuff.
1. I agree that if we have the data of whether a throw is a break-mark, we should add value to it (+2, perhaps). I think it is too much to ask of the stat-taker to record whether the break-mark leads to further gains downfield, so we will treat all breaks the same. This does beg the question (a la Jim), should reception of a break-mark be worth less, since they're not really covering you there? Anyway, I don't expect to have this data.
2. I agree that my method undervalued turnovers. The (negative) value of a turnover should be proportional to the likelihood that it will lead to a score against you, and (1-dbr) is the closest thing to that indicator -- so I would weight turns by that factor, as Dennis has suggested.
3. Now come the number games. How do we weight these turns relative to the yardage indicators? It still makes sense to me to think of a score as equivalent to some number of "yards," and the question is how many? I don't know yet. More later. Another question is how much should an offensive score count positively compared to the negative of a turnover? If the offense is expected to score and the defense is NOT expected to block (i.e. osr > 1/2 and dbr < 1/2), then an offensive score should be worth LESS than the negative of a turnover. So perhaps O scores should be weighted by (1-osr) as well? The more the offense is scoring in general, the less value added a score is.
Finally, I wouldn't call the resulting index a scoring index anymore, since we're no longer counting how many scores the person is responsible for. Now it's just an overall offensive index.
1. I agree that if we have the data of whether a throw is a break-mark, we should add value to it (+2, perhaps). I think it is too much to ask of the stat-taker to record whether the break-mark leads to further gains downfield, so we will treat all breaks the same. This does beg the question (a la Jim), should reception of a break-mark be worth less, since they're not really covering you there? Anyway, I don't expect to have this data.
2. I agree that my method undervalued turnovers. The (negative) value of a turnover should be proportional to the likelihood that it will lead to a score against you, and (1-dbr) is the closest thing to that indicator -- so I would weight turns by that factor, as Dennis has suggested.
3. Now come the number games. How do we weight these turns relative to the yardage indicators? It still makes sense to me to think of a score as equivalent to some number of "yards," and the question is how many? I don't know yet. More later. Another question is how much should an offensive score count positively compared to the negative of a turnover? If the offense is expected to score and the defense is NOT expected to block (i.e. osr > 1/2 and dbr < 1/2), then an offensive score should be worth LESS than the negative of a turnover. So perhaps O scores should be weighted by (1-osr) as well? The more the offense is scoring in general, the less value added a score is.
Finally, I wouldn't call the resulting index a scoring index anymore, since we're no longer counting how many scores the person is responsible for. Now it's just an overall offensive index.
Sorry, one more addition. As Dennis points out, the relative value of different length passes depends nonlinearly on the osr. I wouldn't want to define an index which needs to be computed on a computer, so I'd rather have those relative values fixed by a "standard" osr, such as 70% (nowadays, 58% would be unacceptable).
I think Dennis' proposed calculation to figure out the (negative) value of a turnover should use a lower baseline than "average." An average player does not have zero value. If you took him away, you would have to replace him with someone who was below average, since your above average players are already getting their share of opportunities, and talent is not freely available. Games are lost for want of "average" players. A player contributing 100 yards of offense at 99% of the average rate is worth more than someone who doesn't play.
I would use a scoring percentage of something like 40% to set the replacement value. In 1994, according to this, the average Nationals team scored 39.1% off the pull. This would give a huck turnover value of -4. The completion rate for short passes would be 89.2% (which does seem a little low) and the short pass turnover value of -8.3. If we use 50%, then it's -6 for a huck and 91.7% and -11.0 for a short turnover. If we're living in a very high scoring efficiency environment, I could live with the latter numbers.
And Zaz wrote:nowadays, 58% would be unacceptable.
Nowadays, 58% will win you championships, I'd say. Do you have data suggesting otherwise? I don't think scoring efficiency has changed significantly over the last 10 or 15 years.
I would use a scoring percentage of something like 40% to set the replacement value. In 1994, according to this, the average Nationals team scored 39.1% off the pull. This would give a huck turnover value of -4. The completion rate for short passes would be 89.2% (which does seem a little low) and the short pass turnover value of -8.3. If we use 50%, then it's -6 for a huck and 91.7% and -11.0 for a short turnover. If we're living in a very high scoring efficiency environment, I could live with the latter numbers.
And Zaz wrote:nowadays, 58% would be unacceptable.
Nowadays, 58% will win you championships, I'd say. Do you have data suggesting otherwise? I don't think scoring efficiency has changed significantly over the last 10 or 15 years.
Here's a table showing the marginal scoring percentage and the cost of long and short turnovers (along with the necessary short completion percentage needed to complete 8 passes):
Marg% HuckTO Short% ShortTO
10% -0.7 75.0% -3.0
20% -1.5 81.8% -4.5
30% -2.6 86.0% -6.2
40% -4.0 89.2% -8.2
50% -6.0 91.7% -11.0
58.4% -8.4 93.5% -14.4
60% -9.0 93.8% -15.2
70% -14.0 95.6% -21.9
80% -24.0 97.2% -35.4
90% -54.0 98.7% -75.4
Marg% HuckTO Short% ShortTO
10% -0.7 75.0% -3.0
20% -1.5 81.8% -4.5
30% -2.6 86.0% -6.2
40% -4.0 89.2% -8.2
50% -6.0 91.7% -11.0
58.4% -8.4 93.5% -14.4
60% -9.0 93.8% -15.2
70% -14.0 95.6% -21.9
80% -24.0 97.2% -35.4
90% -54.0 98.7% -75.4
To flo's points. This is part of why you add a factor which weights a turnover by the opponent's chance of scoring. This is what (1-dbr) does, albeit crudely, since it's your chance of not getting the disc back (though calculated using the defensive squad).
In the book we said "A good rule of thumb for an average team is that 10 yards of field position increases the scoring percentage by 10 percent, as does being in the end zone instead of on the goal line (for better teams, use a percentage increase of 5 percent)." In reality teams do much worse than 90% from the goal line, but that is because end zone offense sucks.
somewhat off topic but...
wow. Cojones scored 68% of their offensive possessions after turning it over? Including scoring 77% of the time after getting the disc back. Am I reading that right?
What accounts for such huge numbers?
wow. Cojones scored 68% of their offensive possessions after turning it over? Including scoring 77% of the time after getting the disc back. Am I reading that right?
What accounts for such huge numbers?
somewhat off topic but...
"wow. Cojones scored 68% of their offensive possessions after turning it over? Including scoring 77% of the time after getting the disc back. Am I reading that right?
What accounts for such huge numbers?"
They were an effin' phenomenal team. And I'm pretty sure they were undefeated all year, except when they played DoG.
Anyway, I like the new additions.
I think Jim is right about lowering the bar to 50%. The rest is subtle tweaking.
I also like:
UISI=(tlp+rlp+2gt+2gc)/18 +(blocks-turn overs)*tsr
(Perhaps, I like (1-dbr) rather than tsr -- but we're getting there.) It's very, very close. But there's still one more problem. Given this ratio, it is significantly easier to get points on D rather than O. When you start on D, you have an opportunity to get that huge +1(tsr). When you start on O, you have that opportunity to get that huge -1(tsr).
A defensive person comes in, plays every point of D (deep in zone) and gets three deep turnovers (thrown to him) -- and he essentially never picks up the disc.
That's a lot of perfect offense required to match that value. 4 goals thrown, 4 goals caught, three full fields of error free throwing and receiving points equals (3*7 + 8 + 8)/18 = 37/18 = little more than 2.
Hmmm. Maybe reduce the denominator to 15 or 12?
-_Dennis
"wow. Cojones scored 68% of their offensive possessions after turning it over? Including scoring 77% of the time after getting the disc back. Am I reading that right?
What accounts for such huge numbers?"
They were an effin' phenomenal team. And I'm pretty sure they were undefeated all year, except when they played DoG.
Anyway, I like the new additions.
I think Jim is right about lowering the bar to 50%. The rest is subtle tweaking.
I also like:
UISI=(tlp+rlp+2gt+2gc)/18 +(blocks-turn overs)*tsr
(Perhaps, I like (1-dbr) rather than tsr -- but we're getting there.) It's very, very close. But there's still one more problem. Given this ratio, it is significantly easier to get points on D rather than O. When you start on D, you have an opportunity to get that huge +1(tsr). When you start on O, you have that opportunity to get that huge -1(tsr).
A defensive person comes in, plays every point of D (deep in zone) and gets three deep turnovers (thrown to him) -- and he essentially never picks up the disc.
That's a lot of perfect offense required to match that value. 4 goals thrown, 4 goals caught, three full fields of error free throwing and receiving points equals (3*7 + 8 + 8)/18 = 37/18 = little more than 2.
Hmmm. Maybe reduce the denominator to 15 or 12?
-_Dennis
It would be helpful if people in possession of databases started running the numbers with these definitions (if possible, without tweaking the results in their favor!).
For more on Cojones, see http://www.shelltown.com/~parinell/cojones.htm and /goalsb.htm and goalsa.htm (for an explanation of what it means).
They lost to Double in pool play at Nationals.
One thing that NY had that Boston never was as good at was that they would trounce the bad teams and not let up, like maybe they had caught them stealing food from Mrs. D's table.
For a specific reason, maybe they started a lot of these in good field position. Their 17/22 (77%)breaks down as 3/6 against Double and 14/16 against the lesser teams.
They lost to Double in pool play at Nationals.
One thing that NY had that Boston never was as good at was that they would trounce the bad teams and not let up, like maybe they had caught them stealing food from Mrs. D's table.
For a specific reason, maybe they started a lot of these in good field position. Their 17/22 (77%)breaks down as 3/6 against Double and 14/16 against the lesser teams.
On rewarding break mark throws:
I am not denying that break mark throws are valuable, but there are also other kinds of valuable passes not gaining yardage per se but leading to subsequent gains. Those should be rewarded too. One example is really quick continuation swing which can lead to fast break as well. Defining those passes in advance and the hoping that the stat keeper is able to objectively record those during game is not very feasible.
What if player’s offensive index depends a little on the passes following his pass? The idea is that there is value to put the disc to a teammate who is then able to advance the disc. The pass can be a break mark swing, or a short pass opening the lane, or what ever. There is no need to ask the stat keepers to estimate the value of the throws on the spot.
By no means is this solution without problems. It might overvalue players who only pass to team's best throwers. What if the next pass is turnover, should that be reflected in the previous passer's rating too? Should one only get "reward" if the following pass gains more than your pass? Should the amount of reward depend on length of the following pass? What if the next player does not make the easy pass down field, but decides to dump the disc...
Another issue is that I would like to see the stat keepers to mark down also the intended receiver for a turnover, if that player can be identified (for point blocks it can be a challenge, as well as to some hucks). My limited experience on this suggests that the turnovers tend to concentrate on few intended receivers. This should be reflected in player’s offensive index too, right?
I am not denying that break mark throws are valuable, but there are also other kinds of valuable passes not gaining yardage per se but leading to subsequent gains. Those should be rewarded too. One example is really quick continuation swing which can lead to fast break as well. Defining those passes in advance and the hoping that the stat keeper is able to objectively record those during game is not very feasible.
What if player’s offensive index depends a little on the passes following his pass? The idea is that there is value to put the disc to a teammate who is then able to advance the disc. The pass can be a break mark swing, or a short pass opening the lane, or what ever. There is no need to ask the stat keepers to estimate the value of the throws on the spot.
By no means is this solution without problems. It might overvalue players who only pass to team's best throwers. What if the next pass is turnover, should that be reflected in the previous passer's rating too? Should one only get "reward" if the following pass gains more than your pass? Should the amount of reward depend on length of the following pass? What if the next player does not make the easy pass down field, but decides to dump the disc...
Another issue is that I would like to see the stat keepers to mark down also the intended receiver for a turnover, if that player can be identified (for point blocks it can be a challenge, as well as to some hucks). My limited experience on this suggests that the turnovers tend to concentrate on few intended receivers. This should be reflected in player’s offensive index too, right?
"On rewarding break mark throws:
I am not denying that break mark throws are valuable, but there are also other kinds of valuable passes not gaining yardage per se but leading to subsequent gains. Those should be rewarded too. One example is really quick continuation swing which can lead to fast break as well. Defining those passes in advance and the hoping that the stat keeper is able to objectively record those during game is not very feasible.
"What if player’s offensive index depends a little on the passes following his pass? The idea is that there is value to put the disc to a teammate who is then able to advance the disc. The pass can be a break mark swing, or a short pass opening the lane, or what ever. There is no need to ask the stat keepers to estimate the value of the throws on the spot."
Break-mark is very obvious, conspicuous, and binary. So it's not subjective.
If you are keeping stats with three categories for throws (short -1, middle -2 and deep - 5), it's pretty easy to assign a break-mark pass that leads to upfield flow a "2" rather than a "1".
The other passes you refer to are mostly about keeping flow.
A break-mark often generates flow.
But if you don't want to reward this, you don't have to.
--Dennis
I am not denying that break mark throws are valuable, but there are also other kinds of valuable passes not gaining yardage per se but leading to subsequent gains. Those should be rewarded too. One example is really quick continuation swing which can lead to fast break as well. Defining those passes in advance and the hoping that the stat keeper is able to objectively record those during game is not very feasible.
"What if player’s offensive index depends a little on the passes following his pass? The idea is that there is value to put the disc to a teammate who is then able to advance the disc. The pass can be a break mark swing, or a short pass opening the lane, or what ever. There is no need to ask the stat keepers to estimate the value of the throws on the spot."
Break-mark is very obvious, conspicuous, and binary. So it's not subjective.
If you are keeping stats with three categories for throws (short -1, middle -2 and deep - 5), it's pretty easy to assign a break-mark pass that leads to upfield flow a "2" rather than a "1".
The other passes you refer to are mostly about keeping flow.
A break-mark often generates flow.
But if you don't want to reward this, you don't have to.
--Dennis
Dennis, rewarding only break mark throws is not fair and skews the statistics as there are also other passes leading to gains and generating flow. And you are saying not to give credit to all break mark throws but only those which have a following fast break. Break mark throws might be binary and conspicuous, but what you are saying that there are in fact two things the stat keeper needs to keep track of, not just the break mark. Some of these events will go unnoticed. And some rewards will be given when no reward would be due. The more one requires the stat keeper to make value judgments on the spot, the more mistakes there will be.
Also the player does not get the reward if the receiving player chooses not to pass the disc upfield. (Ok, this would not be rewarded in my suggested solution either, but I would like to see also other kind of positive acts to be noticed on the field than break mark throws). What if thrower sees a player at the level of the disc all open and passes the disc to him knowing that he has better chances in advancing the disc? Not a break mark throw, but generating flow. Or what about a long quick swing or cross to the open side opening a lane?
What I am trying to say is that if we make value judgment of rewarding break mark throws leading to fast break (to be “flow generating throws”), we should also reward other kind of passes leading to fast breaks and gains. We can either make a list of all the possible “flow generating throws” and hope that the stat keeper is able to record all of those during a game, or then we could reward all short gain passes which are followed with yardage gaining passes. I would prefer the latter one. Or then no reward at all.
And brushing aside some throws as being just “flow keeping” as if they would be less valuable as “flow generating” passes is not fair either. I haven’t played against Sockeye this year, but some years ago when playing against them I was truly amazed in their ability in keeping the disc in constant motion. Their style of play relied in keeping the flow, and it was very effective. They did not need very many break mark throws. Their most valuable throws were the quick keep-the-flow passes. We should not include any valuations in the statistics, which are highly dependent on the style of play. (I think we call all agree that gaining yardage and completing passes are generally important to any kind of offence.)
Also the player does not get the reward if the receiving player chooses not to pass the disc upfield. (Ok, this would not be rewarded in my suggested solution either, but I would like to see also other kind of positive acts to be noticed on the field than break mark throws). What if thrower sees a player at the level of the disc all open and passes the disc to him knowing that he has better chances in advancing the disc? Not a break mark throw, but generating flow. Or what about a long quick swing or cross to the open side opening a lane?
What I am trying to say is that if we make value judgment of rewarding break mark throws leading to fast break (to be “flow generating throws”), we should also reward other kind of passes leading to fast breaks and gains. We can either make a list of all the possible “flow generating throws” and hope that the stat keeper is able to record all of those during a game, or then we could reward all short gain passes which are followed with yardage gaining passes. I would prefer the latter one. Or then no reward at all.
And brushing aside some throws as being just “flow keeping” as if they would be less valuable as “flow generating” passes is not fair either. I haven’t played against Sockeye this year, but some years ago when playing against them I was truly amazed in their ability in keeping the disc in constant motion. Their style of play relied in keeping the flow, and it was very effective. They did not need very many break mark throws. Their most valuable throws were the quick keep-the-flow passes. We should not include any valuations in the statistics, which are highly dependent on the style of play. (I think we call all agree that gaining yardage and completing passes are generally important to any kind of offence.)
Here's a way to remove some or all of the subjectiveness of what constitutes a flow-starting or flow-continuing pass. Give a bonus point iff it leads to a stall-zero continuation pass. (or maybe a stall-one, since markers begin their stalls from far away, or any pass that's thrown before there is a real mark on.)
Post a Comment
<< Home
