TeamRankings Football Pick’em Picks Historical Performance Review

TeamRankings Football Pick’em Picks Historical Performance Review

November 3, 2018 – by David Hess

In this post, we’re going to review the long-term, multi-year performance of our Football Pick’em Picks product, based on data about thousands of real-world pools that is submitted to us by our paying subscribers, either directly via the product or from end of season customer surveys that we send out.

To cut right to the chase if you just want the highlights, since the 2014 football season, our Football Pick’em Picks subscribers have:

Won season prizes 3 times as often as expectedWon weekly prizes 70% more often than expectedWon a prize in 58% of the pools they entered

After factoring in the price of the service (less than $30/year for unlimited pools, as part of our Pool Picks Subscription), this level of performance should generate very high long-term returns on entry fees in most types of pools.

Want to get our expert picks for your football pick’em pool? Read more about the product or sign up now.

Football Pick’em Picks Product Overview

In case you aren’t familiar with our Football Pick’em Picks product, here’s a quick overview:

We’ve built proprietary analytics to p out your best picking strategy for winning various types of NFL and college football “pick’em” style office pools (season prize pools, weekly prize pools, or both).Our data-driven approach is based on thousands of computer simulations of pools similar to yours, factoring in details like your pool rules (like whether you pick against the spread), the size of your pool, its prize structure, and your current position in the standings.We use publicly available national picking trends data to forecast how your opponents are likely to pick each game, and then we identify the picks that give you the best chance to beat them.

As far as we know, we are the only site that has built a system that leverages game theory principles to optimize pick’em pool picks for such a broad range of pool rules, and secondarily, to alter pick strategy based on your pool’s prize structure.

How We Measure Football Pool Pick Success

As far as we know, we’re also the only site that collects hard data on how our recommended picks perform in real-world customer pools. And there is no other way to demonstrate with any level of precision how much of an edge our picks are actually delivering to customers. Here’s why.

First, the high level of customization that we apply to pick’em pool picks means that there is no single set of “official” Team Rankings picks that we can track. Our pick suggestion for a given game differs whether you’re in a spread-based or game winner pool, whether your goal is to win season prize or weekly prizes, whether your pool is big or small, whether you’re in first place or outside of the money, etc. We designed a system that objectively evaluates many factors in order to identify the set of weekly picks that gives each individual user the best chance to win in THEIR specific pool(s).

Second, just tracking how often our picks are right vs. wrong doesn’t measure what really matters. In some pools, getting 70% of your picks right in a given year may win the pool; in other pools/years, that performance may not even come close to winning a prize. Put another way, the same score that wins a prize in a $5 buy-in company office pool may not even sniff the money in a $1,500 buy-in contest run by a Vegas sports book. Ultimately, our subscribers will judge our product based on whether it helps them win more pools and more prizes, not based on the average percentage of picks we get right.

Finally, even tracking how our picks do on a big pick’em hosting site like ESPN or Yahoo! isn’t good enough. Let’s say we track a pick set on ESPN Pigskin Pick’em and it finishes the season in the top 3% of all ESPN users nationwide. A performance like that sounds great on paper, but it doesn’t mean those picks would actually win anything for a given customer. They’d probably have a good shot at winning a prize in smaller pools…and maybe a shot at cashing in some 100+ person pools, depending on the prize structure and how smart/lucky other opponents are? Who knows, it’s a guess.

So there’s really only one good way to measure the effectiveness of our Football Pick’em Picks advice. We ask subscribers to tell us how they finished in each of their pools at the end of the season, and every week we ask subscribers in pools with weekly prizes whether they or not they won a prize the week before.

Calculating Pick’em Pool Win Expectations

Knowing how many customers won their pool is nice, but it’s only part of the equation. After all, without our help, customers will still win pools in some years. We only provide value if we help our subscribers do better than they otherwise would have on their own.

So to understand whether our picks are providing an edge, we need to know what the baseline expectation should be. In other words, how likely is it that an average TR subscriber would have won their pool anyway, without our help? Is winning a prize once every 20 years good? What about once every 10 years?

To define our baseline expectations, we assume every player in a given pick’em pool is equally skilled. Then, we calculate how often our customer should expect to win a prize, based on the number of entries they submitted, the prize structure, and the overall pool size.

This gives us the expected win rate for every customer in every pool. It tells us how often our customers would expect to win a prize if our pick advice was not providing any edge in the pool.

(It’s worth noting here that almost everyone is wildly optimistic when it comes to their expectations for winning pools. For example, if you play in a 100-person pool that’s winner-take-all and everyone in the pool is equally skilled, you should expect to win that pool once every 100 years. If you win it twice in your lifetime, you’re doing amazing.)

To calculate the actual win rate, as mentioned above, we simply ask customers if they won a prize in their pool, along with what place they finished in and the value of the prize they won.

Dividing the actual win rate by the expected win rate gives us a “Win Multiplier,” like 2.4 or 3.1. This Multiplier number tells us that our customers won prizes 2.4 times or 3.1 times as often as you’d expect an average person in the pool to win.

If our Multiplier is greater than 1, that means our pick advice, on average, has been delivering an edge.

Weekly Pools

There is one additional wrinkle here — weekly football pool prizes. Pools with weekly prizes are essentially a series of 17 (or 14 for college football) individual pools. So we keep separate data on season prizes and weekly prizes, and for the most part we report them separately.

The weekly win rates you see in the data tables below consider each week as a separate chance to win. A customer who wins 1 out of 17 weeks in their pool would show up in the data above with a 5.9% win rate (1 win divided by 17 weeks), not a 100% win rate (1 win divided by 1 pool).

You might also notice in the data below that the win rates — both expected and actual — for weekly prizes are lower than those for season prizes. That make sense, because most pools have more payout spots for season prizes than for weekly prizes. In pools with both types of prizes, often the top five (or even more) finishers will win a season prize, but only the one top score in a given week wins a weekly prize.

Long Term Performance Of TeamRankings Football Pick’em Picks

At last, the data. Since we started customizing picks for football pick’em pools in 2014, here’s how our customers have done compared to what you’d expect for an “average” pool player.

By Pool

	Season Prize Win%			Weekly Prize Win%			Won At Least 1 Prize
Year	Expected	Actual	Multiplier	Expected	Actual	Multiplier	Season or Weekly
2017	10.5%	44.3%	4.2	5.1%	9.8%	1.9	67.7%
2016	11.1%	16.7%	1.5	5.0%	7.0%	1.4	47.7%
2015	11.0%	26.3%	2.4	4.7%	7.5%	1.6	51.1%
2014	10.3%	39.4%	3.8	4.8%	8.8%	1.8	61.5%
TOTAL	10.7%	32.7%	3.1	4.9%	8.4%	1.7	58.0%

The bottom line results: our Football Pick’em Picks customers have won season prizes 3 times as often as expected, and won weekly prizes 70% more often than expected.

In the rightmost column, you can see the overall win rate for our customers, across all pool types, taking both season and weekly picks into account. Over the past four years, our customers won a prize in 58% of their pools.

By Customer

The data above looks at performance on a pool by pool basis. A customer that enters four pools and wins one of them shows up with a 25% win rate (1 win divided by 4 pools).

Another way of looking at the data is to check performance by customer. Many of our customers enter multiple pools, and winning a prize in at least one pool can often make it a financially profitable season for them.

When we view the data this way, we combine the season and weekly data, and just look at the percentage of customers that won a prize of any sort. So viewing the data by customer, our hypothetical customer from above would have a 100% win rate (1 “winning customer” divided by 1 customer overall).

Here’s our performance by customer:

	% of Customers Winning At least 1 Prize
Year	Expected	Actual	Multiplier
2017	52%	81%	1.5
2016	55%	60%	1.1
2015	50%	64%	1.3
2014	54%	76%	1.4
TOTAL	53%	71%	1.3

Over the past four seasons, on average over 70% of our football pick’em picks customers have won a prize in at least one pool. Overall, they’ve won at least one prize 30% more often than expected. Given that the baseline win rate already started out around 50%, it’s going to be tough to do much better than that, since there’s no way to guarantee a pool win.

More recently, in the 2017 season 81% of our football pick’em pool picks customers won a prize in at least one of their pools.

Football Pick’em Pick Performance Splits

The numbers above show overall performance. However, we provide picks for a wide variety of pool rules and sizes. It’s worth looking at performance by pool type or by other factors, to see if only certain types of pools perform well, or if the edge holds across various types and sizes.

All the data in this section uses grand totals over all the years we’ve been offering custom football pick’em picks, and reports data by pool rather than by customer, unless otherwise noted.

By Type Of Pick’em Pool

First, here is customer performance by type of pool. Note that we support combinations of these types, but if we break it down any further, the sample size gets too small to be meaningful:

	Season Prize Win%			Weekly Prize Win%			Won At Least 1 Prize
Pool Features …	Expected	Actual	Multiplier	Expected	Actual	Multiplier	Season or Weekly
Game Winner Picks	11.2%	39.3%	3.5	5.3%	8.9%	1.7	61.4%
Spread Picks	9.9%	22.1%	2.2	4.2%	7.5%	1.8	52.4%
Fixed Points	9.6%	26.6%	2.8	4.3%	8.5%	2.0	52.7%
Confidence Points	11.8%	39.0%	3.3	5.6%	8.4%	1.5	64.2%
Fixed Game Slate	10.8%	34.0%	3.1	5.0%	8.5%	1.7	59.9%
User Selects Games	10.0%	23.1%	2.3	4.3%	8.1%	1.9	44.1%
NFL	10.9%	32.9%	3.0	5.0%	8.3%	1.7	59.1%
College Football	10.0%	34.4%	3.4	4.8%	9.4%	2.0	50.0%
NFL + College	9.9%	27.9%	2.8	5.0%	10.3%	2.1	56.7%

As you can see, our football pick’em picks have delivered an edge in all types of supported pools. Customers have won season prizes 2.2 to 3.5 times as often as expected, depending on pool type. And they’ve won weekly prizes 50% to 110% more than expected.

Our best performance in season pools (relative to expectations) has come in these types of pools:

Game Winner pools (3.5x)College Football pools (3.4x)Confidence Point pools (3.3x)Fixed Game Slate pools (3.1x) [Every entrant picks the same games, rather that selecting a subset of games to pick.]NFL pools (3.0x)

Our best performance in weekly pools (again, relative to expectations) has come in these types of pools:

NFL + College Combo pools (2.1x)College Football pools (2.0x)Fixed Points pools (2.0x) [Pool entrants don’t get to assign confidence points to their picks.]User Selects Games / “Pick 6”-Style pools (1.9x) [Entrants select a subset of games to make picks on, from a broader pool of allowable games.]Spread pools (1.8x)

By Pick’em Pool Size

Now, here is performance by pool size:

	Season Prize Win%			Weekly Prize Win%			Won At Least 1 Prize
Pool Size	Expected	Actual	Multiplier	Expected	Actual	Multiplier	Season or Weekly
0-24	17.0%	46.4%	2.7	9.7%	15.7%	1.6	72.9%
25-49	11.3%	35.2%	3.1	4.9%	8.3%	1.7	62.7%
50-99	7.6%	26.0%	3.4	3.1%	5.5%	1.8	53.7%
100-249	5.6%	20.5%	3.7	1.6%	3.6%	2.3	38.3%
250-999	2.9%	9.1%	3.2	1.1%	2.9%	2.8	31.0%
1000-9999	1.1%	5.6%	5.0	0.2%	2.8%	13.4	12.0%
10000+	0.0%	0.0%	0.0	0.0%	0.3%	11.0	2.5%

This is a pattern we’ve seen before in our office pool product performance. As pool sizes go up, the absolute win rate goes down, but the edge delivered by our picks goes up. This makes some sense. If you play in a very small pool and start out with, say, 15-20% win odds, realistically there’s an upper bound on how much we can improve that. We also suspect there is more “dead weight” in huge sized pools — players who just make dumb picks because they either don’t know any better or don’t put in the effort required to do so.

One note on the 10,000+ pool size bin, which shows a 0% win rate for season prizes. The sample size in that bin is small enough that even if we delivered a 10x multiplier, we wouldn’t expect to see any wins. A 10x multiplier would move your win odds from 1 in 10,000 to 1 in 1,000. So this sample size is simply too small to tell us anything very meaningful about our season picks edge in giant pools.

However, note that we actually did get a report of some weekly wins in pools of 10,000+. And the reported edge in those pools was greater than 10x, in line with what we see as pool size increases. (Remember that the sample size here is essentially 14 to 17 times larger than the sample size of season pools. It’s not super surprising that we saw a positive result here despite seeing nothing in the season pools.)

The bottom line is, if you want a fighting chance of winning a season prize in a 10,000+ person pool in your lifetime, you better be playing tons of entries.

Four Seasons of Football Pick’em Pick Results: So Far, Very Good

We’ve been offering football pick’em picks customized by pool type, pool size, and payout structure for four seasons and change now.

Based on the data we’ve collected from subscribers, our picks have been consistently profitable over those four seasons. There have been ups and downs, as expected, but even in the worst season, our picks delivered a +40% to +50% edge over the average pool picker, and 60% of our customers won a prize in at least one of their pools.

In the best years for our picks, 75% to 80% or more of our customers may win a prize. Of course, that’s still not 100%; winning a prize in a pool in any given year is far from a guarantee, even with our help. But the fact that our customer base, on average, is winning season prize pools 3 times as often as expected and winning season prizes 70% more often than expected is a clear demonstration of the edge our product delivers.

For subscribers with a long-term mentality, the investment in TeamRankings pick’em picks should pay off very well. Not to mention the fact that we’re the only site that can demonstrate, with comprehensive data spanning multiple seasons and thousands of real-world pools, that we’re helping subscribers win more.

Of course, we’ll keep on working to make the product even better in the future.

If you want to get our expert, customized picks for your own football pick’em pool(s), we invite you to learn more about the product or sign up now.

If you liked this post, please share it. Thank you! Twitter Facebook