Blog Archives

My thoughts about things that are on my mind. Comments are my opinion and do not represent the opinions of my employer.

Latest NCAA Tournament Predictions: Still No Overwhelming Favorite

The latest NCAA Tournament predictions are up and the chaos of the tournament is likely to continue. Although Gonzaga has the highest probability of becoming champion, no team has a greater than 50% chance at making the final four. Benefitting from the losses of Duke and Villanova, Florida now has the second highest chance of making it to the final four, sitting at nearly 41%.

What strikes me is the parity in this year’s top teams. After a relatively un-eventful first round, upsets seemed to happen in the round of 32. Such a result seems due to having a group of very good teams that are capable of beating anyone and another group of teams that aren’t as good. We are now in a spot where only the best teams remain, so expect more upsets to happen for the remainder of the tournament.

The updated model estimates gives the following predictions for the remainder of this year’s NCAA tournament:

TeamSeedRegionElite EightFinal FourFinal TwoChampion
Gonzaga1West0.67560.49520.37210.2655
Kansas1Midwest0.67830.40740.22520.0969
Arizona2West0.82650.30970.17290.0963
Florida4East0.59650.41540.16490.0883
UCLA3South0.54250.31330.17560.0784
Michigan7Midwest0.57730.28050.13730.06
North Carolina1South0.5420.25040.12660.0551
Kentucky2South0.45750.24130.1250.0539
Wisconsin8East0.40350.24650.09320.0452
West Virginia4West0.32440.16650.08390.0428
Baylor3East0.64850.24870.08470.0355
Butler4South0.4580.1950.08620.0321
Purdue4Midwest0.32170.15550.06390.0229
Oregon3Midwest0.42270.15660.06020.0194
South Carolina7East0.35150.08940.02180.0065
Xavier11West0.17350.02860.00650.0012

For more information, please visit my predictions main page at http://jonathantemplin.com/2017-ncaa-mens-basketball-predictions/.

 




2017 NCAA Tournament Prediction: Get Ready for Bracket Chaos

Just in time for the 2017 NCAA Men’s Basketball Tournament, I have put together a new statistical model to assess team strength and make predictions about the NCAA Tournament. Two years ago I used a well known model to make predictions about the NCAA Tournament, with limited success. The upcoming tournament is the first time new model has been used so this year is essentially a live test case.

For more information, please visit my predictions main page at http://jonathantemplin.com/2017-ncaa-mens-basketball-predictions/. To download some of the 10,000 model-generated brackets, please visti http://jonathantemplin.com/sports/2017brackets.

The model gives the following predictions for this year’s NCAA tournament:

TeamSeedRegionGames WonRound of 64Round of 32Sweet 16Elite 8Final FourFinal 2Champion
Gonzaga1West3.5110.9820.8040.6460.4940.3340.246
Villanova1East2.5410.9430.6870.4150.2650.1390.095
Kentucky2South2.3510.9570.6220.3610.2260.1290.058
Duke2East2.5410.9490.7560.4630.220.0940.054
Saint Mary's (CA)7West2.0710.9060.5110.3780.1530.080.047
UCLA3South2.310.9570.6510.3350.2110.1070.04
Kansas1Midwest2.5210.9740.7380.4380.2170.1180.039
Virginia5East1.7510.8310.4510.230.1350.0640.038
Louisville2Midwest1.9910.9490.4650.2860.1650.0930.036
Arizona2West1.9410.9150.4730.3290.1280.0610.035
Oklahoma State10Midwest1.3210.5690.3220.1990.1250.0720.034
Florida4East1.9410.9630.5270.2380.130.0490.029
North Carolina1South2.0310.8860.5650.3230.1510.0740.026
Iowa State5Midwest1.4310.6710.3820.2050.1010.0530.022
West Virginia4West1.5210.7790.4420.1610.0820.0390.021
Purdue4Midwest1.7210.8720.4680.2180.0980.050.018
Baylor3East1.8710.9010.5450.2610.1060.0370.018
Creighton6Midwest1.1210.5490.3120.1370.0720.0380.015
Butler4South1.4510.7110.3840.2080.0910.040.014
Michigan7Midwest0.8610.4310.2040.1140.060.0330.013
Southern Methodist6East1.3710.730.360.1710.0710.0270.013
Wichita State10South1.3210.7490.3050.1420.0790.0330.01
Oregon3Midwest1.5310.8360.4160.1610.0720.0330.009
Kansas State11aSouth1.510.66270.4630.20.0940.0550.0250.009
Minnesota5South1.0410.5160.2880.1410.0570.0260.008
Notre Dame5West1.5410.8880.4950.1010.0390.0130.007
Maryland6West1.7210.8550.6040.1890.0480.0170.007
Wisconsin8East0.8510.5430.1820.070.0330.0130.007
Rhode Island11Midwest0.8510.4510.2380.0930.040.020.006
Middle Tennessee12South0.9210.4840.2530.1240.0420.0130.004
Miami (FL)8Midwest0.9610.6510.1950.0770.0280.0110.003
Florida State3West1.2310.7870.3130.0930.0220.0080.003
Northwestern8West0.6910.5350.0930.0410.0160.0060.003
Arkansas8South0.910.5210.230.1050.0340.0120.003
Vanderbilt9West0.6310.4650.10.0430.0160.0060.002
Virginia Tech9East0.6510.4570.1210.0430.0160.0060.002
Marquette10East0.8410.5940.1590.0650.0160.0050.002
Nevada12Midwest0.510.3290.1160.0370.0130.0040.001
Cincinnati6South0.4910.350.0910.0320.0140.0030.001
Seton Hall9South0.7710.4790.1940.0740.0190.0050.001
Wake Forest11bSouth0.610.33730.1870.0530.0190.0090.0040.001
Michigan State9Midwest0.4410.3490.0630.020.0040.0010
Vermont13Midwest0.1710.1280.0340.0060.0020.0010
Iona14Midwest0.2110.1650.0350.0080.0020.0010
Jacksonville State15Midwest0.0610.0510.0090.001000
North Carolina Central16aMidwest0.80.770.0250.0050.001000
UC-Davis16bMidwest0.230.20.00100000
Virginia Commonwealth10West0.110.0950.0080.001000
Xavier11West0.2310.1450.0740.0090.00100
Princeton12West0.1610.1120.0380.0050.00100
Bucknell13West0.2510.2210.0250.002000
Florida Gulf Coast14West0.2210.2130.0090000
North Dakota15West0.0910.0850.0070.001000
South Dakota State16West0.0210.0180.0040000
South Carolina7East0.510.4060.0740.0140.00200
Southern California11aEast0.580.45470.0980.0190.0050.00100
Providence11bEast0.790.54530.1720.0550.0170.0040.0010
North Carolina-Wilmington12East0.1910.1690.0140.002000
East Tennessee State13East0.0510.0370.0080.001000
New Mexico State14East0.1210.0990.0210.0030.00100
Troy15East0.0610.0510.010.002000
Mount St. Mary's16aEast0.410.41120.00100000
New Orleans16bEast0.660.58880.0560.0110.001000
Dayton7South0.3510.2510.070.0170.0080.0020
Winthrop13South0.3910.290.0750.0230.0050.0010
Kent State14South0.0510.0430.0040000
Northern Kentucky15South0.0510.0430.0030000
Texas Southern16South0.1310.1140.0110.002000



2017 NCAA Tournament Predictions

Just in time for the 2017 NCAA Men’s Basketball Tournament, I have put together a new statistical model to assess team strength and make predictions about the NCAA Tournament. Two years ago I used a well known model to make predictions about the NCAA Tournament, with limited success. The upcoming tournament is the first time new model has been used so this year is essentially a live test case.

For more information, please visit my predictions main page at http://jonathantemplin.com/2017-ncaa-mens-basketball-predictions/.

 




2017 Statistics Workshops: Diagnostic Modeling and Multilevel Longitudinal

Learn from instructors who wrote the book(s) on the topics! All workshops are given in a relaxed atmosphere with ample time for one-on-one interaction with the instructors.

Stats on the Beach is a workshop series taught by Lesa Hoffman and Jonathan Templin (co-founders of AnalyticAble LLC). In 2017 we are offering two workshops, located in Hilton Head Island, South Carolina:
  • Diagnostic Measurement (May 22-25, by Jonathan Templin)
  • Introduction to Longitudinal Multilevel Models (May 30-June 2, by Lesa Hoffman)
Our format for each includes separate morning and late afternoon sessions, with a long mid-day break so you can let the content sink in (or meet with us individually) while enjoying the beach. For more information, please visit our website: StatsOnTheBeach.com
Please send a link to this announcement to anyone who might be interested, and let us know if you have any questions!



2015 NCAA Tournament Predictions: Final Update

Wisconsin pulled the upset of Kentucky, which makes the final of the 2015 NCAA Tournament an interesting match-up between two very evenly matched teams. Throughout the tournament, all predictions indicated that Wisconsin was (a very distant) second behind Kentucky with respect to probabilities of winning the tournament.

Based on the latest (and final) model predictions, Wisconsin is a slight favorite over Duke in a game that is essentially a toss-up:

TeamSeedRegionProb of ChampionshipB-T Effect
Wisconsin1West0.53265.461
Duke1South0.46745.294

Overall Model Results

Although the Bradley-Terry model was accurate at picking the winner for only 71.2% of the games, a deeper analysis shows that the model predictions were well calibrated but that the model itself lacked the prediction power to be more informative. In the figure below, the X-axis represents the model-predicted probability a team would win and the Y-axis represents the expected true proportion of wins for any given probability. The solid red line is the model the dashed black line is where a perfectly calibrated model would fall.

modelpred

The graph shows that the model was, for the most part, well calibrated in its assessment of probability in that the red line is nearly on top of the black line. Where the model missed was with predicted probabilities close to .5 and close to 1. In these cases, the model was slightly over-confident in its assessment of probability.

What does that mean for the final game (with a model-predicted probability of winning of .53)? Based on the curve in the graph, the result is likely closer to a 50-50 game.

What this says to me is that a better model is needed…and one will be forthcoming in time for the next round of predictions…to be continued…




2015 NCAA Tournament Predictions: Final Four Update

The 2015 NCAA Tournament is down to the final four teams. The biggest surprise is Michigan State, who started the tournament with a 1.8% chance to make the final four. Those chances were especially long when compared to the pre-tournament chances of the other three final four teams (Kentucky at 67.9%, Wisconsin at 43.2%, and Duke at 30.3%).

Kentucky is still the clear favorite to win the tournament. Should Kentucky not advance to the final game, however, there will be no clear favorite to win out.

TeamSeedRegionFinal GameWinnerB-T Effect
Duke1South0.730.205.23
Kentucky1Midwest0.700.576.63
Michigan St7East0.270.034.08
Wisconsin1West0.300.205.54



2015 NCAA Tournament Predictions: Elite Eight Update

The final eight teams are set in the NCAA Tournament. From the updated model results (method described in this previous post), of the four games in this round, only Kentucky’s match-up with Notre Dame has a high probability of one team winning (Kentucky at .794). The remaining three games are close to toss-ups.

Overall, Kentucky now has almost even odds to win the tournament. If Kentucky were to lose, Wisconsin, and the winner of the Duke and Gonzaga (the model thinks Duke with a .531 probability of winning) game would be next in line to be the favorite.

TeamSeedRegionB-T EffectProb Final 4Prob Final GameProb Winner
Kentucky1Midwest6.4890.7940.5900.477
Wisconsin1West5.3720.5990.2130.133
Duke1South5.1230.5310.3750.120
Gonzaga2South4.9770.4690.3210.104
Arizona2West4.9510.4020.1120.064
Notre Dame3Midwest4.7830.2060.0850.044
Louisville4East4.3380.5920.1990.043
Michigan St7East3.9420.4080.1050.017



2015 NCAA Tournament Predictions: Sweet 16 Update

Although KU’s loss yesterday means basketball exist to many here in Lawrence, Kansas, there still is a tournament going on and 16 teams left. Here is a brief update on the model predictions for the Sweet 16. Again, this uses the Bradley-Terry model predictions using methods described in this post.

I’ll update the model results for each remaining round and recap the accuracy of the model predictions at the end of the tournament. It is interesting to compare this model’s predictions with those from other prediction systems, such as Ken Pomeroy’s log 5.

Overall, Kentucky appears to be the overwhelming favorite. Their likely opponent will be the team that emerges from the South: most likely Duke or Gonzaga.

TeamB-T EffectProb Elite 8Prob Final 4Prob Final GameProb WinnerExpected #Wins
Kentucky6.420.90830.73220.56410.46324.6678
Wisconsin5.280.79480.50340.19210.1263.6163
Gonzaga4.920.87160.46790.33260.11373.7858
Duke5.010.72640.41030.28890.09873.5243
Arizona4.900.80830.38470.11850.0713.3825
Notre Dame4.640.56530.14490.06450.03382.8085
Louisville4.250.70390.40890.14760.03073.2911
Wichita St4.340.43470.09390.03630.0172.5819
Oklahoma3.900.52190.25130.07840.01342.865
Michigan St3.820.47810.22220.07230.01172.7843
Utah3.920.27360.09970.0510.00982.4341
North Carolina3.770.20520.07150.01240.00432.2934
North Carolina St3.380.29610.11760.02360.00292.4402
West Virginia3.640.09170.0290.00690.00212.1297
Xavier OH3.360.19170.04040.00520.00142.2387
UCLA2.800.12840.02210.00560.00032.1564



2015 NCAA Tournament Predictions: Round of 32 Update

The play-in games and the first round of the NCAA tournament featured a number of upsets. The Bradley-Terry model predictions from my previous post took a beating the first two days but somehow managed to make a number of good predictions later in the first round.

Accuracy of Play-in and Round 1 Predictions

Overall, the model was correct on 72% of all predictions. I’m pretty sure the model and most peoples’ brackets took a beating in the same places- the big upsets of the first day of the round of 64.

The table also lists proportion correct by probability. These are helpful to see if the model’s probabilities are either over- or under-confident. If neither were to happen, we should expect that over the long run, the model will have a proportion correct equal to it’s predicted probability. That is, for all predictions of winning with probability of .9, we would expect the model to predict 90% of the games correctly. The table shows a rough alignment with proportion correct and predicted probability, but then again, our sample of games is very small so far.

Predicted ProbabilityProportion CorrectNumber of Predictions
All0.7236
0.5-0.60.5511
0.6-0.70.676
0.7-0.80.676
0.8-0.91.004
0.9-10.899

Updated NCAA Tournament Probabilities

Using the methods described on the previous post, the updated probabilities for winning are listed in the table below. Of note: the team with the biggest increase in probability for winning the tournament was Gonzaga, which was aided by two upsets in their half of the South bracket. Also interesting is that despite winning their first game, Kansas has a decreased chance of winning in each of the next rounds, likely due to the Big 12 having a terrible showing in the tournament so far.

In the table, the last three columns represent the change in championship winning probability, change in Bradley-Terry effect, and change in expected number of wins over the pre-tournament predictions.

TeamBT EffectSweet 16Elite 8Final 4Final GameWinnerExpected # WinsChange WinProbChange BT EffectChange # Wins
Kentucky6.3710.9370.8020.6700.5350.4144.3580.0040.0170.103
Virginia5.3480.8120.6740.3900.2600.1123.2480.0070.0680.128
Villanova5.2700.8690.5790.3440.2230.0983.1130.0180.0690.313
Wisconsin5.2110.8010.6550.4430.1730.0983.1700.0130.061-0.030
Gonzaga4.8360.7730.6900.4030.1810.0663.1130.046-0.0280.658
Arizona4.8310.8360.6780.3590.1150.0543.042-0.0060.0730.317
Duke4.8960.7720.5470.3350.1510.0492.8530.0140.0700.178
Northern Iowa4.5830.6080.2560.1280.0710.0242.087-0.0260.0810.127
Notre Dame4.5100.6500.3530.0890.0440.0192.155-0.0110.0860.135
Kansas4.4770.5750.3040.0720.0340.0131.999-0.002-0.067-0.096
Maryland4.4110.6960.1440.0740.0310.0121.9580.0020.0730.273
Wichita St4.1720.4250.2060.0490.0220.0091.710-0.0060.1100.365
Louisville4.0890.3920.1310.0490.0230.0051.599-0.0050.0870.214
Utah3.7920.4860.1690.0700.0200.0041.749-0.0010.1060.379
Georgetown DC3.8510.5140.1820.0770.0200.0041.796-0.0010.0880.386
Oklahoma3.8320.6760.1690.0460.0180.0041.913-0.016-0.0500.453
Butler3.8570.3500.1370.0270.0100.0031.526-0.0020.1060.466
North Carolina3.6610.5300.1300.0520.0090.0021.723-0.0030.1090.488
Oregon3.5760.1990.1050.0370.0070.0021.3500.0020.1230.600
San Diego St3.5710.2280.1020.0400.0080.0021.3800.0020.0900.650
Xavier OH3.3730.6550.1780.0460.0070.0011.8870.0010.1061.072
Iowa3.4130.2270.1660.0510.0090.0011.4540.0010.0960.614
Michigan St3.6770.1880.1080.0280.0100.0011.3350.0010.1180.570
West Virginia3.4930.3040.0350.0130.0030.0011.3550.001-0.0260.465
Arkansas3.5060.4700.1100.0370.0050.0011.6230.0010.0610.578
Cincinnati3.0900.0640.0190.0060.0020.0011.0900.0010.0830.605
Ohio State3.0090.1640.0810.0150.0020.0011.2620.0010.1440.752
Georgia St2.7370.3450.0630.0110.0010.0001.4210.0000.1891.166
Dayton3.0440.3240.0500.0070.0020.0002.3830.0000.0341.733
Alabama-Birmingham1.7400.2820.0240.0020.0000.0001.3090.0000.1881.219
North Carolina St3.1940.1310.0350.0080.0020.0001.1750.0000.1110.510
UCLA2.7180.7180.1200.0240.0030.0001.8650.0000.1811.540



2015 NCAA Tournament Predictions: Pre-Tournament

NCAA tournament predictions are pretty common these days, from FiveThirtyEight.com to Bing.com. Not to be left out of the fun, here are some predictions I put together using a fairly simple model and R.

Prediction Methods

Using data from the 2014-2015 NCAA men’s basketball season (posted at http://prwolfe.bol.ucla.edu/cbasketball/scores.htm), I used a modified version of the so-called Bradley-Terry model to estimate a relative strength effect for each NCAA Division I team. The Bradley-Terry model predicts the outcome of an comparison (a game for this analysis) using a logistic regression where the winner of the game is predicted by who played and other covariates. I added in covariates to control for the location of the game (home, away, or at a neutral site). This model was estimated using maximum likelihood with the glm() function in R.

Using the estimates and the asymptotic covariance matrix, I then used a simulation to predict the outcome of the tournament. For each of 10,000 replications, I the team’s strength effect from a multivariate normal distribution. For each game in the tournament, I then proceeded to create the Bradley-Terry model predicted probability of a win. The results shown below are summaries of the simulation.

Time permitting, I will update the results of the tournament as the event progresses. I’m interested in seeing how effective a very simple model for the outcome of games can be considering all the varying complexities of the sport not incorporated into this analysis. For each subsequent analysis, a new Bradley-Terry model will be estimated using the results of any tournament games.

Predictions for First Round (Play-in Games)

  • BYU over Mississippi (58.3%)
  • Manhattan over Hampton (66.2%)
  • Boise State over Dayton (55.4%)
  • North Florida over Robert Morris (a toss-up, really 50.2%)

All Pre-Tournament Results

Perhaps the easiest way to summarize the table is to say it is Kentucky versus the field. Kentucky has an estimated .422 probability of winning the tournament. Only five teams (Kentucky, Virginia, Wisconsin, Villanova, and Gonzaga) have win probabilities of greater than .05.

The results are summarized in the table below. The columns are sortable with a click.

The BT Effect column is the estimated Bradley-Terry model effect. When comparing two teams who may meet in a tournament game, the team with the higher probability of winning is the team with the larger BT Effect number.

The columns for each of the varying the rounds are the probability a given team makes it to that round. For instance, Kansas has a .356 probability of reaching the Elite 8 round.

The wins column represents the expected number of wins in the tournament.

TeamSeedRegionBT EffectRound or 32Sweet 16Elite 8Final 4Final GameWinner# Wins
Alabama-Birmingham14South1.55.072.012.001.000.000.0000.09
Albany NY14East2.26.187.046.006.001.000.0000.24
Arizona02West4.75.940.752.544.295.095.0472.67
Arkansas05West3.46.620.336.090.032.005.0011.08
Baylor03West4.00.791.534.226.087.018.0061.66
Belmont15East1.82.040.007.001.000.000.0000.05
Boise St11aEast3.27.233.112.026.007.002.0000.93
Brigham Young11aWest3.16.287.117.035.010.001.0001.03
Buffalo12Midwest2.92.364.104.010.003.000.0000.48
Butler06Midwest3.75.660.270.104.017.006.0021.06
Cincinnati08Midwest3.01.516.034.011.003.001.0000.56
Coastal Carolina16West1.84.049.014.004.000.000.0000.07
Davidson10South2.98.428.089.028.008.001.0000.55
Dayton11bEast3.02.157.068.012.003.000.0000.69
Duke01South4.83.952.713.530.303.141.0482.69
Eastern Washington13South2.29.210.059.009.002.000.0000.28
Georgetown DC04South3.76.790.444.156.057.016.0031.47
Georgia10East3.23.428.077.032.006.001.0000.54
Georgia St14West2.54.209.078.018.004.001.0000.31
Gonzaga02South4.84.932.751.494.306.153.0582.69
Hampton16aMidwest0.66.003.000.000.000.000.0000.34
Harvard13West2.58.293.109.020.006.001.0000.43
Indiana10Midwest2.85.255.058.016.002.000.0000.33
Iowa07South3.31.572.145.050.013.002.0000.78
Iowa St03South4.32.928.608.287.147.054.0152.04
Kansas02Midwest4.55.921.596.356.091.044.0182.03
Kentucky01Midwest6.36.987.930.806.679.545.4224.37
Lafayette16East1.71.041.010.001.000.000.0000.05
Louisville04East4.00.862.391.139.055.022.0051.48
LSU09East3.07.497.082.025.006.001.0000.61
Manhattan16bMidwest1.40.010.003.000.000.000.0000.67
Maryland04Midwest4.34.769.548.120.060.027.0111.53
Michigan St07East3.56.572.128.064.018.007.0010.79
Mississippi11bWest2.80.172.058.013.002.000.0000.66
New Mexico St15Midwest1.85.079.014.002.000.000.0000.09
North Carolina04West3.55.707.383.108.039.007.0011.24
North Carolina St08East3.09.503.080.025.006.002.0000.62
North Dakota St15South1.92.068.016.002.000.000.0000.09
North Florida16aSouth1.53.023.004.001.000.000.0000.53
Northeastern14Midwest2.06.106.023.002.000.000.0000.13
Northern Iowa05East4.50.818.531.235.119.065.0221.79
Notre Dame03Midwest4.43.895.609.321.080.040.0151.96
Ohio State10West2.87.406.081.028.006.001.0000.52
Oklahoma03East3.88.813.440.137.045.017.0031.45
Oklahoma St09West3.11.417.072.034.010.002.0000.53
Oregon08West3.45.584.128.069.027.003.0010.81
Providence RI06East3.77.611.334.102.031.011.0021.09
Purdue09Midwest2.94.485.034.010.003.000.0000.53
Robert Morris PA16bSouth1.52.025.004.001.000.000.0000.53
San Diego St08South3.47.477.127.063.021.005.0010.69
SMU06South3.82.760.329.129.057.019.0051.30
St John's NY09South3.56.523.153.081.027.007.0010.79
Stephen F. Austin12South2.84.315.124.029.007.002.0000.48
Texas11Midwest3.03.340.098.025.003.001.0000.47
Texas Southern15West1.75.061.016.003.000.000.0000.08
UC-Irvine13East2.05.138.020.003.000.000.0000.16
UCLA11South2.53.240.051.010.002.000.0000.30
Utah05South3.68.685.372.131.050.014.0031.25
VA Commonwealth07West3.33.594.151.067.020.003.0010.84
Valparaiso13Midwest3.02.231.102.009.002.001.0000.35
Villanova01East5.20.959.829.561.331.211.0952.99
Virginia02East5.28.960.788.621.368.246.1093.09
West Virginia05Midwest3.52.636.246.034.011.004.0010.93
Wichita St07Midwest4.06.745.332.174.045.021.0091.33
Wisconsin01West5.15.951.786.638.432.170.0903.07
Wofford12West2.92.380.173.037.012.002.0010.60
Wyoming12East2.75.182.058.011.003.001.0000.25
Xavier OH06West3.27.542.213.067.017.003.0010.84

Let the games begin…