Thursday, November 17, 2016

How wrong was Nate Silver?

Nate Silver has been given credit for basically not being Sam Wang and giving Clinton a 99% chance of winning the Presidency (Silver had it 72%). But in the grand scheme of things, Nate Silver was still dead wrong and dead wrong in a ton of places. Let's take a look:

Presidential prediction (by simulation): Clinton 302 - Trump 236
- Actual result: Trump 306 - Clinton 232
- States missed:  Florida, North Carolina, Pennsylvania, Michigan, Wisconsin

Senate prediction (by simulation) : Democrats 51 - GOP 49
- Actual result: GOP 51 - Democrats 48 (one undecided)
- States missed: Pennsylvania, Wisconsin


Keeps missing wide left!
Now the fact was that this election had a ton of razor thin margins in both the Presidential race, and the Senate race. It's not hard to see why someone would miss several. But once again, Nate Silver managed to "only" miss races where a Republican won and he didn't expect it, now running his streak through another election cycle of "always" getting it wrong by erroneously choosing the Democrat (or Independent) to beat a Republican.

As Nate Silver would tell you, the statistical chances of him just "randomly" missing seven races and missing them all in the same direction is less than one percent. The chances of him running this streak through several election cycles is even less than that. The only explanation that makes any sense is that whatever statistical formulas being used by Mr Silver are quite obviously skewed to favor Democrats over Republicans.

Perhaps he should think about doing something about that?

4 comments:

Indy Voter said...

Any poll-based forecast was going to be wrong. Silver gets credit for acknowledging the possibility that there was such an error. Others, like Wang, were much more dismissive of the possibility the polls were systematically off.

Anonymous said...

But once again, Nate Silver managed to "only" miss races where a Republican won and he didn't expect it, now running his streak through another election cycle of "always" getting it wrong by erroneously choosing the Democrat (or Independent) to beat a Republican.
_____________________________________________________________________________________________________________________________________

and that's the crux of the biscuit.

silver always was nothing more than an intellectually dishonest journ0list hack. how he ever got elevated to the status he enjoys in the world of polling has always baffled me.

wphamilton said...

CH, if you still have the polling data or at least results, from the period during which Silver was making his prognostications, can you now in hindsight remove his "weighting" modifications and determine how much his subjective bias was damaging to the aggregate accuracy?

C.H. Truth said...

WP - sure... if you went simply by the polling averages within each state (as RCP did), you would have concluded that Trump was leading in states that made up 266 Electoral College Votes. Silver had a couple of those that Trump was leading (Florida and North Carolina) in Hillary's column due to his weighting scheme... In those cases, he was moving them slightly to Hillary due to the fact he saw the national polls trending in her favor.

His weighting scheme also pushed Michigan and Pennsylvania further into Hillary's column than the polling showed down the stretch. This was more due to his use of older polls from pollsters he had more confidence in, rather than weighing the most recent polls highest. The most recent polls in those two states had it very tight. Even just going back a week to ten days and there were polls showing Clinton leading by significant margins.

Nobody predicted (or probably could have) Wisconsin. There was almost no polling that showed that race close.

I am sure there was a legitimate method to his madness, but it seemed to me that there was far too many good Trump polls underweighted to be just coincidence.