UPDATED: Was there a Clinton Surge or not?

Updated to include polls through Oct 26th (AM, more polls later in the day on the 26th will be added at the next update):

screen-shot-2016-10-26-at-9-46-18-am

Updated, 25 October AM

As I expected, and demonstrated much to the consternation of everyone, the ever widening double digit lead of Clinton over Trump in an increasing number of polls meme is a falsehood. Here is the latest graphic using the same approach as described below, but updated to reflect additional polls.

screen-shot-2016-10-25-at-8-28-14-am

Rather than a widening, or even consistent, gap, or a gap that is double digit, we see Clinton continuing to lead, but pretty much in the same way that she has led since the conventions. In other words, the three presidential debates, the release of Trump's tax records, the sexual assault tape, the confirmation of many actual groping cases, and the VEEP debate, may have had some short term effects on the polls, and if you look closely and squint, may have actually re-widened Clinton's lead to post convention levels a bit, but for the most part, we are looking at a pretty steady relationship between the two candidates from the end of the convention period to the present.

When the general polls conform to expectations, they matter. When they don't conform to expectations, say "yeah, but what really matters is the electoral college, and in the electoral college ... bla bla bla."

And yes, since we attempt to choose our president using the Electoral College (though that doesn't always work) that is what matters, and it may be the case, though I can not independently confirm this at this exact moment in time (Tuesday AM), that Clinton is either taking or widening the lead in some of the swing states, and some red states are turing less red, as we speak. But, it turns out that we DO look at the general numbers for a number of reasons, including the fact that we expect general trends to conform to state wide trends, as a check on what we are seeing, and general trends may matter down ballot.

The original reason that I wrote this post is that I was concerned that a lot of commenters (and maybe voters) had come to the conclusion that Clinton's lead was growing, nearing or in the double digit range, and that the Clinton campaign need not look back, and could start doing other things, but, my read on the polls was that the debate/scandal swing looked like earlier swings, and I had little faith that it was long lasting. I took a look at the data and saw preliminary information suggesting that this may be the case. And now, that is confirmed. I conclude for now that the three presidential debates, the release of Trump's tax records, the sexual assault tape, the confirmation of many actual groping cases, and the VEEP debate, may have had some short term effects on the polls, and if you look closely and squint, may have actually re-widened Clinton's lead to post convention levels a bit, but for the most part, we are looking at a pretty steady relationship between the two candidates from the end of the convention period to the present.

And yes, I said the part that the incredulous will ignore twice.

I may do another electoral projection to replace this one later today.

Original Post:
America. Democracy. Decency. Thoughtfulness. Everybody and every thing, it feels like.

Everyone is upset this morning about Trump's comment that he will wait and see about the results before he accepts them. His comments are deplorable and astonishing, but I think they are also a distraction. If he ignores the results, it may be a bit messy but he will be ignored. A few militia groups will go and take over a Federal facility or two, but that will be managed. Unless the Congress gets on board with denying Clinton the presidency, nothing really bad will happen.

I'm more alarmed by all the comments he made in this debate, and perviously, about how he would handle wars, the military, the economy, the law, the Supreme Court, trade, ethnic/race relations, and his comments about women (which continued last night). Those are all problems that will ruin us as a country if he wins, and that have damaged us as a country already even if he walks away from this race right now. I'm not all that worried about him having a tantrum if he loses.

And, of course, it is maximally concerning that Trump wins the election, than it is that he loses and refuses to go quietly. This is because it is simply not the case that Hillary Clinton and the Democrats have this sewn up. Let me show you why.

screen-shot-2016-10-20-at-9-45-37-am

This graph shows the daily averaged-out polls, all of them, as listed by RCP's site, since July 1st (plotted on a y-axis of days before the election). There is a 3 day moving average imposed on this (a shorter moving average than usual, but this is an average of averages, and those averages are of polls taken over varying numbers of prior days, so we have plenty of helpful smoothosity on that curve).

Never mind the details for a moment. Notice first that over this time, which starts in the month of the conventions and goes up to the present, there is an overall pattern of oscillation. For much of the time all of the pols are within the margin of error, but Clinton's polls are usually higher than Trumps, when averaged out. If you apply the FiveThirtyEight method, or use similar approaches, to combine the different polls into probability statements, one can be more definitive about Clinton's overall and consistent lead since the conventions.

But, notice that about 50 days out, the two candidate's polling became close before Clinton started to separate again, and also notice, that this cycle of Clinton pulling ahead and then drawing down again seems to be happening one more time. There was probably a lot of pressure separating Clinton and Trump, with Trump's bizarre and generally poor performance in the debates, the revelation of the tape in which he seems to have no clue that sexual harassment is not OK, and the revelations seeming to confirm that he is a serial sexual molester, and the tax story from the NYT, and all of that. But the about 27 days out, that pressure relaxes, and all the numbers regress towards the mean again.

Let me put this another way, as a stark but supportable hypothesis. About 50% of the United States would vote for Trump, and about 50% would vote for Clinton. People talk about the 35% to 40% Trump base, and that's real. And Clinton has a similar base. But the rest of the country, the 20% to 30% that are not part of those groups, are divided roughly in half, in terms of preference for either candidate, and their preference is soft.

If there are no more strong events pushing people away from Trump, the numbers will settle down to where they were between days 40 and 50. this will place trump within about one point of Clinton. And, one point is very very close.

The current widespread rhetoric that Clinton is going to win no matter what may be the exact cause of her losing. How many people will not bother to vote, when they otherwise might have, because they are confident that Clinton will win? If the two candidates are 1% apart, then only 1 in 200 voters have to do that to put Trump in the White House.

Let me note what may end up being the greatest situational irony of our times. MSNBC has lots of great commentators and reporters, like Rachel Maddow and Chris Hayes. They are providing the most thoughtful and coherent analyses of what is going on during this election cycle. But, they are also constantly repeating and supporting the rhetoric that Trump can't win. And, their audience corresponds closely to that subset of people who are going to vote for Clinton.

Unless...

Unless MSNBC and other sources fail to shut up about how Clinton can't possibly lose, and one in 200 otherwise-Clinton-voters stay home.

There are, of course, other possibilities. The apparent closing of the gap we see on the above chart could be an artifact of poling and disappear by itself over the next 48 hours, or it could be real, but reverses because of something Trump does. However, keep this in mind: Trump is being such a distraction from the race that a lot of information that could be used against Clinton (legitimately or not) is currently piling up and not coming into play. It is quite possible that forces that work to push Trump down on this graph could be weak, and forces that work to push Clinton down on this graph could be strong, and we might not be looking at a dangerously weak 1% lead by Clinton when the first week of November rolls around. We may be looking at a distinct Trump lead.

I should mention that today's polls are not shown on this graph because they are mostly not available. Those that are available are in that subset that tends to favor Trump, but they are all showing a virtual dead heat.

Today, tomorrow, through Monday, we should be looking very closely at the polls. If they show narrowing, then my Hypothesis from Hell can't be ruled out and the idea that the race is really about 50-50 between scandals needs to be taken seriously.

More like this

I hope that's not your own chart. A 3-day moving average cannot end on the last data point.

By Steve Easterbrook (not verified) on 20 Oct 2016 #permalink

Is it "deplorable and astonishing" that Al Gore took weeks to accept his defeat.

By Tom Harris (not verified) on 20 Oct 2016 #permalink

Is it true that he did?

They were still counting the votes. One does not concede when the race is close and the counting is still in progress.

Your implication is deplorable.

By Brainstorms (not verified) on 20 Oct 2016 #permalink

Yeah, pretty much. Apparently Al Gore was supposed to concede before the recount in a vote that was barely 500 votes between them in an entire popular state...

How deplorable...

From an electoral college angle, Clinton seems unstoppable despite the close percentage of voter preference shown in your table. Do you agree?
Also, Gore's situation makes me think of a scary possibility: Trump wins the popular vote, but loses the electoral college...he and his supporters are not likely to go quietly in that scenario.

It's also important to remember that Gore did not request the Florida recount: it was mandated by law. At least at the time, Florida was one of many states that requires a recount when the margin is less than 0.5% of ballots cast. Those laws are in place because officials want to be sure that the winner has been correctly determined.

The 2000 Florida recount was never completed because the US Supreme Court intervened to stop it. There have been two other nationally prominent cases I know of where the 0.5% rule has come into play: the 2004 WA Governor race and the 2008 MN Senate race. My memory of the former is a bit hazy, but after a trial held in Chelan County (one of the reddest counties in the state, which is why the Republican Party specifically chose that venue), Democrat Christine Gregoire was declared the winner. In the latter case, election night returns had Norm Coleman ahead of Al Franken by a small margin. After the required recount, Franken pulled ahead and was eventually declared the winner after the MN Supreme Court dismissed Coleman's challenges to the process.

By Eric Lund (not verified) on 20 Oct 2016 #permalink

Trump not willing to accept the presidential election results = The conservative senators not willing to complete the hearing process for the empty SCOTUS seat.

By Brainstorms (not verified) on 20 Oct 2016 #permalink

"Your implication is deplorable."

Come on, it was Harris who said it. "Intentionally dishonest" is the correct description.

You're doing it wrong, you should be making the accusation in the form of a question, like Tom did:

Is it deplorable and astonishing to accuse based off a situation that never happened, but never mention it's falsity?

"Is it deplorable, astonishing, and blatantly self-serving to accuse innocent people using a made-up, fraudulent, intentionally misleading situation of "false equivalence", one that has never happened, and further your malice by intentionally not mentioning its falsity and your own duplicity and bad-faith motivations?"

By Brainstorms (not verified) on 20 Oct 2016 #permalink

Steve, it certainly can if you put the average on the last day, which is a perfectly acceptable way of doing a moving average. On this day, the current three day moving average is as shown.

I prob. should have specified that somewhere, but consider it hereby specified.

Tom, there was a legitimate recount going on. Had the recount been allowed to continue, Gore would have won the election.

James, see the two maps in this post for the Clinton with 358 (blowout) vs 275 (nailbiter) versions:

http://scienceblogs.com/gregladen/2016/10/16/the-current-trump-clinton-…

I am concerned that if things go in the way proposed here as a hypothetical, we could be looking more at the nailbiter, but I'm not sure exactly how it would pan out.

Eric: And, in Mn, (where I recounted some of the votes myself) an important principle of American electoral politics was demonstrated. Coleman was ahead by just over 200 votes. After the recount, Franke was ahead by a couple thousand votes.

The Democratic Party is highly diverse, and one of the earmarks of diversity, is a higher inclusion of people who can't fill ballots out properly, so the machine does not read them correctly. But, on visual inspection, the intention of the voter is crystal clear.

Both candidates lost votes during the recount (machine error) and gained votes during the recount (more machine error) but the gain was higher than the loss.

According to factcheck - Bush would have won under all the requested scenerios. By that I mean what the Florida Supreme court had ordered, but which was stopped by the US Supreme Court - or the Gore requested four county recount, etc.

http://www.factcheck.org/2008/01/the-florida-recount-of-2000/

Here is what factcheck.org says about the non-requested scenerio:

On the other hand, the study also found that Gore probably would have won, by a range of 42 to 171 votes out of 6 million cast, had there been a broad recount of all disputed ballots statewide. However, Gore never asked for such a recount. The Florida Supreme Court ordered only a recount of so-called "undervotes," about 62,000 ballots where voting machines didn’t detect any vote for a presidential candidate.

My fear as well. I do not believe in the polls particularly, but I most definitely feel the situation is a lot closer than the Clinton supporters have been trumpeting. It's ALL about who actually votes on November 7th!

By Mark leue (not verified) on 20 Oct 2016 #permalink

Mark: Democrats vote on November 8th Republicans on the 9th.

RickA: Gore would have won had there been a fair election in Florida.

Isn't this just fatalistic guesswork? There is also a bandwagon effect: people like to vote for the winner. It might work in her favor. Besides: people are already voting, early voting is a great thing if you are ahead!

G127: There is scientific evidence that GOTV is very important. There is very little support for a large "bandwagon effect" when it comes to something as onerous as leaving your house, standing in line for a long time, and not getting ice cream at the end.

Of the 127 million people, or whatever the number is, who will vote, about 2 million have voted.

So, sorry, no.

I think Clinton probably has this race in the bag, as I've said recently on man occassions. But I've never seen the winning side give up the fight so completely as I'm seeing it right now, and that is very dangerous.

But I’ve never seen the winning side give up the fight so completely as I’m seeing it right now, and that is very dangerous.

It was complacency of this sort that caused Brexit. The malcontents are motivated and they voted in droves. The UK was particularly let down by the young who didn't bother to take their pro-European beliefs as far as the ballot box.

We need a Constitutional Amendment that makes voting mandatory.

I'll settle for higher taxes with a corresponding refund if voting rolls show you voted. (Or, reducing your refund if you didn't vote.)

Democracies turn into oligarchies (or dictatorships) when the populace declines to vote.

By Brainstorms (not verified) on 21 Oct 2016 #permalink

"it certainly can if you put the average on the last day, which is a perfectly acceptable way of doing a moving average"

Greg: you missed my point. Even if you do that, the moving average and the last data point can only align if the last data point is also the average of the last three. The average of the last three data points would give you a final gap in the moving averages at least 3 times the size of the gap your chart shows, which undermines your argument that the numbers are converging again.

I'm only pointing this out because it's the same thing that climate denialists often do when they create misleading charts to show some trend has stopped or reversed at the end of the series.

By Steve Easterbrook (not verified) on 21 Oct 2016 #permalink

Al Franken didn't take his Senate seat until July because Norm Coleman kept demanding recounts. That's the situation to compare a possible Trump protest to.

By Christopher Winter (not verified) on 21 Oct 2016 #permalink

#19
But I’ve never seen the winning side give up the fight so completely as I’m seeing it right now, and that is very dangerous.

What I'm seeing is just the opposite – an extension of the fight into otherwise secure red states. I'm seeing the effective use of surrogates, and I expect that Clinton's performance in the last debate will soon give her another little bump.

Early voting seems to indicate a Democratic advantage, and the turnout capabilities of the two campaigns favor the Democrats. In 2012 Obama did better than the polls predicted, and I believe that that will be the case this time too. And this time the Democrats will probably have the added benefit of reduced Republican turnout.

Meanwhile the prospect for a Democratic Senate majority looks better than ever.
https://fivethirtyeight.com/features/senate-update-the-last-week-has-be…

The Princeton Election Consortium gives the Democrats a 79% chance of control.

By cosmicomics (not verified) on 21 Oct 2016 #permalink

"The average of the last three data points would give you a final gap in the moving averages at least 3 times the size of the gap your chart shows, which undermines your argument that the numbers are converging again."

Steve, you can look at the chart and see the points and the line. Nothing is being hidden here, this is not a science denialist tactic.

It really is optionally done this way.

Gary, that chart is not from the same day these data are taken from, uses a different scale, and all my numbers are the percent of Trump v. Clinton even if in a four way race, which is a more accurate prediction of who would win, say, a given state (if applied there)>

I am very proud of everyone for looking so closely at the data and charts and commenting, instead of just taking a graph face value. But it is a but funny that the reason people are looking so closely is that their expectations are not being met.

I'll have a full on analysis as soon as the first new set of polls come out, I expect on Monday and/or Tuesday, reflecting 100% post debate. The fact remains that just as the news reporters were rushing to declare a huge gap, frequently referred to as a "double digit lead," that was growing and would continue to grow until election day, the gap was that widened in the immediate wake of the bus tapes narrowed at least a bit and has not been widening. And, all these charts make the differences look huge, but if you put in margin of error ... Yes, Gary, don't relax! Too much is at stake.

I'm still pretty much sticking with this set of predictions:

http://scienceblogs.com/gregladen/2016/10/16/the-current-trump-clinton-…

But as I said, I'll have a new set out Mon or Tue.

"We need a Constitutional Amendment that makes voting mandatory."

If you do, you MUST allow a "None of the above" which puts government "on hold" and new candidates found to vote for if the "None of the above" vote wins the majority.

And you will probably need some method for coalition governments, specifically to defuse this false dichotomy of two parties.

"The UK was particularly let down by the young who didn’t bother to take their pro-European beliefs as far as the ballot box"

No, it was let down by the elderly who turned out in their droves to vote on a mandate of "I don't like today, it MUST be europe's fault, so lets leave!".

If nobody had voted for exiting, it wouldn't have mattered how many young had not bothered (and the "not bothered" was less than a third, not all of whom actually cared, since there wasn't any information to make an actual damn decision on).

Re 11: yup.

I note it wasn't answered, though, by anyone.

Odd that, eh?