Developing Intelligence

UPDATE: Diebold effect explained?

Marc has an excellent summary of a flurry of Diebold-related discussions between me, “T“, Marc, and Sean.

Sean also has a network model of the apparent Diebold effect.

I think we’ll soon hear from Brian Mingus (who’s running a meta-classifier) and Steve Freeman (an expert on machine-effects in elections) as well.

At bottom is a disagreement over how to infer causality in observational data, and how to diagnose the functional form of a data set.

The good news is two-fold: there may not be a large “Diebold effect” when nonlinear methods are used, and reason suggests that the apparent Diebold effect will be explained through demographics.

The “bad news” is also two-fold: not everyone agrees those nonlinear methods are appropriate, and there’s an alarmingly persistent, consistent, and large Diebold effect when simple – but traditional – inferential statistics are used.

It’s still not clear exactly which demographic feature results in such discrepant results between nonlinear and linear models. (Edit 1/21: An important but previously unconsidered variable is how each precinct voted in the 2004 democratic primaries).

Comments

  1. #1 Brian
    January 20, 2008

    I ran a few dozen experiments with a variety of classifiers and meta classifiers and found that J48, which generates a C4.5 decision tree, and Ada Boost (pdf) achieve the best results using my feature set. RandomForests perform poorly over a variety of parameters. I didn’t experiment much with feature selection (such as by PCA – takes too long) and I’m not sure which features others are using. Send me your datasets and I’ll run them.

    Features used
    Town,Sqmiles,Votes,Municipalwater,Municipalsewer,
    Totalhousingunits,Singlefamilyhomes,Multifamilyunits,
    Manufacturedhomes,Totalpopulation,Medianage,Percenthighschoolgraduates,
    Percentholdingbachelorsdegree,Totallaborforce,Totalemployed,
    Totalunemployed,Percapitaincome,Medianhouseholdincome,Age5andunder,
    Age5to19,Age20to34,Age35to54,Age55to64,Age65andup,Employeesinlargestbusiness,
    MarkObama,MarkClinton,IncomeDev,EducationDev,Contested,PopDensity

    AdaBoostM1
    Relation: diebold

    Correctly Classified Instances 204 88.3117 %
    Incorrectly Classified Instances 27 11.6883 %
    Kappa statistic 0.7647
    Mean absolute error 0.1565
    Root mean squared error 0.3046
    Relative absolute error 31.5858 %
    Root relative squared error 61.2877 %
    Total Number of Instances 231

    === Detailed Accuracy By Class ===

    TP Rate FP Rate Precision Recall F-Measure Class
    0.868 0.098 0.918 0.868 0.892 Hand
    0.902 0.132 0.844 0.902 0.872 Diebold

    === Confusion Matrix ===

    a b < -- classified as
    112 17 | a = Hand
    10 92 | b = Diebold

    J48
    Relation: diebold

    Correctly Classified Instances 206 89.1775 %
    Incorrectly Classified Instances 25 10.8225 %
    Kappa statistic 0.7812
    Mean absolute error 0.1804
    Root mean squared error 0.3088
    Relative absolute error 36.4002 %
    Root relative squared error 62.1423 %
    Total Number of Instances 231

    === Detailed Accuracy By Class ===

    TP Rate FP Rate Precision Recall F-Measure Class
    0.891 0.108 0.913 0.891 0.902 Hand
    0.892 0.109 0.867 0.892 0.879 Diebold

    === Confusion Matrix ===

    a b <– classified as
    115 14 | a = Hand
    11 91 | b = Diebold

  2. #2 HRS
    January 21, 2008

    I hate to add to all this noise, and I have neither fancy statistics nor the time and energy to produce any, but can’t resist making two quick points:

    1. this may be a good example of why Killeen (e.g. Killeen et al. An alternative to null-hypothesis significance tests. Psychological science : a journal of the American Psychological Society / APS (2005) vol. 16 (5) pp. 345-53) has suggested that null-hypothesis significance tests can be very misleading– if you don’t know the prior probability distributions under the null hypothesis (which you can’t usually, and definitely not in this case), Fisher himself said �Such a test of significance does not authorize us to make any statement about the hypothesis in question in terms of mathematical probability� (Fisher, 1959, p. 35). This is a problem we have all tried really hard to ignore because it calls into question much of what we do with statistics, but it seems like a particularly big problem for complex and uncontrolled (in the sense that people were not randomly assigned to conditions, not in the sense that you have not tried to control for confounding variables) correlational studies like this, where we really can’t know the priors. Killeens’s proposed solution is to use the probability of replication (p-rep) instead, but since this is currently just a mathematical transformation of p, I don’t think it really solves the priors problem here (although it seems to has other advantages).
    2. This (http://www.nytimes.com/2008/01/06/magazine/06Vote-t.html?ref=magazine) is a really interesting article in the NY Times about voting machines– basically, everyone worries about them, but the public and the experts worry for very different reasons. The public tends to believe in deliberate fraud, while the experts seem to all agree that the problem is random error and lost votes due to crashes, which could cause very tight races to be cast into doubt. Optical scan of paper ballots is actually the preferred solution, since it enables hand re-counts.

  3. #3 Jaymax
    January 21, 2008

    I was about ready for the stats to show no problem. But then Nashua 5 ward result came out.

    Somehow a systematic (human) error in that ward is supposed to explain the scaling up every candidate BUT Obama by 9%.

    Hillary, Kucinich, Edwards, Richardson all lost around 9% in the recount, Obama went up but by well under 1%.

    Question, to what extent were the models looking for a Hillary effect – I wonder about comparing the Obama vote vs the Others vote, or something, to find maybe a stronger signal if it’s there?

    I have faith in the stats to pick up a problem – and they seemed to, but now they don’t, but this just doesn’t seem reasonable against the explanation being provided:
    1030 x 0.93 [HILLARY]
    405 x 0.93 [EDWARDS]
    9 x 0.88 [BIDEN]
    72 x 0.96 [RICHARDSON]
    673 x 1.01 [OBAMA]

  4. #4 MINDYSloan
    January 16, 2011

    I had got a desire to begin my own company, nevertheless I did not earn enough of cash to do that. Thank God my close mate advised to take the home loans. Hence I used the car loan and made real my dream.

  5. #5 Melva35LEBLANC
    May 15, 2011

    The credit loans suppose to be essential for people, which are willing to organize their own career. By the way, this is very easy to get a car loan.

  6. #6 Odom32Elizabeth
    July 23, 2011

    Cars and houses are not cheap and not everyone is able to buy it. Nevertheless, business loans was created to aid different people in such kind of hard situations.

The site is undergoing maintenance presently. Commenting has been disabled. Please check back later!