To have one of one's papers serve as the sole listed citation for a colleague’s publication is usually one of the more gratifying professional experiences. So it was with high hopes that I began reading Douglas W. Jones’s Mark Sense Ballot Design: Human-Factor Lessons from Florida 2000. [1] Imagine my disappointment in discovering that Jones' decision to cite arose from a fundamental error in statistical understanding; Bush’s Fifth Ace: a Crooked Panhandle [2] actually has nothing at all to do with his work. Unfortunately, Jones’s error is not unique to amateur electoral statisticians and needs to be formally addressed.

Let this not be thought to detract from Jones' contributions. As one of the principal authors in the area of voting machine technology, his knowledge of voting machine types is encyclopedic. His articles are always interesting to read. In Mark Sense Ballot Design: Human-Factor Lessons from Florida 2000, he ably reviewed how voting machine type and ballot design varied from county to county in Florida 2000. Remarkably, however, neither he nor other authors who have written on the subject [3, 4] seem to be aware of one another's work, nor of work by Paul Lukasiak[5] which probably preceded and perhaps unconsciously inspired the work of those who followed.

As it happens, Elizabeth Jordan and I looked at the issue of voting machine type and ballot design in early 2001 on behalf of Lukasiak and happen to be familiar with the issue. There’s no question that confusing ballots lead to more spoiled (or, if one prefers, "miscast"^a) votes. Anyone who suffered through the media’s coverage of the Palm Beach Butterfly Ballot (i.e., any adult who was not comatose during November, 2000) was aware of this point. Furthermore, no one who read and digested the June 2001 report of the US Civil Rights Commission, Voting Irregularities in Florida During the 2000 Presidential Election [6] could have been unaware of the issue of voting machine type. The latter is such a central work in the area that Jones' failure to cite it in this regard (he cites it only over a trivial matter of terminology) is very surprising. 

Indeed, there are many other factors that might affect vote spoilage. Leon County has shown that perhaps the key factor in preventing spoilage is intelligent political leadership, since that directly affects secondary factors such as programs of voter education and maintenance of voting machines. It is striking that the level of educational attainment is not a factor. Key work by Klinkner,[7] cited by Lichtman, [6] provides strong evidence against the hypothesis that "dumb Democrats" spoiled their ballots, while "smart Republicans" didn’t. Addendum 2 of Klinkner also disproves that voter experience was a significant factor in ballot spoilage. Indeed, if the votes of educated people were to count more than the votes of the uneducated, or those of older voters were to count more than younger voters, it might call into question the very legitimacy of American government.   

The reason that Jordan and I chose not to sign on to Lukasiak's work is simply divergence of interests. Lukasiak's work remains worthy of being read (and, one might add, cited). But while ballot machine type and ballot design can affect the average rate of spoilage in a county, if the machine type and ballot design for the top candidates is identical within a county,^b these factors cannot affect precinct by precinct variation in spoilage.  Even if the assumptions about homogeneity of machine type and ballot design within a county are violated, if voting machine distribution and ballot confusion is truly random, the results of an election should not be changed. So, machine type and ballot design are factors of primary interest to those interested in voting technology, but they are of secondary interest to those interested in violations of the Voting Rights Act or other electoral misconduct. 

The error that Jones [1] and at least one other writer in the area [3] have stumbled into is known in the technical jargon as a confusion of the variance between groups vs. the variance within groups. That is a fancy way of saying that ballot spoilage differs not only between counties (with their different voting machine types, machine maintenance schedules, ballot types, political leadership, and so on) but also between precincts within counties. There’s probably no issue of discrimination if 5% of the ballots are spoiled in every precinct of a county. It’s quite a different thing if 20% are spoiled in African American precincts and 1% in white precincts.

As a consequence of conversations with Lukasiak, in 2001 we examined the incorporation of voting machine type and ballot type into our larger analysis of unusual crossover and spoilage but found the effects to be negligible. This wasn’t entirely surprising, since there are so many more precincts than there are counties. The statistical power of the analysis of our two-factor model simply swamps the power of the analysis of technology factors.

But we also found something more probative that variance within counties is what most matters: the level of significance rose as more counties were pooled into a larger meta-analysis of crossover and spoilage. In other words, whatever the importance of ballot design and machine type is in influencing spoilage - and one must assume that these are important factors - it does not provide as good an explanation of the observations as our model. 

Precincts tend to be of roughly similar size, while the counties of Florida range from very large to very small. So, the county-by-county eyeball analysis Jones performs using the diagrams of Mark Sense Ballot Design weights small counties too heavily in estimating the mean. To sort out this issue in detail is time intensive, and I am not volunteering to do it. However, weighting by population is probably one issue to look at. 

Mark Sense Ballot Design has other deficiencies. Without producing a catalog of these,^d suffice it to say that Jones’ methods are classic post hoc analysis. To remain impartial, an analyst should first identify which counties are statistically aberrant and then let the results speak for themselves in guiding interpretation. By going first to what Jones calls the "human-factor variables" and "specific polling place procedures" (and coming up with almost as many variables as counties on which to test them), the investigator places the result too close to the test. Placing the results too close to model design tempts the investigator's unconscious prejudices to rearrange facts to fit conclusions.^e Post hoc methods are so prone to manipulation that the use of them invites suspicion. At the very least, conclusions arrived at by post hoc methods should be challenged by, for example, applying them to other political races or other states. This Jones did not do.

One other point. Jones (apparently in the mistaken belief that Elizabeth Jordan and I are one person, indivisible) stated that "one author" claimed a "massive fraud" had occurred in 2000. In our manuscript, we (two authors, not one) estimated that by one measure about 7100 votes were probably wrongfully converted.^c We said that statistical analysis was not proof of fraud, but that the evidence merited further investigation by law enforcement. Characterizing one ballot in a thousand as "massive" and characterizing a call for investigation as an allegation of fraud was inappropriate.  Indeed, contrary to visions of "massive fraud," it doesn’t take many ballots to turn tight elections. One in a hundred, or in the case of Florida 2000, one on ten thousand is often enough to change the outcome. 

In sum, Jones' paper is valuable to those who want to understand the nitty-gritty of voting, especially the details of ballot design in Florida 2000. He makes useful recommendations on ballot design. However, he has inexplicably disregarded the very powerful evidence compiled by the US Commission on Civil Rights showing that there was both discriminatory intent and discriminatory effect in Florida 2000. This might have led him to consider other hypotheses that are clearly lacking from his work. There are serious flaws in his methodology, some of which are detailed in footnote ^d . These deficiencies vitiate the conclusions. The central flaw, however, is the failure to respectfully consider the contributions of others. 

Postscript.  Prior to publication of this piece, I offered it to Professor Jones for comment, saying, "Should you find any statements that are wrong or demonstrably unfair, I will be happy to amend them before publication."  A lengthy correspondence ensued.[12] Jones's response was to deny that Mark Sense Ballot Design had been published, saying, "I consider what I have on the web at the URL you have cited to be an unfinished work, and it has sat, without any attention from me, for the past year and a half." He said, "I do not consider it credible myself . . ." and "I do not give out this URL without strong warnings about the page being junk, an abandoned draft of incomplete work."

Unfortunately, since Jones is one of the few experts on voting technology in the United States, even what he admits is "junk" could taint the debate. I pointed out to Jones that "The fact that you posted a comment about my work on a publicly-funded, publicly-accessible academic website and left it there for a year and a half makes it difficult to accept this explanation. Anyone who was looking for evaluations of my work would have found it and concluded, wrongly, that you had some basis to attack my work. I think you'll agree that it's only reasonable that I should be able to respond just as publicly to comments that may detract from my reputation."  

I suggested to Jones that he withdraw the piece. Sadly, Jones’s response was to deny that leaving a piece on the Internet for 18 months amounted to publication and to claim - falsely - that it was unreachable by search engine.  While at this writing (3/16/05), it does seem that Google does not find Mark Sense Ballot Design, the piece was originally found by Google and is still found on Yahoo!  Jones’s only substantive response was to say that he would disappear the article: "If every a search engine finds the page, I will change its URL immediately (sic)." I pointed out to him that this looked like covering up rather than owning up to an error in judgment. Apparently this persuaded him to leave Mark Sense Ballot Design online but label the document as an "abandoned draft," whatever that means. I'd rather he would un-abandon it, and do the work right. 

As I told Jones, "I want to encourage real discussion and not just score points. I agree that ballot design is important and needs to be analyzed."

But honor and honesty in academia is also important. If we had more, I suspect we'd have a real discussion of why Florida's electoral statistics are so unusual.

or

References

Footnotes

^a One of the valuable points Jones makes is the difference between "spoiled" and "miscast" ballots.  "Spoiled" is the preferred term when the entire ballot is discarded, while "miscast" suggests that a vote in a single race could not be counted, leading to a "residual vote." Jones notes that "residual vote" is another term that has been used to refer to the difference between the vote and the turnout. The value in making the distinction between spoilage and miscast votes or the residual vote is that "spoilage" serves as a kind of inventory control, serving to measure the gap between the number of ballots distributed and turnout. However, as Jones concedes, the substitution of "spoiled" for "miscast" is widespread and that, indeed, the number of ballots discarded is generally unknown. Failing to see the point in reserving a special term for a variable that, in practice, cannot be measured, we do not make the distinction. 

^b Keeping track of these assumptions is absolutely crucial in making interpretations. In Ohio 2004, apparently, voting machine type may not have been the same within a county, and ballot positions were rotated in some locations. Even when the type of voting machine and the ballot design is identical, as in Florida 2000, the quality of maintenance of punch card machines or provision of the right sort of marking implement can vary at the precinct level.

^c When do electoral shenanigans rise to the level of fraud? Certainly, it is a violation of the public trust and probably of the law to send the worst-maintained punch card machines for the use of "those people," whether "those people" are African Americans or country clubbers. That sort of semi-conscious bias may be less obviously criminal than actual alteration of ballots, but when it is done to people who have suffered historical discrimination, it is a violation of the Voting Rights Act. In the case of Florida 2000, we know the following beyond the shadow of a doubt:

To give a simple analogy, if one found the contents of a cabinet in disorder, it would make one wonder what had happened. If one found a window smashed, it would make one suspicious. If one found both simultaneously, one would logically connect the two and suspect that a crime had been committed. 

Of course, if one found a baseball by the smashed window, one might alter one's conclusions. In the case of Florida, we have a smashed window, a looted cabinet, and known lawbreakers - officials of the state of Florida - standing just outside, whistling as they count currency. We certainly don’t know that they committed a crime. Labored efforts to come up with alternative explanations for what happened in 2000, however, look like attempts to pretend that a rock is a baseball.

Simply put: how stupid does one have to be to not call a cop?   

(A footnote to the footnote: An analogy similar to the smashed window and looted file cabinet was posted on Daily Kos [11] previous to my work. I did, however, arrive at it independently).   

^dWell, OK, a brief catalog of the paper’s defects: The paper was difficult to follow because there was no consistent usage of significant figures, because fixed notation with many padded zeroes was used rather than scientific or engineering notation, and because it assumed that the reader knows which counties scan/count votes in one central location, vs. precinct-scanning, of ballots.  It also duplicated the category of "overvote with discernable intent." The second usage appeared to refer to deliberate overvotes. It also raised many hypotheses which it failed to address, adding to reader confusion.

However, overcoming these hurdles, the reader will find that the paper also:

^e See A Model for Interpreting Voting Patterns with Application to Florida Rev. 1.0 [10] for an explanation of how we insulated parameter selection from actual testing and otherwise attempted to keep on the straight and narrow. The main point is that we developed the model by working with county-level data, and then applied it to precinct-level data. This separated the model-building process from the obtaining of results.   

Copyright, © 2005 M. E. Cowan and Oliver T. Dawshed. All Rights Reserved.
Permissions for reproduction or extensive quotation may be obtained from
     or