Recent years have seen a flurry of voting advice applications (such as Vote Match) which estimate which political party closest matches your political preferences on the basis of your answers to a number of questions (typically 20-30). Proponents of voting advice applications (VAA’s, for short) claim that they increase both political participation and voters’ tendency to vote on the basis of their preferences on policy issues (as opposed to emotional factors such as party leaders’ charisma). However, regardless of whether that is true, one major shortcoming of existing VAA’s is that their estimate of what party best reflects our political preferences seem to be quite poor. For instance, the advice given to me by newspaper VAA’s on the upcoming Swedish election have differed substantially, which shows that at least some of them have to be way off the mark. Also, more often than not, the questions seem to be picked in a quite haphazard and arbitrary manner.
In fact, I think that I am better at estimating which party is closest to my preferences using my gut-instinct or intuition than these VAA’s are. It is true, though, that such meta-judgements of the accuracy of our intuitive judgements are not very reliable. Research has shown that people systematically overestimate the reliability of their intuitive judgements and underestimate the reliability of algorithms (VAA’s are a sort of algorithms, obviously). At the same time existing VAA’s have, as we will see in the next section, several disadvantages in comparison with our intuitive judgements, which lends support to my belief that the latter actually are superior in this case.
VAA’s take a number of preferences as input values and a single preference or judgement as its output value. Many other algorithms which take multiple preferences or beliefs as input values and a single judgement as output value have proved to be superior to human (including expert) intuition. A salient example are the extraordinary simple “statistical prediction rules” (SPR’s) (same link as above) which make predictions on the basis of a set of factors. An entertaining example from the link is the prediction rule that reliably predicts future marital happiness using the difference between the (present) rate of lovemaking and rate of fighting. Even though such SPR’s disregard many factors relevant to the prediction at hand, a great number of them outperform experts’ intuition-based predictions.
In area after area, algorithms which beat intuition have been developed. There is no reason to believe that you couldn’t do that regarding party choice as well. Of course, it can be hard to prove that a VAA is better than intuition at estimating which party best matches our preferences, since, unlike in the case of future marital happiness, there is no independent way of finding out who was right. This is, however, obviously a separate question from whether they actually do so.
Let us now turn to some of the major flaws of existing VAA’s, which put them at a disadvantage in comparison to intuition.
1) The most conspicuous flaw of virtually all existing VAA’s is that they include too few questions. As a result, the questions either tend to be very general (such as “should taxes be raised or lowered?”), or to fail to address many important issues. An obvious drawback of the former strategy is that some voters might want to see some tax reductions (say on income taxes) and some tax increases (say on petrol taxes) and that general questions are too coarse-grained to capture such preference structures. On the other hand, by making the questions more specific (say by asking for people’s view on income taxes rather than taxes in general) you have to leave out large policy issues altogether (i.e. in this case, non-income taxes) if you don’t increase the number of questions.
2) Another problem is that in many VAA’s you’re not permitted to rank some questions as more important than others. Other VAA’s do allow for this, but the weighting functions are typically rather primitive. In particular, it is usually not possible to weigh particular issues very heavily, something some voters (e.g., some animal rights’ activists) would want to do.
3) Most VAA’s don’t pay attention to the relative importance that the parties attach to different issues. It is important to do so, especially since parties focusing on a small number of questions (environmental policy, animal rights, etc) will fight much harder over these issues than over other issues in negotiations.
4) Many VAA’s only consider policy issues and don’t ask for your opinions on non-policy issues such as which party leader you prefer or which party best expresses your identity. Now perhaps such questions shouldn’t influence voting decisions (though, for instance, many would argue that the fact that Barack Obama is African-American was a legititmate pro tanto reason to vote for him 2008, considering the American legacy of discrimination of African-Americans), but voters should at least be given the choice of including them, if VAA’s are to take all preferences voters see as relevant into account.
A better VAA should overcome these flaws. It should include many more questions (possibly there could be thousands of them); in fact all questions which might influence a significant number of voters to a significant degree. Both parties and voters should be allowed to weigh the relative significance of different issues, for instance using a percentage scale, where your overall party match estimate would to x % be determined by your match on property taxes, by y % on your match on gun control, and so on. Finally it should include questions on non-policy issues.
At this point, an obvious objection immediately suggests itself, namely that such a VAA would be far too large. People would not want to answer so many questions. Presumably, it is partly for this reason that more complex and accurate VAA’s haven’t been constructed as yet. I have three suggestions for how to deal with this problem:
1) The VAA should have different layers of questions, ranging from the most general to the most specific. At the former level, you are asked very general questions like “should taxes be raised or lowered?” or “should punishments for crimes be tougher or milder than they are today?”. These would generate “default answers” on lower levels. For instance, if you answered that taxes should be somewhat lowered, this would generate a bunch of default answers at the next, more specific level: that income taxes should be somewhat lowered, that petrol taxes should be somewhat lowered, and so on. You could, however, change these default answers; “customize” them, as it were.
To get an estimate of which party best matches your preferences, you would have to fill in all answers at the most general level, but you could leave the default answers these generate on the lower levels as they are, should you so wish. However, the more questions you answer manually, the more accurate the match estimate would become. You would be given a new match estimate after each new manual answer (beyond the most general level).
2) The VAA site should give you a permanent profile, like members on date match sites such as OKCupid have. Since this allows you to return to the questionnaire, this should increase the likelihood that people will answers lots of questions. People answer huge numbers of questions on OKCupid, but they don’t do so in one go, but rather on many different occassions.
3) As a means to persuade test-takers to answer more questions, they would be given a robustness score; a score measuring the likelihood that your party preference would change if you answered more questions manually. People would also be able to set the system so that it would give them the questions that increased their robustness score most first. For instance, if your number one and number two parties have diametrically opposed views on some question, you’d be suggested to answer that question first, since your answer to that question would make a big impact on which of them you should choose.
The robustness score would progressively improve, obviously, as you answer more questions. Together with the constantly changing party match estimates, it adds an element of gamification to the test. I think that this could do much to make the test more popular, and to convince existing users to answer more questions.
Some further, more practical, ideas on how to make the test more popular will be discussed in the next post. I’d be very interested in reader suggestions on this (as well as on other) point also.
I thus think that there are some tentative reasons to believe that a VAA constructed along these lines could be both accurate and popular. That said, I know what some of my suggestions need to be fleshed out more. For instance, I need to specify the weighting function and how the hierarchy of questions would work. I have some ideas on these technical details, but discussion of them must wait for later. Let me, though, make a call for pragmatism, and note that we should not let the best be the enemy of the good. What’s important is not whether we can find a perfect solution to these problems, but whether we can find a solution that is good enough to allow the VAA to make reasonably accurate estimates of voter-party matches.
I plan to write several further posts on this topic. One post will discuss implementation – various practical issues regarding how to get a VAA like this running. Another will discuss possible effects that a site like this could have on voter behaviour and on the political system at large. This is it for now, though.