Eoin O’Malley (17 February, 2011) Updated with seat projections
We have a tendency to see new polls and take them as indicating some real change in the fortunes of the parties since the last comparable poll was taken, when possibly much of the variation is caused by differences in samples – the luck of the draw. One way to alleviate the tendency to overstate the importance of the most recent poll is to take a poll of polls. Michael Marsh has produced one for RTÉ of all the national election polls taken so far. In doing so he weights each of the polls equally. However all polls are not equal, some polls are more useful than others. We can tell whether polls are any good when they publish their methodology, question ordering etc. Also as the campaign goes on some polls are more up to date. Is it appropriate to count a poll taken two weeks ago with the same weight as one taken two days ago? The results of my weighted poll of polls are
The weighted average estimates presented here are possibly more valid than any single poll because it should average out errors (assuming the errors in the polls are not all systematically in one direction). It means that the results of one poll won’t have a major impact on our overall assessment of the state of the parties. We will expect that some polls will have error above and beyond the margin of error. Even with the best will in the world some samples produced by polling companies are ‘rogue samples’ and produce results that are not good estimates. About one in twenty will be like this.
On these figures, using Michael Gallagher’s method of projecting polls to seats we would expect to see Fianna Fáil return 25-29 TDs (incl. Seamus Kirk); Fine Gael would win 65-69 seats, lower than the estimates currently being touted; Labour would win 35-39 seats and Sinn Féin 16-20 seats. It’s difficult to say much about the smaller parties because local considerations and the geographical concentration of the vote matter most, but we can assume Others will win between 15 and 19 seats.
Even if everything went Fine Gael’s way and it got a much bigger seat bonus than already factored in, it would still be up to ten seats shy of a workable majority. The speculation about single party government then seems irrelevant except that Labour is possibly going to benefit from the concerns some voters might have about such an eventuality, and might hinder Fine Gael as Labour supporters might shy away from giving it transfers.
In weighting the polls I take into account a number of issues.
Time decay People’s opinions change in the course of campaign. So a poll carried out three weeks ago should not carry the same weight as one carried out last week. The one carried out last week probably reflects the mood of the voters better. Therefore I take .05 off the weight given each poll for each day since the fieldwork was completed. so if the fieldwork was finished 6 days ago I take .25 from the weighting.
Weightings Because getting a truly random sample is very expensive and to some extent out of their control (non-responses tend to bias the data) opinion polling companies will weight their raw data for demographics. This is not controversial. However only RedC (as far as I am aware) weights the data for political factors. At the moment Fianna Fáil supporters may not be willing to admit that they support Fianna Fáil. Therefore they weight by respondents’ recall of their vote in 2007 and how this compares to the actual result. Additionally much of the electorate will not vote, so RedC (again uniquely in Ireland) asks the respondent how likely it is that they will vote (probability to vote -PTV). For those who give a less than .4 probability, they are excluded from the analysis. Given that we know that people are indeed less likely to vote as they go down the scale, it might make more sense for Red-C to weight their responses by PTV rather than create this binary of voters and non-voters. Anyway, Red-C, because it does this weighting gets a .2 bonus. Also I don’t use RedC’s headline figures, but what it calls its spiral of silence figures, which I consider to be better because it includes the political weighting. This will mean that the Fianna Fáil figure tends to be higher than in other polls. RedC polls get an extra .2 on their weighting.
Transparency Some polling companies publish their methodology. Although not as detailed as one would like, at least one gets a sense of the methodology they use. It would be nice if they published the actual questions used and the order in which they are asked. For instance if you ask a question about party leaders just before what party they would vote for, you prime respondents to evaluate the parties through their leader. This could bias against parties with unpopular leaders. Red C, OI, Millward Brown and IpsosMRBI (eventually) publish the main details – again RedC is best and most timely. For those that don’t publish at all or are not traceable (none of the polling companies used here), I take .2 off their weighting.
Polls by the same company Some polling companies because of their methods tend to bias in favour of certain parties. If one polling company is used more than any other, then it could skew the poll of polls. To prevent this I take .2 off a poll’s weight if it is the second poll by that company, and .4 off for the third etc.
Sample size sample size has a big impact on a poll’s accuracy. Marsh’s analysis aggregates all the samples and thus takes it into account. Because there are only small differences in sample size – anything used here has over 1000 respondents, I ignore this.
There is an element of arbitrariness in the weighting, but in general it should give a better indication of the state of the parties.
Below are the data and weightings used.