A new poll by Millward Brown came out this weekend, with headline figures of 26% for Sinn Féin and 22% for Fine Gael. This let many news outlets to conclude that Sinn Féin is now the biggest party in the Republic. But this conclusion cannot be drawn from the Millward Brown poll, based on 991 respondents.
First, the difference between Sinn Féin and Fine Gael falls within the margin of error of the poll. While the reported margin of error is 3.1%, this is the margin for any single party. If we calculate the margin of error for the difference between two parties, we find that it is 4.3%. That is just slightly bigger than the 4% gap. This means that even if there was no difference between the two parties among all likely voters, there is more than a 5% probability that a poll of 991 people* finds a difference between SF and FG of 4% or more. Just because of the people that randomly end up being surveyed. This is generally considered inadequate to base firm conclusions on.
Second, this calculation assumes the poll is a random sample from the population. In reality, it is very difficult to get a random sample, because (among other reasons) it is difficult to draw up a list of voters and to get them to participate in the polls. Therefore, the error margin might in fact be larger than the reported 3.1%. This is sometimes called ‘Pollster Induced Error’. The Irish Polling Indicator gives an (arguably rough) estimate of the Pollster Induced Error for Irish parties, which it estimates to be between 1.3 and 2.7 for Sinn Féin. That means that the error margin is likely to be at least 1.3 times the calculated margin. So, probably even the 4.3% margin of error is an optimistic estimate.
Third, polling estimates are prone to systematic error. This is visible in the fact that some polling companies have structurally higher estimates for Sinn Féin than others. In the graph below, we can see clearly that Millward Brown (green dots) generally has a higher estimate for Sinn Féin than other pollsters do. This effect can be calculated to be just under 1%. It is helpful to take these ‘house effects’ into account when interpreting polling results.
Therefore, one should be cautious to base sweeping statements on a single poll. The Irish Polling Indicator summarises information available from all Irish pollsters. This provides a more nuanced overview of the state of the parties. The Polling Indicator suggests that Fine Gael, Sinn Féin and Independents are on similar levels of support; based on the available data we cannot draw conclusions about who is the biggest of the three.
So, is support for Sinn Féin currently bigger than for Fine Gael? Perhaps, perhaps, perhaps**
* And bear in mind that 24% of those 991 gave a ‘Don’t know’ answer, so the effective sample size is in fact 753, which would yield an error margin of 4.9% for the difference between SF and FG.
** Mind the difference with the title of this blog post: the title reflects the fact that we cannot say that SF is bigger based on the Millward Brown poll, but that does not mean that SF is certainly not bigger. It might be. We don’t know.