3 Citizen Engagement Misconceptions: Telephone Surveys

From seniors on the internet to online replacing offline modes of engagement, we showcased three myths about accessibility and online citizen engagement in a previous post.

Today, we’re returning with three misconceptions about telephone surveys. This mode of public opinion collection continues to be deeply ingrained in government agencies and organizations, despite growing concern about the accuracy of the data collected. While phone surveys have their benefits, such as the ability to ask in-depth follow-up questions or provide a personal touch, there are some issues which must not be overlooked.

1. Phone survey respondents are representative

The most obvious problem is that increasingly, people are eschewing having a landline and going mobile-only. At the end of 2015, only 40% of households in the United States had a landline, and the number has surely decreased since then. Caller ID and “Do not call” lists have further limited access to people with landlines.

Mobile-only households in the U.S. disproportionately lean younger, low-income, Hispanic, and were more likely to be renting or have roommates. This means that any phone survey which relies on landline numbers are going to be automatically sampling from a group which doesn’t represent the demographic makeup of the general population at large.

2. Phone survey respondents are random

One commonly cited reason for continuing to do phone surveys is the fact that respondents are randomly selected. People believe that there is more validity in being able to collect information across various demographics, and weigh them to obtain a “representative” sample. The idea is that the feedback provided can be generalized to the population at large without having to survey every single person.

However, people who still have a landline, answer it and are prepared/willing to take part in a survey are a highly self-selecting group. Currently, phone survey response rates in Canada hover in the low double digits, and are sometimes even lower than 10%. While there are significant efforts to include cellphone numbers, response rates are still low and other issues such as over/under-sampling in geographical regions start to crop up. In short, the sampling may be random, but the actual participants who follow through aren’t. They’re actually a very unusual group of people. Therefore, phone survey respondents are not representative, and their feedback cannot be generalized to the general population.

3. Having a random sample is objectively better

In some scientific studies, having a random sample is necessary in order to obtain results which can be generalized to the population at large. However, when it comes to local decision-making or public policy, a random sample of the general population may not be the most efficient or even particularly useful. Consider the communities of interest: those who are most directly impacted, most geographically relevant, etc.

For example, a school district may design a survey about changing the boundaries of catchment areas. It is not particularly useful to obtain a random sample of the population in the school district and find out what they think. Instead, it would be most useful and relevant to survey specifically parents and families living within the catchment boundaries, who will actually be impacted and affected by any potential changes.

So, what are the solutions?

While there is no foolproof solution, recognizing that every method has its limitations is a good start. From town hall meetings to online engagement to phone surveys, each method attracts a self-selecting group of people. Acknowledging this allows for a realistic approach to citizen engagement. By utilizing multiple methods of participation wherever possible, you are more likely to hear from different demographics who are drawn to different methods of participating.

In addition, consider the data you are hoping to collect. Who are the communities of interest that you want to hear from? Who are the people who will be affected by any potential decision? Are you hoping to collect qualitative or quantitative data? Defining your target audience is crucial to ensuring that your organization is hearing from the right people and collecting the data required to make a defensible decision. 

If you found this post interesting and useful, we’d appreciate if you would share and subscribe to our blog.

To get started with your online public consultation, visit placespeak.com.