Measuring Success in Online Citizen Engagement

How do you measure success in citizen engagement? Too often, the response to this challenging question is reduced to an arbitrary number of participants. Metrics such as the number of surveys taken or page views achieved are tangible and easy to understand. While practitioners increasingly understand that numbers don’t necessarily reflect a genuine, in-depth process, many still rely on them to validate the quality of their engagement.

When participation numbers are solely used to determine the efficacy or effectiveness of the engagement process, other crucial elements may be ignored. Here are some other aspects to consider when measuring success in your citizen engagement processes.

1. Quality of responses

Is the feedback collected useful? Are the responses on-topic and meaningful? Sometimes, less is more. Obtaining a small number of thoughtful, high-quality responses can be more useful than hundreds of knee-jerk or even rude/vulgar responses. In addition, not all types of citizen engagement activities will result in the same range of responses. Some methods such as citizen juries and deliberative polls have, by definition, fewer participants while resulting in more in-depth and well-rounded responses.

Solution: Keep your participants on topic by asking clear and directed questions. Ensure that your question is straightforward, easily understood, and unambiguous. If the scope is too broad, people are more likely to stray off-topic or air any and all of their grievances, regardless of relevance.

2. Are participants informed?

While it is unrealistic to expect participants to be knowledgeable on a wide range of subjects, it is difficult to have reasonable and meaningful dialogue if people don’t know much about the issue at hand. Uninformed participants are more likely to resort to knee-jerk reactions, rather than considering all the different factors that affect the final decision.

Solution: Often, background information can be jargon-filled and inaccessible to non-technical readers. Sometimes, it’s simply boring! Recognize that participants may not have the time, know-how, or language level to peruse extensive amounts of background information. Instead, include resources about the consultation in a variety of formats (e.g. text, infographics, images, videos, etc.)

3. Defensibility of feedback data

Does the feedback data used to justify policy decisions stand up to scrutiny? With thousands of respondents, the numbers may look good. However, if most participants are irrelevant to the consultation (and how can you tell?), the feedback collected is not particularly useful for decision-making. Many controversial projects attract widespread attention, and some of the loudest voices may come from activists, lobbyists, or other special interests who will not be impacted by the final decision. Today, the digital equivalent of “busing in” supporters is the growing prevalence of online ballot-stuffing, where bots can spoof IP addresses and submit multiple responses. For example, in the high-stakes fight on net neutrality, tens of thousands of the responses obtained were from bots pushing anti-net neutrality views. With public trust at an all-time low, it is crucial to ensure that your feedback data is accurate and defensible.

Solution: Incorporate some form of digital identity verification to ensure that participants are real and relevant to the consultation. Here at PlaceSpeak, our unique geo-verification technology connects participants to their physical address. For example, decision-makers can limit participation to residents within specific geographical or jurisdictional boundaries, such as ensuring that only residents (i.e. taxpayers) of a city are eligible to participate on an online consultation about the city budget.

4. Diversity of participants

Is the feedback coming from a diverse range of participants? Traditional forms of citizen engagement such as town hall meetings and public meetings tend to draw “the usual suspects”. While their opinion is still valued, they are not necessarily representative of the residents within the affected community. Consider the impact on underrepresented groups or groups who are less likely to participate, such as youth, families with children, and new immigrants, and make an effort to go out of your way to include their perspectives in the decision-making process.

Solution: Include both online and offline methods to reach out to a diverse audience. Utilize multiple types of engagement options, ranging from low-effort (e.g. simple poll, survey) to more in-depth methods (e.g. charrettes, ideation). For example, busy people who don’t have time to participate in a charrette might still be willing to take a quick poll.

5. Project outcomes

Did the final outcome reflect the feedback that was collected? Was the process an attempt to “decide, announce, defend”, or was it a genuine consultation in good faith? The purpose of citizen engagement is to gather feedback from residents to help guide the decision-making process, so this should be an integral part of determining the success of your consultation.

Solution: Demonstrate to people how their feedback made an impact on the final decision. This is crucial to building trust in the process. When people see that their feedback has had a tangible effect on the outcome, they are more likely to stay engaged and participate again in the future.

If you found this post interesting and useful, we’d appreciate if you would share and subscribe to our blog.

To get started with your online public consultation, visit placespeak.com.