Don’t Confuse Metrics with Meaning: Actual Engagement Is What Matters

The Department of Energy’s Office of Environmental Management has been in existence for over 30 years. The established presence of the successful program has made for a group of very smooth and well-practiced teams of communication professionals. They are often known in the communities in which they practice, by the people with whom they engage, and they, in turn, know the communities. That can be a bit of a two-edged sword, captured in the word “change.” As examples, the environmental challenges of today are more chronic, or subtle, than urgent; population growth and transportation infrastructure have brought communities closer to sites that were once remote; demographic shifts have occurred wherein the community members who were actively engaged in cleanup decision-making in the early 1990s are no longer involved; and engagement is more difficult to implement due to changes in technologies and competition for attention. Bringing the same messages with different people using new technologies and tools can create a disconnection.

We’ve been there before

We have all been there—the public meeting about an important project with several team members participating in complementary ways. When the subject matter expert presented, the goal was to speak s-l-o-w-l-y enough that the audience would understand the concepts and support the project. Similarly, the design engineer presenter was certain that if the complex issue was explained one more time, but with a different chart or graph used to explain the same concept, all would be understood. Like the previous two presenters, the project manager was confident that the well-practiced presentation with the vetted slides was bulletproof. No need for Q&A—there couldn’t possibly be any questions, every question had been answered by the retinue of professionals. Time for the PM to check that box. These situations can create barriers to participation.

How do we break down those barriers to participation and provide meaningful engagement? To help find out, an anonymous online survey was conducted over 31 consecutive days in October and November 2023 and included 20 questions. There were 94 participants. The purpose of the survey was to gather information on a number of aspects related to participation and meaningful participation in general, including but not limited to engagement as a concept, what constituted value in having their voices heard as part of a public process, to identify preferences for meeting/gathering locations and types of gatherings, needed accommodations to be better able to participate, and how “meaningful” is determined by the participant (e.g., what needs to happen for someone to believe that their participation was meaningful?).

Table 1. Selected summary-level survey findings

A summary of the survey results is found in Table 1. Several types of questions were asked to gather the information, from demographic to rating to multiple-choice based on preferences or experiences. Despite the author’s efforts and outreach to obtain a broad spectrum of individuals and groups, the dataset is skewed. There is a disproportionate percentage of young college students and overrepresentations of women and college graduates and above.

When asked about barriers to participation, not one of the respondents mentioned the lack of a smooth meeting that flowed well. What did the respondents mention? A “Lack of active listening from the leading party. Not fully listening to ideas being shared.” “When the company/agency is engaging for show and not to obtain information that might be contrary to their plans.” “Social anxiety (fear of being around people who I do not know personally, and fear of judgment based on my suggestions).” “When it feels there is no room for comments and participation. For example, the agenda is all presentation with very little time or mechanism for public comment.”

It is the author’s position that public participation is a metric with purposeful but limited uses. The basis of the position is that while it is possible to count the number of people attending a meeting on a topic of interest to a decision-maker, it is not possible to define presence as participation (let alone meaningful participation), only attendance. Standard approaches to measuring participation are not useful for assessing meaningful participation. A number is not a meaning; it is a value that represents a behavior—choosing to show up.

Two ears, one mouth

If you plan to engage with a community, especially if you are planning to engage meaningfully and build a long-term working relationship, you must listen. It can make a significant difference in your effectiveness and whether a project may proceed as planned, or at all.

There are five types of listening, each with the behaviors and results shown in Table 2 [1]. When applied to community engagement, the first two types are disengaged and are therefore unacceptable. The third listening type should be transitory, and the last two are the destinations for being and remaining engaged. The latter two types of listening are investments of time that reap important rewards.

Table 2. Types of Listening Behavior

What do different generations think about communication and engagement?

Another anonymous survey was conducted in October 2024 to gather input from respondents on points related to communication and community engagement for proposed projects, of any type, recognizing that the underlying element with a project is change. The survey contained 16 questions and received 114 responses. Responses were grouped by generation, aiming to learn how meaningful messaging may be best focused to reach particular audiences. The survey included several open-ended questions, opportunities for ranking responses, and pure metrics. A selection of survey questions with important replies and those with clear generational differentiation are noted in Graphs 1–5 below.

Critical survey findings and recommendations on how to address them

The greatest differentiation between news sources is between baby boomers and Gen Z. Newspapers are important to the former and barely exist for the latter. Social media must be used to reach the latter, who consume it at a 61 percent frequency. (The shift in the type of news sourcing begins with millennials.) Social media has value, but great care needs to be exercised with the medium, especially how it can be manipulated.

Email is the preferred method of reaching people of all ages to let them know about projects in their communities. (Reviewing 2023 data, people indicated that they wanted to participate and that they wanted to be invited. The 2024 survey showed that email is the means to use.) Tailored consideration is needed on how to reach people to invite them.

Community involvement is fairly low among each generation—averaging about 30 percent. The consideration needed here is that a 100 percent positive result of a proposal at a meeting is not reflective of the community. It is biased by who attended the meeting. The highest percentage possible is only 30 percent of the overall population.

All age groups said that in-person meetings were the most effective means of community engagement. This is good to know and supportive of DOE practices.

Of the people who said they would like to be involved in the decision-making process (averaged approximately 32 percent among the baby boomers, Gen X, and Gen Z, but only 7 percent among the millennials), well over half said they would need to learn how to be involved and how to participate. Interestingly, while millennials were the least likely to seek to be involved in decision-making (only 7 percent), they were the most likely (72 percent) to say that they would need help to learn how to be involved. Millennial participation is important and needs to be cultivated. Consider workshops and other means of outreach, including opportunities beyond the project you’re supporting. It will be worth the investment.

“Listening first” mattered before people wanted to hear about projects in detail. People need time to learn to trust you/your team, first—invest that time.

Over 80 percent of each age group said that they had been to meetings where it felt like projects were being pushed on them. This is a significant reply, one we cannot ignore. For example, we cannot make someone understand. Your project schedule is your schedule, not theirs. You can have every approval and permit, but if the public isn’t accepting, they will show up in protest. Social license to operate, while not a requirement in the United States, is very real. A protest, in combination with or fueled by social media, can set you back or cancel your project.

Fig. 1. The pyramid model of community engagement.

In a community-focused perspective, the pyramid model (Fig. 1) [2] presents a well-founded understanding of the process by which a community can come to accept and potentially support a proposed project. This view represents how the concept of a social license to operate is obtained. There is effort involved in attaining trust and reaching the top of the pyramid. The effort is highlighted through the actions identified by the survey respondents (e.g., invite, listen, don’t rush, don’t make assumptions, etc.).

Digging in further

Additional analysis of the 2024 survey was obtained by asking a small group of individuals (two representatives from each generation, one male and one female) for further thoughts on engagement preferences for two types of projects: one a cleanup project and the other a new development (of a nonspecified type). The inquiry was focused on “what would you like engagement to look like?” for each type of project. The results of this analysis were telling, primarily because there was virtually no difference between the age or the sex of what individuals sought within a specific project type. The major differences existed between the types of project (cleanup versus new development) and the type of information sought.

Focusing on cleanup, there were requests for newsletters, postings with QR codes, and social media information on projects so that interested parties could be kept up to date. Detailed information was discouraged for these media; rather, what people wanted to see was plain language, clarity, and “big picture” status. People recommended that the project sponsors seek to learn who is interested versus assuming that everyone is interested. Recommendations were made on finding out what the atmosphere and mood was about the project; literally, how people were reacting to it. Related was the recommendation that the project find out what people are hearing and what they think versus how that comports with accurate information. Complete information was specifically requested, as exemplified from one respondent who commented, “Tell us the whole story, not just the positives.” One other person offered an outline of what they wanted a project to explain, namely what is the issue, how has it affected me and my community, how could it affect me or my community, and what will be done to fix it. Plainly, there is a desire for the facts, clarity and consistency of information, and news to enable keeping current.

Recommendations

The often-expressed phrase that “no one will show up” needs to be dropped from vocabularies and replaced with “what do we need to do to get the public involved?” Especially now. It is short-sighted to believe that decisions can be made in a vacuum without a future consequence. “Meaningful engagement” raises the stakes and the effort for all parties. There is a significant responsibility in shepherding a community through an information gathering and understanding process. Inherent fears and dislikes cannot be swept under the rug; they will linger until they are shared. To share, people need to feel safe. A communicator must embrace the need to create that safe environment built on patience, humility, understanding, and trust. Remember that meaningful engagement does not depend on your interpretation of meaning but on the community’s interpretation. In this regard, meaningful engagement is a precursor to getting your projects built. It is a type of social license:

Fig. 2. Community engagement life cycle.

Time and space (and budget) need to be provided for meaningful engagement (Fig. 2).

“Meeting people where they are” means that accommodations may be needed (e.g., livestreaming, multiple locations, offering transportation, etc.).

Community engagement needs to occur while there is room to change and influence outcomes.

It is essential that you make it clear, repeatedly, that no decision has been made—that “making it clear” comes from each presenter’s words, facial expressions, tone, and body language.

If the steps taken to perform meaningful engagement make people feel like an ingredient rather than a result of the recipe, it’s no different than checking a box.

Despite what some media outlets suggest, people are not disinterested in their communities or “too busy to care”; rather, they are quite interested. Stakeholders need to know that opportunities exist for their engagement and that their voices are welcome. Project sponsors need to get that message out. Overall, people want to be invited to participate in projects, want to learn how to participate effectively, want to help make decisions, want to see their comments and/or contributions noted, and want to see follow-through. It is our responsibility as project sponsors to enable those events to occur. It is how a team or partnership is built.

References

1. Graziano, M., “Using Listening to Increase Presence and Evolve Your Leadership,” Right of Way Magazine (Sept/Oct 2024).
2. Thompson, I. and R.G. Boutilier, “Social License to Operate,” SME Mining and Engineering Handbook, 3rd edition, ed. P. Darling, Littleton, CO: Society for Mining, Metallurgy and Exploration, pp. 1779–1796 (2011).

Lesley Cusick is a community engagement specialist, regulatory process and policy innovator, and a skilled regulatory trainer with RSI in Oak Ridge, Tenn.

This article is based on papers presented at the 2024 and 2025 Waste Management Conferences, presented by Waste Management Symposia in Phoenix, Ariz.