Lies, Damn Lies, and Statistics: The Nonprofit Social Media Data Dilemma

marketing and sales cartoon

Everyone seems to be all about the world of “big data” right now. And – as a data nerd who gets her professional kicks in that same space – I’m not (even a little bit) complaining. I’ve found in my work with IMPACTS that nonprofits are placing an incredibly strong emphasis on data collection and analysis. Ostensibly, organizations paying careful attention to their social media data may seem an encouraging trend, but in our age of information overload many organizations are misplacing emphasis on the wrong metrics – or misinterpreting the meaning of these metrics. In essence, social media metrics are becoming nonprofit (and even business) fool’s gold. 

Social media data is critical to understanding how your organization best engages with the market – and this knowledge is critical to achieving your goals. However, social media data are diagnostic metrics and NOT key performance indicators (KPIs). They inform how your organization is doing on social media…NOT the overall health of your organization. (They are related…but not the same.) Confusing the meaning and rightful application of this data can put organizations on a very arduous, frustrating path. Is a healthy organization active and engaging on social media? You bet. But high engagement numbers on social media mean absolutely nothing if your organization isn’t getting more people in the door, increasing membership renewal rates, facilitating donor-related conversations, or achieving any number of the goals that indicate the solvency and relevance of an organization.

Am I getting too jargon-y with all of this “KPI” talk? Here are some clarifications:

Key Performance Indicators (KPIs): KPIs are used to evaluate the ongoing success of an organization or a particular initiative. Success is often defined in terms of making progress toward achieving the strategic objectives that optimize the solvency of an organization. In other words, KPIs have a direct correlation to desired outputs (fundraising, visitation, etc.). For instance, for our nonprofit visitor-serving partners at IMPACTS, we measure items related to market sentiment that include metrics such as reputation (e.g. top-of-mind metrics), educational value, satisfaction, value-for-price perceptions – and other items that correlate directly to the “health” of an organization and its ability to achieve its bottom line objectives.

Diagnostic metrics: Diagnostic metrics are data points that contribute to KPI performance and aid organizations in pinpointing specific opportunities. In the online space, these metrics allow organizations to observe how effectively they are engaging audiences. However, these metrics cannot “stand-in” for KPIs because they are a sub-measurement of assessment criteria (i.e KPIs) that lead to desired behaviors. For instance, on the surface, certain social media diagnostic metrics may look positive, but if they aren’t elevating your reputation (a key driver of visitation), then…well, a “like” is just a “like.” Diagnostic metrics are also helpful for “listening” to audiences, and informing organizations of opportunities for improvement.

Here’s how they work together (flow chart style):

IMPACTS - KPIs and Diagnostic metrics

And here are three, critical points to consider concerning social media metrics:

1) Social media metrics do not directly measure your bottom line (so keep them in perspective)

A measurement indicating online reach, for instance, only measures online reach. Just because your organization reached a large number of people with a social media status doesn’t mean that anyone paid attention to it, that it was the right message, or that it strengthened any individual’s connection to your organization. Is does mean that the message had the opportunity to build a bit of affinity among a certain number of people. This is not your bottom line. More meaningful metrics include donor giving, membership acquisitions and renewals, and attendance.

2) Even when social media metrics are high, they can still be at-odds with KPIs (making it HARDER for your organization to achieve its goals)

This is a big one. If you are evaluating the efficacy of your digital strategists and social media community managers strictly by Facebook Insights numbers – knock it off (please). These metrics can be purposefully and even accidentally inflated to the detriment of organizations.  “Gaming” this system is child’s play for even the most neophyte of social media professionals.

To cut to the chase: If you’re measuring social media efficacy strictly by social media numbers and rewarding staff based on these metrics, you’re actively setting up your organization to fail. Your team may feel pressure to offer discounts or post superfluous updates that will artificially increase engagement rates (i.e. good for them in terms of their performance evaluation), but these practices will ultimately increase visitor dissatisfaction, devalue your brand, marginalize your mission, and demean your perceived reputation as “expert.”  Have you asked yourself this question: If we’re so popular online,  how come nobody is coming in person?  Chances are that you’ve created ineffective, misleading evaluation criteria based on social media metrics and not true KPIs.

3) You do not control the platforms providing key social media metrics. (They actually control YOU)

TANSTAAFL (pronounced: “TAN – staf –ful”) was a common “word” on campus at my alma mater. It stands for “There ain’t no such thing as a free lunch” (though it came from science fiction writer Rober A. Heinlein, the term was popularized by Milton Friedman, the Nobel Prize-winning University of Chicago professor – hence, the popularity on campus).  Sometimes organizations get so caught up with the ability to report numbers that they forget to think critically about social media metrics. Specifically, they forget about the concept of TANSTAAFL as it applies to social media.

Facebook and You - Product being sold

Over 15 million businesses, companies, and organizations have Facebook pages and sometimes Facebook metrics have bugs. Actually…a lot of the time Facebook metrics have bugs. At IMPACTS, we attempt to correct for bugs by gathering insight information from several organizations and normalizing it, comparatively…but if you’re a single organization, you likely don’t have this opportunity and you are, well, a wee-bit stuck with whatever information or misinformation Facebook shows you. Organizations that run more than one Facebook page likely know first-hand how common system-wide bugs are for individual pages. If you notice a bug in your Facebook Insights, the best that you can do is contact Facebook and hope – over the course of several months – that they will fix the bug. Here’s a thing to remember: Your organization is using Facebook for free or at a low cost (if you aren’t constantly buying ads, or promoting or sponsoring posts) and there isn’t a direct incentive to fix your Insights bug (that you may or may not know that you have). In short, these metrics should not be the MOST important metrics or the ONLY metrics for your organization.

There’s no doubt that social media measurement is absolutely and increasingly critical to effectively engage audiences and remain relevant with the market. These metrics are NOT unimportant. But with social media metrics being relatively accessible to non-expert evaluators, and absent the considered interpretation and analysis of their “true” meaning, organizations risk confusing isolated data points with KPIs.

Bottom line: Social media is a tool for achieving your organization’s goals. Social media metrics help organizations assess how well they are using these tools.  However, these metrics are not the end-all-be-all assessment tool in your organization’s toolbox…and organizations that misunderstand how to evaluate these metrics in terms of larger organizational goals risk confusion, frustration, and may jeopardize their long-term success. 

 

Interested in getting blog posts, tips, and some silly social media geekery periodically delivered in your Facebook newsfeed? Like my Facebook page. Or for more regular sharing of nonprofit marketing information, follow me on Twitter!

Posted on by Colleen Dilenschneider in Community Engagement, Digital Connectivity, Myth Busting, Nonprofit Marketing, Trends 2 Comments

About the author

Colleen Dilenschneider

MPA. Chief Market Engagement Officer at IMPACTS Research & Development. Nonprofit marketer, Generation Y museum, zoo & aquarium writer/speaker, web engagement geek, data nerd, marathoner, nomad, herbivore

2 Responses to Lies, Damn Lies, and Statistics: The Nonprofit Social Media Data Dilemma

  1. Jessica Green

    Great post, and love the flow chart 🙂 It’s so frustrating to always hear “we need more followers!” from nonprofits about their social media sites. You can have a million followers but if they aren’t talking/donating/volunteering with you its hard to say that your getting benefits from it.

     
    • colleendilen

      Thanks, Jessica! Well said. I absolutely feel your pain with the “we need more followers!” perceived urgency! 🙂

       

Add a Comment