Weighing Outputs: Measuring Social Impact in Museums and Nonprofits

Photo from hollis333.wordpress.com

Since writing my last post, I’ve done a bit more thinking about our most recent prompt within the Nonprofit Millennial Bloggers Alliance: How do you measure social impact? Check out some of the responses so far. Don’t forget to follow us on twitter and add your 2 cents by using the tag #NMBA.

Evidence-based policy is not the only thing compelling museums and community centers to come up with some sort of accurate measure of social impact. Donors want to know where their money is going. How is the museum elevating the community? What is the impact of museum programs and exhibits?

…But how do you measure the unmeasurable? Perhaps a certain interactive exhibit at a science center inspired the spark that will fuel a young girl to become a paleoanthropologist in 15 years. How do we measure that– and how do we even know if that spark took place at all?

It makes sense that we have a tendency to focus primarily on outputs (clients served or number of programs performed) rather than outcomes (desired goals) in museum environments. According to Hill and Lynn in Public Management: a Three-Dimentional Approach,

“Outputs may be the only type of measure available, as outcomes may not be available until well after management decisions have been made. The question, then, is to what extent output measures actually correspond to outcome measures?”

Measuring solely outputs in museum environments (especially in regard to community engagement), provides an immediate advantage and a long-term disadvantage in attracting donors. Let’s examine, for example, the fact that the typical output measurement tends to be how many people participate in a program or community engagement event (let’s say that’s 50 people). Our desired outcome is a sparked interest in a certain subject matter (let’s say that 10 years after their visit, 5 people still remember the program and have taken classes in the subject matter, engaged friends in the subject, or passed along the lessons they learned during the program to their children).

Output reporting advantage: The museum may report to donors that 50 people participated in the program. That is 50 potential sparks. The amount reported here is not the amount of people who retained the lessons learned in the program (which we won’t know until years later), but rather the maximum amount of people who could have been sparked by the subject matter during the program.

Output reporting disadvantage: While reporting the output (50 people) may look impressive to higher-level management and potential donors at the time of an annual report, the knowledge of the true outcome of the program (that it altered the lives of 5 individuals in a positive way) is more impressive than the fact that 50 people merely participated. Moreover, the outcome could grow past the amount of original participants if those sparked share their knowledge and with others.

Though output reporting provides an immediate advantage that often proves inaccurate several years down the road (for better or worse), we often have no other choice but to measure outputs because outcomes are not available to us immediately. As more and more museums, nonprofits, and community centers are encouraged to measure social impact through outputs, the old saying still rings true: quality is greater than quantity. It’s possible that outcomes may far exceed (even impressive) outputs.

Posted on by Colleen Dilenschneider in Trends 5 Comments

About the author

Colleen Dilenschneider

MPA. Chief Market Engagement Officer at IMPACTS Research & Development. Nonprofit marketer, Generation Y museum, zoo & aquarium writer/speaker, web engagement geek, data nerd, marathoner, nomad, herbivore

5 Responses to Weighing Outputs: Measuring Social Impact in Museums and Nonprofits

  1. Alessandro Califano

    Brilliant! I liked it very much – I’m retweeting this as “crdav”… Cheers!

     
  2. Caroline

    It’s true. With arts nonprofits especially, it’s virtually impossible to measure the true impact, which is why I think the output number IS useful, because the wider net you cast, the more likely you are to create those “sparks.” But the quality definitely matters, and arts/cultural nonprofits have to be more clever in the ways they report the effect of their work. Instead of being so quantitative, we can collect drawings/essays/video feedback from students and teachers, keep in touch with former interns and get statements from them about how the org’s work affected their lives, etc etc.

    This is something that’s really important to arts/culturals now as outside forces try to belittle the work that we do; now is the time to really get creative and FIGURE OUT ways to “quantity” our unquantifiable work and tell stories about why we’re important. It’s something I’m puzzling over now and trying to figure out how to do with my own organization.

     
  3. Lauren

    Hi there Colleen! (Yes, I have been silently reading your blog). This post reminded me of a talk I went to last week at UW. Lynn Dierking is currently doing some really interesting research focused on longitudinal outcomes for girls who participated as youth in one of several out-of-school science programs. They are interviewing young women who participated in programs up to 10 years prior. Long-term outcomes are difficult to evaluate, but not necessarily impossible!

     
  4. colleendilen

    @ Alessando- Thanks for the tweet!

    @ Caroline- In class and with folks in my program, we talk a lot about how some nonprofits are moving toward “commoditization” and trying to measure outcomes like businesses (measuring things like “units of mental health,” for example). There’s this effort to measure the unmeasurable. That’s certainly a difficulty that the private sector does not have to face so directly with their clear-cut bottom-line.

    @ Lauren- Thanks for reading! Lynn Dierking’s research sounds very thought-provoking. I’m very interested to learn what she finds. Maybe she’ll aid in uncovering an easier way to evaluate outcomes?

     
  5. Pingback: Decline of the MBA, Increase in Social Good? « Colleen Dilenschneider- Know Your Own Bone

Add a Comment