Since writing my last post, I’ve done a bit more thinking about our most recent prompt within the Nonprofit Millennial Bloggers Alliance: How do you measure social impact? Check out some of the responses so far. Don’t forget to follow us on twitter and add your 2 cents by using the tag #NMBA.
Evidence-based policy is not the only thing compelling museums and community centers to come up with some sort of accurate measure of social impact. Donors want to know where their money is going. How is the museum elevating the community? What is the impact of museum programs and exhibits?
…But how do you measure the unmeasurable? Perhaps a certain interactive exhibit at a science center inspired the spark that will fuel a young girl to become a paleoanthropologist in 15 years. How do we measure that– and how do we even know if that spark took place at all?
It makes sense that we have a tendency to focus primarily on outputs (clients served or number of programs performed) rather than outcomes (desired goals) in museum environments. According to Hill and Lynn in Public Management: a Three-Dimentional Approach,
“Outputs may be the only type of measure available, as outcomes may not be available until well after management decisions have been made. The question, then, is to what extent output measures actually correspond to outcome measures?”
Measuring solely outputs in museum environments (especially in regard to community engagement), provides an immediate advantage and a long-term disadvantage in attracting donors. Let’s examine, for example, the fact that the typical output measurement tends to be how many people participate in a program or community engagement event (let’s say that’s 50 people). Our desired outcome is a sparked interest in a certain subject matter (let’s say that 10 years after their visit, 5 people still remember the program and have taken classes in the subject matter, engaged friends in the subject, or passed along the lessons they learned during the program to their children).
Output reporting advantage: The museum may report to donors that 50 people participated in the program. That is 50 potential sparks. The amount reported here is not the amount of people who retained the lessons learned in the program (which we won’t know until years later), but rather the maximum amount of people who could have been sparked by the subject matter during the program.
Output reporting disadvantage: While reporting the output (50 people) may look impressive to higher-level management and potential donors at the time of an annual report, the knowledge of the true outcome of the program (that it altered the lives of 5 individuals in a positive way) is more impressive than the fact that 50 people merely participated. Moreover, the outcome could grow past the amount of original participants if those sparked share their knowledge and with others.
Though output reporting provides an immediate advantage that often proves inaccurate several years down the road (for better or worse), we often have no other choice but to measure outputs because outcomes are not available to us immediately. As more and more museums, nonprofits, and community centers are encouraged to measure social impact through outputs, the old saying still rings true: quality is greater than quantity. It’s possible that outcomes may far exceed (even impressive) outputs.