Chicago Children’s Museum (CCM) collaborated with RK&A to study the impact of Skyline— a National Science Foundation-funded exhibition designed to facilitate family learning of science, technology, engineering, and math (STEM) concepts relevant to building stable structures. Skyline underwent all three phases of evaluation—front-end, formative, and summative. The summative evaluation is discussed here; it was conducted to examine visitors’ overall experience in the exhibition and to determine whether Skyline had achieved its intended impacts on visiting families.
How did we approach this study?
RK&A believes that planning for visitor outcomes, such as increased knowledge or skills (as opposed to outputs like increased visitation) creates successful exhibitions that achieve impact. CCM developed audience impact statements within NSF’s Framework for Evaluating Impacts of Informal Science Education Projects. This framework requires museums to define impacts within six categories plus “other”—awareness, knowledge or understanding, engagement or interest, attitude, behavior, and skills—as well as indicators as evidence of these impacts. Thus, RK&A designed a summative evaluation to explore how successful Skyline was at achieving its impacts. We conducted 100 observations of children, 100 in-depth interviews (50 each with children and their caregivers), and analyzed 43 recordings of families who completed the exhibition’s photo-narrative experience (a story-telling station that allowed structured visitor reflection).
What did we learn?
Notably, Skyline achieved nearly all three of its intended impacts. One intended impact is that families increase their understanding of STEM concepts associated with how buildings stand up (impact categories: awareness and knowledge or understanding). CCM successfully achieved all five indicators as evidence of that one impact. For example, two indicators appear below:
What are the implications of the findings?
Museums can use evaluation to demonstrate impact; the achievement of specific visitor outcomes (or indicators) can, together, create an overall desired impact on one’s target audience. CCM was able to use the results of the summative evaluation to demonstrate to NSF that it successfully achieved most of its intended impacts.
The full report can be read at www.informalscience.org.