Full report coming August 2018!

Evaluation Methods

Local Evaluation

Evaluation of WBT was conducted by a local evaluation team that was proposed by the selected science centers/museums (Hosts) and hired by ASTC to report on program progress and visitor reactions to WBT events. ASTC provided overall guidance about the evaluation, then each potential evaluator submitted a statement of work as part of the selection process.

WBT format
Graph represents the components of the World Biotech Tour

All Hosts evaluated the Ambassador Program (12 of 12), and most evaluated Lab-in-a-Box (LIAB) kits (10 of 12). A total of 10 Hosts collected data to describe those who attended the WBT, and nine made conclusions about whether and how the program helped Hosts work with new audiences. The new audience(s) of interest varied based on the WBT requirement.

Hosts also evaluated several components beyond the requirements, with the total number of components evaluated ranging from three to 10.

Looking across the three years of the project, the scope and rigor of the evaluations was enhanced over time. Earlier evaluations were less reliant on data from participants to help evaluate WBT components, and instead relied on the independent judgement of the evaluation team. Later evaluations, by contrast, often collected data from multiple participant groups to understand the value of WBT.

Summative Evaluation

ASTC appointed an international three-person evaluation team to conduct the summative evaluation and produce a final report. The evaluation team used a meta-evaluation approach, conducting a secondary analysis of two reporting requirements that were completed by each WBT Host: (1) Progress and final reports that were submitted by each Host, using a template provided by ASTC, and (2) Evaluation reports from each Host’s local evaluation team. The evaluation team collected additional data as needed, through interviews and conversations with ASTC, the Biogen Foundation, and Hosts.

Ambassador strengths and weaknesses word cloud
Display: Word clouds that represent the strengths and weaknesses of the Youth Ambassador Program as shared by the science museums and participants