How evaluators adapt their evaluations to the developmental context: Evaluation approach

This is the fourth  post in a series of blog posts based on my thesis. Read the introductory post here which describes the developmental context.

As mentioned previously, developmentally appropriate evaluation requires a culturally appropriate evaluation in the context of youth programs. This means including youth, or at minimum those with knowledge and experience working with youth, in the evaluation.In my study, I asked practicing evaluators to describe the levels of involvement there would be across a wide range of stakeholders including school administrators, teachers, parents, program staff, program designers, district personnel, funders, developmental consultants, math consultants, and the tutors and tutees of the program. In particular, I was interested in the levels of involvement of the consultants, the tutors, and the tutees across the evaluators randomly assigned to the child, adolescent, or adult conditions.Overall, evaluators were less likely to involve tutees in the child condition than the adolescent condition, and evaluators in both conditions were less likely to involve tutees than evaluators in the adult condition. Evaluators were also less likely to involve tutors in the child condition (as a reminder, the tutors in this condition are adolescents) than evaluators in the adult condition. There were no differences in use of consultants across the conditions.One could argue that some evaluators have the knowledge and expertise required to conduct a culturally appropriate youth program evaluation. Thus, I also examined the extent to which their knowledge and expertise moderated the differences. Evaluators in the Youth Focused Evaluation TIG (a TIG focused on youth participatory evaluation) were most likely to involve beneficiaries than non-youth evaluation TIG members and members in the PreK-12 TIG were least likely to involve beneficiaries. Furthermore, evaluators with more self-reported developmental expertise were less likely to involve beneficiaries.These results suggest that evaluators are less likely to involve beneficiaries of the program when they are children and adolescents than when they are adults. Evaluators were exposed to the same exact program, with the only difference being the age of beneficiaries.Stay tuned for more results from my study in terms of the evaluation design and evaluation methods, as well as a discussion explaining these results and next steps!