Image by #WOCinTech Chat - wocintechchat.com

Many of my colleagues at Claremont Graduate University (CGU)are beginning to journey into evaluation consulting with other organizationsoutside of CGU. It’s a new and exciting journey for all of us to see howevaluation is done in other organizations.

We have been spending many of our get-togethers comparingstories about what it’s like to work for other organizations and otherprincipal investigators. What theories or approaches do they typically use?What kind of methods are they prone to using? How do they work with their clients?

One area that we are finding is very different across organizations is their reporting style. The field of evaluation has been learning a lot about effective evaluation reporting strategies: shorter reports, findings up front, effective data visualization, and more. But not all evaluators are aware of these strategies, and some just do not care! Here’s a flavoring of some of the experiences we have had:

• I was hired by one organization specifically for my abilities to improve their reporting and data visualization practices. However, when it came time to turn in my first report, I was told to go back and do it in their organization’s style. Mind you, not the client’s style but the organization’s style. It’s taken some time (over a year), but I’ve slowly been able to work the organization towards more effective reporting that is tailored to clients.
• One of my colleagues was just hired by another organization who is way too understaffed to do all of the evaluation and research projects they have, which is why she was hired. She has been working closely with the main stakeholders of the client organization she was specifically hired to work with. They told her they wanted a one-page report. When she came back to her organization with the report, they made her go back and write up a 50-page report that we all know was not read by either the organization or the main clients.

As I slowly read through the chapters in EvaluationFailures: 22 Tales of Mistakes Made and Lessons Learned, these issues donot seem to be ours alone. Many evaluation failures also dealt with not discussing what clients want up front.We would have avoided these failures if we had done some of the following:

• Discusswhat clients’ needs are from the beginning. Know what they want and lookfor in a report. If your client wants a one-page report and you are not thelead, discuss it at the beginningwith the lead to let them know how you plan to address the clients’ needs.
• Show examplesof your reports. You could also create a template and get confirmation onthe style and length early on. Let them know how you like to report and why it is more effective than the 50-pagereport.
• Communicateresults frequently and in the style of your end report (if applicable). Donot wait until the end to let clients know what the results are, especially if they are negative results.I like to create feedback reports in the style of my end report to get themused to what the end report will look like and get feedback early on about howthe reporting style could be improved.
• In theend, do what the client or lead evaluator wants. Unfortunately, if eitheryour client or the lead evaluator pushes back, you are just going to have to dowhat they want. The alternative—quitting—is not always feasible, particularlyfor newer evaluators. So if your lead evaluator states you need to write the 50-pagereport then you do it; maybe there is a good reason they need that long report.But perhaps you can sneak in the one-page summary at the beginning for yourclient.