Surveying Children: Assessing Reading Level

At AEA 2017, I presented a demonstration session on strategies for collecting better survey data from youth. This session ended up being a huge hit, with a mostly full room and a bunch of people contacting me afterward for the slides and more resources. This series of posts will share some of the highlights from that session. This is part one of five. The other posts will cover wording questions, response options, administering surveys, and pretesting surveys.

Assessing Reading Level

At the simplest level, we need to ensure that the reading level of our surveys is appropriate for our audience. This is important even when you are surveying adults. You may be working with an illiterate population and need to lower the reading level to ensure you are gathering valid and reliable data.Unfortunately, most strategies for assessing reading level are based on age or grade level, but age is not the same thing as developmental level. I routinely work with low income youth in Los Angeles. They often have much lower reading levels than expected for their age or their grade level. Alternatively, we can sometimes use a slightly higher reading level for gifted and talented youth who read much higher than expected for their age or their grade level.Fortunately for you, there are multiple strategies for assessing reading level of your instruments!

1. Microsoft Word (free!)

Did you know Microsoft Word could assess your reading level for you? It’s super simple! Go to File à Options à Proofing à “Check grammar with spelling” and “Show readability statistics”. Now when you click on Spelling & Grammar under the “Review” tab you’ll be able to see readability statistics, including the Flesch Reading Ease, Flesch-Kincaid Grade Level, and the percentage of passive sentences.

• Flesch Reading Ease: This measures the ease of reading and ranges from a score of 0 to 100 with a higher score indicating greater ease of reading. According to this score table, this would suggest that a 10^th to 12^th grade school level should be able to read this and that overall the blog is fairly difficult to read.
• The Flesch-Kincaid grade level can be interpreted just like a K-12 reading level. For instance, the beginning part of this blog post is written between a 9^th and 10^th grade reading level so the average 9^th grade student should be able to read this with ease.
• Both Flesch scores are based on average sentence length and the average number of syllables per word. Based on this, we can characterize my blog as both prolix and sesquipedalian (ha!).

2. Readability Test Tool (free!)

This tool lets you test URLs and direct input text and has more features than Microsoft Word. Notably, it adds a few more readability indices (i.e., Gunning Fog Score, SMOG Index, Coleman Liau Index, and the Automated Readability Index). While the Flesch indices are based on lengths of words and words per sentences, the additional indices also take into account complex words or just number of characters per words. This tool is particularly useful because it takes the average of the indices to provide you a recommended reading grade level.

3. Readable.io (paid; try it for 7 minutes free)

At the time of doing my presentation at AEA 2017, this tool was mostly free. Now, it is completely pay-for-service that is $5/month or $48/year (however, you can try out the tool for about 7 minutes before it kicks you out). It has all the same readability indices as the previous tools, but it also highlights for you in your text where you can improve your readability score. Now, this may not be too important for you considering the aspects of text these indices use to calculate their scores: you can just find your long sentences, long words, and complex words and simplify things. The tool also analyzes the tone (i.e., formal to conversational), sentiment (i.e., negative to positive), provides reading and speaking time for your text, and identifies various types of words (e.g., adjectives, prepositions, verbs).While this will be discussed more thoroughly in a future blog post, these services are not the end-all be-all. Rather, you should also consider pretesting your survey. Cognitive interviews are a particularly useful strategy for testing the readability of your surveys.