Back to top

For Written Information, Are Readability Formulas a Quick Fix?

Contents

I Introduction
II To understand readability formulas, first understand what readability is
III The critics  
IV The advocates
V Readability formula = diagnostic tool
VI More readability-enhancing tricks
VII Test early, test often
VIII The last word on readability formulas
IX References
X Resources

I  Introduction

It’s safe to say that most of us who develop written information are highly literate. And we often work surrounded by colleagues who are also highly literate. This may help explain part of the puzzle as to why many of us “highly literate types” find it difficult to get our heads around the facts: A large portion of the Canadian and American populations are not highly literate:

  • The 1994 International Adult Literacy Survey showed that close to half of the Canadian adult population age 16 and older performed below Level 3 on the prose and literacy scale; Level 3 is the “desired level” of competence for coping with the increasing skill demands of the emerging knowledge and information economy.  [1]
  • The 2003 International Adult Literacy and Skills Survey did not show any marked improvement in the overall literacy performance of Canadian adults since 1994. In the 2003 study, “nationally, 48 percent of the adult population—12 million Canadians aged 16 and over—perform below Level 3 on the prose and document literacy scales (about 9 million or 42 percent of Canadians aged 16 to 65).” [1]
  • Most recently, the first results released in October 2013 of the Programme for the International Assessment of Adult Competencies indicate that “Canada ranks at the OECD average in literacy and interestingly, shows a larger proportion of its population at both the highest and lowest levels of literacy.”  [2]

Given this perspective on literacy levels, it makes good sense that to meet the needs of the general public, experts recommend a goal of writing at the 9th-grade level, with a an even lower goal regarding comprehension of essential health and safety information, such as at the 5th-grade level.  [3] However, as most of us know firsthand, writing at a reading level and a literacy level that is not one’s own is definitely easier said than done. Regardless of whether we are considering hard copy or online information, there is often a mismatch between the needs of the general audience and what our written materials provide:

  • “Approximately 800 studies published between 1970 and 2006 indicate that most health-related materials are written at reading levels that exceed the reading skill of an average high-school graduate.” [4]
  • In one study, “the average reading comprehension of public clinic patients was 6th grade 5th month. Most tested patient education materials required a reading level of 11th to 14th grade, and standard institutional consent forms required a college-level reading comprehension. In the public clinics there was a gap of more than five years between patient reading levels and the comprehension levels required by written patient materials.” [5]
  • Another study notes that “a substantial amount of consumer health-related information is available on the Internet,” it was found that “most of the articles exceeded the 7th-grade reading level and were in the USDHHS ‘difficult’ category. Our findings support that Web-based medical information intended for consumer use is written above USDHHS recommended reading levels.” [6] (USDHHS is the U.S. Department of Health and Human Services.)   

To make the best match, why not just use a readability formula and then simply correct the troubled areas it identifies? Well-known readability formulas include Flesch-Kincaid, Flesch Reading Ease, Dale-Chall Readability Formula, Fry Graph, and Gunning’s Fog Index.

Maybe readability formulas are the answer to making the best match between the general public’s needs and our written information. But then again, maybe not…  

II To understand readability formulas, first understand what readability is

You could think of readability as simply what makes some information easier to read than others; however, what is “easy” for one type of reader is not necessarily easy for another type of reader. In The Principles of Readability, the readability expert William H. DuBay provides a sampling of how readability experts define readability:  

  • A focus on writing style as separate from, for example, content, coherence, and organization by defining readability: “the ease of understanding or comprehension due to the style of writing.” [7]
  • An emphasis on clarity by defining readability: “ease of reading words and sentences.” [7]
  • An understanding of the interaction between the text and the category or type of readers and characteristics like their reading skill, prior knowledge, and motivation, by defining readability as “the degree to which a given class of people find certain reading matter compelling and comprehensible.” [7]

Which brings us to a more comprehensive definition of readability as “the sum total (including all the interactions) of all those elements within a given piece of printed material that affect the success a group of readers have with it. The success is the extent to which they understand it, read it at an optimal speed, and find it interesting.” [7] Indeed, there is a lot more to enhancing readability than just changing words and sentences, so this broader definition is helpful when assessing the value and limitations of readability formulas.

III The critics  

Although there is general agreement about what reading grade level to aim for regarding information for general audiences, there is ongoing debate as to how to get there—especially where readability formulas are concerned.

Critics of readability formulas feel that overall the formulas are too simplistic because they typically measure only two variables: sentence length and word length. In addition, it adds confusion that even when measuring the same information scores can vary widely from formula to formula although the formulas basically function in a similar way. The concern is that, based on the readability formula results, if the writer wants to lower the grade level, the conclusion may be that all that needs to be done is to use shorter words and shorter sentences.

This raises a red flag, as the critics explain that reading comprehension is complex and based on the interaction of numerous variables. To match the text with the reading level of the target audience requires judgement about a range of characteristics of the target reader. For instance, beyond just selecting word and sentence length, the writer must also assess:

  • Characteristics of the document such as organization, tone, length, and cultural appropriateness, to name a few.

…combined with…

  • Characteristics of the target reader such as literacy level, current topic knowledge, state of mind, interest, motivation, and so on…and so on… and so on.

And then there are also numerous variables to consider related to the “look and feel” of the information, such as layout, graphics, and typography, because these factors also influence comprehension.   

IV The advocates

For the most part, readability-formula advocates recognize the formulas’ limitations; however, they don’t believe they should be discounted altogether. The advocates believe there is a place for readability formulas because they are considered effective predictors of the level of reading skill required to read the text. “Formulas are merely tools for measuring the readability level of writing, and as sources of hints for more readable writing. Certainly in neither case are they perfect, nor can their use result in a good style where it did not exist before.” [8] Advocates advise that readability formulas can provide a good starting point for assessing information. In addition, the formulas can provide “before” and “after” results to show how information is evolving.

V Readability formula = diagnostic tool

Lessons learned by combining the critics’ and the advocates’ perspectives can be summarized by thinking of the role of a readability formula as a diagnostic tool like an X-ray. Use a readability formula to help assess the “patient”—your information. Then, just as with effective patient care, it’s important to take a holistic approach to diagnosis and treatment by considering the results of the readability formula as just one piece of information, not the only consideration. As one expert puts it, “The variables used in the readability formulas show us the skeleton of a text. It is up to us to flesh out that skeleton with tone, content, organization, coherence, and design.” [8]

Rudolf Flesch, creator of the Flesch Reading Ease Formula, comments on his own formula: “Some, I am afraid, will expect a magic formula for good writing and will be disappointed with my simple yardstick. Others, with a passion for accuracy, will wallow in the little rules and computations but lose sight of the principles of plain English. What I hope for are readers who won’t take the formula too seriously and won’t expect from it more than a rough estimate.” [9]

Similarly another expert states, “For these reasons, formula scores are better thought of as rough guides than as highly accurate values. Used as rough guides, however, scores derived from readability formulas provide quick, easy help in the analysis and placement of educational material.” [7]

Even with what may be “the next generation” of readability formulas, like the Pearson Reading Maturity Metric—which includes an “artificial intelligence technology that estimates how much reading of various kinds and in what order a person must have accomplished to understand a particular text” [10] —nothing can replace professional judgement about the target audience and overall context.

VI More readability-enhancing tricks

Don’t rely solely on readability formulas; instead, have other strategies in your readability-enhancing bag of tricks, like plain language writing techniques. Based on a comprehensive understanding of characteristics s of your target audience, like their literacy level and reading ability, incorporate plain language writing techniques into your development process and then, based on the results of the readability formula, ask yourself these questions to see how you can further enhance results:

  • Is the information organized logically from the target audience’s perspective?
  • Does it provide only relevant information for the target audience (i.e., information the target audience needs to know, information is focused on the target audience not the sponsoring organization or writer)?
  • Is the information concrete from the target audience’s perspective?
  • Is the tone and language appropriate for the target audience?
  • Is it engaging and in the active voice as much as possible?
  • Is it concise with short sentences, short words, and no unnecessary words?
  • Is it direct, including redundant phrases and repetition only if they add clarity?
  • Does it use idioms, abbreviations, and acronyms selectively based on appropriateness for target audience?
  • Is structure consistent throughout the headings and body text?
  • Is all the information accurate?
  • Overall, is the writing appropriate for the medium (i.e., hard copy versus Web)?
  • Does the layout follow clear design techniques?

VII Test early, test often
 
The best judge of whether your information is effective for your target audience is not solely a readability formula. And it’s not even a readability formula combined with professional judgement plus plain language writing techniques. The best judge is your intended audience itself. Test your information with representatives of your target audience to determine, as described by the experts, “if your audience wants to read your work, if they can read it, or if they can make use of it.” [11]

In addition, experts explain that although “testing is one of the most ignored areas of plain language, you may not have time to test, however, you should make that decision knowing that the document may not communicate effectively and you may end up spending more money trying to ‘fix’ the communication later. You will get feedback. The issue is whether you want that feedback when you can still do something about it or after it’s too late.” [11]

Testing even before you write the first draft and continuing until the final draft is the ideal approach as part of the plain language process. Look for simple and inexpensive approaches to testing; it is possible.  

VIII The last word on readability formulas

Accordingly, use readability statistics about your information only as a helpful starting point, as a guideline regarding the reading level and effort required to read the document in its current language. “We can say this—bad readability scores for the particular group of readers at which writing is aimed do mean reading difficulty; good readability scores do not by themselves mean good writing.” [12] For a “before and after” example that is a based on readability formula results, plain language writing techniques, and target reader feedback, please see the Sample Plain Language Makeover at http://clearlanguageatwork.com/complimentary-resources/clw-materials/

IX References

1.  Building on our competencies: Canadian Results of the International Adult Literacy and Skills Survey, Statistics Canada, Catalogue no.89-617-XIE

2. Skills in Canada: First Results from the Programme for the International Assessment of Adult Competencies (PIAAC), Retrieved April 2014: http://www.cmec.ca/Publications/Lists/Publications/Attachments/315/Canad...

3. Know your readers, Plain Language at Work newsletter, Retrieved April 2014: http://www.impact-information.com/impactinfo/literacy.htm

4.Health Literacy in Canada, Initial Results from the International Adult Literacy and Skills Survey, Retrieved April 2014: http://www.ccl-cca.ca/pdfs/HealthLiteracy/HealthLiteracyinCanada.pdf

5. The gap between patient reading comprehension and the readability of patient education materials, PubMed.gov, Retrieved April 2014: http://www.ncbi.nlm.nih.gov/pubmed/2230677

6.  Readability assessment of internet-based consumer health information, PubMed.gov, Retrieved April 2014: http://www.ncbi.nlm.nih.gov/pubmed/18811992

7. The Principles of Readability, William H. DuBay, Retrieved April 2014: http://www.impact-information.com/impactinfo/readability02.pdf

8. Know your readers, Formula Pitfalls, William H. DuBay, Retrieved April 2014: http://www.impact-information.com/impactinfo/newsletter/FormulaPitfalls.pdf

9. Oxford Guide to Plain English, 2013, Martin Cutts, page 124

10. Pearson Reading Maturity Metric 30 Percent More Accurate, Retrieved April 2014: http://edtechtimes.com/2013/07/02/pearson-reading-maturity-metric-30-per...

11. Plain Train, Testing, Retrieved April 2014; http://www.plainlanguagenetwork.org/plaintrain/Testing.html

12. Limitations of Readability Formulas, George R. Klare, Byron Buck, Retried April 2013:  http://www.impact-information.com/impactinfo/Limitations.pdf

X Resources

Clear Language @ Work website offers resources including the Sample Plain Language Makeover at http://clearlanguageatwork.com/complimentary-resources/clw-materials/

The Plain Language at Work Newsletter includes readability formulas and other information. It is available at http://www.impact-information.com/impactinfo/literacy.htm.

The Plain Language Network website includes information on document feedback and testing at http://www.plainlanguagenetwork.org/plaintrain/Testing.html.