What to do after administering the ORF: A conversation with Dr. Jan Hasbrouck

Anna Geiger: Hello and welcome to Episode 98 of the podcast! Today is the second half of my interview with Dr. Jan Hasbrouck. Last week she helped us understand the concept of fluency and why it's so complicated. Today, we're going to talk about oral reading fluency.

You may be familiar with the ORF norms chart. The chart shows the oral reading fluency norms of students, as determined by data collected by Jan Hasbrouck and Gerald Tindal.

When teachers give their students oral reading fluency assessments, where they check to see how many words they can read correct per minute, they can refer to this chart to see whether their students' oral reading fluency is on track, or whether there's a red flag that we need to figure out what else is wrong that's causing them to read so slowly.

Again, I apologize. My audio for this episode is not the best, but Dr. Hasbrouck comes in nice and clear. We'll get started right after the intro.

Dr. Jan Hasbrouck: There are three publications. There are three sets of Hasbrouck and Tindal norms. The first one, the publication was in 1992, but that's a whole story of the challenges of getting research published sometimes.

I first started working with Gerry Tindal at the University of Oregon, which was the mid 80s. I became a reading coach in 1985, and by 1986 I was working with Gerry. He had come, as I had mentioned, from the University of Minnesota where he was part of a group of people who essentially invented a whole bunch of measures called curriculum-based measures. One of which was this measure of oral reading fluency, of having students read out loud for a minute and scoring their words correct per minute score. That's one of a suite of assessments called curriculum-based measures. But Gerry was a doctoral student working with Stan Deno and folks there who invented this measure.

So it was very new, but Gerry was a big believer that words correct per minute was the thing. You needed to measure that with students, and it was going to tell you a whole lot about who was on track and who wasn't. And of course, it was a brand new measure, I'd never heard of it before, and having been a reading specialist for fifteen years, I was extremely skeptical.

I just thought, as many people still do when they first hear about ORF (oral reading fluency), should we really use a one-minute measure of cold, unpracticed reading, and that would be enough to tell us anything about anything? I was as skeptical as anybody when I first heard about it.

At that time, because it was a new measure, there were no norms to indicate what those numbers should be. For a third grader reading 83 words correct per minute in the winter, what does that mean? We had no norms.

What the original researchers suggested, the University of Minnesota folks suggested, was that schools should establish their own norms, and they had procedures for doing that. They said to test all your kids and create norms for your building.

When Gerry told me that I thought, "Well, that's ridiculous!"

I had spent my entire fifteen years up to that point working in low performing schools. I said, "What's the value in assessing our low performing school? We already know we're a low performing school, who cares about the 50th percentile at a low performing school?" And I said, "We need national norms."

He thought that was a good idea and put me in a little room with a desk and a ruler and a calculator, and brought stacks and stacks and stacks of paper. Very few people back then were doing ORF measures of their students, but there were a few, and through that process of literally using a ruler, going down and typing in scores on my calculator, we came up with some norms for oral reading fluency from second grade through fifth grade.

That was the first study. That's all the data we had. I think in total it was about 10,000 students, which was still a lot of rulers and typing. That was a lot of scores for me to put in all by myself. There were no computers back at that time, or I didn't have one.

So we wrote up a little article about this new way to assess kids called oral reading fluency and what the score should be. I probably finished that work in 1987 or 1988, and it took us until 1992 to get it published because every time we submitted it, people's reactions were like mine. They were horrified! Why would we assess kids on a one minute measure? That's ridiculous. Who cares about that?

It was finally published, not in a research journal, but in a special ed teacher practitioner journal called "Teaching Exceptional Children." They saw the value of it and they published it.

Then over years the reading world discovered it. The internet started, I guess, in its early stages, and people started learning about it. It became a measure that people were interested in, but there were no national norms. There was no DIBELS or aimsweb or Acadience or FastBridge or easyCBM. In those early years, people were just creating their own assessments, so they wanted norms.

Then in 2006, we did it again. By then we had access to a quarter of a million scores from first grade to eighth grade. We did the study again and published that in "The Reading Teacher." We chose that, not because it's a reading research journal, but because we wanted to get this information into the hands of as many teachers as we could.

Then ten years after that, we thought it was time to update it, and so in 2017 we compiled our newest set of norms. That one has over six million students. This time we only had access to first through sixth grade, for a lot of reasons.

But those norms, when you look at those three sets of norms and other norms all kind of coalesce around the same. Now, of course, we have the publishers who have created commercially available assessments from that research, and looking at all the norms together, they kind of coalesce around the same. There are certainly differences between aimsweb benchmarks and easyCBM benchmarks and DIBELS benchmarks, a little bit. People using those commercially available products sometimes do go then to the Hasbrouck and Tindal norms, because ours are compiled from multiple measures. Ours are not just aligned with a single assessment.

But let's get to the second part of your question about how we should use them.

They really are to help teachers get a sense of where their students are in terms of their acquisition of automaticity. That's a very important piece of information, and to be able to acquire that piece of information in one minute is pretty extraordinary. It's not, and never was intended to be, the only measure. It doesn't diagnose reading problems. It doesn't tell us a lot. It actually doesn't really tell us about a student's fluency. It tells us about their automaticity.

There are many people who have tried hard to consider changing the name of that assessment from oral reading fluency. Because fluency, as we've talked about, is much more complex. It has expression and prosody. It is connected with comprehension. A sixty second assessment doesn't measure that, but it does measure automaticity.

So those folks back at the University of Minnesota were absolutely right that they were onto something. We now have close to forty years of research where the words correct per minute score has been shown to correlate or predict comprehension, almost better than anything else we have. And it takes a minute to do that!

So we do know that, in general, those students who have not reached the 50th percentile on an oral reading fluency measure when reading unpracticed grade level material are not on track for future success in reading.

That's the primary way I feel that those norms should be used. They should be used to check on individual students' relationship to the normative progress that we know students should have.

Some very recent research came out that I'm citing a lot. It was Wyatt et al. that did a study where they went back and looked at NAEP scores, the National Assessment of Educational Progress, and wanted to know what ORF scores, or words correct per minute scores, best predicted NAEP scores.

They were looking at fourth graders, as the first NAEP test is given to fourth graders. The fourth graders who were at the top of the NAEP, which is the advanced level, were reading at the level of the 75th percentile on Hasbrouck and Tindal norms. The students who were at the advanced level, which is where we want students to be, advanced basic level, were reading around the 50th percentile of the Hasbrouck and Tindal norms. And the kids who were not successful on the NAEP, which is a comprehension test, bottom line, were below the 50th percentile.

So that suggests to me that what we've been saying for years based mostly on hypothesis, that you need to get kids to the 50th percentile, is true, and there is some advantage to being as high as the 75th.

But that study really to me feels quite conclusive in saying that those kids who are just reading super fast don't seem to have any benefit for comprehension. Which makes sense to me as a practitioner, but now we have some good clear evidence about that.

Anna Geiger: Yeah, I can definitely speak to that with one of my kids who, of all six kids, he's the one that least prefers to pick up a book. He can read "very fluently," but I'll ask him what it was about and he doesn't always know. So I'm working on slowing him down. It's that thought that reading is a race, don't worry about expression, and that's not where we're headed.

For a teacher that does the oral reading fluency with their students and then sees that some are lower than where they should be, what's the next steps? I know you've talked about how ORF is a thermometer. It doesn't diagnose. So what comes next? If a teacher notices that someone is low, what should the next steps be?

Dr. Jan Hasbrouck: It's a very good question and it's a sophisticated question that not a lot of people are always asking.

A lot of administrators are uninformed about that idea that ORF is a thermometer and that's all. Just like in the world of a physician, if they use a thermometer to take your temperature and find you have a fever, they don't treat the fever. I mean, sometimes this might be, but the CAUSE of the fever is not identified by the thermometer. So they're not going to fix you by plunging you in a bath of ice water to lower the fever. It's not the score that's important, it's what it could indicate.

So in the medical world, a fever is an indicator that something is amiss. But in conversations that I have had with physicians about, "When you see a high fever, what does that mean to you?" They start talking about all the things that can cause a fever: it can be flu, it can be COVID, it could be a ruptured appendix, it could be other kinds of infections or inflammations. It means something's not right, but what it does for a physician is then trigger a whole other set of assessments called diagnostic assessments.

That's exactly what should happen with us too. Our students have an academic fever if they are not strongly at the 50th percentile on unpracticed grade level text.

What caused that fever? Often when I'm trying to explain this or help teachers understand this, I will pull out Scarborough's Rope again. It helps to point out that those students who seem to be stuck at the middle of Scarborough's Rope where the pieces are being woven together, they can read, but they're not reading at that tightly woven rope. They're not reading well.

Is that loosely woven rope caused just by the fact that they're not yet sufficiently fluent? Should we work on just reading more to help them become fluent? For some students, yes, that's it. They read quite well, but all they really need is to practice, practice, practice, practice to become more fluent. However, that's only a subset of those children.

For some of those children, we go back to the beginnings of Scarborough's Rope and look both at the language piece and the word recognition piece. What I have found for a lot of my students who struggle with becoming fluent readers is that it's at the bottom part of Scarborough's Rope. It's the word recognition. They have some gaps or weaknesses in the foundational word recognition skills. Like it could be they're still struggling with phoneme awareness or they're still struggling with some aspects of word recognition. And if you're struggling with both of those things, you're going to struggle with the acquisition of sight words.

A lot of our students who aren't fluent, although they may be pretty good readers, have not acquired sufficient sight vocabulary. Their orthographic mapping process is faulty. If it is faulty, it's likely some deficits in phoneme awareness and phonics, and vocabulary we know plays a role in that. We are all better at turning words into memorized sight words if we know the function or the meaning of that word.

So if a student is struggling with their fluency as measured by words correct per minute, we should do some diagnostic assessments quickly. We don't need to send them to a school psychologist for a two or three-hour deep dive. It's more that we should do a little check of their phonics or do a little check of their phoneme awareness.

If you suspect that insufficient vocabulary academic language is an issue, we have some ways to take a look at that. That's where, in most cases, you're likely going to find some things that we need to be working on along with fluency, to make sure that we do get eventually to that tightly woven rope.

Anna Geiger: I know you mentioned that this is not always understood. How are people, in your experience, misusing the results of the ORF? If they see a certain reading words correct per minute, what are you seeing some schools do that you would not recommend?

Dr. Jan Hasbrouck: Well, I would say the main thing is using it as if it were a measure of fluency that you have just used diagnostically to assess, and then saying that the treatment for that is to do lots of intensive work to help you become a more fluent reader. In too many people's minds means that means a fast reader, and that's going to be an effort of extreme frustration for everybody if that student is struggling with fluency because of underlying issues.

I think the name of the assessment being oral reading fluency implies that to people. I don't blame the end user. There's a lot of blame to go around on that. We should rebrand that assessment and not call it oral reading fluency.

These days I usually just call it a measure of words correct per minute. Words correct per minute, which is a measure of automaticity, is an indicator of proficiency in underlying skills.

So if they're not proficient, that likely means their underlying skills have some weaknesses that we can go back and do some remediation on. Depending on the age of the student, we are likely going to do that skills remediation while we also are working on fluent reading of text. So we should be doing all of that altogether. But if the intervention is just doing something to get the kids to read faster, that is very likely going to fail and make everybody frustrated.

Anna Geiger: So I've seen some teachers that have a passage that all of their students are responsible for. And at the first day of the week they read it, they check the words per minute, they actually graph it on a little graph, and they just keep doing that all through the week to see how many they get. What would be your response to something like that?

Dr. Jan Hasbrouck: I actually recommend graphing students' oral reading fluency scores over time when I do workshops on best practices in intervention. If we're concerned about students, we really do need to collect some data and monitor their progress.

I also, though, always talk about doing that kind of thing in a differentiated way. I think that we always start with the classroom time that we have and it's never enough, and if we're talking about the entire classroom of students then of course we have great differentiated needs. We've got students who are probably, in almost every classroom, sailing along and doing great. We've got some students who are just making exactly the progress we would expect. And we've got some students who are struggling.

Given the fact that there's never enough time in classrooms for the instruction, which is the most important thing we do, instruction and intervention, I want to be very cautious about how much time is being spent on data collection.

So I think teachers, just like physicians, can differentiate data collection if you think about it.

A lot of us who are reasonably well only go see our doctor once a year. We get these once-a-year assessments and we check our blood pressure and cholesterol and all that. Our doctor says, "Everything looks good. I'll see you next year."

A good friend of mine right now is in the hospital. She just moved out of intensive care following some surgery. While she was in intensive care, she was getting assessments every minute of every hour. The level of assessment goes up when the need goes up in the medical world.

And that should be the same for us too. Weekly assessments of students using words correct per minute is very appropriate if a student has very high needs because it is a very sensitive measure, and we can see relatively quickly, over a period of a few weeks usually, whether our intervention is working or not. And if it's not working, we need to do something different.

I wouldn't recommend weekly assessment of our kids who seem to be doing fine. For those students, especially in our early elementary years, I do recommend those words correct per minute benchmark checks in the beginning, middle, and end of the year. As well as teacher observation of the students as they're working, doing little checks of their spelling, which is a really interesting thing.

A teacher can say, "We've been working on words that have blends at the beginning. You can decode those. Well, now take out a piece of paper or a whiteboard and see if you can spell those words. Can you spell the word, 'slam?'"

We do little checks like that to see if indeed the kids you think are moving along are actually moving along, but that can be done just as part of small group instruction, or even whole class instruction. But I think we need to be cautious about the amount of time we devote to data collection and differentiate that based on the needs of our students.

Anna Geiger: Thank you.

I just have one more question for you. Timothy Shanahan wrote a blog post with some sort of suggestion for a literacy reading block schedule. One thing he said was that every day there should be time for fluency building. What do you think should go in that block? I know it depends on the grade, but just some general ideas.

Dr. Jan Hasbrouck: Yes, and I appreciate that Tim did that. It is important because when we consider that goal on Scarborough's Rope, that tightly woven rope, the only way we're going to get there is practice. So it's not just the grade level or age of the student to consider, much more important than that is their skill development. Where are they in their development?

At the very beginning of the rope, fluency practice is more at the sound and letter level. Kids can be doing some work outside of whole class or small group instruction in partners, or they could be doing some center work where they're practicing that.

Then you have fluency practice with really beginning novice readers. Typically that would be late kindergarten or early to mid first grade who would do fluency practice with decodable text. Again that can be independent, or it's ideal for them to be working in partners or working with an older student or somebody who can listen and do some corrections with them.

Once students, wherever they are in the trajectory, have broken the code, then they really can apply their decoding to text that has more variety to it. For those kids, once they're really quite well-established readers, that's going to be early to mid second grade, that's when sustained silent reading, independent reading, can help you be a better reader.

We do know that there's value in independent reading, but only once you've become an established reader. Before that it's the individual component parts, the word level and the very simple text level. More often that's better done not silently and not independently, but with somebody there who can listen and give you some feedback and practice.

But ALL of that is fluency practice. We sometimes think of fluency practice only for those well-established readers. But we can practice the component pieces, text appropriate for that child's developmental level, and it should be done on a daily basis for sure if we want to get all kids to that tightly woven rope. And we do want to get all kids there.

Anna Geiger: Awesome. Well, usually I edit my episodes quite a bit, but I'm not going to want to cut anything out of all the wonderful things you had to say! Thank you so much for all that you do and continue to do. I just love catching any workshop that you're giving. I always make sure to watch those. Thanks for not retiring yet!

Dr. Jan Hasbrouck: Well, thank you for your interest in my work. I appreciate it.

Anna Geiger: Please head to the show notes for this episode to find links to Dr. Hasbrouck's work, as well as the oral reading fluency norms chart, and links to many presentations that she's generously shared and that are posted on YouTube. You can find the show notes for this episode at themeasuredmom.com/episode 98. Talk to you next time!