On February 27th, 2018 a slate of non-partisan (cough-cough-*Republican*) candidates announced their bid for Frederick County Board of Education (BoE). This slate of four candidates are choices among thirteen vying for 4 spots on the BoE.
And they wasted no time in generating eye-roll worthy statements of ridiculousness in their press release. Right out of the gate, Seth Eisenberg, Edison Hatter, Chaz Packan, and Cindy Rose make it seem like the existing BoE has been actively dismantling the Frederick County Public School system day by day. However, absent in their rant...ahem...press release (right), there is both little actual policy proposals to "fix" FCPS...and a whole lot of misinformation.
For example, the statement I bumped on most was "11,000 FCPS students are unable to read because FCPS does not have a scientifically based reading program."
Let's not go into how they end this paragraph with the statement "reading truly is a fundamental.', and focus this blog post on the statistic they note above.
Wot?
It's one thing to to be upset about the state of education in FCPS, but don't bloviate without the correct numbers to back up your claim. Put another way: don't "massage the data" or "cook the books" or "cherry-pick" the data to make for a more compelling story. Though, kudos to the slate for at least looking at the data to come up with 11,00 student figure. There is some semblance of logic that can bring one to that number...but it is fuzzy logic at best.
PARCC It, Mister!
To dig deeper into this statement, we first have to talk a little bit about PARCC Assessments, or the Partnership for Assessment of Readiness for College and Careers Assessment [Redundant sating 'Assessment' twice isn't it? Yeah, probably. But we're gonna roll with it].
The Partnership (P in PARCC) is a consortium of states + DC, The Department of Defense Educational Activity, and the Bureau of Indian Education that work(ed) to develop k-12 assessments in Math and English based on the Common Core State Standards (CCSS). In layman: PARCC develops tests that assess whether students in K-12 schools have mastered the content we, as a society, believe students ought to have mastered in each grade. For example, the CCSS outline what students should learn in grade 3 - and the the PARCC exams assess whether those grade 3 students have mastered the content such that promotion to grade 4 is warranted. This is why any discussion of "teaching to the test" seems silly to me, because if the test is aligned with the standards...then teaching to the test is teaching to the standards...which is the goal, no? But *that* is a discussion for another time.
At one time, there were 21 states, Washington D.C., the Department of Defense Educational Activity (DoDEA), and the Bureau of Indian Education participating in PARCC. This list has shrunk to 6 states, DC, and DoDEA. I say this to note that PARCC has not been without its criticisms (and, as an advocate of standardized testing, I have my criticisms of PARCC as well).
Criticism aside, students who take the two PARCC subtests in math and literacy earn a Scale Score in each subject area (based on their performance on a number of sub-areas within each subtest: i.e., in elementary math, mastery of topics such as fractions, decimals, factors, multiples etc.). These Scale Scores are then categorized into one of five levels of performance:
- Level 5: Exceeded Expectations
- Level 4: Met Expectations
- Level 3: Approached Expectations
- Level 2: Partially Met Expectations
- Level 1: Did Not Yet Meet Expectations
For more information on interpreting PARCC scores, you can download the handy score report released every spring. Things get a little nuanced in Levels 2 and 3. But we'll address that below.
So...Are There *Really* "11,000 Students Unable to Read" in FCPS?
Short answer? No.
Best answer? Awww hell naw!!! And here's why...
It all comes back to Level 2 and Level 3 of PARCC performance. Recall that each subtest (math and literacy) correspond to the CCSS standards. At each tested grade level (3-11), there are ten (10) standards in English Language Arts (ELA) that students must master - to some extent - in order to "meet" or "exceed" expectations. For example, the standards for grade 5 involve quoting accurately from a text, drawing inferences from a text, determining the theme of a story, determining the meaning of words and phrases, describing how a narrator's point of view influences how events are described, and more. There is a *lot* of nuance here. And there is more than simply "reading" being measured by these PARCC assessments.
This nuance is why the statement "11,000 FCPS students are unable to read..." is utter bullshit.
Here are some numbers to consider for the paragraphs below (note: all data are publicly available on the Maryland State Department of Education website):
Table 1. Frederick County Public Schools 2017 PARCC Proficiency Rates in Literacy, Grades 3-8; Grades 10-11; Three 'Not Met' Categories
Looking at Table 1 above, it appears that our four candidates came up with their 11,000 students not meeting the PARCC Literacy expectations figure by combining Levels 1 through 3 at each grade level (except grade 9, as there were not enough students sitting for the exam to post results while maintaining mandated levels of student privacy) and sum them. Our main focus here is the four right-hand columns (Met+Exceeds; Not Met*; Total Tested; and % Not Met). Here we combined the raw numbers of students that were categorized as "Exceeds Expectations" or "Met Expectations" (i.e, in Grade 3, 166 students were in the "Exceeds" category and 1,388 were in the "Met" category for a total of 1,554 students in the "Met+Exceeds" category) into one column and Levels 1-3 in an second "Not Met" column. In this dichotomy, if I am interpreting the 11,000 student figure in the press release correctly, the slate of BoE candidates categorize the "Met+Exceeds" group as those who are at or above grade level and the "Not Met" group as those students who are below grade level.
When we treat Level 2 and Level 3 as students who are "below grade level" or worse, "unable to read" the press release makes sense as 11,039 students - or 47.9% of the students assessed on the PARCC Literacy exam "did not meet expectations."
But this is misleading.
Students placed in the Level 2 category of "Partially Met Expectations" or the Level 3 category of "Approached Expectations" may read splendidly. However, they might have struggled in the more nuanced areas of the standards such as describing how a narrator's point of view influences how events are described. Not meeting grade-level standards on these specific standards is *not* evidence that students are "unable to read" or "below grade level" - and to suggest as such is foolish.
To rectify this foolishness, we decided to look at the same table of numbers with a more accurate categorization of "at-or-above" and "below" grade level. Thus, in Table 2 we only categorize those students who were categorized as Level 1 - Did Not Yet Meet Expectations - as being below grade level in literacy as measured by the PARCC exam.
Table 2. Frederick County Public Schools 2017 PARCC Proficiency Rates in Literacy, Grades 3-8; Grades 10-11; One 'Not Met' Category
As you will note, this more accurately reflects student performance in Literacy on the PARCC across grades 3-8, grade 10, and grade 11 in Frederick County Public Schools. By this measure, only 2,254 students (9.8%) are below grade-level in Literacy as measured by the 2017 administration of the PARCC exam. [And though it is not illustrated here, the number of students below grade level has declined over the past three years (perhaps we will do another post on longitudinal performance on the PARCC in FCPS next)].
So what?
Data are fickle things and how one presents and interprets them are tantamount to making data-driven decisions - which is a practice being used in more and more corners of society. But I would argue that there is a *right* way and a *wrong* way to present and interpret data (and I am not alone in this argument). In this case, labeling students scoring at the Level 2 and Level 3 PARCC performance categories as "below grade level" or "unable to read" is a false statement.
Making "broad brush" sound-byte seeking statements such as "11,000 FCPS students are unable to read" is careless, and does a disservice to the voters whose trust is trying to be earned. Such farcical statements stoke unnecessary fear and anger in students, teachers, school leaders, policymakers, and stakeholders in Frederick County - when that fear and anger is wholly unwarranted.
We should all be wary of those seeking seats of power who are unable to interpret the very data necessary to make sound policy decisions that will ultimately affect the 42,000+ students counting on our teachers, school leaders, and community to set them off on the best path for their success. From the look of the attached press release, Cindy Rose, Seth Eisenberg, Chaz Packan, and Edison Hatter have a LOT more to learn before they should be placed in a decision-making chair.