Interim State Superintendent Lonny Rivera says, "We Need Testing."

In my in-box today from Ohio's Interim Superintendant Dr. Rivera...

"I know you appreciate the crucial importance of maintaining student progress in an effective education system — for students, their parents, teachers, school administrators and the taxpayers who support your district. Testing shows evidence of student progress. It provides much needed information to classroom teachers and others, so they can monitor and improve our efforts in service of students. Results of these assessments provide the whole community perspective on what their students are able to retain and apply long term, allowing for reflection and improvement. Especially at a time when we must prepare our students for the high-skill demands of today’s workforce, we need testing — and test results — to tell us how to best help our students succeed."

As a teacher in an urban high school, I do appreciate the importance of maintaining student progress. Students in my school often have a multitude of obstacles to overcome prior to being able to make adequate academic progress. Sometimes students have health concerns, family problems, emotional issues, developmental concerns, problems related to poverty, parental unemployment, homelessness, and poor diet, among many others. I'm sure you understand.

Unfortunately, that progress has recently been stymied by a month of testing. 

Here's the thing, though, with all due respect to Dr. Rivera, who I'm sure means well, testing has never helped any of my students overcome the obstacles that prevent their progress. Furthermore, these assessments provide very little in the way of evidence with any real value. They've never really provided me with "much needed information." The way I understand it, school districts will get overall results from these assessments in June, parent reports in July. I hope to get the data related to my students this summer, but I've gotten no guarantees. Even if I do, my sophomores are gone June 1st, and my school is so big, I may never see them again.

As a teacher, the "reflection and improvement" that I make use of most effectively is done moment to moment in the classroom based on student responses, or lesson to lesson, day to day, unit to unit. I use formative assessments almost daily and summative assessment every few weeks. When I analyze those moments, or grade those tests, the results are in my hand, right then. At that point I make informed decisions about instruction. If I am unsure of something, then I discuss matters with the many effective professional educators who I am proud to call my colleagues.

All Dr. Rivera's test results do, is to provide us with information on how we can best help students succeed on tests. And while test scores may improve...I know much has been made lately of our "success" on the third grade reading guarantee...what we have improved is test performance. If success on assessments is our goal, then yes Dr. Rivera, "we need testing." However, if we are truly interested in student progress, and legitimate preparation for the high-skill demands of today's workforce, then we should probably reconsider our priorities.

It is widely understood that these assessments primarily reflect economic status. We don't need standardized tests to tell us who is economically vulnerable. We need remediation of their conditions, solutions to the obstacles that prevent student progress in an effective education system.

The Testing Window Has Opened.

Goal: End unnecessary state testing.

The testing window has opened here in Ohio, and students across the state, at all levels, have begun to sit for 3 hour assessments in a variety of subjects. At the younger grades a lot of districts are giving the tests in two parts because it's easier for an 8 year old to sit for 90 minutes. (I said easier, not easy)

I coach a youth soccer team of 11 and 12 year olds. At practice on Thursday, they were wild before we started. A bit nuttier than usual. We did our lap, had a stretch, and before I introduced our plan for practice I asked them, "So, how many of you had to take a really long test this week?" The majority of them raised their hands and this same group of boys who, 5 minutes prior, had been energetic and joyful, immediately became sullen, if not angry and defeated. I made an on the spot decision to play the 3 or 4 small sided games that they love the most.

It was the least I could do. I certainly couldn't tell them that their 5th and 6th grade tests were meaningless, even though they are, or that their grades on them aren't a real grade. Their teachers won't even see their scores until well after the school year has ended, and when they do, those scores will indicate what they always do, that rich kids outperform the poor.

The state tells their parents that the assessments are valuable so that we can have apples to apples comparisons of students, teachers, schools, and districts through the ever important value-added measure. But this logic has long since been proven flawed.

So, if I were to attempt to provide a legitimate reason why these kids were subjected to a 90 minute test multiple times this week, I simply cannot come up with anything good enough. The whole damn system exists because of the Texas Miracle of more tests and accountability equating to greater performance and lower dropouts. We all know that it was a lie. And yet state authorities persist with a system of assessment that has yet to have much, if any, positive impact on education.

Forgive me, but I'm not going to be the one to explain to these boys that they're taking standardized tests because a long time ago, some powerful adults lied about these tests being a good thing, and some other powerful adults (including the President) promoted the lie and made some laws as a result. Then, once everybody figured out they were all lying, it was too late, and nobody did shit to change the system.

The reasons we administer these assessments simply aren't good enough. I'm advocating for change, and I believe that my team (and their peers) would agree with me.

In the mean time, we're going to have a kick around and try to forget about the testing window.

They want input, but they don't want input.

The kids aren't alright, Jim Wright.

The Ohio Department of Education has made much lately of their interest in having a collaborative process in reviewing standards and creating assessments. I have personally received no fewer than a dozen emails over the last 2 weeks encouraging me to get involved in the review of the ELA and Math standards. I forwarded the information to the English and Math Department Chairs at my school and encouraged them to participate. It is good to feel like a part of the process.

Being a Social Studies teacher at the High School level, I've recently been less concerned with the aforementioned Common Core standards and more concerned about the alarmingly low End of Course assessment scores.

In my last post, I mentioned a few of the reasons why Ohio's new Three Paths to Graduation could be problematic. I admit to not being an expert. My wife, however, is becoming one. She recognized the degree to which I was legitimately concerned about the issue, and spent several weeks studying the report card data for our district and others like it. She also compiled information on the ACT remediation free path, as well as percentages of vocational ed students. She then sent info in multiple short blasts to all Ohio State School Board Members, as well as all members of the House and Senate Education Committees. You can see the scope of her research here...   and here...

Here is a brief piece of her correspondence with the state...

To Whom It May Concern:

While I was pleased to learn that over 100 Ohio educators played an integral role in setting the performance standards for this year’s ELA and Math AIR tests (unlike last year where we let PARCC decide the standards for us), I am still very concerned with the projected number of students who will score proficient on the different components of the high school level state tests. As you can see from the chart below, 41% of kids are expected to score a 1 or 2 on the Geometry test. When you compare that to the percentage that scored a 1 or 2 on last year’s Algebra test (49.1% of kids in my similar district study which I sent you last week received a 1 or 2 on the Algebra I test) you should realize that there is cause for concern. Students need 4 points total in Math to fulfill that requirement for graduation. I think that you can fairly assume that some of these students that scored a one or a two on the Algebra I test could be the same ones that score a 1 or a 2 on the Geometry test. You also cannot assume that these kids will be getting 3’s and 4’s on their ELA tests. 59.9% of the kids in the districts I studied last year received a 1 or a 2 on the ELA I test; 48% of kids across the state are expected to get a 1 or a 2 on the ELA II test. You can assume that the similar districts studied would have an even higher percentage only getting a 1 or a 2 on the ELA II test based on their past performance on state tests and the number of studies done that rightly suggest that students in high poverty areas (like the ones I studied) tend to not do as well on standardized tests. You can see the latest study that happens to address these differences in scores from last year’s Ohio tests here: http://www.cleveland.com/metro/index.ssf/2015/12/poor_kids_do_poorly_affluent_do_better_on_ohios_state_tests_-_again.html

You must also keep in mind that students have to score a total of six points across the Science and Social Studies tests and receive a total of 18 points all together. If there are so many students that could struggle to meet the requirements of each separate component, it will be nearly impossible for them to make up the additional 4 points they will need to get 18 total points.

Her research and concerns are legitimate, and her numbers are accurate. Director of Assessment, Jim Wright, sees things differently. In his patronizing response, he trots out the state's tired old bullshit about raising the bar. By raising the bar he means fewer poor kids and minorities will graduate. He also wows us with the Ohio Department of Education's Holy Trinity, the "Three Paths to Graduation." I guess he figured we were unaware of the fantastic bounty of options awaiting Ohio's students. Go ahead, check it out.

Ms. Jablonski,

Thank you for your email and sharing your concerns pertaining to the high school graduation tests. Ohio continues to raise the bar for students to ensure that they are prepared for postsecondary opportunities, whether college or a selected career. The new graduation requirements offer three pathways to demonstrate their preparedness through either the end-of-course exams, remediation-free scores on a college readiness exam (ACT/SAT), or with WorkKeys and a career credential. For the end-of-course pathway that you are addressing, students have flexibility in attaining the needed 18 points with the minimums in content areas as you describe. Most of these seven courses will be completed during the early high school years, and this will allow for multiple opportunities for retakes. The state high school end-of-course tests are currently offered three times annually, including a summer administration.

Sincerely,

Jim Wright

Director, Office of Curriculum and Assessment

So, it appears that the Ohio Department of Education wants a collaborative process that includes educators and community members when that involves participating in a convoluted survey regarding standards. That way they can show us their many invitations to become a "part of the process," without ever doing a damn thing with the information provided.

However, if the "community" researches a legitimate concern, compiles data that seems to suggest a scenario that will prohibit thousands of students from graduating, the ODE is unwilling to seriously consider the information, or begin a discussion regarding its ramifications.

To make matters worse, in this case, Jim Wright seems to agree that there will be an issue. At least he did on January 26, 2016 when he said as much at a meeting of the Ohio Technical Advisory Committee. Here is a quote from the minutes of that meeting...

Jim Wright noted that there are three pathways to high school graduation, but recognized that the new proficiency cuts for End of Course assessments will be challenging if used in defining high school graduation. 

So, which is it then, Mr. Wright? The email to my wife seems to suggest that the kids are alright if they only follow one of the three magical pathways to graduation. And yet, in the above meeting with your colleagues, you've admitted the situation will be challenging IF the assessments are used in defining high school graduation. Those assessments ARE being used in defining high school graduation. The other two paths are NOT as viable as you and the Ohio Department of Education would have us believe.

To make matters worse, Mr. Wright cc'd all of the State School Board members in the email he sent my wife, as if he wanted to assure them that the kids are alright, as well as discredit her concerns. All he has done is to provide limited and misleading information to community members and the State School Board. The kids aren't alright.

On the Eve of the Testing Window, Let Us Reconsider The Three Paths to Graduation.

As we all know, Ohio has expanded its graduation requirements to include three paths. These include the ever unpopular and still confusing End of Year Assessment route, the remediation free ACT route, and the WorkKeys Industry Credential route.

If you're unfamiliar, the state's presentation on them can be found here.

When referencing the Three Paths to Graduation, the Ohio Department of Education has taken on a reverent tone as if they are recounting the biblical story of the loaves and the fishes, and many people seem to be buying into this line with all the fervor of religious devotees. Where there was once only a single path to graduation, the legislature has acted in the interest of all Ohio children, and now there are many. Amen.

According to the ODE website, "The transition to multiple options for earning a high school diploma is an exciting one for students who will have more flexibility for success in school and preparing for their future after high school."

I don't know about you, but I'm sick with excitement, and to be fair, more seems better than less. We are Americans, and we want 3 burgers for the price of one. This is the land of plenty, and we should all be so fortunate to share in the bounty of freedom and opportunity, hamburgers, and more standardized testing options than one human being should be able to tolerate without a complete nervous breakdown.

What the ODE is assuming, calling on our blind faith or blind patriotism, is that we're going to believe that the state has created a system that by sheer inclusion of multiple pathways for assessment (I mean graduation) is somehow doing what is right for kids. Except that it isn't. I believe, and I hope I'm not alone, that this system is terribly flawed and could cause a sharp decline in graduation rates at least in the first 5 years, and perhaps beyond.

As I have indicated previously, some Ohio districts had fewer than 50% of their students scoring proficient on last year's Algebra tests (among others). That equates to fewer than the necessary minimum points toward graduation per test (2.57) for more than half of the tested students in those districts. Predictably, these are districts with higher rates of students labeled economically disadvantaged. I understand that the numbers will increase year to year over the next 5-10, but how are these students expected to graduate? Again, you cannot assume every kid who struggles with Math will pull brilliant scores in the other disciplines in order to compensate, and earn the necessary 18 points toward graduation. As a matter of fact, many of the students scoring poorly in Algebra will likely struggle across assessments.

Yes, the tests can be retaken in an attempt to earn a higher score. In this scenario we're assuming those same economically disadvantaged districts are re-testing better than half of their students. We are also expecting, regardless of district, a better score testing a second time despite the fact that those students are no longer enrolled in the courses being tested, and where no remediation programs exist because the assessments themselves are too new.

Sometimes when I mention my concerns regarding the new Ohio high school assessment system to people who would listen me, they concede that I may have a point. There is often a brief look of panic that flashes in their eyes as they consider the ramifications. Unfortunately, after that, many simply walk away toward their quiet place, humming to themselves, wishing they had chosen a career in plastics.

What I'm finding more problematic, however, are those few who stick around, and they are few. They take on a hopeful look, smiling sympathetically, as if to say, "You silly, silly, man. You worry needlessly." Then they explain in earnest... "Perhaps you don't understand. There are THREE paths to graduation."

Of course, forever the skeptic, I interject...

According to The Ohio Education Policy Institute, in analysis of state data, only 15.1% of students scored remediation free on the ACT in districts with greater than 90% economically disadvantaged students. In areas with high rates of poverty, rural or urban, this is not a viable path to graduation. In districts with only 10% economically disadvantaged the percent of students scoring remediation free is only 69%. It's not a foolproof option even in the state's most successful districts.

See that report here.

And if the solution is a vocational certification, a 2014 report by the Fordham Institute indicates that only one in four students in Ohio's Career and Technical Planning Districts earned an industry credential. The data in this report was somewhat limited, but the numbers do not seem promising.

Less than 50% proficiency on new assessments, 15% scoring remediation free on the ACT, 25% earning industry credentials. What does this mean? I believe that we have to assume that students who score poorly on Ohio's End of Year tests will also be unable to achieve a remediation free score on the ACT, and less likely to gain an industry credential combined with a satisfactory score on the WorkKeys assessment. This could be terribly problematic for Ohio's most vulnerable students, those who are economically disadvantaged.

My conclusions here are based upon some educated speculation. I teach in an urban high school whose success on standardized tests has never been guaranteed. I have been conditioned to fear the worst where the Ohio Department of Education is concerned. My school became very successful under the previous system (perhaps necessitating a new system under which we can fail), but it took some time. The work of myself and my colleagues will raise the scores over the next 5-10 years, but what about these kids? Their work deserves a diploma, and the Three Paths to Graduation, as exciting as they are, don't seem to lead there.

To be honest, I hope that my speculation turns out to be a misinterpretation of the data. However, it is the utter lack of dialogue regarding the potential problems that I find so terribly frightening. We cannot simply accept the state's line that this will all work out. They have no evidence to support their claims that this system is better than the last. Worse, they seem to be making this up as they go along. For the sake of my students, I'd prefer to at least begin a discussion prior to a potential crisis. 

More Valuable Data from the 2015 Assessments or The Post Where I Sarcastically Pretend to Be an Uninformed Boob in Order to Make a Point.

The figure above appeared in an article Saturday in the Cleveland Plain Dealer. It shows Performance Index scores when correlated with poverty. The article explains, "Performance Index is a composite of test scores across all grades and subjects that summarizes how well kids scored on state tests."

Very simply, the greater the percentage of economically disadvantaged students, the lower the state test scores in a district.

I don't know about you, but my mind is blown. Who would've thought that poor kids would score lower on standardized tests? Maybe we should test ALL of Ohio's students again next month to see if this is a pattern. If it is, wouldn't that mean that a disproportionate number of poor and minority students are being held back in 3rd grade and prevented from graduating from high school? I can't believe that our legislators could, in good conscience, permit such a system to exist.

That was exhausting. 

Students return to testing in a month. I'm going to predict that poor kids struggle, and districts with more of them are labeled as failures. Furthermore, the state will continue to fail to remediate the effects of poverty, praise themselves for raising the academic rigor, and argue that our testing system is providing a fine apples to apples comparison of school districts.

I will continue to teach, and try to direct my rage into something productive.

A Numbers Game?

5 American History classes.

146 students.

12 days until Spring Break. I should be pleased at things. The built in breaks enable us to maintain the level of energy and commitment necessary to teach successfully. We have a week and a half off. This should be plenty of time to prep our energy levels for the end of the year.

This year is different. The standardized tests in the state of Ohio are changing again. The sophomores I'm teaching are adjusting to a new assessment system for the second year in a row. This year the tests are in one window, which is great, except that the window occurs after that week and a half off, they are taking brand new assessments no more valid than last year's, and they're still working toward a high-stakes endgame, enough points (18) to graduate.

3 (2.57) is the average score a student needs on each assessment to reach 18.

79% is the highest percentage of students to score 3 or better on a given exam in my school. It was Geometry which was taken by all honors students last year.

Some of the percentages on other tests, like Algebra and American History were far lower. It is difficult to say exactly what this will mean for the prospective graduation of my students, their peers, or others in schools that typically struggle on these assessments (read urban schools with high rates of poverty).

I am trying to be optimistic.

We're all shouldering a great deal of anxiety, and grinding ahead in the interest of our students.

My plan is to wrap up a unit on Civil Rights this week, administer the SGM (another state mandated assessment) next week, and recap the first semester prior to break. A brief review is only reasonable, considering these kids are going to be assessed by the state on some material we haven't mentioned since September, and some from their World History course in 9th grade.

Then, in a leap of blind faith, I am going to hope that the best preparation for brand new assessments on which your graduation depends is to take a week and a half vacation. Then I'll pretend that I believe it's a good idea to assess all students on a variety of computers despite the research that suggests otherwise.

I will not, however, suggest to anyone that I believe this testing system is much better beyond shorter. I also will not be convinced that high-stakes measures like graduation should be tied to any assessment. The federal government doesn't require it. Most states don't either. 

At minimum, Ohio needs further revision of the system going forward or the graduation rates will plummet. For many politicians, education is a numbers game driven by election years. For those of us in the field, it's about these kids looking for a bit of understanding.

The Evidence-Based Set and Online Testing.

The Simulation below contains a set of 6 sources and corresponding sample questions from the state of Ohio's American History End of Year Assessment. It is an excellent example of the American Institutes for Research's directive from the state to create PARCC-like questions. The description of the set is described by the state as follows...

"An Evidence-Based Set is a group of several questions associated to one or more common stimuli. Evidence-based sets allow students to work with primary source materials to show deep understanding of social studies topics. The questions in these sets will assess a range of skills and content in the content statements."

The premise is academically sound as it facilitates the analysis of multiple sources, asks the student to draw on relevant ideas, and combine them in order to come to conclusions. 

My students and I recently undertook a similar process in analyzing the decision to develop and drop the atomic bomb. The activity took place over two days within our study of WWII, and involved an analysis of writing from Oppenheimer, Eisenhower, and the War Department, as well as President Truman. We watched interviews with Japanese survivors and members of the crew of the Enola Gay. Throughout the process we discussed the issues, and the effect the bombings had on the world going forward. Students wrote reflective essays and defended their thesis with facts from the sources we'd investigated.

Attempting to recreate this process as a part of a standardized test, while understandable, is contrived at best, and at its worst is another example of educational inequality. The problems begin with the fact that standardized testing takes any information out of an authentic context. As you'll see below, the assessment writers attempt to remedy the issue with a contextual introduction. This does something, though not much, to set up the simulation.

The real issues relate to the fact that 80% of Ohio's students will complete this process online, while the rest will complete it on paper. Online the 6 sources included here must be accessed individually using a drop down menu. In other words, students taking the test on computers have no opportunity to view the sources side by side. This creates a clear disadvantage, and was one of the many criticisms of the PARCC assessments last year. Many people assumed PARCC was the issue, but these problems persist regardless of vendor.

As the Cleveland Plain Dealer recently reported, states looking at comparisons are finding that scores from computer test takers tend to be lower than their paper counterparts. The issue above might begin to explain this phenomenon. Research suggests that the physicality of paper versus screen print promotes a tactile relationship that actually improves a reader's long-term understanding. What is also at play here is a student's familiarity with the technology in question. Just as many have argued that all standardized tests, regardless of subject, are reading and writing tests, they are now becoming assessments of technological skill.

As with most educational issues, economic inequality exacerbates the problem. Students in high poverty areas will obviously have less opportunity to develop computer skills in the home, so will suffer disproportionately as these tests become required online. Standardized assessments have already proven to measure economic standing better than any academic measure. The use of computers for testing seems as if it will further solidify this issue. 

Even if we could assure equal technological skill, which we cannot, the hardware will differ from district to district. It should be clear then that a student's ability to maneuver the assessment will differ depending on their use of a desktop, a device with mouse or without, a laptop, a tablet, Chromebook or otherwise. Furthermore, anyone who has used multiple devices knows that they are not all created equally. Student computer skill (or lack thereof) combined with a multitude of unequal devices on a variety of systems creates a recipe for further educational inequality. 

School districts are currently funding cost-effective devices in order to comply with the coming mandate of all online testing. With a lack of overall funds, it is easy to assume that these purchases are being made at the expense of the arts, music, physical education or other non-core areas. Naturally, time once devoted to these subjects can now be used in the interest of increasing a student's screen time in an attempt to assure improvement on state tests.

What is especially painful is the fact that success on the simulation below and the tests overall decide whether or not a student graduates from high school in the state of Ohio. Other online assessments determine a 3rd grader's promotion, a school's reputation, and a teacher's rating, among other things. These stakes are far too high for an assessment system like this.

A scenario that includes the number of issues that I have delineated here is at least deserving of some discussion. We cannot simply continue to grind students through an assessment system constructed without a thorough analysis, with little regard to its impact on students, that clearly exacerbates the educational inequality that is already terribly problematic.

Unfortunately, as the testing window nears, this is what we're preparing to do. 

The state of Ohio has released some resources for students and teachers in all tested subjects which are available on the ODE's website. I would encourage all stakeholders to analyze the materials. For our purposes here, I have provided screenshots of the sources, and then the questions from the simulation I have described. Consider the issues I've presented, check out the simulation below, and see what you think.