The Evidence-Based Set and Online Testing.

The Simulation below contains a set of 6 sources and corresponding sample questions from the state of Ohio's American History End of Year Assessment. It is an excellent example of the American Institutes for Research's directive from the state to create PARCC-like questions. The description of the set is described by the state as follows...

"An Evidence-Based Set is a group of several questions associated to one or more common stimuli. Evidence-based sets allow students to work with primary source materials to show deep understanding of social studies topics. The questions in these sets will assess a range of skills and content in the content statements."

The premise is academically sound as it facilitates the analysis of multiple sources, asks the student to draw on relevant ideas, and combine them in order to come to conclusions. 

My students and I recently undertook a similar process in analyzing the decision to develop and drop the atomic bomb. The activity took place over two days within our study of WWII, and involved an analysis of writing from Oppenheimer, Eisenhower, and the War Department, as well as President Truman. We watched interviews with Japanese survivors and members of the crew of the Enola Gay. Throughout the process we discussed the issues, and the effect the bombings had on the world going forward. Students wrote reflective essays and defended their thesis with facts from the sources we'd investigated.

Attempting to recreate this process as a part of a standardized test, while understandable, is contrived at best, and at its worst is another example of educational inequality. The problems begin with the fact that standardized testing takes any information out of an authentic context. As you'll see below, the assessment writers attempt to remedy the issue with a contextual introduction. This does something, though not much, to set up the simulation.

The real issues relate to the fact that 80% of Ohio's students will complete this process online, while the rest will complete it on paper. Online the 6 sources included here must be accessed individually using a drop down menu. In other words, students taking the test on computers have no opportunity to view the sources side by side. This creates a clear disadvantage, and was one of the many criticisms of the PARCC assessments last year. Many people assumed PARCC was the issue, but these problems persist regardless of vendor.

As the Cleveland Plain Dealer recently reported, states looking at comparisons are finding that scores from computer test takers tend to be lower than their paper counterparts. The issue above might begin to explain this phenomenon. Research suggests that the physicality of paper versus screen print promotes a tactile relationship that actually improves a reader's long-term understanding. What is also at play here is a student's familiarity with the technology in question. Just as many have argued that all standardized tests, regardless of subject, are reading and writing tests, they are now becoming assessments of technological skill.

As with most educational issues, economic inequality exacerbates the problem. Students in high poverty areas will obviously have less opportunity to develop computer skills in the home, so will suffer disproportionately as these tests become required online. Standardized assessments have already proven to measure economic standing better than any academic measure. The use of computers for testing seems as if it will further solidify this issue. 

Even if we could assure equal technological skill, which we cannot, the hardware will differ from district to district. It should be clear then that a student's ability to maneuver the assessment will differ depending on their use of a desktop, a device with mouse or without, a laptop, a tablet, Chromebook or otherwise. Furthermore, anyone who has used multiple devices knows that they are not all created equally. Student computer skill (or lack thereof) combined with a multitude of unequal devices on a variety of systems creates a recipe for further educational inequality. 

School districts are currently funding cost-effective devices in order to comply with the coming mandate of all online testing. With a lack of overall funds, it is easy to assume that these purchases are being made at the expense of the arts, music, physical education or other non-core areas. Naturally, time once devoted to these subjects can now be used in the interest of increasing a student's screen time in an attempt to assure improvement on state tests.

What is especially painful is the fact that success on the simulation below and the tests overall decide whether or not a student graduates from high school in the state of Ohio. Other online assessments determine a 3rd grader's promotion, a school's reputation, and a teacher's rating, among other things. These stakes are far too high for an assessment system like this.

A scenario that includes the number of issues that I have delineated here is at least deserving of some discussion. We cannot simply continue to grind students through an assessment system constructed without a thorough analysis, with little regard to its impact on students, that clearly exacerbates the educational inequality that is already terribly problematic.

Unfortunately, as the testing window nears, this is what we're preparing to do. 

The state of Ohio has released some resources for students and teachers in all tested subjects which are available on the ODE's website. I would encourage all stakeholders to analyze the materials. For our purposes here, I have provided screenshots of the sources, and then the questions from the simulation I have described. Consider the issues I've presented, check out the simulation below, and see what you think. 

Sweet Fancy Moses, Lonny is at it again.

From a press release from the Ohio Department of Education today...

“Teachers, parents, superintendents and education leaders believe that all students should take the tests, and their communities are better served when achievement is measured. After collaborating with education stakeholders and our partners in the General Assembly, we also know it’s important to provide schools and communities with additional information on the impact of student participation on the report cards,” said Interim State Superintendent of Public Instruction Dr. Lonny J. Rivera. 

What the hell, Lonny? Haven't you been listening?, or is this just more bullshit spin like I heard in the teleconferences you participated in on the wonderful improvements in Ohio's testing (See this blog's "Phone Forum Propaganda" 10/27/15 for more on Lonny's spin).

In the interest of full disclosure, Lonny, as a parent and teacher, I DO NOT believe that all students should take the standardized tests. I believe that they should be abolished, especially those tests that tie in high stakes decisions like the 3rd grade guarantee and high school graduation tests. Now, I know you've no interest in abandoning the tests, so how about ditching the high stakes and bringing Ohio's system of assessment down to the federal minimum. As you should know, as a professional educator and acting superintendent, the greatest predictor of test scores is economic standing.

I believe that you could save us all some trouble...teachers, parents, superintendents, and education leaders alike...oh, and students as well (who you forgot to mention), and use tax records instead of test data.

Furthermore, Lonny, I'm willing to bet that lots of other stakeholders would agree with me on this. I think you'd better start paying attention.

Greg Ring

Cheers to Lorain County ESC Superintendent Greg Ring for his opinion piece in the Lorain Morning Journal. Published in print this morning, it provides an informed and realistic look at Ohio's system of charter schools.

Highlights include...

"Ohio's charter schools take far more kids from school districts that outperform the charters than the other way around."

"Consider that in Ohio, no entity audited by the state misspends money at a higher rate than charter schools..."

Check out the entire opinion piece here...

http://www.morningjournal.com/article/MJ/20160202/NEWS/160209881

420

Years ago I had a student in class who liked to quote himself. He'd say something like, "Well, if I could quote myself..." and then go on to quote himself in defense of his argument. His method was brilliantly absurd, comedic genius. It was impossible to argue with his airtight misguided logic.

In the spirit of that guy, I'd like to quote myself, 

"Over the last several years an increasingly vocal contingent of administrators, teachers, parents, and students have criticized our system of standardized testing. In the state of Ohio, this activism has forced the hand of politicians. Recently they decided to eliminate PARCC, and cut one of the testing windows. It is not enough. The assessments created by AIR are not any better. Three hours per test is still inappropriate. We are testing beyond the federal minimum of hours and content areas. While we squander instruction time for the sake of assessments, the tests will continue to measure what they measure best, which is the relative economic well being of the students taking them."

I spoke these words publicly to the school board in my district this past fall in the interest of maintaining a critical dialogue regarding education and assessment. Should I be punished for this critique of standardized tests, or the many others I have made? According to an interpretation of Substitute House Bill 420, I might be.

As many of you already know, the bill prohibits teachers from "suggesting" that students opt-out of state assessments. Does the above critique make that suggestion? Well, I don't know, but if someone had it in for me, it might. If I were found in violation of the law because of my interest in collaborative intellectual dialogue on the subject of testing, then I could be charged with a minor misdemeanor, have my teaching license revoked, and lose my teaching job.

And I thought quoting oneself was absurd.

This provision is clearly an attempt to muzzle educators, is likely a violation of our first amendment rights, and is completely, professionally unacceptable.

This is not a paranoid scene from a dystopian educational future. It is happening.

I have contacted ALL members of the House Education Committee, and I encourage you to do the same. Below, you'll find the letter I sent. Copy and paste, revise it to your liking, then send it.

Representative So and So,

I recently sent an endorsement of HB 420 to your email. Unfortunately, I do not find the substitute bill in the same spirit. To be fair, even the original was only minimally appealing in its very limited scope and short term focus on limiting the impact from opt outs on last year's tests. In Elyria, where I teach and send my child to school, the refusals were statistically from higher performing students, and the original bill did nothing to address the effect this has on overall scores.

As always, I feel the level of Ohio's testing, well beyond federal minimums, combined with unnecessary high stakes (3rd grade guarantee, graduation requirement, teacher evaluation, etc.) to be the real issue.

The substitute bill's inclusion of language forbidding the "suggestion of opt outs" by teachers and other employees makes this legislation unacceptable and likely unconstitutional. I have never been an outspoken advocate of refusing tests. However, I do believe it is a parent's right. As a parent and teacher, if I opt my son out of assessments, am I suggesting opt outs? Furthermore, if I engage in a public dialogue that is critical of the assessment system, as I have done, am I suggesting opt outs? The bill does nothing to clarify, and so leaves educators open to unwarranted attack. These are only two scenarios that illustrate the point.

The language in question removes the most informed stakeholders in educational policy, teachers, from the discussion. Education is a collaborative endeavor when it functions best. Collaboration has already been threatened through high stakes assessment, pitting public schools against charter schools and other public schools, an evaluation system that pits teacher against teacher, school versus school. A system that thrives on cooperation and collaboration suffers in this environment. The marked decline in Ohio's national education ranking is proof of that. Substitute HB 420's systematic muzzling of teachers will only make matters worse.

I have not even begun to reference the constitutionality of the measure, or the hostility evident in the subsequent punishments if one is found in violation of the statute. I know that much of this came out in testimony.

I am not normally an advocate of completely dismissing work or using cliches that involve babies and bathwater, but in this case I encourage you to throw the baby out with the bathwater. House Bill 420 is lost and unnecessary. Dismiss it, and move on. Much is wrong with Ohio's education system. 420 is not the answer to any of it.

As always, thanks for your time, consideration, and work for your constituents.

Matt Jablonski

I never would've guessed.

From the article...

The Ohio Department of Education agrees that the trend is clear. Chris Woolard, the department's head of accountability, said showing that pattern is a goal of the report cards.

"These are aspirational measures that are pointing out a problem," Woolard said. "Not all kids are leaving high school ready for college or work."

By showing districts how many kids are not meeting the state's goals, they now know what they have to work on, Woolard said.

To be fair, the statistics suggest that the problem is poverty, what needs to be worked on is a remedy for poverty. Are Mr. Woolard and the ODE suggesting that continued excessive assessment is necessary in order to point out who is poor? The IRS already does that. Or, worse yet, is he suggesting that my new role as a public school teacher is to somehow remediate the effects of poverty on a child's life? Is that what I know I have to work on?

I consider myself a fairly successful teacher, but I question the expectation here.

What we've illustrated once again with a hefty round of high stakes assessments: Poor kids score poorly, affluent kids score far better.

Again, we should consider the implications when it comes to all high stakes use of this data...

     the 3rd grade reading guarantee

     High School graduation tests

     ACT/SAT scores

     School Ratings

     District Ratings

     Teacher Ratings

Let's remember we're measuring poverty. Since we've already done that, let's abandon a few of these meaningless assessments and their high stakes punishments.

And so, the release of more meaningless data OR three more reasons I'm happy to be an Elyria teacher.

So, the ODE released the K-3 Literacy Rating as well as graduation rates, corresponding scores and some other data. By all counts, there are serious issues with the numbers. In some cases they don't even make mathematical sense. Districts all over the state have been hazing the Department of Education over their release of the information before it clarified the numerous formal appeals and corrections.

The ODE's response is predictably absurd, according to an article in the Elyria Chronicle Telegram,

“According to law, we are required to release the report card,” said Toby Lichtle, assistant director in the ODE’s Office of Communications and Outreach.

Why yes, Toby, the law does require that. Do you know what else the law requires, Toby? The law requires that the state provide equal educational opportunities for its children. It requires the effective and accurate rating of charter schools and sponsors using ALL relevant scores. It also requires an investigation where a public official may have defrauded the public. The law requires that schools be governed by locally elected boards of education. As a matter of fact, the Ohio constitution also requires an equitable funding system for public schools. In short, Toby, the ODE and the Ohio legislature haven't seemed very interested in adhering to the finer points of the law.

Clearly the report cards are far more important than these other issues in that they can be used in the ongoing process of discrediting, defunding, and destroying public education.

Fortunately, many of us in the public schools aren't simply going to accept questionable ratings from an invalid assessment system that diminishes the work we put in on a daily basis. My boss, Elyria Superintendent Tom Jama weighed in on these things this week in articles in the Chronicle and the Lorain Morning Journal. I'll call these quotes three more reasons that I'm happy to be an Elyria teacher, and public school teacher in general.

"These results are just not an accurate assessment of what is being done,” he said. “We are never going to run away from areas of concern. But I know what our teachers and staff are doing each and every day from central office on down to improve the life of our kids. This information is nothing more than one snapshot that is not giving the whole story.”

“The report card isn’t a true and accurate reflection of the district’s performance,” said Dr. Tom Jama, superintendent of Elyria City Schools. “You can’t use a test that has been decided not to be used again, and use the results of the test to label a district. 

“We don’t want to be labeled on a test that included opt outs, negativity. Students who took the tests didn’t take them seriously.”

“As the superintendent, I couldn’t be more proud of the teachers, principals and central office team for what they do each day for kids in this district,” he said. “To have one test or one piece define who we are is absolutely ridiculous. I’m just proud of the entire district. And there’s no doubt in my mind that, that is taking place in all the schools in the county.”

Right on.

Quotes and other info here was taken from the following articles...

http://chronicle.northcoastnow.com/2016/01/14/county-superintendents-wary-of-state-evaluations/

http://chronicle.northcoastnow.com/2016/01/15/districts-question-ratings/

http://www.morningjournal.com/general-news/20160114/superintendents-ask-state-not-to-release-test-results

I decided to be a teacher so that I can spend my free time defending public schools, dodging verbal abuse, suffering uninformed criticism,disseminating information to the contrary, studying pending legislation, and lobbying legislators.

I get asked often why I became a teacher. To be fair, there are a lot of reasons. 

To be honest, it wasn't a decision I was equipped to make at 18 years old. I feel very fortunate that a decision that I made as an old child (more than a young man) has worked out as well as it has.

My favorite answer to the question is that, like a lot of kids, I didn't have an easy time with high school. It wasn't terribly problematic, but it wasn't a walk either. 

While I was there, though, I had the good fortune of encountering some teachers who treated me with interest and respect. I saw that it didn't matter to them if a student was different, didn't have money or wear the right clothes, play the right sports, or quite fit in. They also seemed to genuinely enjoy their jobs, to be having some fun with it. Yeah, some content you had to grind through, but that didn't mean you couldn't keep it entertaining along the way.

On a very basic level, these people were reasonable and decent enough human beings to make time for kids, and not seem put out by the effort.

So, in deciding my career I suppose that my thinking went something like, "Hey, I'd like to be a reasonable and decent human being. Those guys were good people who made me feel like I was good people. Kids have a hard time. I'm going to be the kind of teacher that makes school a more inclusive and fun place to be."

You'll notice the relative lack of depth in my process. I certainly didn't consider the fact that there were absolutely no jobs for history teachers. 

What I think is reflected here is what I valued most in my education, all academic advantages aside, which is the value of relationships. I'm sure that somewhere in there lies the importance of education to a democracy, and a desire to promote critical thought and individuality. However, I got involved in this business to help kids, and have some fun in the process.

This is a terribly simplistic view of a teacher's job. Don't get me wrong, I also understand the professionalism involved. If I took myself too seriously I would reference my degrees and years of experience. I might even provide a copy of the electronic documentation of my satisfaction of the state of Ohio's professional standards for teachers through the albatross that is the Ohio Teacher Evaluation System, the evidence, the artifacts, the pre and post tests, the all important data that supposedly proves my worth. But these things hardly demonstrate the value of a good teacher intent on providing an environment that facilitates the relationships necessary for a quality education.

Coming out of 2015, I feel as if I've spent an inordinate amount of time and energy in a political fistfight to defend the ability of public school teachers to do just that.

It's too much.

This is not why any of us got into education.

So I guess this is about a New Year's resolution, then. 

In 2016, I'm going to focus on being a decent and reasonable human being, accepting of the students I encounter, focused on making school a fun (or at least more tolerable) place to be, just like many of my teachers did for me.

I'd like those in legislative, and other power, positions to consider the same resolution from their own perspective. Strengthen the ability of public school teachers like myself to provide a quality education.

If 2016 is another year of excessive high-stakes testing, the demonization of teachers, unconstitutional school funding, the absurdity of championing a failing charter school system, furthering a process that facilitates the privatization of public schools, and then purposefully misleading the public (I'm looking at you Dick)...

Well, if that's the case, I'm not afraid of the fight.