Math Gaps

Three weeks ago educators wondered if conclusions from the Response to Intervention (RTI) research might be applicable to mathematics.

Educators gathered here last week to discuss a recent federal “practice guide” on response to intervention for students struggling in mathematics agreed that applying the RTI approach to that subject is challenging. But they also suggested that doing so was worth the effort.

It is instructive to discover the reasons applying the RTI approach to math is challenging.

Response to Intervention involves six steps: Screen, Teach, Intervene, Probe, Chart and Adjust.

Screen
Valid screening measures predict who is, and who is not, at risk for future reading difficulty. These measures are administered to determine if a child is at risk for failing a state's "high stakes" end of year achievement test, by which the state measures a school's overall performance. Children considered to be "at risk" are expected to experience difficulty responding (not keeping up) in the core curriculum as traditionally delivered in the regular general education classroom. Note: Due to the desire to capture all children who are truly "at risk," the false positive rate of early screening may be as high as 50 percent. In other words, as many as half of all the children who are identified as "at risk" by early screening may not be truly "at risk."
Teach
Core curriculum in the regular general education class should be research-based and field tested. This means, based on evidence from converging research, that the core curriculum contains all the elements found necessary to effectively teach reading and has a known track record of success. Such curriculum is to be delivered by "highly qualified" teachers sufficiently trained to deliver the selected instruction as intended, i.e., with fidelity to design. My note: Notice the language “trained to deliver the selected instruction as intended.” I knew one highly competent first-grade teacher in California that refused to deliver the selected instruction (whole language) as intended. It was a good thing because her students ALL learned to read even as California fell to 49th place in reading during the whole language period. The fad lasted until 1995 when phonics was reinstated in the curriculum. The change in the role of the teacher as indicated by the language I have noted is problematic.
Intervene
Provide "at risk" children with enhanced opportunities to learn, possibly including, but not limited to, additional time exposed to the core curriculum in small groups (3-6 students), other supplementary instruction, or special education.
Probe (progress monitoring)
Progress monitoring tests are brief measures of specific reading skills that are administered to determine if the child receiving intervention is responding as intended. They are given frequently, at least once every two weeks.
Chart
Progress is regularly charted to provide a visual record of actual rate of gain in specific reading skills in relation to a specified goal. The goal of intervention is for the child to improve relative standing and perform at or closer to grade level standards and is individualized according to the unique needs of the child.
Adjust
Depending on whether the child is achieving a rate of progress determined by his or her individualized goal, the manner and intensity of intervention will be adjusted. The cycle of progress-monitoring and adjustment of intervention will continue, even if a determination for special education eligibility is made.

Math educators met June 10, 2009 to explore whether the same six steps would be just as effective in math as in reading.

Educators at last week’s event said that fitting math into an RTI framework is hard, but that they believe it is now vital to improving math performance for struggling students.

One of the main difficulties is finding suitable math education materials.

Judith Russ, the mathematics curriculum supervisor for the 134,000-student Prince George’s County district in Maryland, said for her part that finding the right materials is hard.
The instructional materials “are not looking at building conceptual understanding. That’s one of the challenges we have,” she said.

The first step is to screen.

Karen D. Cheser, the assistant superintendent for learning support services for the 20,000-student Boone County district in Florence, Ky., said her school system started using RTI in reading two years ago, and had initially planned to leave math for later. But indications that students were becoming weaker as they reached higher-level math classes, among other factors, pushed the district to act.

Ms. Cheser said the district created its own universal screening program, which allows teachers to dig into what was going wrong for many students. It turned out that many students needed to focus so hard on computation that they were unable to grasp more sophisticated concepts, she said.

The panel found that children's gaps in mathematical understanding were fairly predictable.

... remediation for students in grades K-5 should focus on the properties of whole numbers, like counting, addition, and subtraction. Older students, up to 8th grade, should learn rational numbers in depth, including the meanings of ratios, decimals, and percentages, the panel recommends.

Another recommendation is that all students who need extra math assistance should work on fluent retrieval of basic arithmetic facts, like simple addition and multiplication. Higher-level mathematics often assumes that students can quickly recall facts like “3 times 9” or “11 minus 7,” when such operations may be difficult for those lagging behind their peers, the panel found.

The current reliance on the calculator does nothing to promote fluency with math facts. In fact, some teachers say that, with the ubiquitousness of calculators, it is no longer necessary to memorize math facts. But I know that students who cannot readily retrieve math facts struggle with algebra, and the research is confirming my observations.

Mathematics instruction has emphasized procedural competence over conceptual understanding. You do not necessarily have to understand the mathematics underlying long division as long as you can perform the operation. Sadly, students who perform mathematical operations reliably are told (through test grades) that they understand math when the reality is that they may have no idea why they do what they do. They do not “understand” math. The math materials in our schools “are not looking at building conceptual understanding,” as Judith Russ, mathematics curriculum supervisor, noted. But at least in times past, students were expected to memorize math facts in order to complete the procedural operations. So the situation is that students are not understanding math concepts, nor do they have the basic tools, math facts, for mechanically solving problems.

Years ago we used to call screening “diagnosis” and the best teachers have always made diagnosis part of their teaching practice. I used to diagnose struggling algebra students one-by-one. I still diagnose one-by-one, but nearly always I find the same gaps. They do not understand the function of place value (even if they can name a digit's place). They do not understand whole number properties (and fail to apply properties to numbers that do not look like “numbers,” especially numbers containing variables. They do not understand the difference between one and zero (such as when the “cancel” and say something like, “That's zippo.” They certainly do not understand fractions. Once I address these four abysses of knowledge, algebra suddenly becomes straightforward and even beautiful.

Is Quality Education a Lost Cause?

I love reading old stuff. Did some expert pundit's analysis and future projections come to pass or not. A quarter century ago, there were lots of articles wondering what we would do with all our future free time. The answer turned out to be---work even more. A quarter century ago, lots of articles wondered how America would solve its looming math and science teacher shortage. The answer turned out to be ---not much. Many universities are importing math and science professors from other countries because America is not producing its own. So it was interesting to reread an old article from Time entitled "Help! Teacher Can't Teach" dated Monday, Jun. 16, 1980, nearly thirty years ago. It could have been written yesterday.

Like some vast jury gradually and reluctantly arriving at a verdict, politicians, educators and especially millions of parents have come to believe that the U.S. public schools are in parlous trouble. ..Experts confirm that students today get at least 25% more As and Bs than they did 15 years ago, but know less.

Society holds the teachers responsible.

the new complaints about teachering also arise from a dismaying discovery: quite a few teachers (estimates range up to 20%) simply have not mastered the basic skills in reading, writing and arithmetic that they are supposed to teach.

Even as criticism abounds, Time (and all of us) recognize that 20% does not equal 100%.

Of course, among the 2.2 million teachers in the nation's public schools are hundreds of thousands of skilled and dedicated people who, despite immense problems, manage to produce the miraculous blend of care and discipline, energy, learning and imagination that good teaching requires. ...The best-educated and most selfless teachers are highly critical and deeply concerned about the decline in teaching standards and educational procedures. Their frustration is perhaps the strongest warning signal of all.

Testing tends to be the first line of defense. Many states began mandating teacher competency tests, only to find that far too many practicing teachers were unable to pass these tests. Lest one should think that teacher competency tests were perhaps too hard, most required math typically taught between the eighth and tenth grades, and English at corresponding levels. Any teacher, presumably all college graduates, should be able to pass easily. But they do not.

I was astonished to be the first one done with the California Basic Educational Skills Test (CBEST). I put that test away in less than two hours. It was supposed to be a four-hour test. I once took the National Teachers Examination (NTE) in early childhood education cold, no study, review or preparation of any kind. I had been a secondary teacher for many years. I scored at the 86th percentile. I was not happy. My score was too good for someone like me who had taken a test outside of my field. According to the normative data on my score report, the vast majority of test takers were graduates of early childhood education programs. I did better than 86% of them. Not good.

Education Secretary Arne Duncan's support of merit pay for teachers, though framed as a way to pay teachers more, is really just a disguised way of saying if teachers taught better, our schools would be better and maybe more money would motivate teachers to teach better. One of the problems of merit pay is the unstated assumption that teachers are not already doing their best.

Okay, let's face the issue of teacher quality head on.

1.Teaching credentials are no assurance of teacher quality.
2.Schools do not hire the best qualified candidates, but the cheapest.
3.School of education attract students of lower academic ability than other academic departments.
4.Graduates of colleges of education must often take basic teacher competency tests many times before they pass.
5.Math teachers often do not possess a profound understanding of fundamental mathematics.

Some schools, like The Equity Project charter school in New York*, is determined to acquire high-quality teachers. They are offering $125,000 per year and the application process is a grueling four-step process. The charter school has created such a grueling process because the usual documentation, university degree with or without a state teaching credential, is worthless.

It did not used to be like this.

In 1900, when only 6% of U.S. children graduated from high school, secondary school teachers were looked up to as scholars of considerable learning.

Things were going swimmingly as high schools graduation rates steadily improved to a high of 70% by the 1960's. Sputnik was a huge surprise in 1957.

Almost overnight, it was perceived that American training was not competitive with that of the U.S.S.R. Public criticism and government funds began to converge on U.S. schools. By 1964, achievement scores in math and reading had risen to an alltime high.

Let's repeat that: Public criticism and government funds began to converge on US schools. Though only a child, I remember that time well. Society did not simply complain and moan; society demanded action and the government responded. The result, which directly benefited me, was that by 1964 achievement scores in math and reading had risen to an alltime high. Only genuine achievement would do because society had a stake in knowing accurately if education was working. There was no interest in the statistical juggling so common now. Want SAT score improvement? In 1995, the College Board simply added 100 points** to everyone's score.

Over the last thirty years, there has been plenty of societal moaning and complaining, but no demand, no collective will. So society had the education system it wants. What did now Research Professor of Education at New York University Diane Ravitch say thirty years ago?

Diane Ravitch: "It is really putting things backward to say that if children feel good about themselves, then they will achieve. Instead, if children are learning and achieving, then they feel good about themselves."

Colleges of education are still teaching a backwards concept of self-esteem.

Although the motivation was not the noblest, beat the Soviets to the moon, my generation was the last beneficiary of America's once legendary education system.

Ever since the mid-1960s, the average achievement of high school graduates has gone steadily downhill.

...snip...

Many teachers have come to see themselves as casualties in a losing battle for learning and order in an indulgent age. Society does not support them, though it expects them to compensate in the classroom for racial prejudice, economic inequality and parental indifference.

In 1957 it was Sputnik. What will it take today for society to set aside complacency, ideological wrangling, or perpetuation of social status quo?


*The Equity Project's 4-stage application process

**The purpose of recalibrating scores was to realign the scores to the population so that a score of 500 would once again be average. According to the New York Times article,

The average verbal score today is 424; the average math score, 478.
So the College Board officials have decided to "recenter" the scale, changing it so the average student will once again get scores of 500 on the verbal and math tests...
In 1941, when the current norms were established for scoring the S.A.T., the world was a very different place. A small group of middle- and upper-class Americans attended college...

As colleges diversified in the 1960's, opening their doors to more poor and first-generation Americans, S.A.T. scores began a steady drop. By 1969, the average verbal score was 462; today, it is 424.

Teachers are Widgets

So President Obama wants to get rid of the bad teachers.

"We need to make sure our students have the teacher they need to be successful. That means states and school districts taking steps to move bad teachers out of the classroom. Let me be clear: if a teacher is given a chance but still does not improve, there is no excuse for that person to continue teaching. I reject a system that rewards failure and protects a person from its consequences. The stakes are too high," Obama said.

Nobody wants bad teachers. But exactly who are the bad teachers? How do we go about identifying them? Oh, I know. Let's take a look at the teacher evaluations. Surely the evaluations, many of them professionally designed, will point out the bad teachers. After the bad teachers are identified, all that remains is the battle with the teacher's union, whose mission is to make sure bad teachers teach until retirement. (snark alert).

Every school I have ever seen has a program (at least in the employee handbook) for annual evaluations. In my career, I have been evaluated five times by supervisors (principal, vice principal, or department chair). And something else everyone knows, the evaluations are worthless. Now a new study by the New Teacher Project confirms what everyone knows. Entitled “The Widget Effect,” the study show that teachers are fungible.

The study illustrates that teacher evaluation systems reflect and codify the “Widget Effect”—the fallacy that all teachers are essentially interchangeable—in several major ways:

All teachers are rated good or great. Less than 1 percent of teachers receive unsatisfactory ratings, even in schools where students fail to meet basic academic standards, year after year.

Excellence goes unrecognized. When excellent ratings are the norm, truly exceptional teachers cannot be formally identified. Nor can they be compensated, promoted or retained.

Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation.

Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process.

Poor performance goes unaddressed. Half of the districts studied have not dismissed a single tenured teacher for poor performance in the past five years.

The worthlessness of evaluations creates a major disconnect in the school policy.

Though it is widely accepted that a teacher’s effectiveness matters more than any other school factor in student success or failure, it is almost never considered in critical decisions such as how teachers are hired, developed or retained.

Teacher effectiveness cannot be considered because teacher effectiveness is unknown. What's more, researchers have no consensus as to the characteristics of an effective teacher.
I would like to address the first two points.

All teachers are rated good or great. And because all teachers are good or great, Excellence goes unrecognized.
At best, evaluations are worthless. In many schools, an evaluation is a pro forma process, if it happens at all. The busy administrator visits the class for a few minutes, walks out and writes the glowing report.

At worst, the evaluation is a retaliatory or evidence-fabrication tool. I am reminded of the young elementary art teacher whose reputation for excellence was well-known by staff, and parents alike. Teachers dropping off their class at her classroom often lingered and teachers retrieving their class often came early to observe and hopefully glean some useful tips. One fine April morning the vice principal came to observe a class period and stayed for the whole class. His one comment after the class left: he did not like that the students were allowed to chat with their neighbors as they worked.

The resulting evaluation was a disaster. On a 5-point scale, her average came to 2.7. She objected to the principal and he allowed her to write a rebuttal. But the rebuttal went nowhere. All that survived of the evaluation was the average which appeared on a list of all the teachers with all their 4.X averages. The school submitted the list to the district office.

She complained bitterly to the principal who told her not to worry—it would have no effect on her future career. She complained to her colleagues, some of whom interceded for her with the principal. You see, this young teacher had rebuffed the vice principal's advances at the school Christmas party. Her colleagues suggested the principal replace the vice principal's evaluation with one of his own, but he refused, saying it would unseemly to override the vice-principal.

This story is not a fluke. Evaluations, if done at all, are often undertaken only because the teacher has entered the administrator's radar for some reason. In such situations, greatness cannot help but go unrecognized. I would go so far as to say that Teachers of the Year are not necessarily the top teachers. They are teachers with time. Many of the best teachers are simply too busy to fulfill the onerous essay and video requirements to be considered for a Teacher of the Year award.

What, you say. You thought Teachers of the Year were nominated for doing their jobs every day. Most Teachers of the Year are self-nominated. Typical is the application for the Arizona Teacher of the Year.

Teacher nominees/applicants must submit a written application that is reviewed by a panel of judges consisting of educators, students and members of the business community. Ten finalists are selected from the written applications.

The 10 finalists are asked to prepare a 15-minute videotape. The final selection process includes review of the videotape, an interview and an impromptu speech by each of the 10 finalists. Following that process, the Teacher of the Year is selected along with four “Ambassadors for Excellence” and five finalists. The Teacher of the Year and Ambassadors have multiple opportunities during the year to make public appearances throughout the state, speaking to professional, civic, educational, parent and student groups. (my bold)

The written application includes 13 double-spaced pages of essay material:

Educational History and Professional Development Activities (2 double-spaced typed pages) – 5 points
Professional Biography (2 double-spaced typed pages) – 5 points
Community Involvement (1 double-spaced typed page) – 5 points
Philosophy of Teaching (2 double-spaced typed pages) – 15 points.
Education Issues and Trends (2 double-spaced typed pages) – 15 points
The Teaching Profession (2 double-spaced typed pages) – 20 points
National Teacher of the Year Message (1 double-spaced typed page) – 10 points
Arizona Teacher of the Year Message (1 double-spaced typed page) – 10 points

The school administrator must agree in writing to approve up to 30 days for a substitute teacher to allow for newly-minted teacher of the year public appearances.


You may be the greatest teacher in the world, but if you are, and you are honest, you are very likely not going to write essays that will get you selected as teacher of the year, especially when you write about “education issues and trends” and “the teaching profession.” John Taylor Gatto was voted New York's Teacher of the Year in 1991 and immediately, with his acceptance speech, began telling everyone about the insidious goal of compulsory education to de-educate students. He has written several books, all with the same message. Somehow I have trouble believing the message he proclaimed from the Teacher of the Year platform was the same message he told the committee he would proclaim when he wrote his application essays.

Neither the complicated, multifaceted, self-selected Teacher of the Year evaluations nor the run-of-the-mill annual evaluations performed (or not) in most schools succeed in any meaningful way.

Good Question about Math Education

How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

International comparison studies typically focus on the comparing the scores achieved by same-age students in different countries. Also typically, students from Asian countries tend to outperform US students over and over again. Each time a report like that comes out, just as predictably there will be an out-pouring of the same old tired excuses. Their students are different from our students. Their culture is homogeneous whereas ours is diverse. Their schools are allowed to teach whereas our schools must meet social, medical and nutritional needs. Their parents value education whereas our parents, not so much. On and on. The excuses act as a sedative to put society back to sleep. Okay, society says, there are understandable reasons for the differences in performance. The results are not really comparable. Apples to oranges. What a relief. So we stop thinking about it.

Could there be something more?

Sean Cavanaugh of Edweek
reports:

A host of recent studies have examined how U.S. students’ mathematics skills compare against those of their foreign peers. Now, a new analysis probes a more precise question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

Let's repeat the question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer? It does not matter whether the results are comparable or not. No matter the reason our kids come out second rate, other kids are beating our kids in the worldwide competition. Remember, President Obama said that if we want our kids to out-compete the world, we must out-educate them.

So how does the math covered stack up?

A study released last week finds that elementary students in Hong Kong are exposed to more difficult and complex math than pupils in Massachusetts, an elite scorer on national and international exams. The analysis, published by the American Institutes for Research, in Washington, examines the math content of Hong Kong and Massachusetts by comparing the two jurisdictions’ standardized tests in 3rd grade math.

We're talking about third grade, part of the foundation of the rest of a child's academic career. The study did not look at scores on a specially designed test for international comparison purposes. The study did not look at the content of such a specially designed test. The study examined jurisdiction's in-house test, the standardized test for Massachusetts and Hong Kong. Even more interesting, the study had no interest in the children's scores on these tests. The study studied the tests themselves. And why Massachusetts?

Massachusetts is also a consistent elite-scorer on the primary U.S. domestic test, the National Assessment of Educational Progress.

What the study found is that the Hong Kong test emphasizes number and measurement concepts. The test also contains a larger percentage of constructed responses rather than chosen responses. The Hong Kong test questions were more complex, requiring the application of knowledge and non-routine, multi-step solutions over simple recall. From the foundations, children in Hong Kong are tested on higher-order thinking skills than American children, even “elite” American children.

Do Chinese teachers teach to the test?

(Steven Leinwand, one of the study's authors), said the authors chose to examine test content in Hong Kong and Massachusetts because the two jurisdictions' early-grades math curricula were relatively similar—and because state tests in the United States tend to guide math instruction.

American educators “pay attention to the tests,” he observed. “If you change the state tests, it’s a powerful lever for what goes on in the classroom.”

In the US, the favorite quick and dirty way to reform education is to redesign the tests. That's what Arizona did in the 1990's with their AIMS test. Arizona created high-stakes tests for fifth, eighth and eleventh grade, as if new tests automatically change educational philosophy and encourage innovation. Even honor students flunked these tests. The overwhelming response to high-stakes tests is to teach to the test, a response well-documented by No Child Left Behind. When a test reflects existing educational philosophy, there is no need for sample tests or practice materials.


Liping Ma has documented the emphasis Chinese teachers place on concept development over computational procedures. James Stigler reiterated many of the same points. Chinese math education, exemplified by Hong Kong, already valued conceptual understanding and the test reflects that value. The US, regardless of all the pretty talk in the media, values computational procedures and the Massachusetts test reflects that value.

How did Mr. Leinwand put it? “... state tests in the United States tend to guide math instruction.” That's a large part of the problem. We are suppose to test what we teach, not teach what we test. The US mistakenly thinks testing drives instruction.

The Uncomfortable Conclusion

Laying solid foundations in the early years matters.

Hong Kong’s use of more difficult and complex test items could be connected to a higher proportion of its test-takers, 40 percent, scoring at the “advanced” TIMSS level, than Massachusetts, at 22 percent. Just 10 percent of American students, on the whole, reached that level, the authors argue. In addition, research shows a “strong correlation” between nations’ math performance in early and later grades, they say.