Anecdotal Evidence is NOT Worthless

When I started this blog, I intended to cite objective stories and draw conclusions from those stories as rationally and dispassionately as possible. I knew I would be examining some controversial issues and I wanted very much to avoid inserting myself into the stories. I wanted to be an objective observer of education issues. I might use personal experiences as a springboard, but that would be all—just a story starter. I even thought about writing under a nom de plume in order to distance myself as much as possible.

Another reason I wanted to write from the perspective of an observer rather than a participant is because so many of my experiences contain poignant, sad, bitter or otherwise negative elements. I wanted to avoid being dismissed as just a disgruntled teacher. Our society does not understand that important positive change begins with disgruntlement. A third reason was to protect my privacy. I had been targeted for retaliation before.

It was difficult to write so coldly; my own experiences kept clamoring to intrude. It would take me a long time to write a blog entry what with the constant distraction of the memory movie going on in my head. I needed a couple of uninterrupted hours in order to concentrate properly, hours that were hard to come by. Consequently very little writing got done and too much time passed between entries.

I have many questions. My questions are prompted by my own experiences, but I already knew that experience is easily trumped by research. Critics will discredit “mere” anecdotal evidence from the gate. I decided to design research to examine the validity of the anecdotes. Were my experiences only flukes or were they a representative sample of the experiences of others?

When I tried to ask people, other teachers, administrators, whoever, they became defensive or evasive. I felt fairly isolated. Were these questions under some sort of taboo? Or was I truly the only one? I spent most of my thirty odd years of teaching in a kind of bewilderment. Maybe I just was not seeing the big picture.

The internet with the various online forums were a boon, coming along at a fortuitous time. I had been lurking around a number of forums since 2004. Frequently I came across news articles or comments that suggested that I was not alone. Any number of people are having the same puzzling experiences, but hey, most of it is still just “anecdotal.” I have decided that dismissing anecdotal evidence out of hand is a simple way of invalidating experience that might conflict with the story the information purveyors are seeking to build. In fact, sometimes forum moderators and the moderator's friends are so threatened they accuse anecdotal evidence of being fabrications.ⁱ

The research implementation was not going well. The research is problematic for a number of reasons. Most of what I want to study is not easily accessible; it is not at all certain that schools would be willing to divulge the kind of data I want. I do not have proper university affiliations and those with proper affiliations do not want to collaborate either because my research would challenge cherished preconceptions or because my research would not “further “their own areas of research. The obstacles to securing freelance research funding are nearly insurmountable. What entity is going to fund the research of a nobody who is asking difficult questions about the very foundations of our education system?

All of my questions can be summarized by one wide-ranging question: Do our unexamined assumptions have any basis in fact? For example, everyone says we need smaller class size so that the teacher can give more individual attention to each student. Do students really require individual attention to achieve? Japanese elementary students seem to do great in classes of forty-five. For example, student misbehavior is often interpreted to be a bid for attention. Is attention the real motivator or an assumed one? For example, education students often complain that most of their education professors have little real classroom experience. Is this true? What would constitute “real classroom experience”? I want to tackle some sacred cows: special ed, bilingual education, self-esteem, standards.

I generally like the National Council of Teachers of Mathematics (NCTM), but once I challenged them to provide the research they claim supports their recommendation for calculators. First I checked their website, nothing. After an hour on the phone, they still could provide nothing. Nothing, and this a major topic of the “math wars.” I checked their website six months later certain they had remedied their earlier oversight. Still nothing. So I called—nothing, but they continued to assert research backing for their position.ⁱⁱ Fine, I thought, I'll gather the research myself. I gathered every bit of research on calculator use that had ever been done. I wrote a forty-page paper categorizing and summarizing the research. Briefly, the research does not support the recommendation. I sent the manuscript to the NCTM journal. One of the peer reviewers rejected the manuscript on the grounds that NCTM had never made any such recommendation. But there it was on page 79 of the NCTM publication Principles and Standards for School Mathematics (PSSM), with the added caveat that instruction with calculators should proceed “under the guidance of skilled teachers.”ⁱⁱⁱ Why would the peer reviewer use such a bald-faced falsehood to reject the manuscript? What hot button had I touched? Why were teachers still being told the research favors calculator use in the primary grades? What was really going on?

I have so many more questions about so many more issues and what I conclude from reviewing the research literature is that the answers are simply not there. So I am going to change the style of the blog and fill it with anecdotal vignettes. Perhaps if enough anecdotal evidence can be collected, a case can be made for undertaking some serious research on the very foundations of the American education system instead of the trivial stuff that makes up the dissertations of too many PhD's and fills too many journals.

i. Charging fabrication is a common ruse. “...a writer argued that I had simply fabricated Rule 240 out of thin air.”

ii. I checked the NCTM website just recently and found a large number of references under the search term “calculator use.” Their current online version of Principles and Standards also contains a number of reasonable, qualified statements regarding calculator use.

iii I can no longer find the specific sentence in the online version of PSSM.