What VAM Can Never Measure?

It does not matter how many papers are written discrediting VAM. If it was possible to shove the American Statistical Association Statement on Using Value-Added Models for Educational Assessment in front of Andrew Cuomo’s face, it would make no difference to him. He would toss the paper into his circular file because reason has nothing to do with his so-called reform. VAM is not a rationale, but a belief to the reformers. The basis of their belief has nothing to do with mathematics, but everything to do with the tall tales the reformers give as anecdotal evidence to justify their beliefs.

This tall tale, often told by Teach for America types go something like this. After five weeks of training, this natural born teacher who just graduated from Harvard comes to work in a high needs school and immediately he is able to motivate every student in his 7th grade class located in the most poverty stricken area of Chicago. Each lesson captures the imagination of every student in the classroom. These highly inspirational lessons are differentiated toward every student in his class. He “teaches like a champion” as he was taught in those five magical weeks. He breaks down every academic behavior happening in the classroom. His students never knew that they have to face the teacher, give him direct eye contact, and have their feet planted squarely on floor. For the first time someone told them that they must have pencil in hand ready to write. Not only that, he is at school at 7:00 AM in the morning working with students one-on-one to catch them up to grade level. He tutors individual kids during his prep, during his lunch, and after school until nine at night. On weekends, he spends Saturday and Sunday at a local library working with even more of students. When April comes around, this class now has 100% of his students at level three or four on the Common Core ELA and Math Assessments. Just think, the previous year, when these students had that lazy burned-out unionized teacher who came to school at 8:40 and left exactly at 3:00, only 4% of these students even reached a level two on the assessment. Therefore, this “superman” teacher is rated using VAM as highly effective while that shriveled up union hack next door is deservedly ineffective and must be fired. Once every teacher in America is just like this Harvard wunderkind, every student will be on grade level headed toward college. There will no longer be any poverty in this great nation.

Disney could not have come up with a better fairy tale. All we need is for this teacher to sing a happy tune and a dozen Chicago pigeons will fly through his classroom window and tidy up his classroom. No mention is made of hunger, poor health, drug abuse, neglect, violence and homelessness that such students face every day. No mention is made of the lack of books, pencils, paper, or even desks and chairs found in such schools. No mention is made of broken lights, peeling paint and rodent droppings in these classrooms. No mention is made of about the lack of support and even terror initiated by many administrators of such schools toward new teachers. No mention is made that the only piece of technology these classrooms have is maybe a single outdated computer with intermittent internet access. No mentioned is made that even the most determined teacher will burn-out working 80 hours with no social or family life. No mention is made that this teacher’s meager salary cannot afford the price of a city studio apartment, food, transportation as well as teaching supplies for himself and his students. No mention is made that his salary will not be enough to survive and he must moonlight a second or even a third job to make ends meet while, at the same time, producing pages of lesson plans and taking additional college courses. Can VAM measure the stress and exhaustion of a working teacher trying to meet the demand that every child in his/her class grow academically against such odds?

But the main thing that VAM can never measure is what is within a good teacher’s heart. Only another anecdote can describe the heart of real teacher that no algorithm can compute. In the year 1965, I was ten-year-old fifth grader at PS 186 in Bensonhurst, Brooklyn. I was not a great student. I struggled with reading because at that time the city used what was called the look-say approach to reading instruction using those old Scott Foresman basal reading series (commonly referred to as Dick and Jane). This approach was a precursor to what would someday be called whole language. I often had difficulty pronouncing words and therefore I was not a fluent reader. My parents were concerned and I remember sitting in the principal’s office with them. His name was Mr. Gladstone and he was the epitome of the old fashioned male principal. He was tall, wore a suit, and had distinguished looking gray hair, but had a kind face. He had me read for him and my parents. I do not recall exactly what was said, but as a result of the meeting, my class was changed. My new teacher was Miss Burke. She was this older Irish woman who always wore plaid skirts and high button blouses. To me, this teacher was Mary Poppins, Maria Von Tramp, and Cinderella’s fairy godmother combined into one living, breathing person who did change my life—not by magic but through caring, determination and love.

In her classroom, no one was allowed to make fun of any student who had a learning problem. She taught her students to help one another. When we read silently a book of our choosing, she always came over to me and had me read very quietly to her. I did not realize it at the time, but she was teaching me a host of strategies that helped me to become a fluent reader. I would not realize until I became a teacher myself that she was giving me phonetic and word analysis tools to improve my fluency. I recall that each time I read a page without error, she would have a big smile on her face and say “good job.”

During that era, each class was required to put on a play. She had each student read some lines of script and then said that I had the most expression and asked the class if I should have the main part. To my surprise, the whole class agreed with her. That was the first time in my school life that a teacher and my classmates showed confidence in me. However, I was scared to death and when I got home I cried to my parents that I could not do it. My mother called her at the school the next day and told her of my fears. I was afraid that I could not remember all the lines; I was afraid of making mistakes; and I was very much afraid of making a fool of myself in that giant auditorium in front of every student, parent and teacher. That evening, Miss Burke came to my house and spoke to my parents and me. She and my parents came up with a plan how I would learn my lines and practice a little bit every day. With her encouragement, I did it. I performed the main role of silly play about eating the right type of foods. I recall that I had to perform not only in the morning, but also again in the afternoon for another group of students. At that time, many of us went home for lunch, but I recall Miss Burke saying to my mom that I should have lunch with her (she was afraid I would not come back).

Whereas today, the common core teaches fifth grade students to compare the structures of drama, poetry and prose, we lived it. I learned how hard it is to put together a play with scenery, cuing for each stage direction and the details of choreographing a single dance. I learned how to stand, project my voice and even walk on a stage. Today, students learn that scripts have italicized stage directions, but I learned why and how each of those directions was important.

Miss Burke also ran the school’s chorus (glee club in 1960s jargon) with another teacher. I auditioned and soon found myself learning a medley of songs from Mary Poppins. We worked hours memorizing those songs, learning how to breathe and how to perform on cue using various hand signals. I remember that our chorus was chosen to perform in Lafayette High School. I was amazed that we were bathed in light while the audience was in total darkness. That was fine with me because it made me less nervous. To this day, I can still sing Supercalafragalisticexpialadoshus and Chim Chimney.

In addition, at that time, the Brooklyn Museum had an orchestra. This orchestra taught school children music appreciation. Every week, the fifth grade classes of our school went to the museum and the conductor and his orchestra taught us about all the different types of musical instruments and the role each instrument played in a concert. The week we were learning about woodwinds, I remember him calling me to the stage to try to blow through a tuba. No matter how hard I tried, I could not produce a single note. At the end of four weeks, all those instruments came together and played for us all different types of classical music. That was my first introduction to Beethoven and Mozart. I would learn the complexity of such music.

During the year, we went on a trip to the New York World’s Fair in which Miss Burke would explain all the different exhibits from many different cultures. Before we entered the Vatican exhibit, she explained to us the whole history behind the Pieta. For the first time, I was introduced to the Renaissance and Michelangelo. At the end of the year, because the fifth grade was the graduating class, we took a trip to Philadelphia to Independence Hall, the Franklin Museum and Betsy Ross’ House. What I remember most form that trip was sticking my head inside the Liberty Bell and sitting, in Independence Hall, at the desk of Thomas Jefferson. That motivated me to begin reading everything I could about our third president.

In terms of learning, I improved my writing because Miss Burke got each of us a penpal from Europe. I ended up writing to a young girl in Czechoslovakia. My letters got longer and more descriptive as the year progressed as I tried to tell her everything that we American children did throughout the day. She even had us write to an author of a book we read. I wrote to the author of a children’s biography of Eleanor Roosevelt. I was so proud when he wrote back to me and I have that letter to this very day. By the way, I still have that old play script in an attic box. A few months ago when we were cleaning out the attic, this sixty-year-old man refused to part with it.

That script is more than a bunch of faded rexograph papers. I cannot part with it because it represents the richest educational experience I ever had. It is amazing that all this was done in an average New York City public school by a teacher who taught with love and care. Miss Burke taught in a very traditional manner. To tell the truth, I cannot recall or even visualize a single lesson, but what I do recall is the warmth every time she spoke to either me or the class. I recall her smile and her pat on my back every time I did a good job. Can VAM measure any of this? Can VAM measure the love this teacher showed me and the other students of that class? Can it measure all the wonderful experiences this woman gave to me and my classmates that year? If you notice, I have not mentioned a single test. I do not recall taking any type of formal standardized test that year. Instead of hours of test prep, I had real learning, great learning. I learned through experiences, through song, through dance, through art and through purposeful writing. After that year, I no longer had any significant academic difficulties. What this teacher did for me is really how one is made college ready. A child is made college ready when you instill in them curiosity and a love of learning. Hours of high stakes testing and days of test prep create just the opposite—a hatred of learning. As I am writing this, my eyes are very moist. My tears represent the happy memories instilled by Miss Burke as well as tears of sadness for what has been lost—the true magic of teaching. Teaching is a human act, a complex act that cannot be measured by any algorithm. The passion and complexity of a human interaction cannot be measured by a single snapshot. When reformers describe teaching, it is nothing more than a mechanical act. One cannot measure something that comes from your heart and soul. One cannot measure an act of love, for that is what teaching really is.

Reflection after Three Months into Retirement

I hate to be asked how I am enjoying retirement.   People who ask this question have created a fantasy world of what it means to be retired.   After listening to my friends, family, former colleagues as well as perfect strangers, I have synthesized all their dreams, hopes and aspirations by creating a make believe world they call Retirement Heaven.   In Retirement Heaven, you can wake up late every morning—no earlier than 11:00 AM.  You never ever set your alarm clock again.   Upon waking up, your spouse will deliver to you breakfast in bed that is no less than 2000 calories.  After breakfast, it is time for a relaxing shower.  After a shower, you put on shorts, a t-shirt and sneakers for either your daily run or your two hour workout at the gym.  By the way, in Retirement Heaven, it is always summer. At about 2:00 PM, you make your way home for lunch and a few hours of television.   You’ve recorded on your DVD only mindless television programs to watch for the next few hours.  Now it is time for dinner.  In retirement, you are not allowed to make dinner, but instead, every evening you go to a different restaurant to eat meals that have no fewer than three courses.  You return home, take out your trashy novel and read until you drift calmly to sleep at about one in the morning.

For someone like me, such a retirement heaven would be like that old Twilight Zone episode where a gangster dies and is given every one of his heart’s desires.   He is given wealth, beautiful women and the ability to successfully commit every crime known to man without consequence.   In the end, he is completely bored out of his wits and asks to go to the other place only to be told he is in the other place!

To me, retirement means having the freedom to choose the type of work that I want to do.  It is also the ability to focus on those endeavors that are really important. On the day that I retired from teaching, unfortunately all the problems that existed before my last day of work really did not disappear.  Several people said to me you shouldn’t care what happens to the world of education.  One person told me that if all the public schools blew up tomorrow, you will still be retired on your nice pension.  Would I?

First of all, if public education ended tomorrow and was completely privatized, it would make it all the more easier for those in power to end pubic pensions.   Public education and my pension, which I earned through hours of hard work as a teacher, both represent a social contract. It is a social contract between the government and the people it represents.  If you end one, you can easily end the other.   The philosophical basis of public education derives from those Enlightenment thinkers who believed  that power derive from the governed.  The governed allows power to be given to a government so that it can establish policies that will benefit the collective.  The 18^th century Philosophes believed that public education’s purpose was to create a sophisticated citizenry who would be able to immediately spot tyranny and remove it from power—either through constitutional democratic means or by force if necessary.   The concept of public pensions also has its basis in the 18^th century idea of a social contract.   Public servants will serve society for a number of years at a lower rate of compensation so that the state can benefit from their skills.  In return, the public servant will receive a pension paid at the end of their public service so they can live the rest of their life in dignity.   But the purpose of a pension also is based on sound economic and social principles.   Instead of the retired worker being a drain on society, the retired worker will have enough income to continue to purchase goods and services (providing employment to others) and still be able to contribute to society (and the economy) in other ways, such as through volunteering or part-time employment.

Therefore, in retirement, I now have taken on a new job—saving public education.  One way is by working part time in the school where a spent the last third of my career.   For the majority of my 36 year career, I had a job that came into existence because New York City grossly violated the rights of disabled students back in the late 1970s.  Federal law mandates that disabled students need to be evaluated every three years and if there has to be a change in services, any reevaluation must be done in a timely manner.   Thus, in the late 70s, the city was sued because it had a backlog of close to 30,000 reevaluations. It resulted in the court ordering the creation of a team of professionals in each school whose job it was to make sure assessments and subsequent IEP conferences were held in a timely manner.  My job was as case manager and Educational Evaluator.   I became highly skill in conducting norm-referenced, criterion referenced and curriculum-based assessments.  The purpose of these assessments was not to punish teachers but to diagnose the educational needs of disabled students so as to help teachers meet their IEP goals. (Yes, this is the real purpose of testing.) A quarter of a century later the city was in relative compliance.   Because most city schools have relatively large populations, the team needed a psychologist to conduct intelligence and projective tests, an educator who was a skilled diagnostician and a social worker.  This ended in 2003 when Mr. Bloomberg embarked on reforming special education in New York City, which really meant he was trying to find a way to save money.   He made my job disappear overnight and gave the case management piece and educational piece to the school psychologists that now are so overworked, they are forced to cut corners in order to remain in compliance.   As for me, because Bloomberg violated so many of the contractual rights of Educational Evaluators (who were teachers), he was ordered by an arbitrator to create the job of IEP teacher.  I held this job for the final eleven years of my career. When the job was created by  the arbitrator, it was undefined. It was up to a building principal to define what IEP teachers would do. Bloomberg hoped that most principals would make the lives of these new IEP teachers so miserable that many would leave the system.   The opposite happened.   Most of us ended up doing many different albeit necessary educational tasks within our schools.  As for me, I became in charge of compliance, testing, data, and academic intervention services.

When I retired, the new principal would ask who did this and who did that.  My name was mentioned each time and before I knew it, I was back in the school a couple of days a week.  However, I am doing what I loved doing the most—working with kids.   Unfortunately, the city schools have a simple philosophy—the minimum is the maximum.  Because the union contract stated that IEP Teachers would only be funded for those who were former Educational Evaluators like me, once I retired, the money dried up and the position disappeared.   It disappeared even though the school would now have no one to provide state mandated academic intervention services for those students who received level 1 on those wonderful Common Core State Assessments in ELA and Math.  As for all my other jobs, I am helping to train three other professionals to do different pieces of my job.   When I started as an IEP Teacher in 2003, testing was just a little piece of my job, but as we all know, it turned into a monster with three head and twenty arms.  It took up so much of my time that I often could not work with students.   Now I am trying to give students the skills to do better on these assessments (notice that I did not say pass).   For example, it is not enough to say to a level one student that they need to use context clues.   What I do is to try to give them four or five strategies to help them try to figure out the meanings of so many unknown words on passages that are always above their grade level.  Yes, I hate the test, but I have to do something.  Many of these students are former English Language Learners who supposedly reached proficiency in English on a state assessment measuring second language ability.  By the way, I am in one of the few middle class school districts within the city.   However, we have many immigrant families.  The parents of these children work two, sometimes three jobs so they can live in a nice area.   However, because our scores  are a little better than the state average and we have fewer students on public assistance, we get less money than other schools around the city.   And knowing this, the city cut funds to hire just one academic intervention services teacher.

In addition to working part time in my school, I am also tutoring and I am collaborating with someone in writing a review text to help students try to pass these horrible state assessments.  Some might say that I am hypocritical trying to help students pass these assessments when they should be done away with.   For years, I have tutored students to pass the SAT even though I hate everything these assessments stand for.   However, by not helping these students, their low grades stand as a barrier preventing them from getting their foot in the door to enter the world of higher education.   As long as these institutional barriers exist—common core, SAT, etc., I will help student acquire the skills to work the system while at the same time advocate for change.

What I found most interesting these last several months was watching the types of teaching jobs posted by various online employment services.    Last June, I rewrote my resume and posted it on an employment site.   Now my email is inundated with lists containing scores of teaching jobs.   The first thing I noticed that most jobs posted today are for charter schools.   In the New York area, the one charter that comes up all the time is our favorite—The Success Academy.   Interestingly, one of Eva’s charter schools has been looking for a SETSS teacher since June.  I have a simple theory.  No one wants this position.   Who would want to work for a school where you are on-call 24 hours a day, paid low wages, and then spit out after two years.  Sometimes I think about applying as a lark.   I am sure once they realized my age and the fact that I had a 36 year public school career, my resume would end up in the circular file.  Interestingly, a well-known tutoring company saw my resume and wanted to interview me.  I asked what their pay was.  I   laughed when they said $15.00 an hour.  I told them that when I first started SAT tutoring in 1987, I worked for a college preparation tutoring service and was paid $20 an hour.   I added,  “When you have a high turnover rate, you end up getting what you pay for.”

I would never work for a charter or such a tutoring agency because they violate our society’s social contract.    I believe in and will fight for public education because every cent of public money must go to the child.  And yes, paying public school teachers decent wages benefit children.   A well paid professional feels invested in the system and will work hard for those under their tutelage. A well paid professional wants to dedicate their lives to public service.   The social contract is broken when education is privatized.   The privateers view teachers and students as human capital whose purpose is to create profit.  I call these privatizers education pimps.   Students cannot benefit when your purpose is greed and not the creation of a well-rounded individual who is able to think and make sound life decisions.   The purpose of these corrupt and greedy charter operators is to throw a few crumbs to their school’s students and teachers while they hoard our public dollars.  I want my tax money to be invested honestly and completely into each public school.  Charter operators will be quick to say that they are capitalists taking risks.  Yes, when one invests private capital, risks are taken, but what is being invested is our public money.   It is public money that is being given to them by elected officials who are in their corrupt little pockets.   These officials are also pimping our dollars for private gain and must be ousted.  Our elected officials have forgotten that they serve us and derive their power from us.

Getting back to that retirement fantasy world I mentioned at the beginning, I ended up doing one thing that I rarely did during my work years.  I have watched a lot more television.   One thing I ended up watching was Ken Burns’ documentary on the Roosevelts.   It reminded me that our society is again in a Gilded Age where a few wealthy men have taken control of our government and its institutions.   Ken Burns thinks that it was the power of these singular individuals that changed America in the first half of the twentieth century.   What he does not understand is that these great individuals could not have done anything if people did not organize and petition first on the local level and then nationally for change.  We have to regain control of all levels of government to make it again, as FDR believed, a force to create a just and fair society in which everyone has some share in the economic wealth of this great nation.   I know we have a hard fight ahead of us, but we are making headway.  There is an old adage:  The Ocean started as single drops of water.

Pearson’s Totalitarian Test Security

It has been several months since I have posted to my blog.  It is not because I haven’t wanted to, but because the educational reforms wrought by a binding arbitration between the New York City Department of Education and the United Federation of Teachers have made my job this year one of endless preparation, paperwork, and drudgery.  In my last year of teaching, I have worked harder than those Hebrew slaves that built the cities of ancient Egypt.  To finish my career as an effective teacher, I have to do well on 22 Danielson rubric points, which include 8 artifacts that will justify the generation of enough paper to cause the death of, at least, one hundred trees. 

But this is not the purpose of my little article.  Its purpose is to describe a demeaning test security system wrought by Pearson to safeguard and protect their profits.  Last week, I started testing children for New York City’s gifted and talented program.  I have been doing this activity for many years.  Originally individual districts tested students for their local gifted programs, but eventually the testing became a citywide endeavor.   Originally, children four to seven were tested using the OLSAT along with another test that measured academic readiness.  However, due to criticism that very few minority and ELL students got into the program, last year, the test was changed to just using components of the original OLSAT along with a nonverbal section.  This year, the test was changed once again—in my view—to make it even easier. 

However, our friends, or should I say enemies at Pearson, do not like adverse publicity and embarrassment whenever test flaws are revealed.  Pearson just hated when newspapers revealed that common core test questions had to be thrown out, a passage about a talking pineapple was incomprehensible, that a fourth grade passage was also used on a third grade assesment, and that the illustrations for many passages contained marketing logos that were paid for by the highest corporate bidder.  Therefore, they decided on a solution to solve these problems.  Instead of creating a valid and reliable assessment that would be subject to reviews and study by psychometricians at the university level, they would increase test security. 

Originally, in the good old days, test security was in place to prevent students from getting a hold of a test in order to cheat.  It is for this reason that tests were shrink-wrapped and placed in the Principal’s safe until the day of the test.  However, today, when a Pearson test, such as the Common Core ELA and Math assessments come in at least 75 boxes, that safe at the bottom of the Principal’s small storage closet does not work too well any longer.  Now we had the problem of having many, many tests in several supposedly secure rooms, but once the assessment started, copies ended up all over a building.  Anything could happen.  A page could be scanned into a readability program causing the discovery that a third grade passage was on an eighth grade Lexile reading level or that a passage described the nutritional benefits of a Whopper.

Therefore, Pearson concluded that the only way to prevent such errant discoveries was to collect any electronic device that could copy the test and prevent anyone from even talking about the assessment.  As a result, when I was trained at my testing site this year, I was told that if I was alone with the assessment or even the assessment’s directions for administration booklet within a classroom and had an electronic device capable of reproducing the assessment, the supervisor had the right to immediately fire me.  See, I planned to bring my Ipad to do some lesson planning and a little wifi reading between students, so I now thought all was lost.  However, there was a solution at hand.  All proctors would sit in the hall, on small classroom desks, with their electronic device. while the classroom door was locked with the tests inside.  There would be a school aid sitting on a chair at the top of the hall, watching that we would not enter the classroom to perform any misdeed with our electronic toys.  Another school aid would come with kids, unlock our door, and we would proceed to assess the student with our smart phones, Kindles and Ipads sitting quietly in the hall. 

What I described is nothing compared to the DOE’s Assessment Manual for 2013-2014.  The DOE mandated that every teacher be trained in this manual before December 20, 2013.  Not only would teachers be forbidden to even talk to each other about the tests, but also the name of every proctor would have to be sent to Pearson.  The manual lists at least 50 forbidden actions that a teacher cannot do when testing, and if discovered, the teacher would be prosecuted to the fullest extent of the law.  I guess even the slightest malfeasance would mean death by hanging, not just for the possible culprit, but for every teacher within earshot. 

And to do away with any other controversy, Pearson says on the state’s website that whatever you thought was controversial in the past is no longer controversial.  They are now doing everything on purpose.  Pearson has stated that they will use the same passage on tests at different grade levels (but with different questions).  In addition, they will use passages found within their textbooks, but again with different questions. And finally, they will use controversial pieces of text that will make certain students upset and agitated. But, according to them, students have to develop a stiff upper lip and take it like a man (or woman).

All this, of course, is for the sake of profit.  Let us prosecute and even jail any teacher who dares to analyze a test using psychometric research techniques.  What are a few careers, when billions of dollars in the hands of a few is at stake? At least J.D. Rockefeller used to give children dimes.  Pearson, on the other hand, gives our precious youth, anxiety disorder—and is definitely proud doing so. 

Interesting, during another time, the old New York City Board of Education had an office that used to review and critique different assessments from different publishers.  I know this for a fact because I used to be one of the reviewers.  We used to research tests using Buros’ Mental Measurements Handbook and the ERIC database to tell prospective buyers an assessment’s strengths, weaknesses, reliability and validity.  In this way, we helped schools and clinicians make wise and informed decisions about different diagnostic instruments.  All wiped away by the likes of Bloomberg and Klein so that their friends in the testing business could get sweetheart contracts and monopoly control.  The result is now the creation of a looking glass world in which the perpetrators make billions while those who question anything run the risk of criminal prosecution.  Let us hope for better days ahead.