I wrote this today to a building data person who has begun working with our WL team to discuss the scores from the DPS district fall pre-test. She came into our PLC yesterday without knowing the research, which prompted this long missive from me, so that our weekly department discussions with her could in the future be held within the context of current research:
Natalie since you will be working with our WL team on data analysis in our building I would like to add one important bit of information from our discussion yesterday. You should know that we have spent a lot of time at the district level talking about what percentage the skills of writing and speaking should be given in the overall student scores.
Both of these late emerging output skills can only happen after years of reading and listening input. This is based on relatively new research which the data gathering teams in our school and at the district level must take into consideration.
In the past, before the current research was available, teachers used to think that they could teach and assess writing and speaking concurrently with listening and reading. Those “four skills” were the standards in the last century and few questioned their order of emergence, except for Dr. Stephen Krashen and a few others. But the current research has changed all that.
Our DPS WL team of eight people has been writing the district WL assessment and refining it for over five years now each summer. In June of 2011 we came up with a weight on the overall score of 10% for speaking and 15% for writing. Therefore only 25% of a student’s district assessment is based on the output skills. To some of us on the writing team, even that number is too high. In my opinion it’s like asking a two year old to speak and write properly and then test her on it – it’s not a very reasonable thing to do.
It is important to note here that the district data team with whom we worked to create the overall assessment distribution of percentages relative to overall scores completely accepted our decision and even encouraged it. This was encouraging because it represents the first time nationally that a major metro district in the U.S. has aligned district testing and instruction with current research under the direction of classroom teachers.
It is this acceptance of our professional decisions about what counts most in the first few years of a language that should influence our discussion on data and on teacher evaluation. If a teacher who is aligning with the research by avoiding focusing on the output skills for the first two years were to be held responsible for data that is not connected to what the research says is true, there would be problems.
To repeat a crucial point already made here – in the past, tests on writing and speaking carried the same weight as tests on listening and reading. Actually they counted much more, since the instruction at the time was based on the textbook, memorization, verb conjugation and output in the form of writing sentences. Those days are over.
Obviously, the new research is causing strong pushback in districts that rely heavily on the textbook. One would expect this. Teachers who rely on out-of-date methods of instruction would naturally want to keep data analysis in their own hands.
Such teachers, who nationally currently represent the vast majority of language instructors at the secondary and university levels, would prefer to keep all WL assessment instruments connected to the testing of how many verbs a student can memorize, how many vocabulary words they can memorize, etc.
Thus these teachers will clearly try to push state, district and building data teams in that direction, to align with chapters in a book, which is, for most of them, their pacing guide, which is most unfortunate given what we now know about how students acquire languages.
We can therefore see that the data teams involved in foreign language analysis must be aware of the larger picture when they discuss test design and test results with WL teams at the building level. Any writing or speaking score in a level 1 language class, in my opinion, is basically meaningless for the reasons described above.
I know that district and building data teams can work carefully with the foreign language instructional teams to assure that the assessment instruments designed align with the current research. This is the work at hand now.
To support these points, I attach the following text, written by one of the WL directors of the Ohio State Department of Education to a teacher who was seeking clarification about assessment of teachers in the new Ohio redesign of World Language assessment that is now going on in that state as well as in many states:
This letter is from Ryan Wertz, the World Language Consultant for the Ohio Dept. of Education. It is written in response to a request for clarification about the future of teacher assessment in Ohio from my colleague Chris Roberts who teaches Spanish in Ohio:
Hmm. You’re not the first person who has shared this very scenario for me. Nobody likes change and the additional work it creates, but to your colleagues who want to engage in the “same old same old,” I would say that resistance is futile and their jobs will hang in the balance if they intend to continue with their focus on grammar at the expense of building their communicative abilities. Your points about the new AP test are very well taken (and articulated). I have begun training administrators what to look for in terms of best practices in the classroom. Over-reliance on textbooks, a grammar focus and general lack of treatment of the 3 modes of communication are all things administrators are being trained to look out for as signs of ineffective instructional practices.
With regard to your work around SLOs (student learning objectives), I would merely point out that assessments measure achievement – not student growth (which in our discipline refers to growth in language proficiency as characterized by the ACTFL Proficiency and Performance guidelines). Period. It is not characterized by an increased ability to conjugate verbs, spew memorized vocabulary words in memorized chunks, and so forth. In our field, the only way we at the ODE believe you can adequately measure students’ proficiency growth is using standards-based SLOs (as found in LinguaFolio) in combination with integrated performance assessments. Anything that measures “grammar/vocabulary acquisition” is an achievement measure and a moot effort. Period.
I daresay you’ve got your work cut out for you, but I would strongly encourage you to hold to your guns. That old school thinking among your colleagues is going to blow up in their faces. There is major change going on, and unlike in the past when we in WL were able to hide behind our closed doors and ignore what was going on, the new educator effectiveness system will very directly affect our discipline and will bode very negatively for those who would resist embracing modern communicative language pedagogy.
Ryan