Unlock with PatreonThe Problem with CI
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
When Attacked
Share:
Facebook
Twitter
Pinterest
LinkedIn
Ben Slavic
Get The Latest Updates
Subscribe to Our Mailing List
No spam, notifications only about new products, updates.
Related Posts
CI and the Research (cont.)
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Research Question
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
We Have the Research
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
75 thoughts on “When Attacked”
This test makes me want to vomit. There is practically nothing but explicit grammar on it. Is there any way this Latin teacher is going away some time this century? Hold on to that test though, that seems good evidence if they continue the ridiculous assaults on your rigor. I expect your true false questions might have some Latin in them and that your reading passages are set up so the kids might actually read them, not consult their handy guide to what all of them mean while expecting them to properly do grammar.
I hear that Eric. What John has that this Latin teacher does not have are two things:
1. Actual research about how people acquire languages, which the grammar teacher does not have.
2. A Roman legion whispering in Latin behind his back, a legion that he can’t see when he is teaching, hanging out, listening in on class, enjoying hearing the language again, protecting him.
You have an excellent argument that this test does not use Latin in context. Make your own test, asking for translations from Latin to English (showing comprehension), using the verbs given in this test. I’ll bet that your students will do better than the students of the other teacher. Without attacking them, you could say that everything you do in class is contextualized. That your goal is to have students who understand Latin in context. So of course they do badly on tests that are completely out of context.
The other teacher(s) might not want to play the game and let their students take your test, which only goes to show that they don’t think their students would do well on a test of Latin in context. And that would give the administration something to think about. Just a suggestion. I know that depending on the personalities involved and the relations they have with your administration, it can be difficult to suggest a different form of test without raising hackles. Still you could give your test to your students at the end of the year, and produce it as proof that they have learned to understand Latin in context, that is in stories and passages which have meaning. Listing forms of verbs has no meaning and can not be used to communicate.
Whew! And I thought our HS placement exam was bad. This kicks it up a notch.
I’d restate to everyone that your goal is communication/proficiency, so that is what you would assess – their ability to interpret and/or produce messages. (backwards design).
So, you’d include passage-long readings and listenings with familiar language and have kids responding to them with T/F, multiple choice, summary-writing, etc. If it includes unfamiliar language, then you’re now assessing a different skill: inference.
State that you want to align with ACTFL – then grammar and vocabulary no longer get their own test section. Just the 3 modes. And if you want to align with the ACTFL Proficiency Guidelines, then there’d be very little role played by specific grammatical elements (the ONLY explicit mention of grammar in the guidelines is that at the advanced level there will be control of the 3 major time frames!).
You probably won’t even have to bother to tell them that explicit grammar instruction does nothing for language acquisition, so long as you focus on alignment of goal and test and alignment with ACTFL.
If the test has to include discrete grammar, then it should be doable without knowing without all that grammar terminology (unless the goal is to know that terminology). So, the question prompts all need to be changed or supplemented. E.g. If it says to conjugate the verb “hablar” in the first person singular present tense, it should INSTEAD or ALSO say to “write the word for ‘I talk.'” That way, there is some focus on meaning.
One way to check if a test is mechanical (no attention to meaning necessary) is to replace the test items with nonsensical items. If you can still get the right answer, then clearly meaning is not important. E.g. Conjugate “hablar” – hablo, hablas, etc. Substitute “shablar” and you’d still be able to get “shablo, shablas, etc.”
About the question of whether this kind of test be prepared for with CI, I don’t see how, reasonably. I don’t even understand the directions. (I really don’t know all that many grammar terms even about English or Chinese.) The whole test is looking for language analysis, not comprehension. Even the Latin-to-English translation apparently includes quite a few words the teacher knows they’ve never seen, so lists them below the short reading. It’s not really about comprehending the reading, though, but about the mechanics of translating it.
For me, the equivalent test would be one entirely dependent on memorized handwritten characters and fancy grammar terms for parts of speech with blanks to fill in with Chinese words. My students wouldn’t do well on that kind of test, either. This is an awful thing to be judged by (for the students or you, John).
I agree that this is egregiously bad. Eric’s analysis is spot on. It does nothing to test genuine language proficiency or performance. There is certainly no authentic resource, either.
Genuinely asking how this information addresses the ACTFL Standards, not just as a rhetorical device but as an opportunity for this teacher to reflect on his teaching, might open dialogue, but I doubt it.
John and this teacher obviously have different goals for their instruction, and this test illustrates that. The question then becomes, which goal of instruction is closer to the ACTFL goals and to the purpose of language? Eric has done a good job of elucidating this.
This teacher is obviously teaching a class in Latin Grammar Appreciation, not Latin as language or Latin for communication.
Just being snarky, I have a couple of nitpicky comments:
1. Since when does Latin have umlauts? I assume those are supposed to indicate long vowels, but I always thought the diacritical was a line over the vowel, not an umlaut. Unless the convention has changed, why is this teacher showing students something wrong when his emphasis is on accuracy? (Hypocritical inconsistency?)
2. I have always heard the term “principal parts”. Being capable of passive-aggressive behavior (I learned from a master), I as a student would fill in “all possible parts” as follows:
canto, cantare, cantavi, cantatus, cant, 0, a, as, amos, etc. (Aren’t all of those “possible parts” of the verb “to sing”?) I am simply following the directions given. I suppose I could even go so far as to consider individual letters (c – a – n – t) as “possible parts” of the verb.
And just as Robert found inaccuracies in the instructions, the same happened on the dreadful HS placement exam I have to give. The exam focuses on accuracy down to a single letter difference and yet there were punctuation and spelling mistakes in the instructions!!! Hypocrisy! I showed the mistakes to a reading teacher and she asked: “Did the person writing the test know English?” Answer: Yes. . . HYPOCRISY!
Now, my kids actually did just as poorly on the test as the other 4 middle schools. The only school that did somewhat better has committed 100% to teaching the textbook syllabus and placing kids higher into the high school. So, short of teaching to the test, kids do just as badly, as long as they can understand the instructions, which is why I suggested supplementing that grammar terminology with ways not biased to knowing those terms.
I find it hypocritical to the extreme when a teacher demands absolute accuracy from students on an exam that contains errors in the instructions or other portions of exam itself. Most teachers will then excuse their own errors with something like, “Well, everyone makes mistakes.” This is definitely a double standard and deserves condemnation. (Proverbs 20:10 – “Differing weights and differing measures are both detestable to the Lord”, and we aren’t talking about just economics here. I wonder how many teachers think about the moral and ethical component of how they grade, or even realize that there is one.)
I like the suggestion to engage with the test around ACTFL standards. I think that is the tack I will take when engaging with the HS teacher. I have been feeling him out over the course of the year, but soon the debate will begin. I have been searching for the right way to “frame” the conversation, which I assume (and hope) will be a long, ongoing one.
What a terrible test! Ugh!
lol my favorite part of the test was the last question: “Give to (sic) methods whereby the Romans told the future.” What terrible grammar! haha! The irony…
Can you cram them for a test like this during the last month of school, then give them a test just like before they leave you as a “final”? You would then save their results to show that such and such a percentage were good at this grammar stuff when they left your class, at least.
James, I saw that, too, and sighed.
I think James’s suggestion is possibly an acceptable concession. Once students have started to acquire the language, give them a little terminology for their own protection in a hostile environment. Just be sure to remind them that learning that terminology has nothing to do with acquiring and using the language.
Maybe “sic” should be included in HF words for HS.
John, I couldn’t make it past the umlauts. We’ve been able to type macrons (using the Maori keyboard) for years, not to mention how recent Apple products make it easy to get those marks onto the page. THAT is a sign they are way, way, way behind. Almost as bad as getting a syllabus in college with the wrong semester, year (decade?) left on from a previous course.
That effing test. Those effing teachers.
Lance, if I had the opportunity to read the teacher’s exam out loud to him, I would read every single one of those as a German umlaut. When he objected, I would “innocently” point out that I was reading only what he had written. If he can’t be bothered to write an accurate test, how can students possibly be expected to give accurate answers? (See my comments above about the double standard.) Of course, as a student I would be wise enough not to do that.
Thank you so much for the thoughtful replies everyone! Here is my plan based on what has been said thus far…
A) Gather stronger evidence of what my students can do in Latin. Keep copies of completed sample assessments that are align with ACFTL.
B) Cover my butt. Use James’ idea of cramming and dumping for this kind of hideous assessment, photocopying the completed assessments, and using them to defend against attacks.
C) Frame discussions around how our assessments align to ACFTL rather than on their aggressive ignorance about SLA.
D) Remind myself that I am not crazy! You all have masterfully identified how wretched this type of assessment really is. Not only is it impossible to prepare students for this, it is simply not worthy of preparation.
You all are the best!
I have my two classes of French students heading into a similar situation. Here is the diagnostic test they get in second year French. Seeing this Latin test made me mad and this French test does too. It has intentionally trick questions like :
Choose the correct response:
Il fait hiver. Lit: It does winter.
Il fair froid. (This is the correct.) Lit: It does cold.
Il fait gèle. Lit: It does freezes.
Il fait neige. Lit: It does snow.
SO TRICKY!! WHY IN GOD’S NAME?!?!?!
He gives this to the kids in the first week of school…welcome to French, kids!
We are meeting in January with the Spanish teachers at the high school so we will probably hear all about how the kids aren’t “ready” for second year Spanish after two years with us. I am new to my school, but it is a constant refrain, I hear.
One thing that I know from having taught French as an assistant/temporary instructor at PSU before getting my license and starting in K-12 is that we were always lamenting in the department that the students were not prepared from their high school language classes…but we did not have the relationship to go track down all the HS teachers that the students came from and blame them for their former students’ lack of acquisition. (Some were totally clueless about French after two-four years of HS study). But since we work in the same districts as HS teachers, we MS teachers hear it all the time that we are not prepping the kids for HS.
It happens in all the subject areas too. I know that HS English teachers blamed us all the time for the kids’ lack of correct grammar in English. And the Social Studies teachers blamed us because kids did not master the five-paragraph essay or truly master citing sources. So it is not just in languages. And it used to make me real mad as an English Language Arts and Social Studies teacher too. I think I get mad easy…
Oops here is the test, forgot the link I was soooo mad! 🙂
https://drive.google.com/file/d/0B-czUmSZdXyddExjazZYaS1GZEk/view?usp=sharing
My HS department admitted to purposely making the questions “tricky” – it’s part of the process of pulling out the bad weeds.
Yes, this. They seem to have a “gatekeeper” mentality, like catch the bad seeds before they sprout into (GASP) third-year students! I just had the thought that the HS teachers LIVE for those third-year classes of grammar survivors and maybe are actually invested subconsciously (or not) in weeding out the “dumb” kids.
…and maybe are actually invested subconsciously (or not) in weeding out the “dumb” kids….
This is a serious charge and one that I agree with. Whether it is done consciously is not the point. The point is that they are discarding kids at a time in their lives when even one small success in an elective class can be a critical turning point in their lives. We must not underestimate the importance of what we are doing. We are daily bringing a sense of accomplishment, of enjoyment of life, of a feeling that they can actually be skilled at something, to young people when much of the things in their lives are made out of plastic.
OK, I admit that I had a grammar rockstar French teacher and I was a grammar rockstar myself and got a ton of happiness and satisfaction out of French in HS. But my French teacher was an island of love for me and I remember crying in her room during lunch many times my senior year, when I was suffering from what I now know was a short bout of depression. I loved loved loved Mme Coggins and how special I felt her room. Admittedly, it was because I was super-fast at memorizing rules and spitting out pretty correct French…but I LOVED French class and my French teacher.
So, I wholeheartedly agree that a “mere” elective teacher can be extremely important to a kid.
Many kids, like me, have looked forward to taking a language their whole life. Others, like one of my students recently wrote to me, never wanted to take Spanish in the first place but they love how the time flies by, and it does not feel like school at all, and “all we do is talk about me and my friends!” That was a very important comment to me and I will remember it my whole life.
But, yes, elective teachers can, in my opinion, have a great impact. Kids see us as “special” because they are choosing to be in our class…or even if they don’t really want to be there, we can quickly show them that we *ARE* in fact special. That kid would be struggling in a traditionalist’s class, like he does in many other regular classes, but instead he is loving Spanish and eagerly participating.
Of course, he is heading off to the HS where it is grammar and themes and “real” school. Sigh. I get my hackles up when I think of him sitting in my colleagues’ classes. Maybe one day…
Algún día, quizás…
Todo es posible, Billy…
Sr. Wooly is such a fount of inspiration. Or maybe it is just that his songs are ridiculously catchy.
Ben said: “We must not underestimate the importance of what we are doing. We are daily bringing a sense of accomplishment, of enjoyment of life, of a feeling that they can actually be skilled at something, to young people when much of the things in their lives are made out of plastic.”
That I argue is my #1 goal. Just yesterday I included this in an email I shared with my district:
“So, what are my goals, listed in order of priority?
1) That students feel confident and enjoy class.
2) Language acquisition (= the creation of an unconscious internal grammar)
3) Oral fluency (= comprehension and comprehensible production with speed and quantity of connected discourse that is spontaneous, unrehearsed, and of familiar and unfamiliar contexts)”
Also notice what is NOT in this list of goals.
I’m quite fed up with bad teaching and then these same bad teachers avoid any professional dialogue – no respectful discussion of difference. It’s common for people to all claim the same goal (communication) and then think that a teacher can teach with whatever approach he/she wants and that every approach can prepare people for the same level of communicative competence. I said this in that same email to the HS department, which stresses to me that their goal is communication. . .
“So, we can proclaim the same goals, but when theoretical perspective, testing, approach, methods/tools, and content are all different, then there’s no logical way we could achieve the same outcomes. Hence, the extent to which we achieve our goals will be different.”
Once again, I have to agree with Eric here.
If anyone has listened to Tea with BVP, you hear VP say something very diplomatic when he addresses questions from the audience. He says something like, “Well, that depends on what your goals are?” If your goals are acquisition, or communicative competence, or creating an implicit linguistic system, or developing some kind of fluency or proficiency than you should or shouldn’t…” FILL IN THE BLANK.
In many cases classroom teachers lack the background in SLA fundamentals to understand HOW to reach goals. I am not ready to go on the hunt for BAD teaching but it does exist and it mostly exists because others do not know any better!
Well, we got to the heart of the problem in my district. If I explain my program, then all differences get PERCEIVED as “you think you’re better” or “you’re giving us advice” when in actuality all I’m saying is “this is my way” and “this is what ACTFL and one SLA perspective say.”
When I shared my goals, I got an email accusing me of saying that those weren’t also the HS goals, telling me that I was offending them by defining terms like “acquisition” and “fluency” and that when I share any SLA it gets perceived as “you don’t know any SLA” and “your experiences don’t matter.” I never said any of that. Perception.
Before we can provide information to others, we need to create that professional community that openly shares. Hard for us to create and lead, if not impossible. It’s aggravating how there is no way to describe what I do without it offending. And there’s no way to include facts about ACTFL and SLA without being told I’m insulting them. The channels of communication are clogged with the holy imperfect subjunctive! 🙂
Eric wrote, “when I share any SLA it gets perceived as “you don’t know any SLA” and “your experiences don’t matter.”
Experiences of individual teachers should not cloud sound teaching and learning principles. I know that it is not good getting into a back and forth matter but the perception of what many think is happening is a narrow-view of the reality. This is why we all, as a profession or PLC need each other. One mind can’t possibly see, experience, and articulate of the information about SLA.
Furthermore, what SLA do these teachers at the HS actually use for guiding principles?
If these teachers (usually it is just one leader and the rest are followers) credit their classroom teaching strategies to something specific I would love to know. My guess is that they would try and pull of little bit of this idea with a splash of that and sprinkle of edubabble.
Leaders make decisions. Leaders search for the answers and make conclusions and then follow through. At the same time, leaders should be held accountable for the decisions they make…if a department chair ain’t getting done with their leadership it should be exposed!
Sorry, I am feeling ranting today 🙂
I’m enjoying your comments, Mike.
My first couple years teaching Chinese I was unaware of ACTFL, standards, SLA as such, etc. (I had some training in language acquisition ideas before I moved overseas. Not theory directly, but TPR-like ideas and unconventional ways to acquire language outside of formal classes. So I knew of TPR from then.)
I remember how I felt when I first heard terms like the modes of communication, and read the 90% target language use statement. Panic and/or feeling stupid. So I empathize with those teachers’ response. But, I followed up with learning about those things, and continuing to attend professional development so I’d grow. I think that’s the necessity — can we not close ourselves off from growing as teachers? Can we investigate when hit with new info instead of get defensive? I would guess it’d be much harder to do that if I’d been teaching for 20 years instead of only 1 1/2 (as was the case for me).
These people admit they don’t know any SLA. I know I could make a better theoretical argument for a skill-building perspective than they could and I also know why that perspective is flawed. Yet they somehow think that’s okay, because their “experiences” are enough. But your experiences don’t convince me at all if what you are measuring is some non-communicative skill and not under Monitor-free conditions (time pressure, impromptu, focus on meaning).
WE on this blog are of a different type. If someone came to us today and told us about a new and better way to build proficiency, then we’d be asking about it, trying to learn more about it, asking to observe it, etc. I can’t even understand people who don’t respond that way. How can you have zero interest and even get hostile about anything different?
Last year my district had a “book club” (of unfortunately short duration) and asked teachers to read the book Mindset. It’s an interesting read, presents some good ideas, and provides food for thought.
I believe that the people on this blog all have a growth mindset whereas those who become hostile to new methods, ideas, etc. have a fixed mindset. My identity is not invested in my method, so I do not feel threatened if someone tells me there is something that works better than what I am doing. I want to learn and grow because I know that intelligence and ability are not fixed but rather dynamic and able to be developed. It seems to me that people who react defensively or with hostility feel that what Eric and others bring to the table threatens their very identity, so of course they must defend themselves. (They are defending themselves, not their program or method.)
Just a couple of thoughts on the matter.
I love your rants Michael.
…The channels of communication are clogged with the holy imperfect subjunctive!…
Eric I was baptized in and still worship at the alter of the pluperfect subjunctive, myself. I just couldn’t keep doing that because most of my students, I figured out, didn’t belong to the Subjunctivist Church. If they did, I could have continued in my old AP ways, teaching to the few. But they didn’t, and I had to change. The sad disconnect perhaps lies in the fact that many teachers don’t really know what the needs of their audience are. They want everybody to embrace Subjunctivism. They want everybody to get excited about relative pronouns. They don’t know that correct grammar is properly spoken speech and properly written language. That’s where they put their jobs in jeopardy. What used to fly is no longer going to fly any more in WL education. It’s like the old videocassettes. They are history. Your colleagues there on your island are history. No blame and this is not an attack on them, just a statement of fact. They hear your words as attack but you are really merely stating your truth. Such is the nature of all change. Real change hurts. Real change is messy. Messy describes the last 15 years of my career. But I wouldn’t have it any other way because just now I received a bunch of Christmas thank you notes, a tradition at our school, and here is a sample of the messages I am getting from my students: “Thank you for being a very fun French teacher and I hope that after 7th grade next year we extend (diplomat family) one more year so that I can be with you for three years.” or “I really appreciate all that you’ve done to make our classes as fun as possible.” Funny – I never got such cards from students when I taught the old way. I just felt a big empty hole in my professional heart. Man am I glad those days are over.
Exactly, Robert! Growth mindset, identity & method are separate.
In my exchanges, I included this:
“Dialogue when there is difference of opinion does not have to be uncomfortable. We can disagree and still love each other. If I disagree with your philosophy, then it doesn’t mean I think any less of you as a person and as a professional. Were we to have more discussions I’m sure we’d learn a lot more about teaching and grow each professionally much more were we to have differences, than were we to already agree. So, how do I explain what I do, how do I comment on SLA and ACTFL, without it being perceived as offensive?”
I was playing the “under-the-radar” game, but it wasn’t going to change anything. Probably only retirement will bring change in some cases. So I recently decided to speak more of my truths and bring some ACTFL and SLA facts to the table.
John Daley was talking to me last night and he said it perfectly: “They don’t know WHY they do what they do.”
Oh wow! That test is simply horrid. Of course your students are “well prepared” for that test or for a class that runs that way. How could anyone be?
I must also say that it so rude, unprofessional, narcissistic and in inappropriate to blame a colleague for student not being ready for your class. I would NEVER have the audacity to do that. How self-absorbed does one need to be to openly assert that one’s fellow teachers only exist to serve as prep factories for one’s own classes? Regardless of subject area, this is appalling conduct.
A professor at UConn once told me something like this:
The HS teachers complain about how poorly the MS teachers prepare the kids.
The undergrad professors complain about how poorly the HS teachers prepare the kids.
The grad professors complain about how poorly the undergrad professors prepare the kids.
After a semester or two, the grad professors throw their arms in the air and say, These kids can’t learn language.
Laurie says that we have to teach the kids we have, not the kids we wish we had.
I meant to reference your PSU experience, Tina. It reminded me of this comments (which is paraphrase of the exact words from over 25 years ago.
The whole normed testing movement was supposed to take the guesswork out of what should be covered and tested. We have all seen how that went!!
These old habits are hard to eliminate. To my mind, teacher preparation at the university level is our best hope, together with our own classroom, conference and PLC work. There’s no impetus or incentive for dinosaurs to evolve…
PPS is about to start work on end-of-year proficiency testing through a grant we got for the world lang. department. I am hopeful that this will show differences in students’ proficiencies and therefore be a nudge to the traditionalists. Anyone have experience with this? I want to help use this proficiency testing as a tool for change. I would love help with that!
Is PPS your school, Tina? I might be doing a similar thing this year, developing a proficiency test that is more suited to CI than textbook format.
I’m required to give some kind of “outside” test to my level 1 students, with the idea that it shows something about the program’s effectiveness and student learning. Last year I got my school to delay this required testing. After re-investigating the test options for Chinese, all of which I find objectionable for various reasons, I asked if I could get one more based around proficiency made instead. Thankfully, I’m allowed to have it created rather than use one of the Chinese tests that’s out there. (I even have an email from a STAMP test company representative who agrees that we shouldn’t use their test until year 2.)
So where it is now: I made a big list of sentences and questions that the students in level 1 this year or last year will have seen, as samples of language they’ve heard and read. Reed Riggs and perhaps others at the Univ. of Hawaii will be adapting that into a listening and reading comprehension test that will be given by computer. Since I teach based off of high-frequency words relatively agreed-upon among CI Chinese teachers, I use Terry Waltz readers in fair measure, and Haiyun Lu’s readers have a high degree of overlap in language used … well, the aim is to create a test that could be used by many other CI Chinese teachers if they are required to get testing data on their students.
The idea is I’ll give my students in level 1 & 2 (maybe also 3?) this test in the spring, around the same time that Spanish & French students take the National Spanish or French Exam.
Diane have you talked to Diana Noonan? Tina as well take note – she has at least ten years of experience in developing cutting edge CI assessment instruments for the Denver Public Schools. We’ve taken it a long way. Such a wealth of knowledge there. Of course, the actual getting together to compare notes with her and avoid reinventing the wheel at Valor and in Portland is the hard part.
Ben, I did not contact Diana. I think it didn’t even occur to me, because DPS materials came up in conversation in the PLC some time ago, and there were funding and other reasons that they were proprietary. But I didn’t consider just consulting her. I’ll follow up on your idea.
By the way Diane, in case you haven’t heard, after a number of years of assessing speaking, DPS is no longer doing that. We just dropped it*. I’m not sure if that includes upper levels, though. Diana is a good source of where to start, even if the tests can’t be shared. (Diana has even looked into selling those tests, since they are so good, but I don’t know where that went.)
*I can’t imagine why. With my own children, I very much enjoyed testing and quantifying their speaking at yearly intervals from birth to the age of 5. They did pretty well! It’s because I used TPRS on them.
I hadn’t heard! That’s awesome that the speaking tests were dropped. I really don’t think testing speaking at level 1 (at least, maybe not ever) is helpful. I know where my students’ speaking skills are because we interact so much during class, when they are not so nervous and they speak because they want to, as much as they wish. I could write a descriptive paragraph about each of them that would say much more than a letter grade would, and it wouldn’t punish them.
Yet, I succumbed to the expectation to test speaking skills at the end of this semester. It’s a department policy: speaking is 20% of their grade. However, I think I did things in a way that was free-form and had two kinds of support to reduce anxiety. I was not going to let them fail. Actually, I already knew they could do what I asked for. I wrote about it: http://tprsforchinese.blogspot.com/2015/12/speaking-assessment-and-rubric.html
Yeah I agree Diane about level 1 testing of speech as a waste of time. I personally take it further to include level 2 as well on that point. Linda Li’s 2nd year students prove instantly that I am wrong on that point, but I like the point I am making here anyway.
I also thought of a good way to assess the students’ speaking abilities for whenever you do start testing it, and it doesn’t even involve the students:
Just keep a log of how much time you spent in good quality L2 (few if any interruptions by L1 and interesting/compelling content) over the course of the year. If you spent LOTS of time in the TL during the year, then give your students high marks on their speech output. Even if you haven’t heard it yet – rest assured that it is there. (It may be below the surface and not measurable, but that doesn’t mean it’s not there. Just because we can’t measure something doesn’t mean it’s not there.)
If you spent SOME time that year in the TL and it was of a decent quality as measured by the above criteria in parentheses, you can say that your students’ speech output is going to be pretty good, once it emerges, even if you haven’t heard it yet, because, as stated, its automatically going to be there, just not yet measurable, as long as you created a scenario that the kid wanted to listen to.
The time you spent in interesting comprehensible input in the form of reading and listening will be the only determining factor in what they eventually produce in terms of their output. And of course, we must add here that we can’t predict when one kid will start speaking compared to another, although I am aware that such a statement goes against the current model of “every one learns at the same rate”, but so be it.
If you spent LITTLE time in the TL that year with lots of blurting happening in the classroom, with low to average interest from the kids (I did eight years of that), then you can say that your students probably won’t be speaking much at all in the TL until they have heard yet more CI. Garbage in, garbage out as Robert said the other day.
Assessing them in this way, without assessing them, saves time and money in test development, and is probably more accurate than any test for levels 1 and 2 that we could develop anyway. And it helps us avoid false results, which happen when students are terrified of being put in a situation where they have to speak with their teacher in a private setting and so clam up, which certainly skews results.
At the same time this way of testing honors the somewhat brash idea by the renegade Dr. Krashen that we all learn languages in a natural way and output will emerge as part of a natural and unpredictable process.
I like my new way of testing speaking without testing the students! I feel good about it. I think it is a very accurate way to test the kids. And so easy that you don’t even have to do it!
And that how I’ve approached assessment of student spoken output, happily. I still use the speaking self-evaluation rubric with level one & that was their semester speaking grade. (It doesn’t require any output.)
But my school, and many other schools, require numerical data points. So I tried to find the best possible way to do that. Really what I did was ask students to do what I already could tell from class was possible for them, so they were all successful.
I don’t assess first year speaking. 2nd years get one oral interview at end of course. I knwo how they are all doing anyway. Give me 15 seconds of PQA and I can tell you exactly where that kid is.
On Wed, my French colleagues all did their “unit oral tests” out in the halls, with the kids dutifully memorising their dialogues and the teachers dutifully giving them feedback on verb issues and pronunciation. Yesterday and today the French classes are watching movies. I guess it must be “too late to start another unit” pardon the intense sarcasm.
Meanwhile Leanda’s French 3 kids can now write 700-900 word stories in 3 tenses, my top 1s are at ~600 (despite us not ever practicing writing), and everybody can speak as well as the “communicative” kids (despite not having to speak). Go figure.
Keep reporting your results, Chris. They remind me I am on the right track. I just haven’t got that far down the track.
If you could just remind of of a few details in order to provide context for what you are doing, like… How much time are they allowed for the 600-900 word writings? By three tenses do you mean three time frames (past, present, future) regardless of tense used to express in each time frame? Or do you mean, e.g., pret / imp / present?
(And it is too late to start another unit. This is understandable since they will not remember it after they return from the holidays.)
I just emailed her this week and I’m hoping we can collaborate!
Me too, just emailed Diana.
Yes. PPS is Portland Public Schools in Portland, OR. Where are you?
I’m at Valor Christian High School outside of Denver, Colorado. (Independent, 9-12 grades). Come visit, Tina!
Diane if Tina shows up prepare for an energy tornado
I happen to take a quick look at what the AP Latin exam has on it. Just like in the other languages, there are no direct grammar features/questions that ask for explicitly learned stuff like rules.
——–Spanish, French & German (Latin)——–
Reading (authentic texts, literature, and dialogues)
Listening (interviews, podcasts, radio broadcasts)
Speaking (simulated conversations, 2 minute cultural comparison)
Writing (formal letter or email and persuasive essay)
No EXPLICIT grammar is assessed
If AP is considered to be at the top…why are the classes under this hierarchy not “getting in line?” All conclusions lead to classroom strategies that promote READING in the TL.
Exam Overview
AP LATIN EXAM: 3 HOURS
Exam questions are based on the seven learning objectives and assess all themes.
Format of Assessment
Section I: Multiple Choice | 50 Questions | ~ 60 Minutes | 50% of Exam Score
Syllabus Reading: Vergil and Caesar (~20 questions)
Sight Reading: Poetry and Prose (~30 questions)
Section II: Free Response | 5 Questions | ~ 120 Minutes (includes 15 minute reading period) | 50% of Exam Score
Translation: Vergil (1 passage) and Caesar (1 passage)
Analytical Essay (1 prompt)
Short Answer: Vergil (5-7 questions) and Caesar (5-7 questions)
http://apcentral.collegeboard.com/apc/members/exam/exam_information/4558.html
You are dead on, Mike! I tried to make a similar argument to my high school colleagues and then they complained to my department chair and I got in a heap of trouble. That’s how you know that this is a really strong point 🙂
Same goes for the AP Spanish test. It’s more like a Spanish SAT. No grammar. But rather than change HOW and WHAT, grammar teachers will be grammar teachers and think the way to prepare kids for that is to do MORE of what they were already doing. Kids must work more and work harder. More homework! 😉
Exactly the situation in Latin AP. Teachers are convinced that the only way to acheive reading proficiency for AP is through lots of explicit grammar study. But it’s a classic situation of post hoc non propter hoc (after this not because of this). These teachers think that they are teaching reading comprehension through explicit grammar and memorization of vocab out of context. Rather, they create a filter through which only 1%ers can pass. Then the only ones who make it to 4th year will be the ones who are able to do well on this test (memorization, test taking skills, deciphering skills, all the results of a life of privilege). Then all the students in the “super” class of 8 students get a 5 (Even though only 12% of all students who take the AP get a 5), then this teacher takes all the credit, and schools are complicit because they like to brag about high scores.
The Latin AP requires NO production of Latin. It is 2 things: 1) literal translation of passages, 2) short answer and essay about passages, in English.
Teachers like John’s colleagues have been destroying Latin programs through the use of AP as justification for Victorian pedagogy for too long. But this will not change without an admin with the guts to put those teachers in their place, and evaluate them by the same standards as the other language teachers, AP or no AP.
And this brings up Robert Patrick’s oft-expressed concern, viz., declining rates in language, especially, classical languages.
How long will there be admins who will accept classes that small?
Which is a better indicator of success? High scores among small groups? or large enrollments? Are we to be applauded for a “severe dropout rate”? or for a high retention rate? What is percentage of AP scorers out of the original cohort?
Relevant to this is the study done by John Lawson (1971). John was a school supt in Shaker Heights, OH. He “reported that the dropout rate may be more than 85% by the third year and more than 95% by the fourth year. Few students–less than 5% who started in a second language–continue to proficiency.” (Cited in Learning Another Language, Asher, 2-1, 3-8). My guess is that this study is the source of our term four percenters.
That was the origin of the term, Nate 😉
http://files.eric.ed.gov/fulltext/ED039802.pdf
Thanks, Eric. There must be another study, dated 1971 (the EDo390802 has a 1969 date of publication).
But this document does underline the attrition issue about which RP is expressing concern:
“In 1959 Dr. Conant suggested that at least the upper 20 percent of academically talented students should study no less than four years of one foreign language and yet in 1969 the number who do is less than seven percent. Two-thirds of our students drop out after two years of language study and nine-tenths after three years. It is becoming increasingly difficult for administrators to document the accelerating need of funds for a discipline that affects so few.”
…If AP is considered to be at the top…why are the classes under this hierarchy not “getting in line?”….
That is a svelte question, Michael. I think immediately of the four percenters who hijack the class, unintentionally of course. They bring the passing scores. The teacher is as Tina implied unconsciously complicit in that. The result is that a few pass the test and everyone is happy thinking that the others were just not capable. This is one illusion that TPRS/CI will crack wide open in the coming years.
It’s going to take the little ones currently being brought up by Bracey and Piazza and the rest of that New Roman Legion (NRL) to jolt that false thinking apart, to explode it. ALL kids can pass the AP exam as you imply Michael. But nobody is going to believe that until the NRL does it in Latin and the rest of us in our languages.
I’m reminded of the poverty kids at George Washington High School who destroyed the IB kids in the next (segregated) building in that high school on the DPS end of year exams – Reuben Vyn’s kids. That’s a story for the ages.
It’s happening.
We did the Center for Applied Linguistics SOPA (Student Oral Proficiency Assessment) for 4 years (SPANISH – 4th, 6th and 8th grades; 6th & 8TH GRADE FRENCH) – It has it’s problems (output based) but if you want more on that experience lemmee know…
Michael said:
…all conclusions lead to classroom strategies that promote READING in the TL….
And reading is elegantly set up with stories. That’s why reading is Step 3 and not Step 1 in the TPRS schema. Latin dominance on the AP exams by CI trained kids will start to occur – it may already be happening but I’m not in that Latin Best Practices loop (Piazza, Patrick, etc.) Those CI trained Latin kids will read so well because of the power of the Three Steps. And it won’t be four of them in a school. Look at places like Kansas City where James Hosler has shifted language enrollment in recent years to where there are more Latin students in his school than Spanish students.
The irony is that we can take kids at any level. Native speakers can be actors or tutors, FPs same, and even if we have mixed-level classes we are good to go, as long as the kids understand everything.
Blaine Ray said “figure out what they don’t know, and start there.” Words of wisdom.
Chris
Chris said:
…Blaine Ray said “figure out what they don’t know, and start there.”….
And this can apply to the moment-to-moment experience of teaching. It doesn’t have to be all planned. What is more accurate than instantaneous formative assessment? What is more effective than finding those soft spots in their knowledge and addressing them right when they happen, right there in class, with a little Circling here and a little personalization there? That realization has made me feel one big new feeling about this work – relaxed. I don’t plan anymore. I just wait for the class and then teach. I guess I could plan. Well, no…let me think about that. Ahh….no! I’m good! But thanks for the offer.
I just received this link from my district curriculum coordinator, and with Ben incommunicado I wasn’t sure how else to get this out to the PLC for some feedback. “When attacked” seemed like the proper place to put this, because to me it feels like an attack, although of course it’s nothing compared with what many of you are dealing with. Anyways, here it is, and any response would be gratefully received:
http://mainedoenews.net/2015/12/17/effectively-using-the-ncssfl-actfl-can-do-statements-in-world-languages/
The article sucks.
1) It is just like every other ACTFL document that claims “research says” but has NO citations of any research.
2) This is a HUGE ABUSE of what was originally intended to be an evaluation tool. From the ACTFL Proficiency Guidelines: “The Guidelines are not based on any particular theory, pedagogical method, or educational curriculum. They neither describe how an individual learns a language nor prescribe how an individual should learn a language, and they should not be used for such purposes. They are instrument for the evaluation of functional language ability” (p.3). The bullet points in the article are in EXACT contrast to this statement by ACTFL!
3) The article suggests that the power in these Can-Dos is in the motivation and direction it gives language learners. The idea is that if you have a goal and recognize your progress towards the goal, then that will be motivating. Well, TCI/TPRS teachers do that in their own way! We try to make students feel successful (comprehend) and see their progress (retells, rewrites, pointing out the number of words in the reading, etc.) every day and are constantly praising those gains (e.g. Clapper Kid student job).
4) If the power in a Can-Do is the motivation, then one ought to read Krashen’s “The End of Motivation.”
http://www.sdkrashen.com/content/articles/2015__end_of_motivation.pdf
Most of us are motivated to understand the content, rather than motivated to master the language. And the ability of TCI/TPRS to provide this motivation (compelling) is second to none. What we do in TCI/TPRS is all about motivation! I doubt an ACTFL self-assessment is going to add much, if any, to that motivation.
5) The ACTFL proficiency guidelines are just one way to evaluate proficiency. There is nothing “holy” about the ACTFL scale. Their main strength is to get teachers to focus on what learners can do (comprehend and produce) with language rather than what they know about language. But we in TCI/TPRS already do that!!!
When will our field realize that before we can talk about goals, assessment, method, and content, we have to all answer: How is language acquired?! The answer we have to that question will guide everything else. The analogy of language with mathematics at the end of the article makes me cringe.
A passive aggressive type nudge Anne, like they want more edubabble in your lessons or something? I’ve personally heard great things about what your students CAN do.
I think the author is right on here:
“Research indicates that the identification of clear learning targets is a strong support for language learning.”
If only they had stuck to the “Learning” mode and not tried to confuse later on with actual language acquisition. They say both but don’t distinguish. I’ve no use for the ACTFL CAN-DO statements personally.
It’d be nice to hear from Michele since she was presenting with Mira last year on the Can-Do topic (which I did not get to see.)
It is always a pain thinking that there is some new level of paperwork or format to display what we are doing. That being the state of things, here are a few thoughts that come to mind:
1. Can-do statements are suggestive. There is a blank at the end of each group which is a way for you to state it for your students.
2. Carol Gaab refers to Can-Do statements in her 2015 ACTFL presentation w/SK (it is on her website). She uses them to serve her purposes of teaching World Series ring wearers to acquire language. What does she want them to do? If you get a chance to watch it you see how she weaves it in just enough to state her direction.
3. The question is what can you students do, let’s say with the current story/activity you are doing? Can they understand the story, interact with story, or tell the story? Can they do a speed write of 150 words in 5 minutes? Can they respond to all question words with the proper information (give a “what” for a “what” question, a “who” for a “who” question, etc.)? Can they understand 100 highest frequency words? the top 200? Can they use them in writing? (Bryce talks about distributing a list of top 400. Students check off the ones they know. This is a Can-Do.) Can they understand complex sentences like “There was a Tiger called Bob that wanted to eat mosquitos?” Can they process in the past time frame? Can they recognize or say what every student in the class likes to do in their free time? Can call every student in the class by their FL moniker? Can they ask what something means when they do not know? Our focus should not be on what ACTFL suggests, but on what Blaine, SK, BVP, and Anne suggest.
4. What do the kids want to be able to do with German? Can they do it? (Most people of all ages say they want to understand and interact in the language. )
5. This is about self-assessment: “Can-Do statements are for student self-assessment and are not intended to guide instruction and assessment.” Is this a HW assignment activity? Is this a bail-out move?
6. The examples are heavy on production and light on interpretation. There needs to be a lot more Can-Do at the comprehension side and at the interpersonal side. Possibly a spectrum is needed. The more you think about this, the more unwieldy this assessment tool becomes. Every knight will need an armor bearer. Every professional will need a secretary. There will be no end to it. So we have to pare this back to what do we want kids to do and can they do it?
7. None of this includes paradigm replication or conjugation copying. So the goal is to be lauded, but we do need to focus on being comprehensible and compelling. Is it the desire of the coordinator to focus energy and time on something which distracts from C&C? Are there deficiencies at the the exit end of your program? Can Can-Do statements remedy them?
What can a Can-Do do? Offer self-reflection for students. We should not expect more of them than they are capable of doing. It is just a tool in the tool box. “If I had a hammer…would I treat everything like a nail?”
Maybe your coordinator is looking to your knowledge and experience to get some feedback about what you should be doing with this. We want to do C&C and get to know the kids in L2. Will this help us?
This article claims these statements can be “used” for self-assessment doesn’t explain how. The articles states “learners need to be taught how to self-assess; it should not be considered to be automatically possible on the part of a language learner.” Right… so how do these statements help TEACH them to self-assess again?
You can’t just throw “kid-friendly” objectives at students and expect them to know what to do with them. The wordy Can-Do statements are convoluted and impede self-assessment. Too much scope-and-sequence, too little “I do or do not understand this story/conversation/lesson” …which is what self-assessment should be.
Can-Do statements muddy the water. The question needs to be: do you understand? 10% ? 50% ? 90%? Literally using a numbered scale or even smiley / frowny faces (my kids re-created hilarious pain-rating scale one year) would be more efficient. Anything BUT the blah-blah-blah of the can-do statement.
Right now, I ask kids to write a take-away or a question (L1 or even copying a hard new vocabulary word from L2) on a post-it and stick it on the red (hard)-yellow-green (easy) light at the door. Easy and effective. Kid’s don’t have to limit what they learned to the can-do statement, and they can follow-up their self-assessment by seeking clarification.
Claire,
You said “The question needs to be: do you understand? 10% ? 50% ? 90%?” I think that “Can-Do” is so wide-open that it is up to us to define what it means for our classes. This is an interpretive Can-Do: I can understand 90% of the story. ACTFL is weak on the beginning stages of acquisition, like most of us in TCI once were or most non-TCI teachers still are. We need to make it clear that this is a Can-Do. It is the most important Can-Do. It is the foundational Can-Do.
Thanks for this. Yes indeed it is the foundational “can-do!” I have not put up any “can do” statements in my room. It is a requirement, but I have not done it yet. This makes it super easy. “I can understand 90% of what I read” and “I can understand 90% of what I hear.” I can ask for clarification when I don’t understand.”
???
They are a crock, if you ask me. Another framework/hoop to jump through, for those who have to. A square peg for a round hole, a stack of dusty papers, a feel-good buzz word/phrase for hapless adminz, but nothing that aids or leads to acquisition. Many have found diplomatic and clever ways to rebrand them so they fit our needs, but it’s just a shell game.
Utter waste of time. Y’all caught me in a mood…
Thank you all, this helps so much. My brain is fried right now and I couldn’t even mentally formulate a response to this article. There is also a part of me that balks at expending energy on this stuff at all, so thank you for doing it for me!
Best wishes to all for the holidays.
Gratefully,
Anne
BTW when we had to come up with an Overarching Essential Question (OEQ) for our Wiggins McTighe UbD exercises, we had, “Do you understand the message and can you show me?” This was posted in those classrooms where there was fear that not posting it would lead to a ding on the teacher evaluation. It’s still our OEQ.
Awesome EQ, Alisa!
Yes!!!!! I am going to plaster my space with this one!