Unlock with PatreonThe Problem with CI
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
A Student's Nightmare
Share:
Facebook
Twitter
Pinterest
LinkedIn
Ben Slavic
Get The Latest Updates
Subscribe to Our Mailing List
No spam, notifications only about new products, updates.
Related Posts
CI and the Research (cont.)
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Research Question
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
We Have the Research
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
55 thoughts on “A Student's Nightmare”
There is a mix a hopelessness and righteousness that sets in for me around now each year. I have started with the projects and reading and culture and stuff, like Ben mentioned earlier. But this warms my heart, Eric.
An update: my 8th graders took the grammar section.
They need an 80%+ to get into Honors Level 1.
2 of my brightest students only got a 59% and a 62%.
If the test or the cut-off doesn’t change, then . . .
The difference of 1 letter or the absence of “the” or “to” on the test gets you a wrong answer. Absolute garbage.
Not to mention 30 conjugation questions.
Absurd. It feels like we are a voice crying out in the wilderness. It doesn’t help we’re in Lent, too. Or, maybe it does.
It is a very real possibility that some teachers would internalize failure on their part from this. That would be a serious mistake. But the majority of teachers still think that the absence of a preposition here and there makes a difference and is a true indicator of lack of learning. They are gloating now over those stats, Eric and I call that nothing less than a professional tragedy. The implication is that we should go up to every three year old on the planet and get in their grills for screwing up the grammar. This is a very serious issues.
One thing I like about the Jackal – he’s not afraid of anyone. And he gets it about who needs to change. And it isn’t him. My advice, Eric, is to stay focused on your teaching. If you go after these people it won’t be good. Let them have their day. It is a lot closer to sunset for them than they think. But we are in the vast minority – like 1% of teachers. Think about that and lay down that sword for now, my brother. They will end up falling on their own swords.
I find it helpful to reference not only the mistakes that three-year-olds make with their native languages, but professional adults as well. If you looked for them, you would find a decent amount of “mistakes” in many of the comments posted on this blog, for example. Mistakes that in no way hinder the message, but that would get points taken off if they were submitted as answers on Eric’s school’s placement test. For kids going into Spanish I. I also enjoy the idea of asking other adults if they can conjugate “to be” in English, or give an example of a definite or indefinite article.
Funny you should mention that Leah, because actually the test instructions and some of the questions contain English MISTAKES! I showed them to another teacher and they asked if English was the second language of the person writing the test. Ha. Nope.
English errors:
-take you time
-chose (3xs) instead of choose
-mi instead of my
-las instead of the
-missing periods and capital letters
-run-on sentence
There are even a few Spanish errors:
-play as jugar when it should be to play
!!!
It’s just so horribly and ironically hypocritical…
I happened to be subbing in a foods class the other day, team teaching situation, and as we’re sitting there around the table a kids says something that started out “Me and David’s team…” and the other teacher who is actually an English teacher there scoffed at him and kind of mocked him and then proceeded to fumble the correct form herself saying “David’s and mine team… errr… David’s and my team”. She still kept her superiority-complex despite the hiccup. I say exactly what that kid said, because the other form doesn’t sound natural to me, and who cares, unless I’m writing a formal letter or something (that’s my opinion anyways). Perhaps it’s not a coincidence that the small group of kids kept getting dinged when they’d put their elbows on the table (I noticed that evening that both my partner and I had our elbows on the table… we’re just savages I suppose!) End of rant.
I get what you’re saying, Ben. But I’m seizing this moment. They’re forcing my hand.
This isn’t just a student’s nightmare, it’s a CI teacher’s. I’m so sorry it’s this bad now, Eric.
I think Eric understands the need for each and every one of us to change when we get new information. Eric has goggled up that information like a professional sumo wrestler, too bad it sounds like his colleagues are lightweights.
Eric, how are things coming with the development of proficiency/fluency tests… I think I remember you saying you were having some explorations into this a while back. Man, wouldn’t that be a stark contrast, an actual assessment that measures their proficiency and fluency vs linguistic knowledge/performance. I would think admin would jump at the chance to measure useful stuff (ok, maybe that’s me being real naive).
Jim, those tests of level 1 reading, listening, writing, even a test of acquired textbook grammar are complete. I shared them with the department head. She didn’t use any of it.
Are these tests that existed already or new ones?
Which tests?
Mine have been worked on since June of last year.
And the high school just made this test 2 weeks ago, but they apparently used to use this format and content on the placement test a few years back.
Which tests? “level 1 reading, listening, writing, even a test of acquired textbook grammar”
I’m not familiar with anything you created, nor the popular standardized ones most districts use. Did you share it? Maybe I’ve missed something.
At least Eric, your students have learned HOW best to learn/acquire a language. Hopefully they’ll be able to hold their next FL teachers accountable. Profe slow down, repeat, write the L1 on the board etc (Talk about …crushed souls. How sad.)
I am starting to feel some push-back of a different kind. The Spanish 1 and French 1 teachers are all saying that their students are saying, “I’m just taking Latin next year because I hear it’s easier.” You can imagine the stress brewing in our department. Everyone, including me, is feeling inadequate.
Bust out the timed writes James… and remember, it SHOULD feel easier!
What James? You expect to not encounter opposition? That’s what this work is all about. We are walking lightning rods. Far from inadequate, your work is making people think. I am so proud of you. Let the lightening strikes happen. That word lightening is related to the word enlightenment, is it not? You can handle this my friend.
In one of my other posts some time ago I used an analogy that applies very well.
When I worked for Medieval Times, I often did educational presentations for which we took a horse in a trailer to schools, etc. One day the usual vehicle was unavailable, and I had to pull the trailer (with horse inside) with a four-cylinder pickup. On the way back to the castle, the clutch went out because of the overload, and I had no gears so could go nowhere. Fortunately one of the knights who went that day had chosen to drive his V8 pickup. We unhitched the trailer from the castle’s pickup and hooked it to his. His pickup easily pulled the horse and trailer back to the castle. Did the load change? Was the trailer suddenly lighter? No. The difference was the power of the vehicle doing the work.
It’s the same way with language learning and acquisition. When we place the burden on the conscious mind with its +/- 7 items of cognitive ability, it’s like asking the four-cylinder vehicle (sometimes perhaps a 1-cylinder Trabant) to pull the heavy load. When we place the burden on the unconscious mind with its immensely greater power, it’s like hooking the trailer up to the V8 (or even a Mack truck): there’s plenty of power to carry the load, so language acquisition suddenly seems easy. But it is still every bit as difficult as it ever was, and students still must struggle with the rigor of our classes (i.e. sustained focus, depth and integrity of inquiry, etc.)
Students, however, speak from from perception, not reality, so they label these more rigorous classes as “easy”, confusing “rigorous” with “onerous”. The tragedy is that language and education professions continue to do the same thing when they should know better.
Absolutely love this, Robert. I’m totally stealing it.
Robert I love your analogy…I have read it several times today. Thanks for sharing it!
“I am starting to feel some push-back of a different kind. The Spanish 1 and French 1 teachers are all saying that their students are saying, “I’m just taking Latin next year because I hear it’s easier.”
I have lived in this nightmare for the past two years, James. “Easier” really means “No mind-numbing busy work, pointless memorization, humiliating forced-output activities, and demoralizing assessments.” Also keep in mind that most kids don’t want to voluntarily take a crappy class, regardless of how “easy” it is. Most normal kids in a traditional classroom experience the pain of working hard and not finding success. These kids then internalize lack of success and say that the class is “too hard.” Students come to your class and succeed and then say that your class must be “easy”. None of this information will give comfort to your colleagues.
If they aren’t total psychopaths, their growing resentment will remain nothing more than professional jealously. It is much easier to say that James is taking our kids by bribing them with easy A’s and no homework, than to say that James is doing something exceptional that we aren’t doing.
If they ARE total psychopaths, they will try to make your life a living hell. Just try and remember that language teachers are very different from other teachers. Our job security is based exclusively on enrollment numbers. No negative evaluation or reprimand will affect your job in a meaningful way. Jealous colleagues and bone-headed administrators can’t yank kids out of Latin and plop them into another language out of spite.
If I haven’t been assassinated by Spanish and Mandarin teachers in my school when they see the enrollment numbers for next year, I would love to hear how this plays out and offer any help that I can.
Stay strong, brother!
I’ve seen this test that Eric refers to and, even for a “traditional” test, it is a horrific document. Example from my (probably poor) memory.
They have
a. tienan b. tienen c. tenan d. tiene
It was also 12 pages long.
I have no idea how ANY student could do well on that. It is the type of test that will narrow the available student field down to 1% of the population.
Eric is right, a conversation needs to take place. Key questions:
What are the goals of the department? What are the goals of this placement test?
How do those goals align with district goals? With state goals? With ACTFL goals?
In the past 10 years, what have scores been on this test?
How are students able to use the ‘knowledge” demonstrated on this test in order to communicate?
What is our retention rate? What would we like to see as a retention rate?
Wishing you the greatest of luck Eric!! (You’ve got everything else you need!)
with love,
Laurie
Eric,
OK, so no surprises in the test results.
It’s both painful and infuriating to see one’s passionate work and mission measured and weighed with the wrong tools, like this.
The silver lining (though it doesn’t feel like it, I know) is that you’ve begun a crucial conversation with the Decision Makers – and they can’t go back to a state of innocence once you’re done with this. This will effect change.
At some point, someone (who gets paid to make the big, uncomfy decisions) will either square the research with the instruction & assessments,…or not.
I can’t see why they wouldn’t want ‘more data,’ – in an era of data for data’s sake – allowing you to assess, then showcase what it is your Ss CAN DO with the language….
You have given it your all; you have presented your cogent argument based on the state-of-the-art theories, replete with examples, documentation, the works. Now you must wait and see, providing illuminating feedback whenever there’s an opening for it….
Future Possibilites Include:
-Parents and/or Ss may get involved & ask HS Ts and/or admin Qs about the dissonance between lower levels and HS (legacy) approach;
-Receiving T’s may notice a need for an alternative class since their new T/CI Ss know way too much to be in regular Spanish 1;
-Fed-up HS WL legacy Ts may mosey over to see what the Herminator is spinning in his class, and seek support/mentorship.
These things can and have happened – we read about them on the blogs….
Continue to do what you do – it makes a difference in kids lives (that’s what we signed up for!!!) and your support and contribution to the profession are beyond measure!!
Perhaps strategize w/the administrator (principal? Sup’t?) about ways to address these issues – see if there’s any insight…
Though you prolly feel yucky for the misrepresentation of your work and your Ss, I see you as a Pioneer.
BRAVO!
Thank you Laurie, Alisa, and others for the kind words and support.
In my request for a meeting, I was asked to be more specific about my concerns to which I responded that I wanted to better understand the goals of the HS program. Whatever the goals, that is what you are supposed to assess. If the goal is “proficiency” or “fluency” then an untimed vocabulary and grammar placement test is the wrong tool, since it measures neither. If the true goal is to know grammar, then in the future I will tell kids, parents, and admin that the HS goal is not to be proficient nor fluent.
Like Mike and I said, you gotta get teachers to put goals into terms of what they want kids to be able to do with the language. Right now, truthfully, the goal is probably to finish the level 1 textbook content or to have kids do well on a college placement exam. Neither of those are goals written in terms of what a kid can do with the language.
That anyone would think this to be a good test is scary. When I show it to non-FL teachers, they see right away how ridiculous it is. Like I said, many questions are testing the difference in 1 letter. E.g.
The first book.
a. El primer libro.
b. El primero libro.
c. El uno libro.
I am going to read a lot of books.
a. Yo ir a leer muchos libros.
b. Yo voy leo muchos libros.
c. Yo voy leer muchos libros.
d. Yo voy a leer muchos libros.
100 questions just like that.
One of my superstars said today: “I felt so demoralized after taking the test.”
That same student started laughing out loud at the below multiple choice question testing whether the kids know not to capitalize the day.
Sunday
a. Domingo
b. Sábado
c. domingo
d. saludo
I had this conversation with a former student of mine who is now in his 2nd year of high school Latin…
Student: Mr. Bracey! I just failed a quiz on pronouns.
Me: Honestly…I don’t think I would pass a quiz on Latin pronouns. Actually, I’m pretty certain I wouldn’t pass a quiz on English pronouns.
You’re the man, Eric. You are fighting to make your department better and to protect your students from that abusive, inappropriate and demoralizing assessment. I know how frustrating it can be to achieve authentic success in your classroom, only to have it devalued by some hideous assessment (or teacher).
Just know that you have support here for whatever moves you decide to make.
I too have seen this test and I agree with what Laurie said…It is HORRIFIC!
I support Eric for going up against this kind of mistreatment. That is what this is…mistreatment. We all deal with some kind of misguided approach of people we work with. My personal opinion of looking at this test is that if Eric had to, he could get his students to do well on such a piece of crap but he shouldn’t have to.
I do second something Ben said though. I think when you go after these guys Eric…the results won’t be good. You will win this and when the dust settles there will be some wreckage. The wreckage will be some embarrassed high school teachers and possibly some admin that have to take sides.
That being said when Eric is wound up…good things come of it! Be careful buddy 🙂
I have thought about how careful I need to be in order to not come off as condescending or embarrass anyone. Taking the discussion into research and theory would do that.
What theory of second language acquisition do you base instruction on? – Ummmmm.
Are you familiar with Krashen? VanPatten? Elley? Nation? Mason? – Nope.
What do you know of the Natural Order and Developmental Sequences? – Nothing.
I could destroy in a battle, but what of the war? Thing is, these teachers would never come to me for help. All I really need is a fluency/proficiency-based assessment and then let results speak for themselves. But what of all those kids in the meantime who are to be demoralized and live the grammar nightmare?
Dude. Just pick up and throw back in the starfish you can. We have to learn how to work within the limitations of our jobs. Focusing on the classrooms we have been given is the work. You said it yourself, those teachers won’t listen to you. That is a fact. Some of those teachers have been crafted from a mold that just doesn’t work any more and never did. Imagine – trying to get away with 20th c. methods in the 21st c…. it can’t work. But you don’t have to get all bent out of shape about it. They will grow into this work if they want to keep their jobs. Don’t go up and rip off their masks. Many of them will leave the party soon, anyway. In Denver Public Schools scores of them have left in the past five years, forced out by Noonanism*. The party is getting better at its own rate of speed. We can’t rush it. We can teach well in our classrooms.
*There is nothing attacking in Noonanism. It is about training each DPS teacher to be the best they can be and give them full support and training. Noonanism is a natural principal of reform.
Eric,
Good luck with the challenge you are facing. You of all people have the courage and conviction to get the cogs of change moving. If there is anything you can share about the CI-friendly assessments that you have developed, I would be eager to hear it as I work on crafting my first final exam.
Ben and Others,
adding on to my request to Eric above, are there examples/principles of CI assessment available anywhere on the blog for perusal and plunder?
To Angie and Jim, I’ve dropped comments on this blog here and there and I’ve shared my tests and test background information. I know lots of that is over on the forum (see links below). Lots can also be searched for on moreTPRS. I hope this summer to write a guide and include ready-made level 1 tests of “Acquired Competency ” or maybe better called “Fluent Proficiency Assessments.”
Acquired Grammatical Competence for those who MUST assess level 1 textbook grammar:
benslavic.com/blog/forum/general-discussion-1/elicited-oral-imitation-test-level-1-spanish-version/
benslavic.com/blog/forum/general-discussion-1/a-twist-on-tf-quizzes-an-acquisition-check/
CI-Friendly Test of reading, listening, and writing:
benslavic.com/blog/forum/general-discussion-1/ci-friendly-test/
benslavic.com/blog/forum/general-discussion-1/lextutor-the-future-of-test-design/
And I’ve written about changing the tests and critiqued some of the standardized tests on the ACTFL listserv:
community.actfl.org/communities/viewdiscussions/viewthread/?GroupId=439&MID=6863#bm0
Thanks Eric for digging this all up again. I’ve followed much of what you’ve written about assessment (and loved the ACTFL post when you dropped it!), just hadn’t seen the actual tests (or rather the process outlined as you’ve done, leaving room for teachers to create their own). You’re leading the way Eric! I look forward to working with what you’ve outlined and see if I can’t create something that shows our area colleges where my students are at with respect to receptive proficiency.
Angie I found this that you had written about assessment to a supervisor. I thought it was well written so republish it here:
Possible Response #1 to Supervisors Requesting to See Forced Speech Output in our Classrooms – Angie Dodd:
Dear Supervisor,
Thank you for visiting my classroom and for offering feedback. Collaboration and coaching among colleagues, both in the classroom and online, have been essential to me in the development of my teaching practice. As you know, many of us in the building are using comprehensible input based methods of foreign language instruction. I have seen positive gains both for my students and myself since adopting these techniques. Within this framework of instruction, the goal for novice language learners follows the Natural Order of Acquisition. Students will be able to understand complete sentences in the target language and respond to that input with gestures, single words or short phrases. Their responses clearly communicate to me whether comprehension has occurred, and I do my best to accommodate my instruction to their level of acquisition. One of the basic tenets of this type of instruction states that forced output at early levels of acquisition actually has a negative effect. Therefore, I strive to provide lots of comprehensible input to my students and their short responses, along with other assessment tools, assure me that I am succeeding. Far from preventing students from acquiring speaking ability, this kind of training prepares a fertile soil from which the speaking skill will emerge at the intermediate and advanced levels. Contrary to popular opinion, the research shows that we actually learn to speak by listening and we learn how to write by reading. If you are interested, I would be happy to provide further materials and links to the research that has inspired this pedagogical approach.
Angie one caution and I’m certainly not suggesting that you test in any other way than what you see as best for your kids:
Summative testing is a bit of a red herring to me. We think that we need to do it. I don’t think we need to. All it does is be weird. The fact is that if it weren’t for formative testing with the quizzes I wouldn’t know how to assess my students other than take some writing samples or by – and here is a novel thought – looking them in the eye. I can generally tell if someone is hearing me (the verb entendre in French has both meanings – to hear and to understand). In my own view constructing an actual summative test that is accurate at the end of year may be in some ways impossible to do well. I also don’t test 4 year old kids on how they can speak and write. Seems like a reasonable idea to me to wait on that.
In private conversation with Diana Noonan I have gotten the feeling that she is very frustrated with having just spent hundreds of thousands of tax dollars on designing the DPS tests that I can’t share here because we, or at least I, perceive them as not really accurate indicators of student achievement. All that money and time spent sitting in a school in June over the past many years and what we have to show just ain’t that great. (How can those DPS tests be really accurate, given that most of the students aren’t really motivated, but that is another story.)
Maybe that is why I just do a story with a reading for my final exams. You’d be surprised how I can make doing a story during the final make look like a real summative exam. It just takes a little creativity. Plus I don’t lose all that review and testing time away from CI for my kids. Plus they are so stressed from their other exams, and I don’t see exactly why they should be, except maybe because we in our country have bought heavily into some kind of mass testing psychosis.
I like it when my kids really enjoy the exam story sessions. It’s like a big break from exams for them, bless their bored hearts and shamed minds. They always leave the exam room a little less stressed than they would if I had made a big deal out of testing.
I guess I should be a harder ass, so my supervisors would think more of me. I don’t know. Well, I kinda do.
I want to show the local colleges, in exam lingo, what my kids can do when they leave with me. I’m getting the creeping feeling that there is some negativity happening with regard to my my program. Maybe it’s me being paranoid. I don’t know. I’d like to say though, “yeah, X student took this test (give it some sort of fancy smart sounding name) and did this on it”. I don’t have to grade them on it (I follow your process of assessment in my program Ben, and love the final being a story… I do no new structures, somewhat free flying plot, recycling of what they’ve heard), but just to have something for receptive skills that is not defeating to them, but rather uplifting.
I get everything Ben is saying.
There are also good reasons to develop the best test we can to measure what has been acquired – what can be used spontaneously in familiar and unfamiliar contexts (proficiency) and can be used in real-time and in abundance (fluency). If we don’t do it, our traditional teaching colleagues will do it for us.
It was motivating and at the same time a kick-in-the-butt to some students when they took the mid-year Fluent Proficiency Exam and could see progress on an exam that parallels the one they took at the beginning of the year. I did not grade it. I did not hand it back to the kids. I did comment on the progress and achievement on the kids’ report cards. And this gave me good evidence some admin and parents expect to see.
A good test can’t be studied and crammed for. You can’t cram acquisition.
Jim,
I totally here what you are saying about using test lingo. I also love what Ben shared about using stories as an assessment. I think these ideas combined with the terms that Eric has been using could be brilliant.
“Acquired Competency Language Exam”
Of course textbook tests are forced upon us (at least in my world). But what I really value is how they do on an “Acquired Competency Exam.”
ACE’s need a reading and writing section and when appropriate speaking sections. I do like the format of Bryce Hedstrom activities.
http://www.brycehedstrom.com/wp-content/uploads/LA-CHICA-QUIERE-CAFE1.pdf
http://www.brycehedstrom.com/wp-content/uploads/LA-CHICA-FANTASTICA.pdf
Can you imagine giving these assessments to traditonal classes? Well I have…whenever I sub for one of my colleagues even in upper level courses, I do stories and end with writing, the work is below average when compared to my beginning TPRS students.
Acquired Competency Exams are the instruments for making change.
I am very precise about how I use terminology.
Acquired competence = unconscious language use, i.e. what language can be comprehended and produced in free, spontaneous, unrehearsed situations.
Many “proficiency tests” do not satisfy all the criteria and make it more possible to “monitor,” i.e. conscious language use.
I am also very precise about what is and what is not an Acquired Competency Evaluation. Anything that allows the 3 conditions for monitor use to be satisfied (know the rule, thinking about the rule, and time to apply the rule) is NOT an ACE (I like that acronym – nice work, Mike! but maybe it should stand for “evaluation” and not “exam”). I am conceptualizing an ACE to be a measure of proficiency AND fluency.
Doing a story with the class as a “summative test” is a wonderful idea.
I glanced at those Bryce lesson plans and they look great.
But neither, class story or Bryce’s evaluation with a cloze test and comprehension questions, qualify as an ACE, at least in the way I am defining it. These are assessments of “performance” and are untimed. They’re fabulous, but not for the purpose of measuring fluent proficiency, not for the purpose of measuring acquired competency.
Fluency writes, when a student does a timed rewrite of a story, is measuring performance. Familiar context, based on recent vocabulary, and when done regularly, our kids get the practice in writing narratives (class stories).
Now, give the kids a new story, one they’ve never seen, and only allow them time to read it once, maybe twice, then collect the story and have kids do a timed rewrite. Now we’re tapping more into acquired competency.
Eric wrote…
“Now, give the kids a new story, one they’ve never seen, and only allow them time to read it once, maybe twice, then collect the story and have kids do a timed rewrite. Now we’re tapping more into acquired competency.”
I think this stuff is really tricky. I like what Ben said he did for a test. He said he would do a story (fresh exposure to CI) and a reading for an exam. This is similar to what I sent you privately. In my example, a class story script (that has taken me several days), followed by a free write, followed by a fresh new story with TL questions as an assessment piece.
The way I see it, is that students are using their “acquired” language, (I am using that term loosely) to demonstrate implicit knowledge by whatever they are asked to do with the reading.
I think what you suggested above is again rewarding the students that have these great memories. The language being used might be acquired competence but rewriting a new story has to be short-term stuff.
I also think of the term Micro-fluency that we both love so much. The Acquired Competence Evaluation (I agree, “evaluation” is smoother) must be in the realm of the language that was provided when students were interacting with CI in the TL.
I don’t agree that reading a story once or twice then rewriting it is a fair assessment. That being said, I am open to being convinced. 🙂
My opinion is that an ACE has to have lessons that stay in bounds of mirco-fluency if we are to keep expectations reasonable.
We are perhaps talking about 2 different things.
You have some good ways to assess your most recent targets and story. Assessment of targets after you’ve just spent hours working them doesn’t show they’ve been acquired. Delay the test, add some time pressure, and now we’re talkin’.
I want a way to assess the level of acquisition independent of any particular curriculum. I am NOT suggesting my means are a way to frequently assess. I am talking about the summative (end-of-semester or year). I imagine texts that are leveled based largely on word frequency. E.g. level 1 kids are dealing with texts with 95%+ words from the highest frequency 250 words.
Also, I am looking for assessments independent of teaching method. Grammar-grind kids can read (slowly) by analyzing form. Make it timed and don’t let them look back at the reading and a traditionally-taught kid is going to have a lot harder time than a TCI-taught kid.
Speedreading is one way – an excellent way – to see what has been acquired. Kids do a timed read of a story, then the story is removed, and they answer comprehension questions. Knowing they have to answer the questions focuses them on meaning and timing the reading makes it about reading fluency.
Controls for memory:
1. Questions are on the main ideas (not the details).
2. These are supposed to be short readings (about 500 words).
3. They use all the language targets you want to assess that semester, so the words should all be familiar. This is not meant to be a test of inference skills.
4. In the case of rewriting a short story they read, they don’t get assessed based on telling the story exactly as before. Yes, they have to try to retell the same story they read, but that is really just a “cover.” We are assessing their language production with analyses of fluency and accuracy.
5. When looking at individual progress, then memory is held constant.
6. The Oral Narrative Test was supported in research to measure implicit knowledge. Subjects read a 300 word story twice and then have 3 minutes to orally retell it. I prefer that the language modes match (read then write, listen then speak). The procedure whereby you take away the reading is how they do it in research studies (that was guidance I also received from Beniko Mason).
7.Even if memory had a small effect, I’ll take it if the trade-off minimizes focus on form.
I also think testing situations that are reasonable for the masses of students that we have, do allow students to use there monitor. I would argue free writes allow for the same. Ever see a a student cross somthing out or use their eraser?
Speed writes and speaking face-to-face are the only ways I see that we can even begin to cut out the use of the monitor. I think it will almost always be there.
Can students be competent with language that they have acquired? Yes!
What does that look like?
To Angie and Jim, I’ve dropped comments on this blog here and there and I’ve shared my tests and test background information. I know lots of that is over on the forum (see links below). Lots can also be searched for on moreTPRS. I hope this summer to write a guide and include ready-made level 1 tests of “Acquired Competency ” or maybe better called “Fluent Proficiency Assessments.”
Acquired Grammatical Competence for those who MUST assess level 1 textbook grammar:
http://www.benslavic.com/blog/forum/general-discussion-1/elicited-oral-imitation-test-level-1-spanish-version/
http://www.benslavic.com/blog/forum/general-discussion-1/a-twist-on-tf-quizzes-an-acquisition-check/
CI-Friendly Test of reading, listening, and writing:
http://www.benslavic.com/blog/forum/general-discussion-1/ci-friendly-test/
http://www.benslavic.com/blog/forum/general-discussion-1/lextutor-the-future-of-test-design/
And I’ve written about changing the tests and critiqued some of the standardized tests on the ACTFL listserv:
community.actfl.org/communities/viewdiscussions/viewthread/?GroupId=439&MID=6863#bm0
And you know well Eric that I am not saying to cool your jets. I mean, those tattoos aren’t the temporary kind. They’re on there. And people need to read them. And you were made to give forearm shivers* to people so you can lay it up on the side of their heads and tattoo their temples with this new information about CI. Both sides are true. We take it to them, and we also focus entirely on our classrooms. It’s just that way in this work.
*Urban Dictionary: A technique done by linemen in the sport of football to deliver devastating blows by using their forearms on the opposing team. Players wore big forearm pads that went out over their elbows and they would practice their craft by thrusting their forearms into blocking bags. Some guys would go so far as to ram their forearms into walls.
“I mean, those tattoos aren’t the temporary kind.”
What are you trying to say about temp tattos man?
🙂
If you are looking for a “middle of the road” test, or a “moving down the road ” test, look at these :
Level 1 (1 year HS or 2 year MS program) http://www.nysedregents.org/loteslp/
Level 3 ( 3 year HS or 2 yr MS+2 yr HS) http://www.nysedregents.org/regents_lang.html
They are not story-based per se, however they are competency-based. This is the format for the exam that we are required to give in NYS. ( The state stopped funding the creation of the exams several years ago, passing that responsibility to the regional level.)
with love,
Laurie
Good term “moving down the road.” I agree. I have used the NYS SLP and there were lots of things I did not like, but it’s also a huge improvement on a 100 question discrete grammar test.
One of my recent revelations is that the term “proficiency” is vague and so-called “proficiency tests” are actually measuring in different proportions learned competence and acquired competence.
Eric you gotta write a book (or at least a Primer?) about assessment. Assessing acquisition, assessing proficiency, assessing the 3 modes, assessing depending on the # of hours and/or developmental level (age), assessing programs (I.e., vertical alignment). And of course, assessing assessments!!
You could prolly dictate the book during a workout….while reading in Portuguese….
Yes please. This is an area that hasn’t been fully addressed yet by CI teachers, I think. So many of us are being held to testing that doesn’t reflect our goals or results.
Update:
The HS did not anticipate that this test for 1 percenters was going to be so hard. It’s like giving someone a test on the mechanics of a car engine in order to test their ability to drive a car. They are now going to grade them and after decide on the cut-off to get into Honors 1. I doubt kids of any elementary school, unless the teacher was in fact teaching to this test (and even then), made the original 80% cut-off. I actually had 1 student get a 77% on the grammar section and I never taught, practiced, or tested this crap. This student incredibly got correct every masculine/feminine exception (e.g. el mapa) !!!
1. Research gives us an upper hand. It is important and necessary. There are plenty of teachers (CI and non-CI) who believe their experience and degrees are enough and that they don’t need research to support their claims.
2. Traditional teachers have a restricted concept of “communication.” What they really care about is communication with that unit’s grammar. If grammar is on the rubric you can be sure they’re only marking the grammar they explicitly taught (and the students know that so they know what to study and focus their monitoring on).
3. Traditional teachers are not testing proficiency (spontaneous language use in unfamiliar, unrehearsed situations) and it’s not unannounced, delayed testing, and it’s NOT independent of curriculum. When you just finish a unit on possessive pronouns and then you announce a test on them and include a spontaneous speaking assessment, what do you think the teachers are listening for? What do you think the kids are practicing and trying to consciously use (something they would not do in a real-life interaction) during the test task?
4. The traditional goal is to master the concept and its application. This flies in the face of what we know about Natural Orders, but traditional teachers truly believe the students are mastering each grammar aspect in communication, too. Again, another reason why some SLA education is needed.
5. There is belief that the major textbooks must have it right. How could so many textbook companies have it wrong? When I suggested this, it made me look like I was crazy. I need to dig up the quotes from Nation, VP, and Krashen who have all criticized textbook curriculums.
6. I was not able to have a conversation about goals, because questioning the program only made things worse. If a few kids pass the AP Exam and a few place out of first year college courses, then a high school feels successful. How many and what that actually means a student can do with the language may be secondary. I was accused of having assumptions not based on any observation of their classes. What should have followed was an invitation to observe their classes. I have had an invitation to observe my classes pending since 2 years ago. Later this year, the HS has agreed to observe a class. I’m still waiting on my invite.
7. There is lots of teacher turnover such that the textbook vocabulary and grammar benchmarks are the only way to make sure kids are ready for the next level. I wonder at all if teacher practices and curriculum have anything to do with teacher retention . . .
8. There is personal conviction that grammatical accuracy FROM THE BEGINNING is important. It makes the program “academic.” Love of the language and conversational skills can be our only focus in K-6 programs and in the lowest track of every HS level, but grammar has to become a part of it in 7th grade and of all higher tracks at every level. A HS program isn’t doing it’s job if kids are graduating and making “simple” grammatical mistakes. I commented that WE teachers don’t always know whether a word is masculine or feminine. Of course, I couldn’t care less about those mistakes, since they have little to no communicative value. There is a key difference: my primary (sole) language goal is communication – conversational fluency. (Probably also the goal of the vast majority of students and parents).
9. Agree to disagree on the merits of a grammatical syllabus.
10. Whatever the reason, a HS feels that accuracy in writing is important. Grammar knowledge does help in that case.
11. Acquisition/immersion is misunderstood, I think, to mean that it requires exposure 24-7 to the language. I tried to say that there are no shortcuts, but that we can enhance in a classroom the elements that make the natural process work. I pointed out that as it is, the only kids who have a chance at really acquiring the language require study abroad. I think that goes over heads. When you bring up the 90%+ TL in the classroom statement, teachers will say they try for that.
12. HS programs do a lot of great cultural projects, but I don’t know how that furthers proficiency. Those projects, especially if they involve a field trip, are time away from CI.
13. Apparently, AP test graders tell FL teachers that if students make “simple” mistakes, like use the wrong definite article, then their scores suffer. If true, the AP test talks the talk, but doesn’t walk the walk.
14. I loaned a copy of Foreign Language Education: The Easy Way by Stephen Krashen and a pamphlet review of Warwick Elley’s FVR studies.
15. I also at some point made it clear that the ACTFL proficiency guidelines do not list what grammar has to be mastered at each level and that Int-Mid proficiency still means kids mix up the major tenses. I wonder if we could get teachers to state their goals in terms of the proficiency level (e.g. Int-Mid) they expect of a 4-year program. I would be interested in becoming OPI-certified in order to assess students on the ACTFL gold standard.
16. There is the idea that “we respect all methods” as equals. Realistically speaking, different methods aren’t going to give the same outcomes. And I said that content is closely tied to method such that if I were to teach to the high school content it would constrain my method. It should be clear from research or after teaching a while, which methods give which results. That is, if students are properly tested.
Despite knowing my stuff and writing strongly about my opinions, I’m not good at this kind of confrontation in person. It’s a weird spot to be put in to try and be nice while also trying to suggest a different and better way to do things. I called Laurie Clarcq afterward, since she offered to be my ear. She made some great points:
1) Wait until I have 3 years, which gives me “professional status,” so that I can’t be fired, before I say anything more.
2) People who end up loving a language fall in love in elementary and middle school, that no matter their experience in HS, many will continue their language goals regardless.
3) Many of the big-name TPRS’ers who have helped many districts shift to TCI were never successful at it in their own district. We can continue to make a difference for our kids and the kids in other districts.
I asked and Laurie also told me that I should not get parents involved. She said the only time to talk to the parents about programs and contrast my goals with the HS’s is if the parents come asking those questions of me. Blaine once told her that an admin said a teacher’s goal was to keep parents out of his office.
Eric, you wrote
13. Apparently, AP test graders tell FL teachers that if students make “simple” mistakes, like use the wrong definite article, then their scores suffer. If true, the AP test talks the talk, but doesn’t walk the walk.
If this is true, it is diametrically opposed to what the AP test graders I know have told me. I have colleagues from German, French and Spanish who are readers for AP – a couple of them are “table leaders”. They said that College Board’s instructions are to ignore any mistakes that do not impede communication. I suppose a hyper-grammarian reader could interpret that to mean something like, “This student wrote ‘la mapa’ instead of ‘el mapa’ and that meant I had to think a moment about the correct form; obviously that interfered with communication, so I will grade down.” According to my colleagues, though, that is decidedly NOT what is intended; grading is to be done holistically and based on clarity of communication. Grammatical accuracy comes into play only when it makes communication unclear.
Robert,
I find your clarification full of hope and encouragement!! Imagine that…HS teachers having to catch up with the College Board!
Laurie is a GEM! She said…
1) Wait until I have 3 years, which gives me “professional status,” so that I can’t be fired, before I say anything more.
3) Many of the big-name TPRS’ers who have helped many districts shift to TCI were never successful at it in their own district. We can continue to make a difference for our kids and the kids in other districts.
These are very poignantly stated tips for us all. #1 reminds us that we should be thoughtful and cautious about sharing our passion for CI and to stay employed. #3 is inspiring because we all want to share our passions for SLA so that we can create positive changes in the lives of people in the world but we all have obstacles. It great to know that even the best teachers and practitioners of TPRS have challenges and battles for these righteous endeavors.
In fact, I find there to be a stronger fellowship among us in this PLC because we all face these mental, emotional, and physical impediments…
Really glad you shared this Eric!