Unlock with PatreonThe Problem with CI
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Question
Share:
Facebook
Twitter
Pinterest
LinkedIn
Ben Slavic
Get The Latest Updates
Subscribe to Our Mailing List
No spam, notifications only about new products, updates.
Related Posts
CI and the Research (cont.)
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Research Question
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
We Have the Research
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
15 thoughts on “Question”
When we use CI (vs. memorization, which brings false results) student growth can only really be shown through reading. It is because speaking takes years and thousands of hours that we don’t have, and writing the same. We can give them listening tests, but they are high anxiety producers. Besides, the research indicates strongly that different people absorb a language at different speeds, making it impossible to truly measure what any one person understands through their ears.
So reading. In NTCI we have them read the stories that they create in class. A nice post test at the semester would be to do a mock up of stories done during the semester. They will be able to read it. It will be a big confidence builder as well. (We don’t do the class novels as a class but they read them in Free Choice Reading time to start class.)
This kind of assessment is further discussed in ANATTY and maybe ANATS – I’m not sure on the latter.
Just a comment, honestly not meant to be snarky:
When we start measuring progress simply by asking kids if they are excited about what they are learning and if they want to continue on next year, we will have learned something.
Do Timed Writes and track the growth. There is always growth.
One time in New Delhi I stopped the CI – in one class only – to teach grammar (they were that kind of class). The other four classes continued on with the CI. This lasted three weeks, and when I did a free write for all classes, that one class showed no growth and the others did. Their graphs went down. So much for grammar study.
This is very helpful to me! I am creating small stories in class with my French 2 students, and they are loving it. I can definitely give them readings to do that would show growth. The only problem there is that we need to show how the students get better/show more progress; and I would want to give them different levels of reading material as we go along. Does that make sense? I mean that instead of saying Johnny scored 1 out of 4 and by the end of the year he should score at least 3 out of 4 on the same/similar material I’d rather state that Johnny will score the same high score he starts out earning on more complex reading material. In other words, I don’t want to let them score poorly using this model. I want CI class to be confidence-building, enjoyable as it can be, engaging, interesting. Our student learning objectives don’t usually follow this kind of pattern.
It’s the end of the school day, and I am rambling. If this helps or if you can help me, please reply. Thanks
Jennifer said:
… the only problem there is that we need to show how the students get better/show more progress; and I would want to give them different levels of reading material as we go along. Does that make sense?…
Yes absolutely. I appeal to the group. My own thinking on this is that the solution to honor Johnny’s efforts at every point in the year lies in creating easy texts all the way and sneakily grading them differently. The grade book as you go through the year would reflect Johhny’s hard work, but the documents submitted would be doctored. The research indicates that we cannot measure what they know and if that is true then we are within our rights to manipulate numbers when we assess. Why within our rights? Because they are using a grossly flawed model to make us do something that is completely in conflict with the research. The people who would check this out can’t read the language anyway so how could they ever come in and challenge us? And they never do. It’s all a big flawed game.
Here’s a correlation: the leveled reading system that just about every elementary school uses in the lower grades across the country. Students start reading level AAA (or something like that) in Kindergarten, when they are learning to read. The advance over a few years to level Z. Teachers mark their reading fluency and comprehension, and if the student is reading well a level F book, then they advance to level G.
I assuming that these leveled readers follow a line of progress that children go through in L1 acquisition. Important to note: I assume. The whole system could be flawed. I don’t know. The L2 acquisition for our older students is different. We don’t have to make sentences super simple nor only use monosyllabic words for our beginning level L2 students. Our students are capable of more complexity of thought. Also, the leveled reading program in English for little kids starts with easier sounds to make, like “m” in “mama” and moves to harder sounds like “k” in “skunk”. We don’t have to worry about that with our students in their acquisition of L2.
Now, Jennifer, some say that we can assess students acquisition of high frequency vocabulary. With more instruction, students will acquire more vocabulary on the high frequency lists. The top 10 in the first month. The top 100 in the first year. The problem is that our stories and our conversations become real dry and boring if all we do is use the Sweet 16 verbs. Our students need comprehensible and compelling. We need some low frequency but highly compelling words. Like today in one of my classes, Alhaji from Saudi Arabia (character made up) didn’t just drink the Sprite with lemon in the desert but sipped the Sprite with lemon. Sipping adds a wonderful image, rather than drinking. So, our stories differ from teacher to teacher and classroom to classroom. My level 1 students will show understanding of my stories that I create with them but not the stories of another level 1 teacher. Levels 2 & 3 as well. Even level 4.
Over time, however, I will speak to my students at greater length in the L2 and with greater complexity (i.e., greater number of short sentences used at a time, greater detail, longer narration, faster rate of speech, and longer sentences). And the readings will reflect these auditory narrations.
So, can we measure this growth? As much as Eric Herman spent so much time on this question and published a book helping teachers implement an assessment program for reading fluency and comprehension, called Assessing Spanish Proficiency with Stories, I’m afraid, as Ben says above, language is too nuanced to measure. After all of Herman’s work, I’m afraid we are still left floating in our sail boats with nothing but the wind (comprehensible, compelling input) to blow us towards land. In this case, land would be the intermediate level where students can start to read easy readers on their own and gains in language start to take that exponential curve so long as they are reading a lot.
Perhaps I’m contradicting myself here. It’s a tricky issue. Hard to explain other than as Krashen and Ben say, language acquisition cannot be measured. Language is too complex and nuanced to measure. So, save yourself some time and don’t fuss over it. Eric Herman was seeking the highest marks on his teacher evaluation from his admin. His admin wanted to see data. So Eric spent an ungodly number of hours trying to create a reading assessment system using Speed Reads with Comprehension questions to mark growth over the year. Personally, I’m okay with getting lower marks on my evaluation so long as my students feel good about learning in my classroom.
Well after 15 years I certainly and searingly oppose the class reading of novels for a million reasons I won’t go into here. So the only reading that level one and two kids should do is in the form of (a) reading their stories and (b) reading individual things that they choose in class (FCR).
Is the system flawed? Yes. It is because fifteen years of seeing mega dollars available for novels has (no blame) spurred a torrent of novels that are dubbed to be written at certain levels in TPRS, but the high frequency lists they use to write them differ and there is no way we have what they have in elementary reading scenarios.
… our stories and our conversations become real dry and boring if all we do is use the Sweet 16 verbs….
BAM!
Don’t get me wrong, I have great admiration for the work Eric has done. I remember very well his time here a few years ago. With all his work on finding ways to measure proficiency growth in the first few years, aside from Fluency Writes – which have limitations, in all seriousness – if what we are to get out of all his work is an affirmation that we can’t measure language acquisition, then, Lordy, what other spotlights do we need to shine?
I’d love to ask Eric that question: based on all the research you’ve done, do you think we can measure proficiency in a standardized manner?
Sean what you said here gave me hives:
…do you think we can measure proficiency in a standardized manner?….
WOW!
My answer is no, because of the unconscious piece, but I am not an expert like Eric Herman. Remember how when he was active here for those years he was formulating/refining his thinking on all this, mainly quoting Krashen and Paul Nation? I couldn’t keep up.
Fun fact: He was not accepted into the graduate program at Michigan State to work with VanPatten. I am almost sure it was because he represented a threat to the department. They probably tolerate Bill, but two Bills would be too much for them. I kind of have more respect for Eric than for VanPatten. The dude can interpret research better than anyone. I miss him, but get that he had to move on and start writing books and all. He has a heart of gold to go with his superior intellect.
Regarding measuring acquisition or proficiency I agree that it’s a chimera esp for novice low through intermediate low. So here’s something else we can do to get the monkeys off our backs. Give a beginning of yr assessment on a story that we will spin or tell or read – and the Ss will score poorly because it will have verb-containing chunks that they don’t know yet. The assessment will have random sentences from the story – (make them up – the story will be asked or collaborated – it’s just a rough stab) but the sentences won’t be giveaways – they will depend on comprehension of the verb (i.e., targeted).
Then we ‘teach the story’ – use those verbs that would come up anyway – and re-give the ‘test.’ Scores go up. If you must repeat this charade again later, then the next story contains the initial batch of verbs plus a buncha new verb chunks – same idea – growth over time. If you have bean counter maniacs, you can count up the words, the verbs, the tenses, the persons, the prepositional phrases, the clauses the gerunds and the indirect object pronouns – anything you want to demonstrate increasing linguistic complexity….
The idea being, to assuage the Data Monsters and get back to work. I hate doing any of this plus I think it misleads the data crunchers, but they don’t seem to comprehend the non-linear and unconscious nature of language acquisition anyway…. I had one say to me once, “If you love it, you can measure it.”
This is so good starting with a zero base line and then measuring later on is good. So let me make sure I understand – we just start the year in the first week with a story that they can’t possibly understand and then give the same story months later. No establishing. meaning a la TPRS, right? I think that this could even work at level 2.
Our JOBS are to predict the kids from predator data people who don’t get that what our students are acquiring is IMPOSSIBLE TO MEASURE BC IT IS UNCONSCIOUS. So we could even justify CHEATING by making the second story much easier. I did stuff like that all the time.
Ironic that those who are paid BIG BUCKS – esp. here in Denver bc we are a Gates Foundation metro district awash in money – divert funds from teacher hiring and training. But certainly nothing new.
Ben, I know what your answer is regarding the question on whether we can measure acquisition in some standardized way. And I get it. But Eric did some beastly work on trying to figure out if we could, and so I wonder what he would say to that question now, a couple of years after creating that Assessing Spanish Proficiency book.
I’m going to guess that he would say that it can be measured so long as you follow his guidelines in his book. (I skimmed through it recently.) But it would be so unnatural for me to follow his guidelines. I would be trying to cover high frequency lists and using cognates that he has listed in the book. In fact, by trying to cover these lists I would lose the compelling features of the stories I otherwise co-narrate with my students, and by losing the compelling features, I would slow down acquisition.
It would be powerful to hear Eric say that, “Truth be told, after all my research, I have to say that acquisition can not be measured.” I wonder.
“Regarding measuring acquisition or proficiency I agree that it’s a chimera esp for novice low through intermediate low.” What a beautiful word, Alisa, chimera. Assessing acquisition is a chimera!!
The reason I know that Eric would indeed say that acquisition cannot be measured after writing a book on measuring acquisition is that acquisition cannot be measured. His mind could set up an assessment scenario because he is a mental beast, but it’s not enough. Plus, you mention the loss of compelling input that results from trying to measure acquisition and that is a big deal. I think Krashen uses the term “restraint on interest”. You also said that “it would be so unnatural for me to follow his guidelines.” Eventually, we all must just relax. We have the research and, unlike other fields, in my opinion only, we don’t need any more. The late 20th c. was the time for the research to be done, the 21st c. is for its application. My own application of the research led me to the natural approach books away from TPRS which I consider stilted and I wish we would discuss the new stuff more here. I started a FB group dedicated to those books but am having tech issues but it should be running in a few weeks.