Unlock with PatreonThe Problem with CI
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
April Test Prep 1
Share:
Facebook
Twitter
Pinterest
LinkedIn
Ben Slavic
Get The Latest Updates
Subscribe to Our Mailing List
No spam, notifications only about new products, updates.
Related Posts
CI and the Research (cont.)
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Research Question
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
We Have the Research
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
20 thoughts on “April Test Prep 1”
I’ve used Mark Davies’ Frequency Dictionary (2006) to guide my “content.” The verb visualization PPT has in the first few slides the rank frequency of the first 136 verbs all from the top 500 most frequent words. Then, back on the PVA article, I listed a few verbs that Davies’ listed as more common in speech.
As I become more and more aware of these high-frequency words and verbs, I realize my own speech in class doesn’t always include some of those words from the high-frequency list. I’ve gone through and marked those words I need to give more input on. I recommend teachers regularly read over these lists. I also realize that TPRS Readers do a great job including these structures! I’ve been strictly plowing (read-aloud 1-2 chapters/day) the books, choosing students to read the dialogue, and students stomp/clap/make funny sound when they don’t know a phrase/word. Now, when I don’t think everyone is following along, then I follow that chapter up with a 2-sentence dictation taken from that chapter. And what do you know, the kids are asking for more clarification 🙂
I’m really interested in creating my own CI-friendly input & proficiency-based tests, tests that best reflect real-life language (based on the high-frequency vocabulary, not biased to thematic vocabulary). You can’t share the DPS tests, but maybe you could share a sample or give us the format and some guidance on how to go about writing our own tests? For example, where do the listening and reading passages come from? Purrrrty please 🙂
Yeah, it’s very interesting that you guys in DPS are working on a district wide assessment where 70% of the score is based on input. Is this just reading input or listening input? And how in the world are you scoring it!? “Purrrrty please”
Sean it’s 35% listening and 35% reading. We work meticulously in June as two separate (French and Spanish) teams to create the tests. It’s amazing how hard the work is. Then we have these data people (in DPS we are awash in data money) run all kinds of validity tests on each item. I can’t understand any of it but somehow each year Diana figures out which items to throw out and rewrite for the next year.
I am still pissed that 30% was given to output. I don’t know why. That decision was made three years ago. Oh well.
I hope this kind of answers your question. Let me know. The real person to ask on that is Diana but she is so busy setting up this year’s testing and working with Carol on iFLT that she doesn’t read much here.
I think I get it. Each individual teacher administers this test in their classroom. So the listening test is where the teacher speaks, like, describes a picture, and the students identify which picture they’re describing. Then the reading section involves answering comprehension questions of a written story – maybe T/F, Multiple Choice, fill-in-the-blank kind of questions. Am I wrong?
If I’m describing this accurately then it would look a lot like what Carol Gaab has in her textbooks.
Depending upon how thoroughly you want to go into it, I think the questionnaires could be a great resource for deciding what three things you attach to each student. By basing them on students’ responses to the questions, you get even greater personalization.
Good idea Robert. Anne has always gotten a ton of mileage out of her questionnaires each year. If I were teaching next year I would get deeper into using them.
Ben, so you’re going to try to review – get a significant amount of reps in – the 100 top frequency verbs in 2 weeks? If you do 5 min on each verb (i.e. Eric’s 5 min power verb rep sessions), that’s 500 min, which is 10 x 50 min class periods. But you certainly can’t do back to back 5 min power verb sessions for so long. It would get painful.
I don’t know, how are you going to slam this all into such a short period of time? And without story-asking? Don’t you think that there is a limit to how much we can go on with our PQA/PSA without students wanting to direct the flow of the conversation? We risk shutting them out… they don’t care to listen if they lose interest in whatever message we may be trying to communicate in a one-directional force-feed.
Perfectly said Sean. I wrote the above before testing it, and when I tested it yesterday what you describe above is exactly what happened. It took all period to talk about Monica who trained for months for tennis and then in her first match this year broke her ankle in the first ten minutes of the match. It was then that I realized that it would really take all year to do what I wanted to do yesterday as a verb review. It was an excellent thing to learn, and you described it here perfectly:
…they don’t care to listen if they lose interest in whatever message we may be trying to communicate in a one-directional force-feed….
It’s just another argument in favor of Krashen’s Non-Targeted Hypothesis.
Honestly, when I read about all the gyrations here we got through in assessment, and how it impinges upon the natural flow of language which is so essential to our success in the classroom, I wonder (again and again) how it is possible that we can even do this work in schools. There is just too much bullshit impacting our instruction. That is why those who teach paying adults can rock the house. They aren’t thinking of anything but teaching, and the testing doesn’t enter into it. That is also why, as Susan Gross has told me many times, no teacher who uses CI should ever have her students do the National French/Spanish/German exams. There is a fundamental disconnect between teaching for fluency and teaching for a test. The X factor that creates that discrepancy is the problem of available time.
I happened to catch Bill Moyers interview Diane Ravitch last night. It is a powerful interview:
http://billmoyers.com/episode/public-schools-for-sale/
“There is just too much bullshit impacting our instruction,” as you say. Ravitch describes the onslaught of the bullshit with sobering clarity. A major point she’s making is that public education is a $500 billion industry that Wall Street types want to tap into. And they’re doing so with charter schools and using test scores to lure families into sending their kids there.
With that being said, I am still very interested in the district-wide assessments that you guys in DPS are working on. I certainly would like to be able to compare the scores of my students with students’ scores from other CI teachers and talk about what those differences might show. I do fear, however, that certain administrators and other non-educators would turn these scores against us.
Diana had to write them, to turn the pedagogical focus of all 100 WL teachers in our district to CI. Without having control over that change in how we test over the past five years, I doubt that Diana would have been able to change the number of CI teachers in DPS from about 5 six years ago to at least 80 now. Since the exit tests are CI based, any teacher still teaching grammar is screwed. But Sean what you say here:
…certain administrators and other non-educators would turn these scores against ….
is absolutely true. Most building administrators, perhaps all, only see the bar graphs, item analysis charts, and overall scores by teacher (teacher ranked against teacher per language). They don’t know or care about the details, the nature of the test, whether some teachers are new to the method, etc. They see the rankings (the bar graph ranking teachers by name is the most striking) and they basically judge the effectiveness of the teacher by that graph.
If the teacher is below average, that is how the label fits and the teacher has to wear that label in the minds of the administrators and in the minds of all teachers in the district since all teachers have access to the scores. It’s like you teach hard all year knowing exactly what the test is like, since your kids took a parallel pre-test in the fall, and then on one day an email comes and there are your scores telling you whether you are a good teacher or not. So teaching to the CI test is not an option – it must be done.
We have established in this discussion that teaching to tests is not something that is exactly friendly with true language acquisition, or at least I said that, because I believe that using the language to teach the language is by nature a very natural and lilting process, a soft one, and not one connected to evaluation of kids and teachers. Learning is soft and natural, and the best CI is non-targeted, but the exit tests require that we target most frequently occurring words, which is unnatural. So knowing that a test is coming in April is not a good thing for me at least. I’m just speaking for myself here – not many people agree with me in DPS on this point.
But many teachers in the district end up being nervous all year about getting all the words on the high frequency list taught (that list is on the DPS website). Targeting vocabulary is not something that I believe in, even when time limitations require it in the minds of some in our district. Diana made that exact statement to me yesterday in a discussion with Annick, that we must address the frequency list since we have but a fraction of the time first language acquirers have).
So, do you feel like you might be doing more harm than good with these common assessments?
I’m doing more harm to myself when in March each year I start thinking about the tests. I start trying to figure out what I didn’t teach, since I really don’t know, since I don’t plan. (I’m an upper right quadrant person on the Myers-Briggs and planning to me is not possible.)
Harm to my students? I don’t know. Is it helpful when a teacher starts teaching certain verbs and force feeding them because they are on a list but the teacher didn’t get enough reps on them during the year? You very eloquently talked me off that ledge a few days ago, and I really appreciate it, because you were reminding me of my tendency to compete with other teachers, which was a very dark part of my career when each year my kids had to win it all – and did – at the state and national levels of the National French Exams. That was some very unhealthy stuff going on with me there for a quarter of a century.
Isn’t it enough to just go in and hang out in the language with our kids? Can’t we just do that?
and as Eric below reminded me of what you wrote above, these common assessments may do harm if you try too heavily to teach to them, but they do tons of good by nudging all the foreign language teachers to teach CI.
I wonder if maybe Diana Noonan could make it so principals don’t get to see the data on these common assessments? I bet it would make for a much more productive conversation among the teachers if they knew that their data wasn’t being shared with their principals.
“Diana had to write them, to turn the pedagogical focus of all 100 WL teachers in our district to CI. Without having control over that change in how we test over the past five years, I doubt that Diana would have been able to change the number of CI teachers in DPS from about 5 six years ago to at least 80 now. Since the exit tests are CI based, any teacher still teaching grammar is screwed.”
Yesss! This is exactly what I have been saying! We all need help in writing CI-friendly, proficiency-based common assessments! That has been a catalyst for change in Denver and could be for teachers everywhere! It will give a lot of aspiring and current CI teachers the freedom to try out and teach with CI. So, I really wish the PLC members of this listserv would help to create a template for this CI exam. Like it or not, testing can spark change.
I have heard what people have said that listening is best tested with translation, but how do you do that except with dictation? And then when I suggested dictation, I was told that puts focus on form, rather than what we want: focus on meaning. So, I suggested reading a story 3xs and having kids write in L1 the story elements. I also want to know where we are supposed to get our listening/reading passages from or else, what are we to use as our guide in writing them? I figured the more high-frequency words included in the passage, the more accurately it represents real-life language. Somebody please critique my ideas. I need some constructive feedback. Don’t worry at all about being direct. That is what I want. If I were ever to propose an idea that was not CI/acquisition friendly, I would WANT people to call it what it is and I would never be offended!
I think testing is ONLY helpful if used to give feedback to the teacher to validate or modify instruction.
“Targeting vocabulary is not something that I believe in, even when time limitations require it in the minds of some in our district. Diana made that exact statement to me yesterday in a discussion with Annick, that we must address the frequency list since we have but a fraction of the time first language acquirers have).”
To clarify, Ben, you aren’t talking about whether non-targeted CI vs. targeted CI is better, but rather whether one set of vocabulary should be targeted vs. it not being important what vocabulary we target.
I thought it was you who has made this same statement to me before: We don’t have the instruction time to use non-targeted CI. For that reason, you were all about targeting input and all about optimizing the repetitions.
I understand that what you are saying here is that targeted CI is the way to go in a FL classroom, but “what” vocabulary is targeted shouldn’t matter. I agree with the latter, since the more we try to target a pre-prescribed set of words, the more constraint we put on interest and the more constraint we put on conversation in general. And conversation is not pre-planned as you often remind us with that quote from the French 🙂
“the best CI is non-targeted, but the exit tests require that we target most frequently occurring words, which is unnatural.”
Haven’t we been told that the zen part of this work is that if we don’t shelter our grammar, we will automatically and naturally include those high-frequency words? I know I’ve heard before that we just need to speak naturally with the target structures and we’ll be including the high-frequency words. Except that that is not completely true, since when I went over the high-frequency list, I realized a few words I tend not to use very often. Additionally, I think it takes a deliberate effort to include some of the high-frequency verbs. In order to best include the high-frequency verbs, seems they have to be part of the target structure. At least I am finding that the TPRS readers can fill in some of the “verb gaps,” those high-frequency verbs I haven’t yet included in my CI.
I think frequency lists, while extremely useful, can be somewhat deceptive in that the source material matters so much to the frequency. My best German frequency list is based on newspapers, so there are certainly a few forms or words on it that are somewhat less common than it used in common speech. Additionally, regional differences make a difference (in German at least).
You have me thinking, Eric, here about what vocabulary structures I choose to target. (Yes, I do target, or pre-prescribe as you say, specific vocabulary with my two Spanish 1 classes. It sounds like maybe you don’t. It seems like it would be hard to keep track of which vocabulary structures you’re covering over the course of the year with your classes if you don’t pre-prescribe them.)
Perhaps it’s not the best way to teach but I have NOT been using any story scripts from Anna Matava or Jim Tripp. I create my own story scripts mostly because I want to use a pre-prescribed set of vocabulary structures to 1. teach the top 20 high frequency verbs as soon as possible, 2. teach relatively important nouns for kids (i.e., family, clothing, pets, places, etc.), and 3. teach other words that help students communicate higher order thinking (Bloom’s) like: compare/contrast (i.e.; is similar to or different than, is more important than, is more popular than), value (i.e.; I like, I love, I don’t care about, I’m interested in), evaluate (i.e.; good, bad, beautiful, ugly, scary, strong), or analyze (i.e.; always, never, sometimes).
So, I guess, being that I pre-prescribe vocab structures as much as I do, if I had a colleague also doing TCI we could then pre-prescribe together, test our students at the end of the year with a common assessment and then have some meaningful conversations about the data this test would show.
I notice too, Ben, that the flow of the class is so much better when I am not focused on making sure we go in one direction, but rather leave that up to the kids. I feel they learn more when it’s what they want to learn versus us force feeding them vocab and ideas. I also love the freedom of picking stories based on what I think will interest that particular class, and not sweating over the fact that I did not touch upon a potentially popular phrase. Like you have always said, the students pick up on this stuff and the more laid back and relaxed we are, the more relaxed and open they are as well.
I know in the future my department will also be focusing more on data, but for now I am enjoying the freedom to teach as I wish. We have discussed merely having the kids write for a period of time each quarter to look at student growth in this fashion, but it is all in the infancy stage. I must say I dread the day when our vocab and grammar seems more targeted and forced. Who ever thought this was a good way to teach any subject? As you said, Ben, it is a lot of bullshit and when I can avoid it I will!
And it’s a lot of bullshit for a very good reason: if people think that they are being “taught at”, then their experience remains one of having their heads pumped with information, just as it happens in their other classes. But when we trick them into focusing on the message, simply because they are interested in it, because it is about them and things that interest them, their heads stop being receptacles for knowledge that we must teach them and become instead active participants in building the growing language system deep in the mind, where it stays. Honestly, I think we all have downplayed the importance of high octane personalization for far too long. Thank you for that lovely comment of support, Polly.
Holy shit, Ben. This is poetry to me:
And it’s a lot of bullshit for a very good reason: if people think that they are being “taught at”, then their experience remains one of having their heads pumped with information, just as it happens in their other classes. But when we trick them into focusing on the message, simply because they are interested in it, because it is about them and things that interest them, their heads stop being receptacles for knowledge that we must teach them and become instead active participants in building the growing language system deep in the mind, where it stays.
Ok, to speak from another angle… I do very much target vocabulary, and I have a plan (now, after a few years figuring out how I wanted to proceed) based on highest-frequency verbs, connecting words, and sentence structures. In fact, in 2013 Haiyun Lu asked a few CI Chinese teachers to help her work out a high-frequency list of structures and vocabulary we felt good for Chinese 1. But… there’s a lot of flexibility in the list. A whole subset of the list is “Spice” words — very high interest but possibly lower-frequency words and rejoinders. And high-frequency was defined by natural conversation of native speakers, also by the interests of our US students, and somewhat by topics we know that thematic, communicative teachers think is level 1 (that being the least important consideration though).
Anyway, I really like it. I feel free within some comfortable structure and that I’m getting the most out of the limited time with them. They will come out with real fluency (and a smaller number of frivolous nouns learned). I feel free to explore within the framework with my students how to use these high-power terms and phrases. A lot of my curriculum adapts with each group: all the discussion and the readings created from oral work will be at least somewhat re-developed with each class. I have begun using easy Chinese books as end points, though, and I don’t therefore have to create every single thing we read anymore. That’s very liberating, too. But I use the readers as a cap to encourage more reading after we’ve read our own class-created stuff. I think Chinese needs more repetitive yet varied readings. It just takes more time to catch characters in the unconscious mind, but it happens when supported well. There is still plenty of room for personalizing content.
Chinese reading is not visually obvious without a good bit of time with supported reading work, so I think it’s inappropriate to submit a beginning student (say, at least year 1 & 2) to any reading with any unknown characters in it, so there goes the truly untargeted input, I guess. In a language with cognates, you can count on many students “catching” those cognates, but not in Chinese. In Chinese you can (on rare occasions) put in familiar characters used in a new word, or use familiar words in a new sentence structure, but not a brand new character. I’d say the same about someone teaching Chinese from pinyin. Nothing new in reading without a lot of oral time on it first. Imagine teaching a language that has only a handful of cognates (and they are ALL food and technology loanwords to or from English) and that has a written system without phonetic indicators (at least until you’re really advanced). One has to proceed with greater care, I think. Does this makes sense to those who teach languages with the same alphabet and a lot of cognates?
So this means I am, at least now, feeling the benefit of targeting high-frequency language first. I do think it’s another area where Chinese instruction has to contextualize CI instruction differently from alphabet languages. I would love to hear from other Chinese teachers on this point (Liam, Tamula, you out there?), and our honorary PLC member, Annick.