To view this content, you must be a member of Ben's Patreon at $10 or more
Already a qualifying Patreon member? Refresh to access this content.
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
To view this content, you must be a member of Ben’s Patreon at $10 or more Unlock with PatreonAlready a qualifying Patreon member? Refresh to
Subscribe to be a patron and get additional posts by Ben, along with live-streams, and monthly patron meetings!
Also each month, you will get a special coupon code to save 20% on any product once a month.
12 thoughts on “CI Friendly Common Assessments – 3”
I hope people will offer their suggestions here for CI-friendly assessments.
I was just informed that the local, grammar-oriented high school plans to write an exam in the next few weeks for 8th grade students (I teach grades 3-8) and this will determine their HS placement. The Freshman HS teacher is sold on TCI and will be able to influence the format and content of the placement test. So I need your suggestions fast.
I need the content to NOT depend on extensive knowledge of thematic vocabulary. Recommendations for how to select/write the listening & reading passages?
Who else has ever tried the “speed translate”? I just started doing them at the end of a cycle of TCI instead of doing the speed writes. The kids are reading and the time pressure limits monitor-use. I can re-read their translations and see how precisely they are processing the input. This has been a great confidence builder. And when the kids know they are going to be held accountable for every word, then they are so much more attentive and likely to ask for clarification. Just like the speed writes, these get assessed for quantity (they count the number of correctly translated words). I don’t grade these. At the end of the semester I’ll look at growth. If the kid is already translating at 80+ words/5 minutes, then I expect to see them maintain that rate.
At the end of a round of Reading Option A, which I do all in one 87 minute class period, I have students spend 10 minutes translating a section of the story. I’ve thought about how it might be beneficial to show my English translation of the story the next day and hand their translations back so they can see how well their translations match with mine. I haven’t done this yet because I’m not certain how I would want to assess/ grade this review of their translations. So far, I certainly don’t see much need to do anything more to motivate students to be on task during these translation sessions at the end of RoA than merely collect them, spot check them for quantity and quality, give a generic grade, and pass them back. But recently I’ve been frustrated with students’ lack of punctuation in their translations, so I’ve been marking their papers for that.
But yeah, grading these translations has been a nebulous affair for me. Perhaps the best way to assess is, as you say, have students keep their copies of their translations throughout the term so that they can see and reflect on their growth. As we know, students produce at very different speeds, and translate at very different speeds.
Coming back to the idea of showing my English translation of the story the next day… maybe I should ask students to make 10 corrections, and only 10. They get to choose which corrections they want to make and they must make them using a different colored pen (or something).
Come on people. All the great minds on this listserv. There has to be a better response to “What is the most CI-friendly assessment?” than just “the best assessment is no assessment.” While I agree with that logic, the world (outside of Denver) isn’t ready for that. I beg of you all.
I was just handed the HUGEST piece of crap Placement Exam that I have to give to my students, which was written by the grammarians at the high school. It’s all fill-in-the-low-frequency-thematic vocabulary word and pin-the-tail-on-the-verb. Understanding the instructions require the students know a 3rd language (linguistics). I kid you not, here is an example of the instructions: “Choose the correct conjugation of the following preterite tense boot verbs.” I’m sorry if I just sent you all to the bathroom to vomit.
I may not be able to change that exam, but I CAN give a separate test to my kids that would actually demonstrate their level of proficiency. With that separate test, I’d show everyone the results and I’d challenge the grammarians to give their students my proficiency exam.
I need a test that is practical (time and cost efficient), as proficiency-based as possible, and won’t be affected by teacher administration/grading bias. It needs to be a new context for everyone so that “practice” won’t skew the results. Is this possible? I know these conditions are not even close to ideal. Maybe I need some “communicative task-based learning” assessments (the goal is successful completion of the task).
I’m facing a similar situation, only I’m the one on the team. Our district has decided to create benchmarks for all the languages at the same time, and I am part of the consult. This gives me the advantage of being able to counter any specific grammar questions. After all, when the Spanish teachers want to test knowledge of the difference between preterite and perfect, I can counter with the fact that German doesn’t observe the same distinctions; German has no progressive verb tense; there is only one verb for “to be”; no “por/para” problem.
On the other hand, the adminsz (to borrow Chris’s convention) are stuck on “communicative approach” and want thematic units. I need to be able to present alternatives because I recognize – although others seem to be oblivious – that setting the benchmarks will set the curriculum for at least a decade and probably longer. At least I have a verbal acknowledgment from the person who will be “in charge” of the consult that we need to address ACTFL guidelines, 21st century skills (with emphasis on then/now comparison), CA State Standards, AP Vertical Teaming (i.e. 3 Modes of Communication), sla research, brain research, and Common Core. We will have some discussions on theory and philosophy early on. Could get interesting.
As far as placement exams are concerned, I think you need to isolate as much as possible the three modes. As has been noted by several TPRS practitioners, the only way to assess Interpretive Communication is by asking questions in English to make sure the issue is with the “text” and not with the instructions. I would suggest something like the following:
Interpretive:
1. Students receive three short texts (preferably narratives), each at a different level (Novice-Low, Novice-Mid, Novice-High) using high-frequency vocabulary. Student then chooses which text to interpret and either answers questions (in English) about the text or does the Essential Sentences assessment (choose the 1-3 Essential Sentences in the text and then illustrate them). Both the difficulty of the text chosen and the proficiency shown affect the placement. If a student chooses the Novice-Low text and simply nails it, the student should be in Novice-Mid; same thing if a student chooses the Novice-Mid and does reasonably well or Novice-High and is “Below Basic”.
Presentational:
1. Students receive three picture stories, choose one and, after a short period of preparation, present the narrative. This should show where student performs in oral presentation.
2. Students receive a choice of three topics and write a short essay or story about the topic.
Interpersonal
1. Students are given their choice of several “situation cards” and then have a conversation either between two students or with the assessor. (N.B.: This would probably be similar to the Classroom Oral Competency Interview.) Situation cards would be something like:
a. You are in a store and want to buy something. Ask an employee about the item.
b. You are talking with a friend about your plans for the weekend.
c. You are telling your father/mother about your day at school.
Here are the challenges:
1. Finding or creating appropriate texts
2. Creating appropriate situation cards
3. Calibrating the rating: in keeping with both ACTFL and AP guidelines, the grading needs to be done holistically; grammar errors are significant only when they interfere with comprehensibility.
4. Getting the grammarians on board with this kind of test
5. ??? (What did I forget?)
Here’s a little quiz I plan to have ready for my fellow consult members when the subject of knowing grammar comes up:
Please give the correct form for each of the following verbs:
1. to drink – 3rd person neuter singular present perfect active
2. to go – 2nd person plural future perfect active
3. to hang – 1st person singular future perfect passive
4. to speak to – 3rd person plural pluperfect passive
5. to equivocate with the idiom “to go” – 3rd person feminine singular future continuous active
6. to hang – 3rd person neuter singular pluperfect passive
7. to hear – 2nd person singular pluperfect passive
8. to lay – 3rd person masculine singular future perfect progressive active
9. to lie (= be in a horizontal position) – 3rd person feminine singular present perfect active
10. to be – 1st person singular pluperfect active subjunctive
Bonus: Use the verb in #10 in a conditional sentence.
If they can’t do well on this grammar quiz, how can they possibly consider themselves educated speakers of English?
I hope other weigh in on the topic.
I recognize that I didn’t address the issue of grader bias other than calibrating the rating. That’s part of the problem with this kind of assessment as opposed to discrete-item grading: language is not “cut and dried”, right or wrong. Even on the old AP exam the cloze section recognized that there were multiple right answers in many cases.
Awesome Robert!
I agree that these examples of the 3 modes would be a HUGE improvement.
The problem is, as you mention in the 2nd comment: grader bias. In my unideal situation, the test almost has to be discrete-item, with partial credits or multiple right answers, in order to control for the different ways teachers will interact with students and apply the rubrics.
I’m not a big fan of “situation cards.” These tend to be the situations that were rehearsed in traditional classes and they can very easily be designed to favor traditional themes. On the ACTFL OPI, a role play would not be utilized unless the person were demonstrating at least Novice-High level proficiency. Any other good options for oral interpersonal?
You’ve identified a lot of the major challenges. What texts to choose are also an invite to the traditional teachers to choose artificial texts that favor their thematic vocabulary. Maybe we should choose authentic texts (screen them to make sure for they include only the most frequent vocabulary words) and take a top-down embedded reading approach, creating embedded reading versions that match the ACTFL proficiency levels.
Since vocabulary has been found to correlate with proficiency, how about testing vocabulary? That’s probably the mindset behind the SAT/GRE, right? It’s quick. It’s easy to grade. And it’s unbiased. That would be right up the traditionalists alley, only the vocabulary we would be testing would be of the high-frequency kind. That would have a phenomenally positive backwash effect, leading to teachers to adjust the content of their curriculums to the high-frequency words.
I would love to get access to vocabulary formats and tests (e.g. Vocabulary Levels Test) designed for research studies, but in the FL (Spanish in my case). Check out these free vocabulary exams, which are among the most common in research: http://www.lextutor.ca/tests/
The idea is to test vocabulary from different frequency bands (e.g. 0-500, 500-100, etc.).
I remember that dictation was the most-suggested means on this blog of accurately testing listening. And I wrote a dictation almost entirely consisting of the 100 most frequent words. The problem is that it gets students focussed on form, rather than meaning. Well, dictation tests have been used to test vocabulary: http://www.victoria.ac.nz/lals/about/staff/publications/paul-nation/2000-Fountain-Dictation_test.pdf
Each paragraph of the dictation is testing words of different frequency bands, the listening chunks get longer and faster, and the grammar gets more complex. There is no translation aspect of the test. . . Cool.
That is a clever approach. It tests listening, sound-symbol correlation, and understanding of language in context.
A lot of students who show expertise in what you called “fill-in-the-low-frequency-thematic vocabulary word and pin-the-tail-on-the-verb” tests would do poorly at this.
I am considering something along the idea of word-frequency lists for my professional goal for next year. This would certainly support that. Thanks for including the link to the research paper.
I was wondering Eric about the necessity to have increased speed and larger chunks as one moves into a less frequent band. On the other hand, one could increase speed and have larger chunks within the same band of frequency. This is more a linear step-wise movement. Doing both (increasing speed/chunk-size while decreasing band of frequency compounds the level of difficulty, making it harder to know if a student is thrown by the vocabulary or the speed/chunking.
I agree. If I were to do this to test 1 variable (e.g. vocabulary), then I’d try to keep the rest controlled. Unless, maybe, if the variable was proficiency. . .
To test grammar not as knowledge ABOUT language, but to test what has actually been acquired and is available for language use, then you need to test grammar in context, in spontaneous, unrehearsed and unfamiliar conditions, and in ways that do not prime students to pay attention to grammar.
-Contextualized -Spontaneous -Unrehearsed -Unfamiliar -Unprimed
One surefire way to know if your test is proficiency-based is deciding whether or not 1 night of studying and cramming would help the students. If the answer is yes, then it’s not a proficiency-based test and you’re probably measuring performance or knowledge about language.
If any of the above conditions are not met, then you canNOT conclude that the language being evaluated has been acquired. For instance, knowledge about language stays that way unless teachers give the material depth (extensive reading/listening, concentrated reps in multiple contexts as in TPRS, etc.) and unless teachers develop fluency skills (increase speed of use with what has already been acquired, e.g. Quickwrite). Also, if the students know the focus is on a certain grammatical aspect, then they will be actively monitoring their output, something they more often than not cannot do in real-life. So don’t announce to them what grammatical aspect you are testing and don’t isolate 1 aspect of grammar to test!
We need to stop looking at language as isolated parts, but look at it holistically. Language is for communication. So evaluate communication. Evaluate grammar only in terms of how it affects meaning – how it affects communication. That is EXACTLY what the AP Exam does and it is what the ACTFL Proficiency Guidelines have been stressing since the early 90’s or before.
I’m thinking that one of the best ways to test grammar is to prompt students to do Quickwrites in various tenses. I’ve seen a PICTURE TASK used in SLA research that could work for us in either the speaking or writing skills. Students are shown a picture they’ve never seen before and they have to make up a story that answers: What happened before? What is happening? What will happen? Then, add some time pressure, e.g. give students 10 total minutes total, and emphasize that you are evaluating quantity of comprehended sentences. De-emphasize accuracy and do not let your grading overemphasize accuracy. Maybe give 1 point for each sentence you can comprehend (try to grade “comprehended sentences” as those that sympathetic, native speakers would understand). Like the NYS SLP, you could give bonus points for overall accuracy, fluency, and complexity, but those bonus points should be a tiny percentage of the final grade.
A level 1 TCI/TPRS-taught student would blow any traditional-taught student out of the water on such a test. Our students would not only write way more, but they would be way more capable of writing in multiple tenses. And in such proficiency-based conditions, I would expect our students to show superior grammatical accuracy.
“the adminsz (to borrow Chris’s convention) are stuck on “communicative approach” and want thematic units. I need to be able to present alternatives”
Two alternatives that come to mind are 1) a structural approach and 2) a word frequency approach.
1) By structural, I am thinking of the the structures that are so germane to TPRS, e.g., can do, wants to do, has to do, needs to do, likes to do.
2) By frequency I am thinking of the word frequency lists. The top word on some Spanish lists
is “que” (that/who/which) which his necessary for describing (the man that is tall), comparing, (taller than), reporting (she said that she was coming), influencing (she prefers that he go), reacting (it’s good that he’s coming), and so forth.
Structures are more communicative than “thematic vocabulary.” They are also global, across both human language and human experience.
Same goes for the 100, 200, etc. most frequent words.
It seems, thus, that they are something that could be agreed upon by all parties involved. How could someone argue against focusing on the most basic, most frequent, most communicative vocabulary?
The use of high frequency thus brings together words in natural but less predictable ways. One has to use TPRS skills to add interest to this core (in order to build the core): interesting nouns (vaca), verbs (hugs), and adjectives (interesting, green) /Proper nouns/exaggeration, bizarre, etc. But interest is the job of the individual teacher/class.
I am confident that I have said nothing that you do not already know, but perhaps the perspective will be helpful.