PERTEMUAN 13
ASSESSING LISTENING AND ASSESSING SPEAKING
(CHAPTER 6 & 7)
A. ASSESSING LISTENING
DEFINITION
• The nature of listening (Fang: 2008), means that the learner should be encouraged to concentrate on an active process of listening for meanings. Using not only the linguistics cues, but his non linguistic knowledge as well.
• In the modern view of listening, O’Malley and Chamot (1989) defines that listening comphrehension is actual and couscious process which the listener constructs meaning by using cues from contextual information and existing knowledge while relying upon multiple strategic resources to fulfill the task requirement.
LANGUAGE SKILLS:
1. Listening
2. Speaking
3. Reading
4. Listening
Listening is receptive skills. The importance of listening:
1. Often implied as a component of speaking.
2. Input in the successful of language acquisition.
3. Applicable in many fields workplace, education, home-context.
Assessment of listening must be made because we neither observe the actual act of listening nor the product.
WHAT MAKES LISTENING IS SO DIFFICULT?
• Clustering:
• Redundancy
• Reduced forms
• Performance variables
• Colloquial language
• Rate of delivery
• Stress, rhythm, and intonation
• Interaction
BASIC TYPES OF LISTENING
• Heaton (1988) argued that developing skills can be done through testing listening comprehension test.
• He, therefore, listed the two categories of auditory test:
1. Test of phoneme discrimination and of sensitivity to stress and intonation.
2. Test of listening comprehension.
• Brown (2004) stated that effective test or appropriate assessment designing must begun with the specification of objectives or criteria which can be classified on several types of listening performance.
THE PERFORMANCE OF LISTENING
MACRO
Extensive listening: developing the gist, a global/comprehensive understanding.
Selective listening: determining meaning of auditory input.
Responsive listening: understanding pragmatic context. MICRO
Intensive Listening: comprehending language structure elements.
DESIGNING ASSESSMENT TASKS: INTENSIVE LISTENING
• Phonological and morphological elements recognition
Phonemic pair, consonant
Test-takers hear: He’s from California.
Test-takers read: (a) He’s from California.
(b) She’s from California.
Phonemic pair, vowels
Test-takers hear: Is he living?
Test-takers read: (a) Is he leaving?
(b) Is he living
Morphological pair, -ed ending
Test-takers hear: I missed you very much.
Test-takers read: (a) I missed you very much.
(b) I miss you very much.
Stress pattern in can’t
Test-takers hear: My girlfriend can’t go to the party.
Test-takers read: (a) My girlfriend can’t go to the party.
(b) My girlfriend can go to the party.
One-word stimulus
Test-takers hear: vine
Test-takers read: (a) vine
(b) wine
• Paraphrase recognition
Sentence paraphrase
Test-takers hear: Hello, my name’s Keiko. I come from Japan.
Test-takers read: (a) Keiko is confortable Japan.
(b) Keiko wants to come to Japan.
(c) Keiko is Japanese.
(d) Keiko likes Japan.
Dialogue paraphrase
Test-takers hear:
Man: Hi, Maria, my name’s George.
Women: Nice to meet you, George. Are you American?
Man: No, I’m Canadian.
Test-takers read: (a) George lives in the United States.
(b) Goerge is American.
(c) George comes from Canada.
(d) Maria is Canadian.
DESIGNING ASSESSMENT TASKS: RESPONSE LISTENING
Appropriate response to question
Test-takers hear: How much time did you take to do your homework?
Test-takers read: (a) in about an hour.
(b) about an hour.
(c) about $10.
(d) Yes, I did.
Open-ended response to a question
Test-takers hear: How much time did you take to do your homework?
Test-takers write/speak:__________________________________.
DESIGNING ASSESSMENT TASKS: SELECTIVE LISTENING
Listening cloze
Test-takers hear:
Ladies and gentlemen. I now have some connecting gate information for those of you making connectings to other flights out of San Francisco.
Flight seven-oh-six to Portland will depart from gate seventy-three at nine-thirty p.m
Flight ten-forty-five to Reno will depart at nine-fifty p.m. from gate seventeen.
Flight four-forty to Sacramento will depart at nine-thirty-five p.m. from gate sixty.
And flight sixteen-oh-three to acramento will depart from gate nineteen at ten-fifteen.
Test-takers write the missing or phrases in the blanks.
Information transfer
Information transfer: single-picture-cued verbal multiple-choice
Test-takers see: a photograph of a woman in a laboratory setting, with no glasses on, squinting through a microscope with her right eye closed.
Tets-takers here: (a) She’s speaking into a microphone.
(b) She’s putting on her glasses.
(c) She’s has both eyes open.
(d) She’s using a microphone.
Information transfer: chart-filling
Tets-takers hear:
Now you will hear information about Lucy’s schedule. The information will be given twice. The first time just listen carefully. The second time, there will be pause after each sentence. Fill in Lucy’s blank daily schedule with the correct information. The example has already been filled in.
You will hear: Lucy gets up at eight o’clock every morning except on weekends.
You will fill in the schedule to provide the information.
Noe listen to the information about Lucy’s schedule. Remember, you will first hear all the sentence; then you will hear each sentence separately with time to fill in your chart.
Lucy gets up at eight o’clock. She has History on Tuesday and Thursday at two o’clock.
She takes Chemistry on Monday from two o’clock to six o’clock. She plays tennis on weekends at four o’clock. She eats lunch at twelve o’clock every day except Saturday and Sunday.
Now listen a second time. There will be pause after each sentence to give you time to fill in the chart.
Test-takers see the following weekly calender grid:
Monday Tuesday Wednesday Thursday Friday Weekends
8:00 Get up Get up Get up Get up Get up
10:00
12:00
2:00
4:00
6:00
Sentence repetition
Sentence repetition is far from flawless listening assessment task. This task may test only recognition of sounds, and it can easly be contaminated by lack of short-term memory ability, thus invalidating it is as an assessment of comprehension alone. And the teacher may never be able to distinguisha listening comprehension error from an oral production error.
DESIGNING ASSESSMENT TASKS: EXTENSIVE LISTENING
First reading (natural speed, no pauses, test-takers listen for gist):
The state of California has many geographical areas. On the western side is the Pacific Ocean with its beaches and sea life. The central part of the state is a large fertile valley. The southeast has a hot desert, and north and west have beautiful mountains and forests. Southearn California is a large urban populated by millions of people.
Second reading (slowed speed, pause at each // break, test-takers write):
The state of California has many geographical areas. // On the western side // is the Pacific Ocean // with its beaches and sea life. // The central part of the state // is a large fertile valley. // The southeast has a hot desert, // and north and west // have beautiful mountains and forests. // Southearn California // is a large urban // populated by millions of people.
Third reading (natural speed, test-takers check their work).
Communication stimulus-response tasks
Dialogue and multiple choice comphrehension items
Test-takers hear:
Direction: Now you will hear a conversation between lynn and her doctor. You will hear the conversation two times. After you hear the conversation the second times, choose the correct answer for question 11-15 below. Mark your answer on the answer sheet provided.
Doctor: good morning, Lynn. What’s the problem?
Lynn: well, you see, I have a terrible headache, my nose is running, and I am really dizzy.
Doctor: Okay. Anything else?
Lynn: I’ve been coughing. I think I have a fever, and my stomach ache.
Doctor: I see. When did this start?
Lynn: Well, let’s see, I went to the lake last weekend, and after I returned home I started sneezing.
Doctor: Hmm. You must have the flu. You should got lots of rest, drink hot beverages, and stay warm. Do you follow me?
Lynn: Well, uh, yeah, but . . . shouldn’t I take a medicine?
Doctor: Sleep and rest as good as medicine when you have flu.
Lynn: Okay, thanks, Dr. Brown.
Test-takers raed:
11. What is Lyyn’s problem?
A. She feels horrible
B. She run too fats at the lake.
C. She’s been drinking too manmy hot beverages.
12. When did Lynn’s problem start?
A. When she saw her doctor.
B. Before she went to tha lake.
C. After she came home after the lake.
13. The doctor said that Lynn _____.
A. Flew to the lake last weekend.
B. Must not get the flu.
C. Probably has the flu.
14. The doctor told that Lynn ____.
A. To rest.
B. To follow him.
C. To take some medicine.
15. According to the Dr. Brown, sleep and rest are _____ medicine when you have the flu.
A. More effective than.
B. As effective as.
C. Less effective than.
Dialogue and authentic question on details
Tets-takers hear:
You will hear a conversation between detective and a man. The tape will play the conversation twice. After you hear a conversation the time, choose a correct answer on your test sheet.
Detective: Where were you last night at eleven p.m., the time of the murder?
Man: Uh, let’s see, well, I was just starting to see a movie.
Detector: Did you go alone?
Man: No, uh, well, I was with my friend, uh, Bill, yeah, I was with Bill.
Detective: What did you do after that?
Man: We went out to dinner, then I dropped her off at her place.
Detective: Than you went home?
Man: yeah.
Detective: When did you get home?
Man: A little before midnight.
Test-takers read:
7. Where was the man at 11:00 p.m?
A. In a restaurant.
B. In a theatre.
C. At home.
8. Was he with someone?
A. He was alone.
B. He was with his wife.
C. He was with a friend.
9. Then what did he do?
A. He ate out.
B. He made dinner.
C. He went home.
10. When did he get home?
A. About 11:00.
B. Almost 12:00.
C. Right after the movie.
11. The man is probably lying because (name two clues):
1. ________________________________________
2. ________________________________________
Authentic listening tasks
a. Note-taking
This is usually done by the students of non-native English users while listening to the classroom lecturers by professors. Their notes will be evaluated in such a way so that it lacks some reliability.
b. Editing
Test-takers raed: the written stimulus material (a news sport, an email from friend, notes from a lecture, or an editorial in a newspaper),
Test-takers hear: a spoken version of the stimulus that deviates, in a finite number of factsor opinions, from the original written from.
Test-takers mark: the written stimulus by circling any words, phrases, facts, or opinions that show a discrepancy between the two versions.
c. Interpretive tasks
The tasks extend the stimulus material to a longer stretch of discourse and forces the test-takers to infer a response by answering a few questions in the open-ended form. The potential stimuli can be used are song lyrics, poetry (recite), radio/tv news reports and oral account of experience.
d. Retelling
The tase-takers listen to a story or news event and simply retell it or summarize orally or written. In order to show a full comprehension, the test tajers require to identify the main idea, purpose and supporting details.
B. ASSESSING SPEAKING
Basic Types of Speaking
- Imitative. It is simple the ability to parrot back a word or phrase or a sentence.
- Intensive. It is the production of short stretches of oral language. Ex: include directed response tasks, reading aloud, sentence and dialogue completion, limited picture-cued tasks.
- Responsive. The tasks include interaction and test comprehension but at the limited level of short conversations, standard greetings, small talk, requests, and comments.
- Interactive. The length and complexity of the interaction are more in interactive tasks than in responsive ones. The task sometimes includes multiple exchanges and/or multiple participants.
- Extensive. (monologue) The tasks include speeches, oral presentations, and story-telling. Oral interaction from listeners is either highly limited or ruled out altogether.
Assessment Tasks: Imitative Speaking
- Word repetition task
- Test-takers hear:
- beat/bit bat/vat
- I bought a boat yesterday.
- The glow of the candle is growing.
- Test-takers repeat the stimulus.
Scoring scale for repetiotion tasks
- 2 acceptable pronounciation.
- 1 comprehensible, partially correct.
- 0 silence, seriously incorrect.
Phonepass Test
- It elicit s computer-assissted oral production over a telephone. Test-takers read aloud, repeat sentences, say words, and answer questions.
- Part A: Test-takers read aloud selected sentences.
- Ex: Traffic is a huge problem in Southern California.
- Part B: Test-takers repeat sentences dictated over the phone.
- Ex: Leave town on the next train.
- Part C: Test-takers answer questions with a single word or a short phrase.
- Ex: would you get water from a bottle or a newspaper?
- Part D: Test-takers hear three word groups in random order and link them in a correctly ordered sentence. Ex: was reading/my mother/a magazine.
- Part E: Test-takers have 30 second to talk about their opinion about some topic that is dictated over the phone. Topics center on family, preferences, and choices.
- Scoring are calculated by a computerized scoring template and reported back to the test-taker within minutes.
Assessment Tasks: Intensive Speaking
- Directed response tasks
- Directed response
- Tell me he went home.
- Tell me that you like rock music .
- Tell me that you aren’t interested in tennis.
- Tell him to come to my office at noon.
- Remind him what time it is.
Test of Spoken English Scoring Scale (Read-Aloud Tasks)
- Pronounciation:
- Points:
- 0.0-0.4 frequent errorrs and unintelligible.
- 0.5-1.4 occasionally unintelligible.
- 1.5-2.5 some errors but intelligible.
- 2.5-3.0 occasionally errors but always intelligible.
- Fluency:
- Points:
- 0.0-0.4 slow, hesitant, and unintelligible.
- 0.5-1.4 non-native pauses and flow that interferes with unintelligible.
- 1.5-2.5 non-native pauses but the flow is intelligible.
- 2.5-3.0 smoothly and effortless.
Variations on Read-Aloud tasks
- Reading a scripted dialogue.
- Reading sentences containing minimal pairs. Ex: Try not to heat / hit the pan too much.
- Reading information from a table or chart.
Read-Aloud Tasks
- Advantages:
- Comparisons between students are quite simply.
- Tests are easy to prepare and to administer.
- Predictable output, practicality, and reliability in scoring.
- Disadvantages:
- It is in authentic, except in situations such as parent reading to child, sharing a story with someone, giving a scripted oral presentation.
- It is not communicative in real context.
Sentence/Dialouge Completion Tasks and Oral Questionnaires
- First, test-takers are given time to read through the dialogue to get its gist, then the tape/teacher produces one part orally and the test-taker responds.
- Ex: (p. 150) short dialogue (p. 151)
- Advantage: more time to anticipate an answer, no potential ambiguity created by aural misunderstanding (oral interview).
Picture-Cued tasks
- A picture-cued stimulus requires a description from the test-taker. It may elicit a word, a phrase, a story, or incident.
- Scoring scale for intensive tasks:
- 2 comprehensible; acceptable target form
- 1 comoprehensible; partially correct
- 0 silence; or seriously incorrect.
A Scale for evaluating Interviews
- Grammar
- Vocabulary
- Comprehension
- Fluency
- Pronounciation
- Task (the objective of the elicited task)
- Example: (p. 158)
Translation
- Translation is a communicative device in contexts where English is not a native language.
- English can be called on to be interpreted as a second language.
- Condition may vary from an instant translation of native word, phrase, or sentence to translation of longer texts.
- Advantages: the control of the output and easily specified scoring.
Responsive Speaking
- Question and answer
- Ex: 1. What is this called in English?
- (to elicit a predetermined correct response)
1. What are the steps governments should take, if any, to stem the rate of de-forestations in tropical countries? (given more opportunity to produce meaningful language in response)
Question Eliciting Open-Ended Responses
- 1. What do you think about the weather today?
- 2. Why did you choose your academic major?
- 3. a. Have you ever been to the U.S before?
b. What other counties have you visited?
C . Why did you go there? What like best about it?
Giving isntructions and Directions
- Ex: how to operate an appliance, how to put a bookshelf together, or how to create a dish.
- Scoring: based on
1. Comprehensibility
2. Specified grammatical/discourse categories.
Eliciting Instructions or Directions
- Test-takers hear:
- Describe how to make a typical dish
- What’s a good recipe fro making___?
- How do you access email on a PC computer?
- How do I get from___to___in your city?
- Test-takers respond.
Consideration of Paraphrasing
- 1. Elicit short stretches of output
- 2. The criterian been assessed:
a. Is it a listening task more than production?
b. Does it test short-term memory rather than linguistic ability?
c. How does the teacher determine scoring of responsibility?
Test of Spoken English (TSE)
- TSE is a 20-minute audiotaped test of oral language ability within an academic or professional environment.
- TSE scores are used by many North American institutions of higher education.
- The tasks are designed to elicit oral production in various discourse categories. (p. 163)
- Ex: sample items in TOEFL (p. 164)
- Scoring: a holistic score ranging from 20 to 60 (performance, function, appropriateness, and coherence)
Interactive Reading
- Oral interview: a test administration and a test-taker sit down in a direct face-to-face exchange and produced through a protocol of question and directives.
- It various in lenghth from 5 to 45 minutes, depending on purpose and context. Placement interviews may need only 5 minutes while Oral Preficiency Interview (OPI) may require an hour.
A Framework for Oral Proficiency Testing
- Four states: Warm-up, level check, Probe, and Wind-down.
- Warm-up: The interviewer directs mutual introductions, helps the test-taker become confortable with the situation, apprises the format, and reduces anxieties.
- Level check: Through preplanned Qs, the test-takers respond using expected forms and functions. Linguistic target criteria are scored.
- Probe: In this phase, test-takers go to the heights of their ability and extend beyond the limits of the interviewer’s expectation.
- Through probe questions, the interviewer discovers the test-taker’s proficiency. At the lower levels of proficiency, probe items may demand a higher range of vocabulary and grammar than predicted. At the higher levels, probe items will ask the t-t to give an opinion, to recount a narrative or to respond to questions.
- Wind-down:the interviewer encourages the teks-taker to relax with some easy questions, sets the t-t’s mind at ease, and provides information about when and where to obtain the results of the interview. This part is not scored.
- Content specifications (p. 169)
- Sample questions (p. 169-170)
Sample Questions of n Oral Interview
- 1. Warm-up:
- How are you? /What’s your name? /What country are you from? /Let me tell you about this interview.
- 2. Level check:
- How long have you been in this city? /Tell me about your family./What is your major?/How long have you been working at your degree?/What do you like your hobby?
- What is your favorite food?/Tell me about your exciting experience you’ve had.
- 3. Probe:
- What are you goals for learning English in this program?/Describe your academic field to me. What do you like or dislike about it?/Desribe someone you greatly respect, and tell me why you respect that person./If you were [president, prime minister] of your country, what would you like to change about your country?
- 4. Wind-down:
- Did you feel okay about this interview?/You’ll get your results from this interview next week./Do you have any question to ask?/It was interesting to talk with you. Best wishes.
The Success of an Oral Interview
- Clear administrative procedures (practicallity)
- Focusing the questions and probes on the purpose of the assessment (validity)
- Biased for best performance
- Creating a consistent, workable scoring system (reliability)
- Descriptions of the Oral Preficiency Scoring Categories (p. 172-173)
Role Play
- It is a popular pedagogical activity in communicative language-teaching classes.
- The test administrator must determine the assessment objectives of the role play, then devise a scoring technique that pinpoints those objectives.
- Ex: “Pretended that you’re a tourist asking me for directions”, “You are buying a necklace from me in a flea market, and want a lower price”.
Discussion and Conversations
- As informal techniques to assess learners, D and C offer a level of authenticity and spontaneity that other assessment techniques may not provide.
- (clarifying, questioning, paraphrasing, intonation patterns, body language, eye contact, and other sociolinguistic factors)
- Games
- Oral Proficiency Interview (OPI) guidelines (p. 177)
Designing Assessment: Extensive Speaking
- Extensive speaking tasks are frequently variations on monologues, usually with minimal verbal interaction.
- Oral Presentation:
- Ex: presenting a report, a paper, a marketing plan, a sales idea, a design of a new product, or a method.
- Rules for effective assessment: (a) specify the criterion, (b) set appropriate tasks, (c) Elicit optimal output, and (d) establish practical, reliable scoring procedures.
- Oral presentation checklist:
3. Excellent; 2. Good 1. Fair 0. Poor
- Content:
The purpose or objective of the presentation was accomplished.
- The introduction was lively and got my attention.
- The main idea or point was clearly stated toward the beginning.
- The supporting points were clearly expressed and supported well by facts and argument.
- The conclusion restated the main idea or purpose.
- Delivery
- The speaker used gestures an d body language well.
- The speaker maintained eye contact with the audience.
- The speaker’s language was natural and fluent.
- The volume of speech was appropriate.
- The rate of speech was appropriate.
- The pronounciation was clear and comprehensible.
- The grammar was correct and didn’t prevent understanding.
- Used visual aids, handouts, etc., effectively.
- Showed enthusiasm and interest.
- Responded to audience questions well.
Picture-Cued Story-Telling
- At this level, a picture/a series of pictures is used as a stimulus for a longer story or description.
- The objective of eliciting narrative discourse needs to be clear. (p. 181) (Tell and use the p. tense)
- For example, are you testing for oral vocabulary, (girl, telephone, wet) for time relatives (before, after, when), for sentence connectors (then, so), for past tense of irregular verbs (woke, drank, rang), or for fluency in general?
- Criteria for scoring need to be clear.
Retelling a Story, News Event
- Test-takers hear / read a story or news event that they are asked to retell.
- It differs from the paraphrasing task discussed above in that it is a longer stretch of discourse and a different genre.
Translation (of Extended prose)
- Longer texts are presented for the test-taker to read in the native language and then translate into English.
- Text vary in forms: dialogue, directions, play, movie, etc.
- Advantages: the control of the content, vocabulary, the grammatical and discourse features.
- Disadvantages: a highly specialized skill is needed.
REFERENCE:
Brown. H.D. 2004. LANGAUGE ASSESSMENT: Principle and Classroom Practices: Pearson.
- Assessing Listening (Chapter 6, p. 116-139)
- Assessing Speaking (Chapter 7, p. 140-184)
https://www.slideshare.net/mobile/irakhwati/assessing-listening-42086143. Accessed on 7 May 2020
https://www.slideshare.net/mobile/kheansokheng52/chapter-7-assessing-speaking. Accessed on 10 May 2020
Tidak ada komentar:
Posting Komentar