Assessing Listening Hanan Alqarni & Yuanyuan Sun Introduction This handout is submitted as a part of requirements for E 638: Assessment of English Language Learners. It is a supplemental material to the powerpoint created for a group presentation assignment in E 638 class. My colleague and I had to research on assessing listening, then reported the relevant information about the language area, a variety of sub-constructs which can be assessed in this language area, the kinds of research that is done in this area, the kinds of tasks a test developer could create, as well as how to score and report scores in this area.
Research on Assessing Listening is: Listening comprehension: A complex listening process to measure ● Interaction of different sources of information: the acoustic input (phonological modification, stress and intonation, etc.), linguistic (vocabulary, syntax, etc.) and nonlinguistic knowledge (knowledge about topic and context, etc.) ● A dynamic, ongoing process: constructing an interpretation of what the text is about, and continually modifying that as new information becomes available (Buck, 2001) ● A multidimentional process ● A very individual, personal process: listeners make different inferences and have different interpretations of the texts they hear (Buck, 2001) Models of Listening (Nation & Newton, 2009) ● Types of Listening: one-way listening: it is typically associated with the transfer of information (transactional listening). e.g. monologues such as lectures in academic contexts. two-way listening: it is typically associated with maintaining social relations (interactional listening). e.g. dialogues such as group discussions in academic contexts. ● Listening Processes: Bottom-up Processes: perceiving and parsing the speech stream at increasingly larger levels beginning with auditory-phonetic, phonemic, syllabic, lexical, syntactic, semantic, propositional, pragmatic and interpretive. Top-down Processes: the various types of knowledge involved in understanding knowledge are not applied in any fixed order.
Listening Sub-skills (Buck, 2001) Theorists have attempted to describe listening comprehension in terms of taxonomies of sub-skills that underlie the process using different approaches, such as Richards (1983) ‘s sub-skills model in conversational listening (cited in Buck, 2001). All the taxonomies should be treated with caution. Factors Affecting Listening (Ockey, 2012): rate of speech, background knowledge about the topic, accent, prosody (stress and intonation, etc.), types of interaction and relationships among speakers. Approaches to Assessing Listening (Buck, 2001) ● The discrete-point approach: it focuses on testing ability to recognise elements of the language in their oral form. Discrete-point tests generally use selected responses (True/False, multiple choice). The most common tasks used are phonemic discrimination tasks, paraphrase recognition and response evaluation. ● Integrative testing: it emphasizes assessing the processing of language (not only know about language but use it). The most widely used tests are tests of reduced redundancy (e.g. listening cloze and dictation) in which elements are removed thus reducing the redundancy of the text. ● Communicative Testing: it emphasizes communicative competence. Characteristics of test items include authentic texts, providing a communicative purpose and authentic tasks. Future Developments in Assessing Listening (Buck, 2001) ● Providing suitable texts: genuine v. authentic, etc. ● Visuals ● Collaborative listening: requiring skills in terms of listener roles in interactive situations: the addressee, participants who are not being directly addressed, over-hearers, etc. ● Computer-based testing
Designing Assessment Tasks: Intensive listening (the discrete-point approach) ●
Recognizing phonological and morphological elements:
●
○
Phonemic pairs. e.g. (is he leaving? vs. is he living)
○
Morphological pairs. e.g. (I missed you vs. I miss you)
○
Stress pattern. e.g. (suspect as noun vs. suspect as verb)
Paraphrase Recognition: ○ Sentence paraphrase ○ Dialogue paraphrase
Responsive listening ● ●
Appropriate response to a question ○ Multiple choice questions Open-ended response to a question (writing/ speaking)
Selective listening (integrative testing) ● ● ●
Listening Cloze (fill in the blanks) Information Transfer (chart filling) Note-Taking
Extensive listening ● ●
Top-down Improve listening fluency (listen to a TedTalk video and write a listening log)
Assessing Listening for Learning (Vandergrift & Goh, 2012) Formative Assessment
Learner checklists, teacher checklists, questionnaires, listening diaries, interviews (think-loud, stimulated recall), portfolios, dynamic listening assessment, quizzes
Summative Assessment
Quizzes Portfolios Achievement tests Proficiency tests Large-scale standard tests
Choosing formative and summative assessment tool for L2 listening must consider five criteria:
● Validity: to what degree does the assessment accurately measure what you want to measure? ● Reliability: to what degree is the assessment dependable? ● Authenticity: to what degree is the assessment representative of real-life language use? ● Washback: to what degree does the assessment provide useful feedback for the learner and influence the teaching process? ● Practicality: to what degree is the assessment amenable for classroom use, given administrative constraints?
Scoring: Selected response tasks: widely used in achievement tests and large scale standard tests, easy to score Limited response tasks: possible partial credits e.g. Gap-filling tests are usually scored by counting the number of gaps that are correctly filled, and using the sum as the total test score. In tests where test-taker has to replace deleted words (listening recall tests), there are two methods of scoring: to score the item as correct only when the actual deleted word is replaced, or to count any acceptable alternative. Extended production tasks: criterion referenced, rubrics
References Buck, G. (2001). Assessing listening. New York, NY: Cambridge University Press. Nation, I. S. P., & Newton, J. (2009). Teaching ESL/EFL listening and speaking. New York, NY: Routledge. Ockey, G. (2012). Assessment of listening. In C. Chapelle (Ed.), The encyclopedia of applied linguistics. Chichester, West Sussex, UK: Wiley-Blackwell. Vandergrift, L., & Goh, C. (2012). Teaching and learning second language listening: Metacognition in action. New York, NY: Routledge.
