2. Assessing Speaking (excerpts from by H. D. Brown, Language Assessment: Principles and Classroom Practices, Chapter 7. Pearson Longman 2004.) While speaking is a productive skill that can be directly and empirically observed, those observations are invariably colored by the accuracy and effectiveness of a testtaker’s listening skill, which necessarily compromises the reliability and validity of an oral production test. How do you know for certain that a speaking score is exclusively a measure of oral production without the potentially frequent clarifications of an interlocutor? This interaction of speaking and listening challenges the designer of an oral production test to tease apart, as much as possible, the factors accounted for by aural intake. Another challenge is the design of elicitation techniques. Because most speaking is the product of creative construction of linguistic strings, the speaker makes choices of lexicon, structure, and discourse. If your goal is to have testtakers demonstrate certain spoken grammatical categories, for example, the stimulus you design must elicit those grammatical categories in ways that prohibit the testtaker from avoiding or paraphrasing and thereby dodging production of the target form. As tasks become more and more open ended, the freedom of choice given to testtakers creates a challenge in scoring procedures. In receptive performance, the elicitation stimulus can be structured to anticipate predetermined responses and only those responses. In productive performance, the oral or written stimulus must be specific enough to elicit output within an expected range of performance such that scoring or rating procedures apply appropriately. For example, in a pictureseries task, the objective of which is to elicit a story in a sequence of events, testtakers could opt for a variety of plausible ways to tell the story, all of which might be equally accurate. How can such disparate responses be evaluated? One solution is to assign not one but several scores for each response, each score representing one of several traits (pronunciation, fluency, vocabulary use, grammar, comprehensibility, etc.). All of these issues will be addressed in this chapter as we review types of spoken language and microand macroskills of speaking, then outline numerous tasks for assessing speaking. BASIC TYPES OF SPEAKING /…/ We have cited four categories of listening performance assessment tasks. A similar taxonomy emerges for oral production. 1. Imitative . At one end of a continuum of types of speaking performance is the ability to simply parrot back (imitate) a word or phrase or possibly a sentence. While this is a purely phonetic level of oral production, a number of prosodic, lexical, and grammatical properties of language may be included in the criterion performance. We are interested only in what is traditionally labeled „pronunciation”; no inferences are made about the testtaker’s ability to understand or convey meaning or to participate in an interactive conversation. The only role of listening here is in the shortterm storage of a prompt, just long enough to allow the speaker to retain the short stretch of language that must be imitated. 2. Intensive . A second type of speaking frequently employed in assessment contexts is the production of short stretches of oral language designed to demonstrate competence in a narrow band of grammatical, phrasal, lexical, or phonological relationships (such as prosodic elements – intonation, stress, rhythm, juncture). The speaker must be aware of semantic properties in order to be able to respond, but interaction with an interlocutor or test administrator is minimal at best. Examples of intensive assessment tasks include directed response tasks, reading aloud, sentence and dialogue completion; limited picturecued tasks including simple sequences; and translation up to the simple sentence level. 3. Responsive . Responsive assessment tasks include interaction and test comprehension but at the somewhat limited level of very short conversations, standard greetings and small talk, simple requests and comments, and the like. The stimulus is almost always a spoken prompt (in order to preserve authenticity), with perhaps only one or two followup questions or retorts: A. Mary: Excuse me, do you have the time? Doug: Yeah. Ninefifteen. B. T: What is the most urgent environmental problem today? S: I would say massive deforestation. C. Jeff: Hey, Stef, how’s it going? Stef: Not bad, and yourself? Jeff: I’m good. Stef: Cool. Okay, gotta go. 4. Interactive. The difference between responsive and interactive speaking is in the length and complexity of the interaction, which sometimes includes multiple exchanges and/or multiple participants. Interaction can take the two forms of transactional language, which has the purpose of exchanging specific information, or interpersonal exchanges, which have the purpose of maintaining social relationships. (In the three dialogues cited above, A and B were transactional, and C was interpersonal.) In interpersonal exchanges, oral production can become pragmatically complex with the need to speak in a casual register and use colloquial language, ellipsis, slang, humor, and other sociolinguistic conventions. 5. Extensive (monologue). Extensive oral production tasks include speeches, oral presentations, and storytelling, during which the opportunity for oral interaction from listeners is either highly limited (perhaps to nonverbal responses) or ruled out altogether. Language style is frequently more deliberative (planning is involved) and formal for extensive tasks, but we cannot rule out certain informal monologues such as casually delivered speech (for example, my vacation in the mountains, a recipe for outstanding pasta primavera, recounting the plot of a novel or movie). MICROAND MACROSKILLS OF SPEAKING /…/ In the previous chapter, a list of listening microand macroskills enumerated the various components of listening that make up criteria for assessment. A similar list of speaking skills can be drawn up for the same purpose: to serve as a taxonomy of skills from which you will select one or several that will become the objective(s) of an assessment task. The microskills refer to producing the smaller chunks of language such as phonemes, morphemes, words, collocations, and phrasal units. The macroskills imply the speaker’s focus on the larger elements: fluency, discourse, function, style, cohesion, nonverbal communication, and strategic options. The microand macroskills total roughly 16 different objectives to assess in speaking. Microand macroskills of oral production Microskills 1. Produce differences among English phonemes and allophonic variants. 2. Produce chunks of language of different lengths. 3. Produce English stress patterns, words in stressed and unstressed positions, rhythmic structure, and intonation contours. 4. Produce reduced forms of words and phrases. 5. Use an adequate number of lexical units (words) to accomplish pragmatic purposes. 6. Produce fluent speech at different rates of delivery. 7. Monitor one’s own oral production and use various strategic devices – pauses, fillers, selfcorrections, backtracking – to enhance the clarity of the message. 8. Use grammatical word classes (nouns, verbs, etc.) systems (e.g., tense, agreement, pluralization), word order, patterns, rules, and elliptical forms. 9. Produce speech in natural constituents: in appropriate phrases, pause groups, breath groups, and sentence constituents. 10.Express a particular meaning in different grammatical forms. 11.Use cohesive devices in spoken discourse. Macroskills 12.Appropriately accomplish communicative functions according to situations, participants, and goals. 13.Use appropriate styles, registers, implicature, redundancies, pragmatic conventions, conversation rules, floorkeeping and –yielding, interrupting, and other sociolinguistic features in facetoface conversations. 14.Convey links and connections between events and communicate such relations as focal and peripheral ideas, events and feelings, new information and given information, generalization and exemplification. 15.Convey facial features, kinesics, body language, and other nonverbal cues along with verbal language. 16.Develop and use a battery of speaking strategies, such as emphasizing key words, rephrasing, providing a context for interpreting the meaning of words, appealing for help, and accurately assessing how well your interlocutor is understanding you. As you consider designing tasks for assessing spoken language, these skills can act as a checklist of objectives. While the macroskills have the appearance of being more complex than the microskills, both contain ingredients of difficulty, depending on the stage and context of the testtaker. There is such an array of oral production tasks that a complete treatment is almost impossible within the confines of one chapter in this book. Below is a consideration of the most common techniques with brief allusions to related tasks. As already noted in the introduction to this chapter, consider three important issues as you set out to design tasks: 1. No speaking task is capable of isolating the single skill of oral production. Concurrent involvement of the additional performance of aural comprehension, and possibly reading, is usually necessary. 2. Eliciting the specific criterion you have designated for a task can be tricky because beyond the word level, spoken language offers a number of productive options to testtakers. Make sure your elicitation prompt achieves its aims as closely as possible. 3. Because of the above two characteristics of oral production assessment, it is important to carefully specify scoring procedures for a response so that ultimately you achieve as high a reliability index as possible. (excerpts reprinted with the permission of the publisher)
Posted on: Fri, 11 Oct 2013 14:08:34 +0000