Yeh, Schwartz & Baule 2011

This article was downloaded by: [Chung Yuan Christian University] On: 05 September 2012, At: 00:22 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Reading Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/urpy20

The Impact of Text-Structure Strategy Instruction on the Text Recall and Eye-Movement Patterns of Second Language English Readers a

a

Li-Hao Yeh , Ana I. Schwartz & Aaron L. Baule

b

a

Department of Psychology, University of Texas at El Paso, El Paso, Texas b

Department of Educational Psychology, Counseling, and Special Education, The Pennsylvania State University, University Park, Pennsylvania Version of record first published: 23 Nov 2011

To cite this article: Li-Hao Yeh, Ana I. Schwartz & Aaron L. Baule (2011): The Impact of Text-Structure Strategy Instruction on the Text Recall and Eye-Movement Patterns of Second Language English Readers, Reading Psychology, 32:6, 495-519 To link to this article: http://dx.doi.org/10.1080/02702711.2010.495621

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/termsand-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden.

Downloaded by [Chung Yuan Christian University] at 00:22 05 September 2012

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Reading Psychology, 32:495–519, 2011 C Taylor & Francis Group, LLC Copyright ISSN: 0270-2711 print / 1521-0685 online DOI: 10.1080/02702711.2010.495621


THE IMPACT OF TEXT-STRUCTURE STRATEGY INSTRUCTION ON THE TEXT RECALL AND EYE-MOVEMENT PATTERNS OF SECOND LANGUAGE ENGLISH READERS LI-HAO YEH and ANA I. SCHWARTZ Department of Psychology, University of Texas at El Paso, El Paso, Texas AARON L. BAULE Department of Educational Psychology, Counseling, and Special Education, The Pennsylvania State University, University Park, Pennsylvania

Previous studies have demonstrated the efficacy of the Text Structure Strategy for improving text recall. The strategy emphasizes the identification of text structure for encoding and recalling information. Traditionally, the efficacy of this strategy has been measured through free recall. The present study examined whether recall and eye-movement patterns of second language English readers would benefit from training on the strategy. Participants’ free recall and eye-movement patterns were measured before and after training. There was a significant increase in recall at posttest and a change in eye-movement patterns, reflecting additional processing time of phrases and words signaling the text structure.

How do readers organize the ideas from a text into memory? Do they just list all the ideas one by one or put them in a meaningful structure? Meyer, Brandt, and Bluth (1980) have identified two strategies that readers typically employ: the list strategy and the structure strategy. Readers who use a list strategy connect the ideas or facts of a text simply according to the order in which they appear in the text. On the contrary, readers who use a structure strategy connect ideas of text based on a more meaningful and hierarchical structure and consequently remember more important information. When signal words, words or phrases that highlight the structural organization of a text, are present in a text, readers are more likely to use a structure strategy. According Address correspondence to Li-Hao Yeh, Department of Psychology, University of Texas at El Paso, 500 W. University Avenue, El Paso, TX 79968. E-mail: [email protected]

495


496

L.-H. Yeh et al.

to the “strategy switch hypothesis,” readers do not regularly encode a text’s overall structure when the signal words are absent, and therefore signal words improve readers’ recall by promoting readers to switch to the structure strategy (Lorch & Lorch, 1995). Since the likelihood of using a structure strategy can be influenced by external cues such as signal words, researchers have investigated whether it can be similarly increased through direct instruction. The Text Structure Strategy (TSS) (Meyer, Young, & Bartlett, 1989) is a training program designed to offer explicit instruction on how to use text structure and signal words to organize ideas during reading. This program emphasizes five basic structures (i.e., “plans” in Meyer et al., 1989) that writers use: description, sequence, causation, problem-solution (question-answer), and comparison. Each structure also contains several signal words that are usually associated with it (Table 1). Numerous studies (e.g. Mayer, 1985; Mayer & Freedle, 1984; Meyer & Poon, 2001; Meyer, Talbot, Poon, & Johnson, 2001; Meyer, Young, & Bartlett, 1989; Meyer, Talbot, Poon, & Johnson, 2001) have demonstrated the effectiveness of the TSS for enhancing recall for a variety of readers from elementary school age to older adults. Although the TSS has been widely tested with firstlanguage readers, surprisingly few published studies have examined whether its benefits extend to second language (L2) readers (Carrell, 1985; Raymond, 1993). Second language reading differs from native language reading in several important ways. First, reading in an L2 requires more working memory resources, and even highly proficient bilinguals have slower reading rates in their less dominant language (Segalowitz et al., 2003; Segalowitz & Segalowitz, 1993). Second, there is now much evidence that when bilinguals read in one of their languages, they simultaneously activate words from their other language; this introduces an added layer of competition (Dijkstra & Van Heuven, 1998; Schwartz & Kroll, 2006; Schwartz, Kroll, & Diaz, 2007). Third, bilinguals have smaller vocabularies than monolinguals, which can hinder their ability to comprehend text (Bialystok, 2001). Since there are several factors that make L2 reading more cognitively challenging, it is possible that reading strategies that take considerable working memory resources to execute may not be well-suited for L2 readers, particularly those in the beginning stages of acquisition. There have been a few studies demonstrating

497

Description Descriptive ideas that give attributes, specifics, or setting information about a topic. The main idea is that attributes of a topic are discussed. For example, a newspaper article describing who, where, when, and how. Sequence Ideas grouped on the basis of order or time. The main idea is procedure or history related. For example, recipe procedures, history of Civil War battles, growth from birth to 12 months. Causation Presents causal or cause-and-effect-like relations between ideas. The main is organized into cause and effect parts. For example, directions: if you want to take good pictures, then you must. . .; explanations: the idea explained is the effect and the explanation is its cause.

Writing Plan and Definition

TABLE 1 Five Basic Organizational Structures and Their Signals

(Continued on next page)

as a result, because, since, for the purpose of, caused, led to, consequence, thus, in order to, this is why, if/then, the reason, so, in explanation, therefore,

afterwards, later, finally, last, early, following, to begin with, to start with, then, as time passed, continuing on, to end, years ago, in the first place, before, after, soon, more recently,

for example, which was one, this particular, for instance, specifically, such as, attributes of, that is, namely, properties of, characteristics are, qualities are, marks of, in describing,

Signals


498 and, in addition, also, include, moreover, besides, first, second, third, etc., subsequent, furthermore, at the same time, another,

not everyone, but, in contrast, all but, instead, act like, however, in comparison, on the other hand, whereas, in opposition, unlike, alike, have in common, share, resemble, the same as, different, difference, differentiate, compared to, while, although, despite,

problem: problem, question, puzzle, perplexity, enigma, riddle, issue, query, need to prevent, the trouble; solution: solution, answer, response, reply, rejoinder, return, comeback, to satisfy the problem, to set the issue at rest, to solve these problems,

Signals

Note. Blanks indicate spaces provided for learners to add signaling words found in their everyday reading. Source. Meyer, Young, and Bartlett (1989).

Problem/solution The main ideas are organized into two parts: a problem part and a solution part that responds to the problem by trying to eliminate it, or a question part and an answer part that responds to the question by trying to answer it. For example, scientific articles often first raise a question or problem and then seek to give an answer or solution. Comparison Relates ideas on the basis of differences and similarities. The main idea is organized in parts that provide a comparison, contrast, or alternative perspective on a topic, e.g., political speeches, particularly where one view is clearly favored over the other. Listing Can occur with any of the five writing plans. For example, listing can occur when groups of descriptions, causes, problems, solutions, views, and so on are presented.

Writing Plan and Definition

TABLE 1 Five Basic Organizational Structures and Their Signals (Continued)



Impact of Structure Strategy on Second Language Reading

499

that, as with native-language reading, awareness of the underlying structure of text improves bilinguals’ recall in the L2 (Carrell, 1992). Second language text recall has also been shown to be affected by clear textual organization. Proficient L2 readers produce better writing summaries for texts using a causation or problem/solution structure relative to texts based on a loosely structured collection of ideas or descriptions (Kobayashi, 2002). Thus, teaching second language readers to recognize text structure should increase their text recall as well. Indeed, instruction of top-level rhetorical structure of expository context has been found to increase English as a second language (ESL) students’ text recall (Carrell, 1985). High-intermediate ESL students received five 1-hour explicit training sessions in a week, while participants in a control group completed vocabulary work. Results at posttest showed that participants from the experimental group had better text recall than the control group. In another, similar study benefits of TSS were observed for bilinguals’ reading in their first language (L1; English) even though the training was conducted in the L2 (French) (Raymond, 1993). In addition to providing converging evidence for the efficacy of TSS for L2 readers, the goal of the present study was to identify the specific cognitive mechanism(s) underlying the benefits of training on the TSS, particularly for L2 readers. One cognitive mechanism that can account for the benefits of TSS training is the freeing up of working memory resources for comprehension by virtue of making the organization of text more clear. Numerous studies have demonstrated that working memory capacity is a key aspect of reading comprehension (Carpenter & Just, 1989; Daneman & Merikle, 1996; Just & Carpenter, 1992). It has been proposed that successful reading involves the ability to integrate newly encountered information from text with previously processed information, and that the mental resources to compute and store this information are limited. Therefore, a reader’s available working memory capacity will affect the amount of information that can be activated and processed. Consequently, readers with less working memory capacity will benefit the most when text is made easier to process, either by virtue of simplifying its syntactic structure or including signal words to aid detection of text structure (Britton, Glynn, Meyer, & Penland, 1982; Lee & Tedder, 2003; Payne, Kalibatseva, & Jungers,


500

L.-H. Yeh et al.

2009; Walter, 2004). In one study (Lee & Tedder, 2003), monolingual readers with either high or low working memory capacity (WMC) were given a text to read in two different presentation styles. The first presentation style, hypertext, was similar to a website, and key terms and titles were highlighted. Readers had to click on links to obtain additional information. The second presentation style took the form of a traditional presentation and followed structures typically used by writers. The results showed that readers with low WMC performed better with the traditional passage than hypertext. On the contrary, readers with high WMC showed no difference in their recall. This result implies that the relative transparency of text is of particular influence on reading efficiency for readers with low WMC. As mentioned earlier, reading in a second language is particularly taxing of working memory resources; thus, how much working memory capacity is available is of particular importance (Payne et al., 2009; Walter, 2004). Another mechanism that has been found to correlate with the benefits of text structure is the enhancement of metacognitive skills during second language reading (Carrell, Gajdusek & Wise, 1998; Carrell, 1988; Carrell, Pharis, & Liberto, 1989). Carrell (1988) found that Spanish ESL students who were aware that using the gist of text, background knowledge, and text organization are effective reading tools were also better comprehenders. She identified three metacognitive abilities linked to better comprehension: (a) the ability to gauge cognitive resources, (b) the ability to monitor reading progress, and (c) the ability to develop compensatory strategies when needed. Although we have discussed the metacognitive and working memory accounts separately, these two mechanisms interact. In a recent eye-tracking study (Burton & Daneman, 2007), readers with low working memory span who differed in their level of metacognitive skills (mature or naive levels) read texts about either familiar or unfamiliar diseases. The pattern of eye movements made through the initial pass of text was similar across the two groups. However, the metacognitively mature readers made a significantly greater amount of look-back fixations (i.e., looking back or rereading) when encountering unfamiliar text. Thus, deficits in WMC can be overcome through effective use of reading strategies.


501


The Present Study The goals of the present study were to (a) replicate the benefits of TSS instruction for text recall for English as second language readers, and (b) examine how readers change their approach as they read through a text after receiving training on the TSS through the use of eye-movement monitoring. We hypothesized that changes in eye-movement patterns after training on the TSS would reflect enhanced metacognitive awareness and use of the underlying structure of text during comprehension. Eye-movement monitoring provides a precise record of online reading behaviors that are composed of fixations (the eyes are still for 50 milliseconds or more) and saccades (the eyes are in motion). Through examination of eye-movement records, one can assess the relative amount of time spent on a given area of text. It is generally assumed that increased time spent on a particular area reflects increased cognitive processing of that information (Rayner, 1998). In an extensive and seminal review, Rayner (1998) lists and describes various reading-related variables that have been shown to influence eye movements, including semantic relationships between words, phonological and lexical ambiguity, and discourse factors. Absent from that listing is the acquisition and use of reading strategies. To date very few published studies have directly examined the link between eye-movement patterns and reading strategies. Hyönä and colleagues (Hyönä, Lorch, & Kaakinen, 2002; Hyönä & Nurminen, 2006) have identified four types of readers based on a cluster analysis of their eye-movement patterns. In their analysis they identified “topic structure processors.” These are readers who frequently looked back to previous areas of text that were pertinent to its overall structure (e.g., subject headings). These readers tended to recall texts better than any of the other groups. Therefore, the field of text processing and reading comprehension will benefit greatly from data that demonstrate if and how the acquisition of reading strategies changes eye-movement patterns. We predicted that, after training on the TSS, readers’ eyemovement patterns would reflect increased processing time on areas of text signaling structure. We further predicted that this increase in time would not come with a cost to overall reading time.

502

L.-H. Yeh et al.

In other words, we expected that, as a consequence of increased metacognitive awareness of the utility of text structure for comprehension, readers would allocate a greater proportion of reading time to text signaling structure rather than simply inflating their overall reading time.


Method Participants Fourteen Spanish-English bilingual undergraduate students from the University of Texas at El Paso participated in the study. These students were enrolled in the program of English for speakers of other languages (ESOL) while this study was conducted. All participants were international students from Mexico and considered Spanish as their native language (L1). All participants earned 100 dollars for attending all sessions of the study. Data from two participants who did not attend all sessions were excluded from the analyses. These exclusions produced a final sample size of 12 participants. All participants completed a Language History Questionnaire (LHQ). As part of the questionnaire, participants rated their proficiency in English and Spanish on a scale of 1 to 10 (see Table 2). Participants reported acquiring Spanish earlier (1.9 years old) than English (13.9 years old), t(11) = 10.77, p < .01. As expected, participants rated their Spanish proficiency significantly higher than their English proficiency (9.6 versus 6.3) t(11) = 8.797, p < .01. It is important to note that their overall TABLE 2 Language Background of Participants Age of Acquisition (years) Reading Writing Speaking Listening Meaning Rating 1On

a scale of 1 to 10.

Spanish (L1) 1.9

Self-Reported Proficiency Rating1 9.6 9.2 9.7 9.6 9.6

English (L2) 13.9 6.2 6.2 5.4 6.9 6.3


503

rating in English was slightly above the midpoint of the scale, suggesting that they were proficient enough in English to successfully compete the TSS training. This was later confirmed by their text recall scores, reported below. Materials Downloaded by [Chung Yuan Christian University] at 00:22 05 September 2012

TRAINING MATERIALS

The materials used in the training program were adapted from Meyer et al. (1989). These materials consisted of instructional aids (e.g., a table listing all five text structures and a list of common signal words), and a variety of authentic texts for practice (e.g., advertisements, newspapers articles). Advertisements were selected from magazines and newspapers, as they provide a short and concise representation of different text structures. Articles of differing lengths were selected from books, magazines, and newspapers. For instance, “Pollution is a problem for our rivers. Polluted rivers are eyesores. They are also health hazards. One solution is to stop the dumping of industrial waste.” Later on, longer and more complicated texts were given to participants. Most of these longer texts had multiple small text structures embedded in a general structure that was suitable for the whole text. Moreover, these long texts usually had signals that belong to different plans, for example, “as a result” belonged to causation plan and “trouble” belonged to problem/solution plan. These longer texts provided participants with the essential practice of not relying solely on signal words or a single structure, but rather considering the text as a whole. Test Materials Passages for text recall and eye-movement analyses. Participants read two texts from Meyer et al. (1989) while their eye movements were monitored, “Infectious Arthritis,” and “How to Tell the Difference Between Geese and Ducks” (see Appendix). “Infectious Arthritis” follows a problem/solution plan, and “How to Tell the Difference Between Geese and Ducks” follows a comparison plan. These texts were relatively shorter than the texts in previous studies (Meyer et al., 1989; Raymond, 1993). There were two reasons for choosing the short texts. First, longer texts require more than one screen to display, which is problematic in eye tracking as the gaze position


504

L.-H. Yeh et al.

would need to be recalibrated for each new screen. This would cause interference to naturally continuous reading behaviors. Second, shorter texts were considered to be more appropriate since participants were ESL students who were not yet highly proficient in English. We confined our analyses to the recall and eye-movement patterns of the problem/solution-based text (“Infectious Arthritis”), since this structure is typically the most easily acquired (Mayer & Freedle, 1984). Second, problem/solution structures are largely used in technical writing today and are most likely to be frequently encountered through college studies (Horowitz, 1985). Finally, in order to draw valid comparisons of eye-movement patterns from pre- to posttest, it was necessary to present participants with the same exact passages at both testing times, recognizing that there would be some effects of retesting on the same materials. APPARATUS

An eye-tracking system (SR Research Ltd. Eyelink2) was used to monitor participants’ eye movements as they read the texts. This is a head-mounted video-based tracker consisting of three cameras mounted on a leather-padded headband. The spatial resolution is 0.01◦ in pupil only mode, and the sampling rate is 500 Hz. The texts were presented on the 17-inch Dell computer monitor. The camera setup included three steps, which were the camera adjustment, calibration, and validation. Camera adjustment ensured that participants’ pupils could be captured by the cameras even when they looked at the corners of the screen. The eye-tracker does not provide absolute gaze position but only measures the relative changes in gaze direction. In order to know precisely what a subject was looking at, a nine-point calibration and validation procedure was conducted to record the value that corresponds to each gaze position. For obtaining valid and repeatable eye movement data, every participant had to go through calibration and validation procedures before reading the text. These procedures took approximately 5 minutes. DESIGN

This study was based on a one-group pretest-posttest design (Shadish, Cook, & Campbell, 2002). In this design, participants take a baseline test before any intervention. Next, participants



505

receive the intervention. After intervention, they take the baseline test again. This design allows the researcher to observe the difference in participants’ performance before and after the intervention. This difference is usually considered due to the intervention. In this study, at pretest all participants read two texts while their eye movements were recorded and then recalled each text by typing in a Word file. After the pretest, participants received the TSS training program. The posttest was conducted three days after the training, and the procedure was the same as the pretest. Participants’ reading behaviors and recall performances at posttest were compared with pretest. Procedure Pretest All interactions with participants were carried out in the L2, English. After informed consent procedures, participants were tested in individual rooms and seated in front of a computer. Participants were fitted with the eye-tracking headband and the experimenter calibrated the system. Participants then read a practice text to accustom themselves to reading with the eye-tracker band on. Participants then read one of the two passages while their eye movements were monitored. When they completed reading the eye-tracking band was removed and they typed their recall into a Word file. Then the procedure was repeated for the second text. The order of the texts was counterbalanced across participants. After the reading task, participants completed a language history questionnaire in which they were asked to self-assess their proficiency in reading, writing, speaking, and listening in English and Spanish on a 10-point scale. The entire experimental procedure was completed in approximately 60 minutes. Text Structure Strategy Training The TSS training consisted of 6 hours of instruction, broken down into three 2-hour sessions, all provided within 1 week. The sessions generally followed the same materials and sequences as reported by Meyer et al. (1989). The goal for this training program was to teach participants to recognize the five main plans that authors


506

L.-H. Yeh et al.

often use to construct a text, as well as to detect corresponding signal words for each plan. Furthermore, participants also learned to organize their recall based on the same plan and signal words. Therefore, successful training on the strategy would result in participants’ success in (a) recognizing the signal words, (b) comprehending the text based on its plan, and (c) using the same plan and signal words to recall. The five plans and their related signal words were first introduced to the participants. Later on, participants learned to identify signal words and to recognize plans from advertisements and short texts consisting of one or two sentences. After this, participants expanded their knowledge of signal words, read longer texts, and organized their recall based on the plans that they recognized. Participants also learned to focus on the content structure of longer texts, which usually had several small plans embedded in an overall plan. Toward the end of training, participants practiced recognizing plans of texts that did not contain obvious signal words. They were also asked to read some “muddy” (terribly organized) texts and to organize them during recall. The sessions adopted a very interactive style; participants were encouraged to provide answers and opinions in the class and they worked collaboratively in groups. Posttest The posttest was administered three days after training. The procedure and materials were identical to those used at pretest. Analyses Text Recall Analyses To assess whether TSS training led to significant improvements in text recall, participants’ recall protocols were analyzed through the Top-Level Structure (TLS) and Quality scoring systems (Meyer, 1985; Meyer & Meier, 2008). TLS is a scoring system ranging from 1 to 9 that assesses whether readers utilize the correct plan or structure for organizing their recall (see Table 3). The Quality scoring system ranges from 1 to 6 and assesses the degree to which participants write down the correct content of


507

TABLE 3 Scoring Criterion for Top-Level Structure and Quality Scoring System TLS Score


1 2 3 4

Criterion Bizarre, mostly unrelated, or nothing from the passage Just description, just problem, or just solution Other structure—no problem/solution Recalls a problem and solution with no clear organization, i.e., has problem and solution in a few sentences embedded in a list of random content from passage, or may give problem/solution in only one sentence Knowing problem and solution but organized using other structure Organized as problem/solution without signaling words Organized as problem/solution with signaling words for problem Organized as problem/solution with signaling words for solution Organized as problem/solution with signaling words for both problem and solution

5 6 7 8 9

Quality Scoring System Score 1 2 3 4 5 6

Criterion Nothing about problem/solution or cause/effect Correct cause of problem or solution only Correct problem only Correct problem with cause in the problem Correct problem and solution Correct problem and solution with cause in the problem

Source. Meyer and Meier (2008).

texts. To gain high scores in the TLS scoring system, participants did not have to correctly remember the content of the text, but they had to recall the correct plan and use appropriate signal words in their recalls. For the Quality scoring system, on the other hand, participants needed to recall the correct content from the text. Independent raters scored the recall protocols with both systems. They closely followed the criteria outlined in the scoring manuals provided (Meyer & Meier, 2008). Raters were blind to whether a recall protocol was from pre- or posttest. The inter-rater

508

L.-H. Yeh et al.

reliability coefficient of TLS scoring was .80 and the inter-rater reliability coefficient for Quality scoring was .88. Eye-Movement Analyses


CRITICAL TEXT AREAS

Two types of critical text areas were identified prior to the experimental period for analyses: critical sentences and signal words. Critical sentences were defined as sentences that stated the specific problem and solution of the text. In the text “Infectious Arthritis,” the sentence “The infectious agent first causes one disease but then spreads into one or more joints, causing arthritis” was considered as the critical sentence reflecting the problem of the text. This sentence not only stated the problem, but the cause of the problem as well. The ability to recall the problem stated in a text as well as its cause are measured within the Quality scoring system adopted in this study (Meyer & Meier, 2008). The critical, solution sentence identified read: “Drug treatment to get rid of the infection usually clears up the arthritis completely, if it is begun soon after the joint symptoms began.” This sentence suggested a solution to eliminate the problem, infectious arthritis. In addition to critical sentences, signal words were also identified within the text. Signal words refer to words that signal the underlying structure of a text, and training on the TSS should help readers recognize these key words as they build up their mental representation of the text. The selected signal words for analyses were not limited to signal words that specifically belong to the problem/solution structure but also included signal words that were suitable for all five text structures learned during the TSS. There were two reasons for this selection criterion. First, before participants know which structure a text belongs to they need to pay attention to all kinds of signal words to ensure the correct structure. Second, although the text had a general problem/ solution structure, there was another substructure embedded within it (causal). Within the “Infectious Arthritis” text, nine different phrases were identified as containing words that signaled text structure. Five of these phrases contained the signal word, “cause,” and one phrase, “leads to,” signaling the cause for the problem in the text. Another phrase, “for example,” signaled the embedded description plan in the text. The remaining two


509

contained different words, signaling the solution part of the overall problem-solution plan (e.g., “drug treatment,” “get rid of”). Results


Text Recall Data TEXT READING TIME DATA

The average reading time of the text in pretest and posttest was measured to test whether participants increased their reading time on the texts. The average text reading time at pretest was 61.2 seconds (SD = 16.8), and the average text reading time in the posttest was 67.7 seconds (SD = 24.6). These reading times were not significantly different from each other (t(11) = .09, p > .05, Cohen’s d = 0.3). This demonstrates that participants did not simply increase their reading time in order to improve recall of text. TOP-LEVEL STRUCTURE ANALYSES

To gain high scores on the TLS scoring system, participants had to recall the correct plan and use appropriate signal words in their recall. Analyses performed on participants’ use of the appropriate text structure in their recall revealed an average score of 5.1 (SD = 2.97) at pretest, which increased to 7.4 (SD = 2.24) by posttest, a significant improvement (t(11) = 2.37, p < .05, Cohen’s d = 0.8). QUALITY SCORING ANALYSES

To gain a high score on the Quality scoring system, participants had to recall the correct problem and solution and also the correct causes of the problem and solution. Participants’ average score at pretest was 3.7 (SD = 1.03) and increased to 4.7 (SD = 1.09) at the posttest. The paired t-test showed that this improvement was also significant (t(11) = 2.23, p < .05, Cohen’s d = 0.94). Overall, analyses of participants’ recall protocols demonstrated benefits of training on the TSS for English as second language readers who are reading in their L2. In addition to a replication, these results also provided a manipulation check, ensuring that the training was indeed effective. Although some

510

L.-H. Yeh et al.


benefit in recall may be expected due to having read the passage before, participants not only showed a benefit in overall recall, but the quality of the organization of that recall as well.1 Having established this, we next analyzed what effects, if any, training on the TSS would have on participants’ eye-movement patterns.

Eye-Movement Data CRITICAL SENTENCE ANALYSES

It was hypothesized that readers who received TSS training would allocate a greater percentage of reading time to the critical sentences signaling the underlying structure of text. Analyses were conducted on two areas of text, the problem sentence and the solution sentence. The duration of all fixations made in a particular text area were added. This sum is referred to as the gaze duration and has been identified as the most appropriate measure to adopt when the interest area is more than one word (Rayner, 1998). The summed gaze duration for each critical text area was then calculated as a percentage of the total gaze duration for the entire passage (the sum of all fixations made throughout the passage of text). Percentages were used rather than the raw gaze durations for critical areas of text, because the latter were likely to change from pre- to posttest as a simple function of reading the passage a second time. Furthermore, percentages would provide a more direct test of our hypothesis that training on the TSS would lead to a reallocation of total reading resources. Separate 2 (TEST) X 2 (READING PASS) within-participant ANOVA’s were conducted on the percentage gaze duration for the critical problem sentence and critical solution sentence. The first independent variable was “TEST,” either pretest or posttest. Since Burton and Daneman (2007) found differences between high- and low-metacognitively aware readers only in rereading (or post-first pass) fixations, we included “READING PASS” (first pass versus post-first pass) as a second independent variable. This variable allowed us to test whether readers would apply the TSS during their first pass through the text, or later during a second-pass rereading. Critical problem sentence. In the analysis of the percent gaze duration for the problem sentence region, neither the main

Impact of Structure Strategy on Second Language Reading 15

511

First pass


Percentage

Post-first pass 10

5

0

Pretest

Posttest Test

FIGURE 1 Percentage of gaze duration on critical problem sentence.

effect of TEST (F (1, 11) = .07, p > .05, MSE = .48, η2p = .01) nor READING PASS (F (1, 11) = 2.7, MSE = 179.42, p > .05, η2p = .13) was significant. However, the interaction between these two factors was significant (F (1, 11) = 6.05, MSE = 142.62, p < .05, η2p = .36). This interaction reflected a change in how readers allocated the total percentage of time they spent reading the problem sentence across first-pass reading versus rereading. At pretest, readers allocated a significantly greater percentage of time reading the problem sentence during their initial pass through the text (11.0%; SD = 5.97) relative to rereading (3.6%; SD = 4.58), (t(11) = 2.60, p < .05, Cohen’s d = 1.37). At posttest, however, they allocated their time evenly across first and second passes through the text (7.3%; SD = 3.27 and 6.7%; SD = 6.01 respectively) (t(11) = 0.16, p > .05, Cohen’s d = 0.08) (see Figure 1). This change in pattern suggests that, at posttest, readers were reallocating a portion of the time they spent reading the problem sentence to later rereading—likely in preparation for recall. Critical solution sentence. For the analysis performed on the percent gaze duration for the solution sentence region, there were no significant main effects of TEST (F (1, 11) = .16, p > .05, MSE = .953, η2p = .014) or READING PASS (F (1, 11) = .67, p > .05, MSE = 83.4, η2p = .057). The interaction was also not significant (F (1, 11) = .17, p > .05, MSE = 7.73, η2p = .015). Thus, unlike with the problem sentence, readers did not reallocate

512

L.-H. Yeh et al.

First pass

15


Percentage

Post-first pass 10

5

0

Pretest

Posttest Test

FIGURE 2 Percentage of gaze duration on critical solution sentence.

more reading time to rereading the solution sentence at posttest (see Figure 2). Signal Words Analyses The duration of all initial pass fixations made on all signal words combined was summed and calculated as a percentage of the total gaze time for the entire passage, as was the duration of all post-first-pass fixations. These percentages were submitted to a 2 (TEST) X 2 (READING PASS) within- participants ANOVA. The main effect of TEST was not significant (F (1, 11) = .1, p > .05), which meant that participants spent a similar proportion of gaze duration on the signal words at both pre- and posttest. The main effect of READING PASS was significant (F (1, 11) = 3.44, p < .05), which indicated that the percentage of gaze duration time at the first-pass reading was significantly longer than the post-firstpass reading time. The interaction of TEST and READING PASS approached significance (F (1, 11) = 4.91, p = .09). Planned comparisons showed that at pretest, a significantly greater portion of the time allocated to signal words occurred during the initial pass through the text (8%, SD = 2.4) relative to regressions back to those words (3.5%, SD = 2.2) (t(11) = 3.42, p < .01, Cohen’s d = 1.97). However, at posttest the portion of reading time allocated to signal words during initial pass versus rereading was evenly distributed (6.6% at initial pass versus 4.7% at post-first


513

15

First pass


Percentage

Post-first pass 10

5

0

Pretest

Posttest Test

FIGURE 3 Percentage of gaze duration on signal words.

pass), (t(11) = 1.01, p > .05, Cohen’s d = 0.58) (see Figure 3). Thus, a very similar pattern emerged in the changes of eye movements made for both critical problem sentences and signal words. This once again suggests that readers were reallocating reading time resources toward rereading in the interest of preparation for recall. General Discussion The findings of the present study extend previous findings, demonstrating that the TSS program can efficiently improve text recall for individuals with limited English proficiency. The improvement of recall in the top-level structure showed that ESL participants had developed a better awareness of the main structure of text. The higher-quality scores indicated that they not only used correct structure but also remembered more pieces of important information from the text. As far as we know, the present study is the first one to not only support the efficacy of the TSS program but to also look at the underlying cognitive mechanism that improves recall. The findings from the eye-movement data analyses demonstrated that participants preferred to look back more often and spend a higher percentage of time on the critical problem sentence and signal words after the training program. These results converge with those reported by Burton and Daneman (2007), in which readers with low WMC but higher metacognitive skills


514

L.-H. Yeh et al.

compensate by increasing look-back times. Although the readers’ WMC in L2 was not measured in this study, bilinguals typically do have lower WMC in their L2 (Service, Simola, Metsänheimo, & Maruy, 2002; Van den Noort, Bosch, & Hugdahl, 2006). Therefore, it is likely that the change in eye-movement patterns observed in the present study was reflective of enhanced metacognitive awareness and use of text structure to improve comprehension. The present results also converge with a recent eyemovement study demonstrating that successful readers spend more time focusing on key words in text (Van der Schoot, Vasbinder, & Horsley, 2008). Similar to Van der Schoot et al. (2008), in the present study participants allocated significantly more reading time to signal words after strategy instruction. This convergence is particularly noteworthy since, in the present study, the critical words were related to structure, whereas in Van der Schoot et al. (2008) they were related to topic. Finally, the fact that the effect of the strategy was reflected in second-pass reading times converges with the findings from Hyönä and colleagues (Hyönä et al., 2002; Hyönä & Nurminen, 2006) that look-back measures tap most directly into strategic processing. One unexpected result from the present study was that participants only showed a significant increase in reading time for the critical problem sentence and signal words, but not the solution sentence. This result can be understood through the metacognitive account. If participants were more metacognitively aware of using text structure while reading, once they reached the problem sentence of the text they would know that a solution sentence would soon follow. Therefore, since the solution sentence was predictable, participants did not have to pay extra attention to read it again. This explanation converges with Calvo’s (2001) study, in which participants spent less time on rereading continuous sentences when the preceding sentence contained predictive information. The reason why the change in eye-movement patterns for the signal words at posttest closely paralleled those observed for the problem sentence is because six out of the nine signal words were highly related to the problem portion of the text and selectively signaled that text (e.g, “cause,” “leads to”). Thus, readers were paying particular attention to any portion of the text related to



515

the problem, in anticipation of using it as the starting point for their later recall. One potential limiting factor in the present study was that, in order to precisely compare and contrast changes in eyemovement patterns after training, participants read the same text at pre- and posttest. One could argue that a portion of the observed improvements in recall were simply due to effects of repetition. However, if participants’ higher recall scores were solely due to repetition, one would only expect a simple reduction in overall text reading time, without any corresponding changes in eye-movement patterns. Participants’ total reading time was actually equivalent at pretest and posttest, and they indeed shifted their reading patterns by allocating a larger proportion of time on rereading the signals and the problem sentence. For future studies, one direction is to prolong the period of time between training and posttest to measure the long-term effect of TSS. Although Meyer et al. (1989) has shown that training effect can last for two weeks, no study has reported whether TSS can have similar, long-term effects on eye-movement patterns. The other direction is to look at whether ESL students can generalize what they have learned to their own academic work. Since ESL students’ academic success is one of the main applications for this type of research, addressing this issue can tell us whether students benefit from the training program, and what is the gap between the texts used in the TSS and in the real class. Moreover, if the gap is recognized, it can provide us with information about what can be done to modify the training program to fit students’ need. Acknowledgment The authors would like to thank Dr. Bonnie Meyer for her assistance throughout the project and the materials she generously shared with our research team. Note 1. In a separate, pilot study, proficient Spanish-English bilinguals who did not receive TSS training showed no significant increase in text recall for the same passage used in the present study when given a second chance to read and recall.

516

L.-H. Yeh et al.


References Bialystok, E. (2001). Bilingualism in development: Language, literacy, and cognition. New York, NY: Cambridge University Press. Britton, B. K., Glynn, S. M., Meyer, B. J., & Penland, M. J. (1982). Effects of text structure on use of cognitive capacity during reading. Journal of Educational Psychology, 74(1), 51–61. Burton, C., & Daneman, M. (2007). Compensating for a limited working memory capacity during reading: Evidence from eye movements. Reading Psychology, 28, 163–186. Calvo, M. G. (2001). Working memory and inferences: Evidence from eye fixations during reading. Memory, 9(4/6), 365–381. Carpenter P. A., & Just, M. A. (1989). The role of working memory in language comprehension. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The Impact of Herbert A. Simon (pp. 31–68). Hillsdale, NJ: Lawrence Erlbaum Associates. Carrell, P. L. (1985). Facilitating ESL reading by teaching text structure. TESOL Quarterly, 19(4), 727–752. Carrell, P. L. (1988). Second language reading: Reading, language, and metacognition. Revised version of a paper presented at the 22nd Annual Meeting of the Teachers of English to Speakers of Other Languages, Chicago, IL. Carrell, P. L. (1992). Awareness of text structure: Effects on recall. Language Learning, 42(1), 1–20. Carrell, P. L., Gajdusek, L., & Wise, T. (1998). Metacognition and EFL/ESL reading. Instructional Science, 26(1), 97–112. Carrell, P. L., Pharis, B. G., & Liberto, J. C. (1989). Metacognitive strategy training for ESL reading. TESOL Quarterly, 23(4), 647–678. Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, 3(4), 422–433. Dijkstra, T., & Van Heuven, W. J. B. (1998). The BIA model and bilingual word recognition. In J. Grainger & A. M. Jacobs (Eds.), Localist connectionist approaches to human cognition: Scientific psychology series (pp. 189–225). Mahwah, NJ: Lawrence Erlbaum Associates. Horowitz, R. (1985). Text patterns. Part II. Journal of Reading, 28(6), 534–541. Hyönä, J., Lorch, R. F. J., & Kaakinen, J. K. (2002). Individual differences in reading to summarize expository text: Evidence from eye fixation patterns. Journal of Educational Psychology, 94(1), 44–55. Hyönä, J., & Nurminen, A. (2006). Do adult readers know how they read? Evidence from eye movement patterns and verbal reports. British Journal of Psychology, 97(1), 31–50. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99(1), 122– 149. Kobayashi, M. (2002). Method effects on reading comprehension test performance: Text organization and response format. Language Testing, 19(2), 193–220.



517

Lee, M. J., & Tedder, M. C. (2003). The effects of three different computer texts on readers’ recall: Based on working memory capacity. Computers in Human Behavior, 19(6), 767–783. Lorch, R. F. Jr., & Lorch, E. P. (1995). Effects of organizational signals on textprocessing strategies. Journal of Educational Psychology, 87, 537–544. Meyer, B. J. F. (1985). Prose analysis: Purposes, procedures, and problems. In B. K. Britton & J. Black (Eds.), Analyzing and understanding expository text (pp. 11–64, 269–304). Hillsdale, NJ: Erlbaum. Meyer, B. J. F., Brandt, D. M., & Bluth G. J. (1980). Use of top-level structure in text: Key for reading comprehension of ninth-grade students. Reading Research Quarterly, 16(1), 72–103. Meyer, B. J. F., & Freedle, R. O. (1984). Effects of discourse type on recall. American Educational Research Journal, 21(1), 121–143. Meyer, B. J. F., & Meier, C. (2008). Scoring manual for top-level structure (TLS) and quality of recalls. Technical Report No. 4 of the Reading Comprehension: Structure Strategy Series. University Park, PA: Penn State. Meyer, B. J. F., & Poon, L. W. (2001). Effects of structure strategy and signaling on recall of text. Journal of Educational Psychology, 93(1), 141–159. Meyer, B. J. F., Talbot, A. P., Poon, L. W., & Johnson, M. M. (2001). Effects of structure strategy instruction on text recall in older African American adults. In J. L. Harris, A. Kamhi, & K. Pollock (Eds.), Communication and literacy in African Americans (pp. 233–264). Hillsdale, NJ: Erlbaum. Meyer, B. J. F., Young, C. J., & Bartlett, B. J. (1989). Memory improved: Enhanced reading comprehension and memory across the life span through strategic text structure. Hillsdale, NJ: Erlbaum. Payne, T. W., Kalibatseva, Z., & Jungers, M. K. (2009). Does domain experience compensate for working memory capacity in second language reading comprehension? Learning & Individual Differences, 19(1), 119–123. Raymond P. M. (1993). The effects of structure strategy training on the recall of expository prose for university students reading French as a second language. The Modern Language Journal, 77, 445–458. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. Schwartz, A. I., & Kroll, J. F. (2006). Bilingual lexical activation in sentence context. Journal of Memory and Language, 55, 197–212. Schwartz, A. I., Kroll, J. F., & Diaz, M. (2007). Reading words in Spanish and English: Mapping orthography to phonology in two languages. Language & Cognitive Processes, 22(1), 106–129. Segalowitz, N., Folk, J. R., Morris, R. K., Binder, K. S., Dopkins, S., Rayner, K., et al. (2003). Individual differences in second language acquisition effects of syntactic category assignment on lexical ambiguity resolution in reading: An eye movement analysis. Memory and Cognition, 31(1), 87–99. Segalowitz, N. S., & Segalowitz, S. J. (1993). Skilled performance, practice, and the differentiation of speed-up from automatization effects: Evidence from second language word recognition. Applied Psycholinguistics, 14(3), 369– 385.


518

L.-H. Yeh et al.

Service, E., Simola, M., Metsänheimo, O., & Maruy, S. (2002). Bilingual working memory span is affected by language skill. European Journal of Cognitive Psychology, 14(3), 383–408. Shadish, R. W., Cook, D. T., & Campbell, T. D. (2002). Experimental and quasiexperimental designs for generalized causal inference. Boston, MA: Houghton Mifflin Company. Van den Noort, M. W. M. L., Bosch, P., & Hugdahl, K. (2006). Foreign language proficiency and working memory capacity. European Psychologist, 11(4), 289–296. Van der Schoot, M., Vasbinder, A. L., & Horsley, T. M. (2008). The role of reading strategies in text comprehension: An eye fixation study in primary school children. Journal of Research in Reading, 31(2), 203–223. Walter, C. (2004). Transfer of reading comprehension skills to L2 is linked to mental representations of text and to L2 working memory. Applied Linguistics, 25(3), 315–339.

Appendix Infectious Arthritis Infectious arthritis refers to the arthritis that some people develop as a complication of another disease caused by a virus, bacterium, or fungus. The infectious agent first causes one disease but then spreads into one or more joints, causing arthritis. For example, one common cause of infectious arthritis is the bacterium that causes gonorrhea. In some people, this bacterium escapes from the genital organs and gets into the bloodstream, which carries it into the joints and leads to arthritis. Drug treatment to get rid of the infection usually clears up the arthritis completely, if it is begun soon after the joint symptoms began. After the swelling has gone down and the infection is gone, some people may need special exercises to rebuild strength in the affected area. How to Tell the Difference Between Geese and Ducks Male and female geese have identical plumage. Large birds with long necks, geese are good walkers. Geese, which molt only once a year, look the same at all times of the year. Nests are placed in hollows on grassy or marshy ground. Geese usually feed on land, grazing on grass and grain; brant, a sea-going species, feed on ellgrass. Though geese are powerful swimmers, they do not dive. Geese are strong fliers. They often form precise V’s or long,



519

irregular lines. They are quite noisy in flight. Male ducks are more colorful than females of the species. Small, plump birds with short necks, ducks have stubby legs and walk clumsily. Ducks molt body feathers twice a year. Most kinds have two plumages—breeding and nonbreeding. Males in eclipse (nonbreeding) plumage look like females. Ducks nest in various sites—some even in tree holes. All ducks can dive. Some species dive in to obtain fish; others eat mollusks or aquatic plants. In flight, most ducks form loose flocks.

Yeh, Schwartz & Baule 2011

Recommend Documents