Dynamic Assessment of Reading Comprehension Ability: Group or Individualized
Tayebeh Fani1, Mojgan Rashtchi2, *
1Department of English Language, Science and Research Branch, Islamic Azad University, Tehran, Iran
2North Tehran Branch, Islamic Azad University, Tehran, Iran
To cite this article:
Tayebeh Fani, Mojgan Rashtchi. Dynamic Assessment of Reading Comprehension Ability: Group or Individualized. Education Journal. Vol. 4, No. 6, 2015, pp. 325-331. doi: 10.11648/j.edu.20150406.11
Abstract: According to Dynamic Assessment (DA) the goals of understanding individuals’ abilities and boosting their development are dialectically unified. The possibility of enhancing English as a Foreign Language (EFL) learners' reading comprehension ability through dynamic procedures which can be used both individually and collaboratively has scarcely been investigated. This study compared the impact of Individualized and Group DA on reading comprehension ability of EFL learners. One hundred and twenty-four undergraduate female students assigned to three experimental groups and a control group took part in the present study. The meditational protocols used in the intervention programs were based on Campione and Brown’s Graduated Prompt Moves Protocol (GPMP). The study revealed that mediation in Concurrent and Cumulative Group DA, as well as Individualized DA positively affected the Iranian EFL learners’ reading comprehension ability. However, although participants receiving Individualized DA outperformed the participants receiving Group DA, the difference among the effects of the three DA approaches was not significant. The present study proved the feasibility of DA in the large group setting as well as with the individuals.
Keywords: Dynamic Assessment, Group DA, Individualized DA, Reading Comprehension Ability, EFL
DA opposes traditional views on teaching and assessment by arguing that teaching and assessment should not be seen as separate activities but should instead be fully integrated. Most traditional assessments are called "Static Assessment" (SA) because they tend to measure learners’ actual development or what the learners have already learned (Feuerstein, Rand, & Hoffman, 1979). One of the main criticisms against SA is that learners’ performance cannot be considered static. According to Feuerstein, Feuerstein, and Falik (2010), SA is constructed in a way that allows no learning to take place during assessment since it measures a learner’s "crystallized intelligence" which does not allow the transfer of the learned principles to new situations, but DA treats one’s intelligence as "fluid" which can be shaped (pp. 90-91). Thus, in SA, the role of the assessor is to look for what is permanent, fixed, and unchanging in the learner. Feuerstein et al. (2010) argued that individuals have the potential to change and if provided with an appropriately-mediated learning environment they can modify themselves. People are not born with crystallized intelligence that remains fixed throughout life. The solution to the problems in SA, based on L2 researchers such as Lantolf and Poehner (2004), is to adopt DA where students are able to receive mediation to promote their potential learning.
According to Poehner (2009), current DA literature lacks sufficient empirical research on the application of DA in the L2 classroom, a context which does not lend itself to the common one-on-one format of DA due to the presence of a group of language learners. This may also be due to the lack of a robust theoretical approach to the concept of group learning and group assessment (Poehner, 2009). Group DA has paved the way in filling in this gap, both theoretically and practically, in psychological and educational settings. The goal of this study was to compare the impact of Group and Individualized DA on reading comprehension ability of Iranian EFL learners.
2. Review of Literature
The major difference between DA and other forms of assessment, as Poehner (2009) stated, lies in the DA basic premise that "fully understanding abilities requires active intervention in their development and consequently shifting focus from products of prior learning to the processes through which abilities are formed" (p. 471). Practically speaking, DA requires teachers and students to jointly carry out activities, with teachers intervening as necessary to help learners stretch beyond their current capabilities. This is quite different from the more familiar assessment model in which teachers observe student performance. This mutual engagement of teachers and students unifies teaching and assessment because teachers simultaneously discover where learners encounter problems and provide support to overcome these challenges (Poehner, 2009, p.472).
Poehner (2009, p.477) stated that Group DA entails understanding the group to be "not merely a context for individual performance but a social system in its own right that might be supported to function in ways that are beyond the present capabilities of any individual member". He goes on to argue that Group DA must engage the group in an activity that no individual is able to complete on his/her own but for which all members require mediation, albeit at different levels. Of course, the forms of mediation needed may likely vary from learner to learner at any given point in time. Bearing in mind the notion of a shifting focus of instruction, teachers may find it useful to track development of the individual’s ZPD within the broader group ZPD by foregrounding the support offered to an individual within the group and to examine his or her responsiveness. However, for Group DA to effectively promote the development of all group members, the teacher must actively engage the entire group in Group DA interactions.
Although there is a robust research literature on DA in psychology and general education (see Lidz & Elliott, 2000 for a review), the approach is relatively new in L2 studies. As Poehner (2008) rightly contended, although there is a growing interest in Vygotskian theory among applied linguists, few studies have explored how DA principles might be used in L2 contexts (e.g. Ajideh & Nourdad, 2012; Alavi, Kianpanah, Shabani, 2012; Antón, 2003, 2009; Davin, 2011; Gibbons, 2003; Kozulin & Garb, 2002; Naeini, 2013; Poehner, 2008; Tajeddin & Tayebipour, 2012). Group-based and one-to-one DA procedures follow the same general principle of offering learners mediation to help them co-construct a ZPD, but they differ in that that Group DA must also take account of the group’s ZPD.
Ableeva (2008) implemented DA in a large university setting with six students studying French. Ableeva wrote that this interaction revealed the learners' problem areas which centered on the inability to recognize known words, and the inability to determine the meaning of new lexical items. She concluded, "it should be emphasized that the problem areas were revealed only on the basis of participants’ performance throughout the DA stage, during which a flexible mediator-learner interaction was involved" (p.73). Antón (2009) also conducted a study in which examined the usefulness of DA in promoting the speaking and writing ability of five Spanish majors university students. The mediation protocol used in this study, similar to Ableeva (2008)’s study was non-standardized. The students were allowed to consult a dictionary and a grammar manual, as well as to ask the mediator questions. The mediator responded to questions by "adjusting intervention to what is needed in each individual case in order to complete the task and show the full potential of the learners’ ability" (p.592). Antón concluded that DA led to a deeper understanding of students’ abilities.
Kozulin and Garb (2002) implemented DA with English language learners. Their design followed a test-teach-test format to assess 23 academically at-risk students who failed to pass the high school English exam. Results showed that while some students were able to greatly improve their performance on the post-test, others did not show much development at all. They concluded that "A dynamic assessment of EFL text comprehension should therefore assess the student's ability to learn, activate and use effective strategies for text comprehension" (p.199). Poehner (2008) was another DA researcher who investigated the effect of DA on university students’ ability to correctly decide between two verb tenses (imparfait and passé compose) in French while narrating a movie. He offered mediation tailored to the needs of his students in both a near and far transcendence task and found that using DA gave him insight into the source of students’ errors. He also found out that the mediation resulted in improved understanding of these two tenses and aspect for the students.
The possibility of improving EFL learners' reading comprehension ability through Group and Individualized DA procedures has scarcely been investigated. The present study is significant because of the comparative investigation of the effects of individualized DA and two approaches of Group DA, namely Concurrent and Cumulative Group DA on reading comprehension ability. In other words, the study was an attempt to explore whether Concurrent Group DA, Cumulative Group DA, and Individualized DA differently affect Iranian EFL learners’ reading comprehension ability.
Over the years, summative tests prevail in college English reading classes all over Iran. The problems with summative tests mostly have to do with their uniform content and rigid criteria. Such assessment modes focus too much on the learning results to the point that they ignore the learning process, and therefore they are not conducive to the promotion of the students' reading competence. The present study focused on the impact of the three approaches to DA namely Cumulative Group DA, Concurrent Group DA, and Individualized DA on the reading comprehension ability of Iranian EFL learners.
The participants were 124 undergraduate students taking a general English course at IAU, Sama Technical and Vocational Training College in Tehran majoring in different fields such as accounting, architecture, computer, and physical education. The convenience sampling method was utilized in the present study because the participants were taken from intact classes for which the first researcher herself was the teacher. The reading section of the Key English Test (KET) was administered to ensure the homogeneity of the participants in terms of the English reading comprehension ability. The reliability of reading subsection of KET estimated through Cronbach’s Alpha was found to be acceptable (r = 0.82). Those participants whose scores fell within one standard deviation above and below the mean were selected. The results indicated no significant difference among the participants’ performance on the reading section of KET. The classes were then randomly assigned to three experimental groups and a control group (n=31). The participants of the three experimental groups underwent three types of DA procedures during the intervention phase named Concurrent Group DA, Cumulative Group DA, and Individualized DA. The participants in the control group followed a non-dynamic traditional instruction.
3.2. Data Collection Instruments
The first instrument was the reading section of KET used to ensure the homogeneity of the participants at the outset of the study. The second instrument was the reading comprehension pretest which was composed of four reading comprehension passages each followed by ten questions (except for test 3 which had twelve questions). The results of the pretest were used to identify the sources of problems EFL learners encounter in reading comprehension in order to develop the mediation program. Based on the results of the pretest, it was found that the participants’ major problems were in four areas of reading strategies including scanning, skimming, summarizing, and sequencing. The pretest reliability was estimated to be acceptable (r=0.73). The third instrument was the reading comprehension posttest which was similar to the pretest in terms of format, topic, difficulty, and length. It was used to evaluate the reading comprehension ability of the participants after the intervention programs comparing the performances of the participants in Concurrent Group DA, Cumulative Group DA, and Individualized DA with that of the control group. The posttest had acceptable reliability (r=0.79).
Beside the five instruments described above, three mediation programs were also designed, piloted, and carried out for the experimental groups during the study (Concurrent Group DA, Cumulative Group DA, and Individualized DA). The meditational protocols were based on Campione and Brown’s (1987) Graduated Prompt Moves Protocol (GPMP). Graduated Prompt (GP), as Campione, Brown, Ferrara, and Bryant (1984) stated, refers to a fixed menu of standardized hints and prompts, ranging from the most implicit to the most explicit one and culminating with the correct answer. The meditational typology in GP is generated a priori and then applied during DA sessions.
The main study was conducted in the first semester of the academic year of 2014-2015. In order to control the migrating effect of teacher variable and to ascertain the elimination of the possible differences in the implementation of the enrichment programs, all the mediations as well as the administration of the tests was done by the researcher. After administering the KET Exam to check the homogeneity of the participants, the classes were randomly assigned to three experimental groups and a control group. Then, the pretest was conducted to diagnose the students’ independent performance abilities. After that, to promote the learners' reading ability, an enrichment program lasting for eight weeks was offered. In this study, Lantolf and Poehner’s approach (2011) for devising meditational typology was utilized in which prefabricated mediating prompts were hierarchically ranked from the most implicit to the most explicit. The precise content of the prompts were tailored to meet the demands of the task and therefore varied somewhat from lesson to lesson, but they were typically organized so that the implicit end of the scale alerted learners that there was a problem, and the explicit end guided learners to correcting the problem. In concurrent Group DA, the teacher called a student to answer the first question while ensuring that the other learners were actively engaged. If the learner was unable to say the correct answer, the mediator offered the first mediation (the most implicit one) intended to help the learner. If she still experienced difficulty, the mediator would shift the interaction to another learner providing the second prompt. The interaction continued with different learners until the correct answer was found. In this group, the teacher interacted with the entire group. Although the teacher might provide mediation in response to an individual, the interaction shifted rapidly between the primary and secondary interactants as one learner’s question, struggle, or comment set the stage for another’s contribution.
In Cumulative Group DA, the teacher called a student to answer the first question while ensuring that the other learners were actively engaged. If she provided an incorrect answer, the teacher would provide that same student with the most implicit mediation prompt. The interaction continued in the same way between the teacher and the same student until she reached the correct answer. In this approach the students took turns "engaging directly as primary interactants with the teacher, with the understanding that each subsequent one-on-one exchange will have the advantage of building on earlier interactions that the class witnessed" (Poehner, 2009, p. 478). In the Individualized DA group, the teacher met the learners individually. The learners had no chance of being secondary interactants, rather, they were primary interactants all the time. The teacher provided the individual learner with prompts from the most implicit to the most explicit one until she got the right answer. Finally, the posttest was administered. The sessions were held once a week for a time period of 11 weeks.
Kolmogorov-Smirnov Test was used to test the normality assumption of parametric tests. It was found that the reading comprehension scores of the tests (the reading KET, the pretest, and the posttest) in the four groups were normally distributed (p>0.05) (see Table 1).
|Concurrent Group DA||Cumulative Group DA||Individualized DA||Control|
|Kolmogorov-Smirnov Z||Asymp. Sig. (2-tailed)||Kolmogorov-Smirnov Z||Asymp. Sig. (2-tailed)||Kolmogorov-Smirnov Z||Asymp. Sig. (2-tailed)||Kolmogorov-Smirnov Z||Asymp. Sig. (2-tailed)|
The degree of homogeneity between the three experimental groups and the control group was statistically determined using one way ANOVA for the continuous variables of the Reading KET and the pretest. First of all, the homogeneity of variances in the four groups was examined using Levene’s test. The results, as presented in Table 2, showed no significant differences among the four groups in terms of the reading KET and the Pretest (p>0.05).
The one-way ANOVA was also conducted to compare the homogeneity of the means of the KET and the pretest. As Table 3 shows the F values for testing the homogeneity of the means were not significantly different (F=1.119, F= 0.505, p>0.05). Therefore, the homogeneity assumption among the groups was met.
|Sum of Squares||df||Mean Square||F||Sig.|
Descriptive statistics, Levene’s test, and a one-way ANOVA were used to compare the mean scores of the groups in the reading comprehension posttest. The results of the descriptive statistics showed that the means and the standard deviations of the posttest (M=23.87, SD=3.75; M=23.65, SD=3.13; M=25.10, SD=3.27; M=19.19, SD=2.96) of the Concurrent Group DA, Cumulative Group DA, Individualized DA, and Control Group respectively, compared to the means and the standard deviations of the pretest (M=14.45, SD=3.08; M=15.26, SD=3.11; M=15.03, SD=2.18; M=14.84, SD=2.19) of the same groups (shown in Table 4) manifested improvement from the pretest to the posttest.
|Concurrent Group DA||Cumulative Group DA||Individualized DA||Control||Total|
|Mean||Std. Deviation||Mean||Std. Deviation||Mean||Std. Deviation||Mean||Std. Deviation||Mean||Std. Deviation|
Levene’s test of homogeneity of variances also showed that the significance level for the Levene’s value was less than 0.05 demonstrating a significant difference among the posttest scores of the four groups. In fact, the variance of the posttest scores in the Cumulative Group DA was the highest (Table 5).
|Concurrent Group DA||Cumulative Group DA||Individualized DA||Control Group||Levene Statistic||df1||df2||Sig.|
The result of the one-way ANOVA (provided in Table 6) depicted that the F-observed for comparing the mean scores of the four groups in the reading comprehension posttest (F=24.000, p=0.01<0.05) denoted significant differences among the mean scores of the four groups indicating that DA intervention did affect the participants’ reading comprehension ability.
|Sum of Squares||df||Mean Square||F||Sig.|
Then, Scheffe’s test was conducted to locate the exact places of the differences between the group means. The results of the post-hoc Scheffe’s test, as presented in Table 7, showed that the differences of the mean scores of the three experimental groups and the control group were statistically significant; however, no statistically significant differences were observed among the posttest mean scores of the three experimental groups. Group mean differences for the pretest and the posttest scores of the four groups are presented in Graph 1.
|I) Groups||(J) Groups||Mean Difference (I-J)||Std. Error||Sig.|
|umulative Group DA||Concurrent Group DA||-1.03226||.73976||.585|
|Individualized DA||Concurrent Group DA||.64516||.73976||.859|
|Concurrent Group DA||Control||5.06452*||.73976||.000|
* The mean difference is significant at the 0.05 level.
The findings of the study revealed that mediation in Concurrent Group DA, Cumulative Group DA, and Individualized DA does affect the Iranian EFL learners’ reading comprehension ability. However, although participants following Individualized DA outperformed the participants receiving Group DA, there weren’t any significant differences among the effects of Concurrent Group DA, Cumulative Group DA, and Individualized DA on Iranian EFL learners’ reading comprehension ability.
The Hawthorne effect was observed in the results of the experimental groups and the control group. According to Brown (1992), the Hawthorne effect referred to the fact that any intervention tended to have positive effect because of the attention of the mediator to the experimental subjects’ welfare. Given the homogeneity of the four groups in terms of reading comprehension ability at the outset of the study, the dramatic improvement of the participants in the three DA groups over those of the control group in the posttest could be attributed to the effects of the intervention.
Our findings were in line with those of Ajideh and Nourdad (2012) who designed a study to investigate the difference between applying dynamic and non-dynamic assessment of EFL reading comprehension ability and examining its immediate and delayed effects. The results of the study revealed significant difference between dynamic and non-dynamic assessment with a statistically significant increase in the reading comprehension scores of the group being assessed dynamically.
In the same vein, Naeini and Duvall (2012) reported the results of a research project aimed at studying improvements in English Language Training (ELT) university students’ reading comprehension performance by applying the mediations of a dynamic assessment approach to instruction and assessment. In their study, DA procedures were conducted with 10 ELT university students. The descriptive and analytic analyses of the results revealed dramatic, measurable progress in participants’ reading comprehension performance.
The findings were also in line with those of Lantolf and Poehner (2011) who examined how a K-5 Spanish teacher implemented dynamic assessment with a large group of students simultaneously. Birjandi, Daftarifard, and Lange’s study (2011) also provided similar results. They investigated whether it was possible to distinguish the quantitative and qualitative effects of dynamic assessment on the items and persons. They used two types of Rasch scaling to scale sets of wh-type questions and scanning items. The data gathered from 42 Iranian university students showed the anticipated quantitative improvement in learners’ performance on the posttest relative to the pretest—for the wh-type questions as well as for scanning items. The rating scale formulation proved to be a useful measure of ZPD as it proved to be a proper tool of capturing the pre- and post-test data simultaneously.
The findings of this study about the effect of dynamic assessment on reading comprehension ability of EFL learners were in line with the results of previous studies in the literature for example Kozulin and Garb (2002), Poehner (2008), Ableeva (2008), Birjandi, et al. (2011), Pishghadam, Barabadi & Mehri Kamrood (2011), and Naeini and Duvall (2012). However, the results about the equal advantage of Concurrent Group DA, Cumulative Group DA, and Individualized DA on reading comprehension ability of EFL learners were among the first attempts of applying DA in the classroom context.
Although different studies have been done on individualized DA, very few studies have examined the feasibility of large group DA in the foreign language classroom (Davin, 2011; Lantolf & Poehner, 2011; Poehner, 2009). Large group DA requires that an assessor work within the ZPD of the group as a whole, and that the assessor provide mediation calibrated to that group ZPD. As Haywood and Lidz (2007) wrote, an assessor must (1) determine whether the learner has the knowledge to proceed with the task; and, if not, (2) develop appropriate mediation for that learner for the task. This type of DA can be difficult with only one student, and even more difficult with a large group of students.
The findings of this study provided additional support for Vygotsky’s ZPD and the nature of development. The participants’ progress from the pretest to the posttest and the decreasing number of prompts needed throughout the study pinpointed the developmental moves of the participants’ ZPD to the zone of actual development. The successful implementation of two types of Group DA demonstrated the feasibility of Group DA in large university classes despite the inherent limitations. Providing learners with immediate mediations compatible with their problems proved to be useful in enhancing their reading comprehension ability. DA technique provided the teacher with a clearer idea of the individual students’ learning while also promoting development for students within the class.
In line with Poehner (2008), the present study advocated the use of mediations compatible with the learners’ level and problem. The findings revealed how Group DA interactions could help establish a community of practice (Lave & Wenger, 1991) in the social space of the classroom and how the students could benefit from the cooperative scaffolding provided by their classmates and teacher to resolve their reading comprehension problems.
Poehner (2009) argued that in order for large group DA to be effective, one must engage "learners in tasks that are challenging to all and [provide] support to benefit all" (p. 477). By engaging a group of students in a task that no individual student can complete alone but that all students are capable of completing with mediation, an assessor, or in this case a teacher, can work within the ZPD of the group as a whole. As Cioffi and Carney (1983) argued standard assessment procedures are best at evaluating the students’ skills knowledge, but insufficient for estimating the students’ learning potential and provide little help for identifying the conditions under which the progress can be made. The present study found that DA was feasible in the large group setting and did provide "support to benefit all".