EPSL-02-09-106-LPRU

The following is a critique of two reports by Joseph Guzman that have received wide press coverage:

Joseph Guzman, 2002. Learning English: New evidence on the effectiveness of bilingual education. Education Next, Fall, 2002. 2(3): 58-65. http://www.educationnext.org/20023/58.html

Joseph Guzman, 2001. Bilingualism and special language programs. http://educationnext.org/unabridged/20023/

Guzman (2002) is an abridged version of Guzman (2001). Data were taken from questionnaires filled out by 12,640 subjects in 1992, and included data from previous surveys done in 1980 when the subjects were in high school. Subjects included about 2,000 Hispanic students. The questionnaires were from the High School and Beyond Study.

Guzman concluded that being bilingual was advantageous, but participation in bilingual education was not. According to his multiple regression analysis, those who were bilingual completed about four-tenths of a year more in school, but those who participated in bilingual education, as contrasted with ESL only, completed about a half-year less of school. Guzman also reported that those in bilingual education were less likely to be in a high-skill occupation and earned less, but the differences in wages earned was not statistically significant.

The strength of this study is that it is a multiple regression analysis; Guzman was able to control statistically for a number of factors, such as socio-economic status, general academic achievement (as measured by tenth-grade math scores), and parents' birthplace. The problems, however, are many.

The Definition of Bilingual Education

The major problem is the definition of bilingual education. Subjects were asked whether they received (a) special instruction in English for non-English speakers, and (b) "foreign language instruction in a non-language topic" (p. 26). Those who answered "yes" to (a) and "no" to (b) were considered to be "ESL only." Those who answered "no" to (a) and "yes" to (b) were considered to be "bilingual education only." It must be pointed out that this is a strange group. It is very rare to have a bilingual education program that does not include an ESL component. It is also a practice that is condemned by virtually every specialist in the field of bilingual education. Quality bilingual programs introduce ESL on the first day, and introduce subject matter teaching in English as soon as it can be made comprehensible.

Subjects who responded "yes" to both (a) and (b), who did bilingual education and ESL, were not considered in the multiple regression analysis reported in the text of the paper. (Note 1) It should be pointed out that including such subjects would not solve any problems: A "yes" answer to both (a) and (b) could mean that the student had a quality bilingual program. It could also mean that the student was in ESL some years, and a low quality (English-free) bilingual program at other times. Guzman's analysis, thus, could mean that participating in a poor quality bilingual program will hurt academic achievement, as measured by years of school completed. The finding that ESL-only students did only slightly better does not necessarily speak highly of ESL alone.

Guzman explains that including "overlapping" ESL/bilingual education students would confound the results, because "they represent diametrically opposed methods. A student who reports receiving both cannot be said to have had an accelerated English transition or a delayed English transition" (p. 32). Guzman is apparently not aware of arguments claiming that proper use of the first language can accelerate second language development.

Additional Problems

Were All Subjects Once English Learners?

Subjects were asked these five questions:

1. What was your first language spoken at home?
2. What other languages did you speak as a child?
3. What language do you usually speak now?
4. What language do people usually speak at home?
5. What other languages are spoken at home?

Subjects responding other than English to any of these questions were then given the ESL/Bilingual education questions discussed above. Note that if a subject only mentioned a non-English language for question five, and none of the others, that subject was then asked the other questions, whether the subject knew anything of the language or not. Also, a subject could answer "yes" to all of the questions, and still could have entered school highly fluent in English.

Possible Confusion With Foreign Language Study

A foreign language student, that is, a fluent speaker of English studying another language, might answer "yes" to (b) if the student took a class in culture or history in the foreign language. As noted just above, such students were not excluded from the analysis.

What Kind of Bilingual Program? How Much?

"Yes" responses to (b) do not tell us how the classes were taught. It is established that some varieties of bilingual education are better than others. Specifically, concurrent translation, i.e., the teacher presenting in one language and then providing a translation, is not effective (Legarreta, 1979). We also do not know how much bilingual education was provided. A "yes" response to (b) could mean as little as one course in another language or as much as ten years of full bilingual education. Also, as noted in previous discussions of this kind of research (Krashen and McQuillan, 1998; Krashen, 1999), we are dependent for evidence on the memories of high school sophomores recalling on their elementary school experience.

Not Much Bilingual Education In Those Days

Jim Crawford has pointed out to me that high schoolers interviewed in 1980 would have been participants in bilingual education in the early 1970s, a time when bilingual education programs were quite rare. A 1969 survey covering states in which bilingual programs are the most prevalent—California, Arizona, Colorado and Texas—found that only 6.5% of the schools surveyed had bilingual education, and these programs reached only 2.7% of the Mexican-American population.

Bilingual education was not up and running until well after Guzman's subjects were in junior high school. It was mandated in Massachusetts in 1971, in Illinois and Colorado in 1973, and in Michigan and California in 1976; and compliance took a long time to achieve. In general, LEP children got little attention in those days; only 5.5% of Mexican-American children in the Southwest received ESL instruction (Crawford, 1995).

This makes it unlikely that significant numbers of Guzman's subjects participated in quality bilingual education or even in any kind of bilingual education. Guzman was aware of this problem, noting that bilingual education was in a "developmental stage" at the time his subjects entered school (p. 37); but he maintains that this is not a problem, citing an unpublished and unavailable paper (Note 2) that "largely validates the accuracy of language program exposures recorded in the HS&B (High School and Beyond)" (p. 37-38). Guzman provides no details of this study.

Conclusion

Guzman's study necessarily excluded students in quality bilingual programs, because it excluded those who reported taking ESL classes. In addition, subjects may or may not have been limited English proficient when younger, and if they had "bilingual education" it could have been one class taught with poor methodology. We have no idea. Also, this bilingual education experience occurred during a time when bilingual education was in its beginning stages in the U.S. and served very few students.

It is important to get longitudinal data on the long term effects of educational practices. Unfortunately, the data set Guzman used does not permit any valid inferences to be drawn concerning the value of bilingual education.

Notes

1. In a brief but fascinating footnote, number 28 on page 32, Guzman states that including "overlapping" cases increases the effect of both bilingualism and the effect of bilingual instruction in his analyses with respect to years of education. This suggests that those who get both classes in the primary language and ESL stayed in school longer. No additional details are provided.

2. The paper as cited in Guzman is: Fermandez, R. (1993). Response inconsistency in High School and Beyond data. Chicago: National Opinion Research Center. This paper is not included in the NORC website.

References

Crawford, Jim. 1995. Bilingual Education: History, Politics, Theory and Practice. Los Angeles: Bilingual Educational Services. Third edition.

Krashen, Stephen, 1999. Bilingualism, Bilingual Education, and Earnings: Comments on Two Recent Studies. Multilingual Education 22(2): 16-17.

Krashen, Stephen and McQuillan, Jeffrey. 1998. Do graduates of bilingual programs really earn less? A response to Lopez and Mora. NABE News.

Legarreta, D. 1979. The effects of program models on language acquisition by Spanish-speaking children. TESOL Quarterly 13 (4): 521-534.

Lopez, Mark and Mora, Marie. (1998). The labor market effects of bilingual education among Hispanic workers, READ Perspectives 5(2): 33-54.

Contact Information:
Education Policy Studies Laboratory College of Education Educational Leadership & Policy Studies Box 872411 Arizona State University Tempe, AZ 85287-2411	Voice: (480) 965-1886 Fax: (480) 965-0303 Email: epsl@asu.edu
Language Policy Research Unit - Mary Lou Fulton College of Education - Arizona State University © Arizona Board of Regents (ABOR)