INTRODUCTION: LEARNERS, TEACHERS, TESTS

A STUDY ON THE RELATIONSHIP BETWEEN TEST SCORES AND ATTITUDES TOWARDS TESTS

Because tests are thought to guarantee a place in the so-called good high schools, and subsequently, good universities and good jobs, they are a big part of students' lives and thus teachers' lives. According to Seo et al, students at Kwanchon Middle School preferred tailor-made materials developed specially for them to the text in class, but they were worried about tests and therefore considered the text-book more  "effective" (Seo et al, 1998: 51-52).	This creates a difficulty, known as "washback", for teachers who want to "negotiate" the syllabus or use their own materials. Martin points out, in a study on writing in American middle schools, that learners do not see broader learning aims and "passing the exam" as connected in any way (Martin 1983: 211). Thus teachers can't help but emphasize tests. But this emphasis can cut short the aims of education, focusing attention prematurely on the result rather than the process, and frustrating those who make a lot of effort.

A CASE IN POINT: MY STUDENT SUNG-EUN

A STUDY ON THE RELATIONSHIP BETWEEN TEST SCORES AND ATTITUDES TOWARDS TESTS

One example of this I have personally observed was my own student, Sung-eun, who studied hard and steadily. Although I gave her considerable praise, she found that she couldn't consistently get good marks. After several tests, I noticed her not concentrating on her studying any more and  pointed out her idleness. To my embarrassment, she burst into tears, for her parents had similarly scolded her with similar results.

But perhaps the reason she did badly lies in the way she studies. And the way she studies may well result from the way she thinks about tests. Perhaps Sung-eun is trapped in a vicious circle: short-term motivation---intensive but narrow and inefficient studying---poor results---even more intense and even more short term motivation---another bad results---scolding from outside---dislike of studying English......etc.

CLASS DISCUSSION: FOR LESS COMPETITIVE AND MORE PRODUCTIVE FORMS OF ASSESSMENT

Sung-eun was not alone in her frustration. After final-term of second semester, I held a class discussion about tests with my students. A number of students made it clear that they thought the discrete point test was a far from accurate measure of productive ability. They argued that it was not appropriate to evaluate the students according to indirect and passive tests in English, or, for that matter in music or art.

Some students mentioned that they preferred direct performance measurement in a pair-work or group-work writing task to the discrete point mid-term and final examinations, which were usually grammar-focused. For example, during the class, first, teachers would show students the topic or situation, about which the students could think and then cooperate to make good text. Group work or pair work would make the activity and the result better organized.

Other learners expressed reluctance to cooperate with dull or different 'level' students. They preferred to have tests like writing by themselves and for themselves from time to time during the class under the topic which teachers suggested previous time. It emerged that they thought writing was the most adequate method to evaluate their productive skill, involving the recognition of vocabulary or grammar and moreover discourse organization at a high level. Translation was also suggested as a possible test.

During this class discussion, students suggested many kinds of test format, but they felt sorry that these formats were not likely to adopted in the real world. I wondered if their suggestions and there preferences were reflective of underlying anxieties and attitudes which might also be reflected in their test scores. So in addition to a number of suggestions, I took away from this discussion three constructs I wanted to examine in the light of test scores: attitudes towards tests, attitudes towards English study, and writing proficiency.

TEST ANXIETY, LANGUAGE ATTITUDE, AND PRODUCTIVE SKILLS: A BRIEF LOOK AT LITERATURE

As a practicing teacher doing "action research", I was unable to do a thorough literature survey to prepare the ground for the following study. But I did notice that the research on anxiety and motivation has generally dealt with the topic generally and not focused on the specific stimulus of testing. (Ellis 1994: 481-482, Chaudron 1988: 101-104).

Gardner and Lambert, for example, took their chief variables from the ESL situation in Canada, distinguishing between people who were "instrumentally motivated" or interested in assimilating to the host culture. These are clearly not appropriate categories for analyzing the attitudes and motivations of Korean middle school learners in an EFL situation (See Ellis 1994: 207-211). Dornyei and Csizer, working in an EFL situation in Hungary, give categories of classroom motivation that are much more appropriate to Korea. They found that interim goal setting by teachers for learners was the most underutilized strategy of all of their motivational strategies. (Dornyei and Csizer, 1998: 220). Teachers often appear to simply rely on mid-term and end-term tests to motivate.

But test-based motivation may be too extrinsic to learners and too abstracted from their day-to-day interests to work consistently. Ellis tells us:

"Whereas leaners' beliefs about language learning are likely to be fairly stable, their affective states tend to be volatile, affecting not only overall progress but responses to particular learning activities on a day-by-day and even moment-by-moment basis." (Ellis 1994: 483)

This suggests, as my learners suggested in our discussion, a day-by-day or even moment-by-moment form of assessment--one that is part of the process rather than simply an evaluation of the product. Chambers, examining various classroom activities which motivate or fail to motivate learners, found that many learners enjoyed project work, pairwork, and groupwork, although he also found that these were rarely used (Chambers, 1998: 238). Similarly, a recent study by Ko Kyounghee showed that many learners believe that less competitive ways of developing and assessing language skills lesson emotional stress and may well be helpful in, for example, listening tests (Ko 1998: 22).

Here in Korea, there is quite a bit of recent evidence that our current forms of assessment favor learners who work by themselves over those who enjoy group oriented activities. Kim Jee-in, working with elementary children, found some evidence that more introverted and intuitive learners tended to do better in English than outgoing, sensitive ones (Kim 1998: 104), and Lee Eui-kap argues that high school learners who are introverted tend to do better if their writing work was analytically assessed, rather than holistically (Lee 1998: 122). We may be unknowingly punishing a particular kind of learning style, and turning a valid attitude to language learning into an ineffective one.

In the following exploratory study, I want to examine the test scores in the light of three constructs: test attitude, language learning motivation, and writing proficiency.	There are of course many disadvantages to looking at complex and very subjective elements like attitude and motivation, particularly in a naturalistic classroom setting. Even though there may be lots of uncontrolled but interesting variables in getting good marks or bad marks on the test (that is, the difference of students' diligence, hakwan attendance, teacher enthusiasm, cheating......etc), I'll neglect these factors because it is beyond my ability to control all of these things and they are very difficult to prove or measure. This, of course, weakens the internal validity of my study, but it increases relevance to classroom conditions, and thus offers external validity.

In addition to examining test attitude and study motivation, I will examine the relationship between test scores and productive ability. At school, we make students have mostly reading and listening mid-term and final-term exams which, I think, are not strongly based on the context or real situation and have limitations as measurements of productive skill like writing and speaking. Kellogg demonstrated a very weak relationship between exam scores and productive ability in English in a study among Korean middle school learners in 1998, but his study was restricted to a comparison of oral output and test scores. Here, I will extend Kellogg's finding to written output (Kellogg 1998, Kellogg 1999; Seo et al, 1998).

SEVEN HYPOTHESES

In order to look at the relationship between test scores, learner motivation, and written output, I formulated the following seven hypotheses, which are provable and disprovable through my data, which consists of questionnaire results, test scores, and output on a writing task.

H1. Students tend to have test-centered motivation in studying English.

H2. Learners who are high scorers on listening and reading tests tend to have future oriented aims (career) or pleasure and interest in English. The students who don't study English voluntarily, that is, those who have short-term motivation in comparison with those who have long-term motivation get low scores on tests.

H3. Learners who are high scorers on listening and reading tests tend to prefer discourse questions to grammar questions or vocabulary recognition questions.

H4. Low scoring students on listening and reading tests prefer vocabulary recognition questions to grammar questions or discourse questions.

H5. Students prefer different test formats which need productive skills to discrete point tests like formal mid-term or final-term exam.

H6. High scorers on the tests prefer direct test such as writing or translation to indirect mid-term or final-term exams.

H7. Passive skills do not accurately predict actual production. That is, learners who score highly on listening tests do not necessarily do well on writing tasks.

SUBJECTS

The subjects were 42 girl students of second grade who are attending at Kwan-eum middle school in Taegu. They have been taught English for one year by me. Some of them have learned from me since entering middle school.

QUESTIONNAIRE

I used a questionnaire to gather data on attitudes and motivation. gathered data from one class which is typical of classes of second grade in Kwan-eum girls' middle school. The questionnaire, which was thoroughly explained before it was administered, ran as follows:

1. What kind of question do you like in the discrete point test?

ⓐ discourse questions

ⓑ grammar questions

ⓒ vocabulary recognition questions

I will show you an example of these questions.

⊙ Here is the example of discourse question, which is based on the discourse and has coherence between the sentences.

⊙ Choose the most suitable answer and fill in the blank.

Jane and Mary were very good friends. Usually they did their homework and played together after school. On the weekend they often went to the park near their houses and had a good time. But now they                     . Their parents don't know why, and they worry about it.

① don't even talk to each other  ② are close friends

③ do their homework             ④ want to study	more

⑤ like to go to the park.

⊙ This is an example of grammar question, which is focused on a grammar point.

⊙ Choose the clumsy one which has a different meaning.

① I am surprised that he didn't come.

= I am surprised at him not coming.

② I am sure that you'll pass the exam.

= I am sure to pass the exam.

③ He couldn't come because of the rain.

= He couldn't come because it rained.

④ Let me introduce you to my brother.

= Why don't you meet my brother?

⑤ How about taking a walk?

=Let's take a walk.

⊙ An example of vocabulary recognition questions

⊙ Choose one which does not belong with the others.

① father   ② sister   ③ friend   ④ brother	⑤ mother

2. What makes you study English?

ⓐ influence (pressure) of teachers or parents

ⓑ fun and pleasure or interests

ⓒ good test scores

ⓓ future oriented aims

3. What kind of assessment do you prefer? (These choices are based on the opinion drawn from the class discussion. Therefore you can make reference to the class discussion.)

ⓐ discrete point test (mid-term or final-term listening reading exams)

ⓑ group activities

ⓒ interview

ⓓ memorizing of contents in the textbooks

ⓔ translating

ⓖ writing

TEST SCORES

The test scores consist of the outcomes of their mid-terms and final terms. These are divided into written tests (which, contrary to their name, actually focus on the reading skill) and listening tests. The tests were written by me.	I didn't want the students to memorize whole words or grammar parts but check how much they comprehend the context. So the discrete point tests have lots of discourse questions.

WRITING TASK

In addition, I made the students do a writing task, which was written about the plan of their winter vacation for twenty minutes during the class. Their writing tasks were evaluated by four separate teachers using criteria such as structure, the number of correct or comprehensible sentences and the length of whole writing task. Each teacher gave one to five points to each writing task. This means that the highest score on the writing taks is 20 and the lowest is 4 points.

ANALYSIS AND DISCUSSION

Below, I will discuss my results by presenting the data in chart form against each hypothesis and deciding whether the hypothesis stands or falls. Let me therefore explain the conventions used in the charts.

The scores given in the charts are labelled as follows:

'oneltest' means mid-term listening test scores of second semester.

'twoltest' means final-term listening test scores of second semester.

'onewtest' means scores of mid-term discrete point written tests of second semester, which are focused on reading ability.

'twowtest' means scores of final-term discrete point written tests of second semester, which are based on reading skill.

'writing' means scores of students' writing tasks which were done about the plan of their winter vacation during the class for 20 minutes.

For ease of analysis, I use pie charts, boxplots and scatterplots, as well as statistical tests (Pearson product moment correlations).

In the box plot, the black line indicates the mean. The large shaded "box" holds fifty percent of the scores nearest the mean. The extended "arms" of the box indicate the fifty percent of the scores furthest from the mean, and any extreme outliers are given as discrete points, with the student number next to the dot. At the bottom of each box, the number of students in the group is given; note that the groups are very different in size.

In each scatterplot, each student score is plotted along two axes, so that a strong relationship causes the scores to appear in a line, while a weak relationship shows them scattered in a "cloud". The strength of the relationship is also indicated by the Pearson correlations in the tables that follow hypothesis 7.

RESULTS BY HYPOTHESIS

H1.Students tend to have test-centered motivation in studying English.

Analysis of data relevant to H1.

This confirms that test scores are a big part of students' lives. If we add the negative motivation of "pressure" (presumably pressure to improve a test score), we get a majority of the students. This means that the majority of our students are extrinsically, not intrinsically, motivated, and that motivation is short-term, and not long-term in nature.

H2. Learners who are high scorers on listening and reading tests tend to have future oriented aims(career) or pleasure and interest in English; that is, they are intrinsically and not extrinsically motivated. The students who don't study English voluntarily, that is, those who have extrinsic motivation, tend to get low scores on tests.

This hypothesis is generally confirmed by the data. Although the mean for those motivated by test scores is only slightly lower than that for those motivated by career interest, the much smaller group which is motivated by intrinsic interest in English has a far higher mean than the small group negatively motivated by external pressure.

<Boxplots of test scores and motivation>

Analysis of data relevant to H2

Interests and pleasure may cause learners to pay more attention to the class or the teachers and to participate in English related class activities autonomously and creatively.

Conversely, we can predict that students who study English because of the pressure from their parents, teachers or others instead of inherent interests or career aims don't do well on the test. The pressure may cause the learners to dislike English and they are not likely to pay attention to English if they are not in the presence of observers or watchmen. It appears that students who have intrinsic motivation are likely to do better on tests than those who have extrinsic motivation. This is a fairly consistent trend across all of the tests, reading and listening.

H3. Learners who are high scorers on listening and reading tests tend to prefer discourse questions to grammar questions or vocabulary recognition questions.

This hypothesis is generally confirmed, although there does not appear to be much difference in level between learners who like questions about "discourse" and those who like questions about "grammar", and the distinction, which is a new one in testing in Korea, may be unclear to most students.

Boxplots of listening scores and question style preference (42 girl students)

Box plots of scores of discrete point tests and the preference of question style.

Analysis of data associated with H3

In general, high scorers on the tests appear comfortable with sentences and situations, and not simply words. Students who prefer discourse questions are likely to read the text or books which have large vocabulary, grammar parts and discourse organization, trying to figure out context. Perhaps this makes students have ability to guess or comprehend. When they contact and acquire lots of text with pleasure, they are likely to learn lots of language. As a result, they can get good scores on the tests.

H4. Low scoring students on listening and reading tests prefer vocabulary recognition questions to grammar questions or discourse questions.

This hypothesis is amply confirmed by the boxplots above.

Analysis of data relevant to H4

Low scorers on the tests appear to prefer vocabulary recognition questions. We can hypothesize that they are uncomfortable with ambiguity, text, links between the words such as cohesion or coherence, and they just put their focus on small bits of familiar language. Therefore, they may learn only small bits of language.

H5. Students prefer different test formats which need productive skills to discrete point tests like formal mid-term or final-term exam.

This hypothesis is completely disconfirmed, as a look at the pie chart will show.

Analysis of data related to Hypothesis 5

During the class discussion, most of the students expressed skeptical attitude to the traditional discrete point tests as evaluation. However, to my surprise, this pie chart shows us that a large number of students selected discrete point tests among the  methods of evaluation recommended by themselves. One obvious explanation for this result is that learners are profoundly conservative--they want what they are used to, or what authority has suggested, at any rate.

H6. High scorers on the tests prefer direct test such as writing or translation to indirect discrete point mid-term or final-term exams.

This hypothesis is strikingly confirmed by the boxplot, where those who preferred translation and writing tests to the discrete point exams got far higher mean scores.

Boxplots of listening scores and preference of test format

One class of second year girls (42 learners)

Box plots of discrete point test scores

and preference of test style(42 learners)

Analysis of data associated with Hypothesis 6

High scorers tend to prefer writing or translating as tests. There are some possible explanations for this inclination. Writing and translating demand the ability to deal with vocabulary, grammar and discourse organization. Students who know lots of language may well feel confident in writing and translating, and choose test format which make distinctive classification between scores.

There was only one learner who preferred group activities, and it was a relatively high scorer. This single example does not of course disprove the findings of Lee and Kim that our current assessment methods favor introverted students.

H7. Passive skills do not accurately predict actual production. That is, learners who score highly on listening tests do not necessarily do well on writing tasks.

This hypothesis was proven so far as listening is concerned. However, the "written" tests, which were actually discrete point tests of reading, have a fairly strong relationship with the score on the writing task, as the correlations and scatterplots that follow will show.

Analysis of data relevant to H7

These correlations and scatter plots show us the relatively strong relationship of discrete point reading tests and the integrative, productive skill of writing. This is in contrast to the findings of Seo et al and Kellogg, who found correlations of only .468 to .603 between standardized discrete point tests and a very coarse measurement of spoken output. All the correlations are statistically significant at the p < .01 level.

It seems that students who have the ability to produce more language than others can get good marks on the reading test which have vocabulary, grammar, discourse questions. As we saw in the scatter plots and the correlation chart, we can predict reading test scores with writing scores. In addition, writing high scorers are usually listening high scorers because writing needs usage of lots of vocabulary, grammar and also discourse organization than other skills. Perhaps, if the tests are well designed, it is possible to take advantage of discrete point tests instead of writing in Korean schools where writing is so neglected because of classroom size constraints.

However, it seems that it is difficult to predict writing scores with listening scores; the correlations of listening scores and writing task scores are relatively weak. In fact, the correlations are very similar to those found by Seo et al and Kellogg in 1998 and 1999 between listening tests and oral production.

CONCLUSIONS AND IMPLICATIONS

Naturally, there are lots of uncontrolled variables at work, as with any study which uses actual learners in classroom conditions. Nevertheless, the factors we have looked at--attitudes towards tests, attitudes towards English, and writing proficiency--do account for some interesting features of the test results.

Learners who are high scorers on  tests tend to have future oriented aims (career) or fun as motivation in studying in English. Intrinsic motivation is apparently more characteristic of high scorers than extrinsic motivation. In addition, learners who are high scorers on written tests of mid or final term tend to prefer discourse questions or grammar questions to vocabulary recognition. When they study English, they read the text or books, trying to figure out the context the books give them rather than checking and learning by heart only the words they don't know. They have fun in the process of doing like this and they can get ability to guess and comprehend. When they like lots of text, they are likely to get lots of information and learn a lots of language. As a result, they can get good scores on the test.

On the other hand, low scoring students on tests prefer vocabulary recognition questions to discourse questions or grammar questions. They are uncomfortable with ambiguity, texts and links between the words such as cohesion or coherence. They just put focuses on small bits of familar language. Therefore, they learn small bits of language. It makes the students get low scores in the vicious circle. We can assume that the ability to deal with input affect output.

Before the survey, I expected that a large number of students wouldn't like to do the discrete point test. However, to my surprise, the result was different. This perhaps indicates that those who don't feel confident in English are worried about the burden of a different evaluation method.

I discovered that productive writing tasks as an evaluation method are relevant to current testing methods. Thus students who got high scores on writing did well on listening tests and discrete point tests based on reading. However, it's difficult to predict that students who are good at listening will do well on reading or writing. Perhaps, as the students argued in our initial class discussion, this is because writing logically implies other skills. Another explanation is that writing is so neglected in Korean schools that only the best students or most autonomous ones go on to develop it.

In Korea, recently the circumstance of evaluation is likely to change into performance assessment. It is desirable trend but it may be possible in small size class. And it has lots of problems like objectivity or burden to teachers who are charge of large class (over 40 students a class) and have lots of classes (more than 20 classes a week) and non-teaching related jobs. If the test questions are well designed and consider various possibilities and not simply one possible right answer, perhaps it will reflect students' performance ability, as the close relationship between the test scores and writing appears to imply.

But perhaps it is necessary to take a more radical view of the necessity of testing. During this study, I have been preoccupied with the thought that the very emphasis on testing may be cheating our students of good test results. I have found that students who have interest in studying English and deal with large amount of English get good result on the tests. If English were not a subject of entrance exams, students would have less pressure in English. They would feel relaxation in dealing with large amount of English and would be able to approach to English through reading books, watching films, listening to music, talking with friends in English in stead of analyzing of grammar points and memorizing of vocabulary. And they won't get afraid of making contact with English or English speakers.

Tests strongly affect students' lives as well as teachers'. We can't reproach students just because they don't do well on tests; such reproach, and such pressure may in itself cause them to do badly. And after all, test scores are a very small part of English ability. Even the lowest scorers can memorize the whole lyric lines of their favorite singers. Both Chambers and Dornyei and Csizer have pinpointed the teacher as being the single most important factor in learner motivation. Thus it is the teachers' role to make students have interest in English and do their best.

REFERENCES:

Chambers, G.N., "Pupils' perceptions of the foreign language learning experience", Language Teaching Research, Volume 2, Number 3, 1998, pp. 231-261.

Dornyei, Zoltan and Kata Csizer, "Ten commandments for motivating language learners: the results of an empirical study", Language Teaching Research, Volume 2, Number 3, 1998, pp. 203-231.

Chaudron, C., Second Language Classrooms: Research on teaching and learning, Cambridge University Press, Cambridge: 1988

Ellis, R., The Study of Second Language Acquistion, Oxford University Press, Oxford: 1994

Kellogg, David, "Pairs work: a comparison of student output in pairwork and teacher-fronted interviews", West Taegu English Language Teaching Research Association, Taegu: August 1998.

Kellogg, David, "HOGS and LOGS: Developing materials for different levels of output generation", unpublished manuscript, 1999.

Kim Jee-in, "Personality Variables and EFL Proficiency", English Teaching, Vol. 53, No. 4, 1998, pp. 93-106.

Ko Kyounghee, "Cooperative Learning in Noninteractive Listening Tasks", English Teaching, Vol. 53, No. 4, 1998, pp. 3-30.

Lee Eui-kap, "The Effect of Temperament Types, Feedback Types, and Writing Modes on EFL Students' Task-based Writing", English Teaching, Vol. 53, No. 4, 1998, pp. 107-134.

Martin, N., "Scope for intentions", in Learning to write: first language/second language, Freedman & Pringle eds., Longman, Harlow: 1983

Seo et al, "Pairwork 중 심 의 수 준 별 교 졔 개 발 및 적 용", West Taegu English Language Teaching Research Association, Taegu: December 1998.