PRESENTER BIOGRAPHY
Myung-sook Jung majored in French and minored in English at Kyungpook National University, graduating in 1985. She's been an English teacher for seven years and now works at Ku-am Girls' Middle School in Taegu. She's interested in writing and teaching higher level discourse skills, and also very interested in testing reform.
MATERIALS

Because tests are thought to guarantee a place in the 
so-called good high schools, and subsequently, good 
universities and good jobs, they are a big part of students' 
lives and thus teachers' lives. According to Seo et al, 
students at Kwanchon Middle School preferred tailor-made 
materials developed specially for them to the text in class, 
but they were worried about tests and therefore considered 
the text-book more  "effective" (Seo et al, 1998: 51-52). 
 This creates a difficulty, known as "washback", for 
teachers who want to "negotiate" the syllabus or use their 
own materials. Martin points out, in a study on writing in 
American middle schools, that learners do not see broader 
learning aims and "passing the exam" as connected in any 
way (Martin 1983: 211). Thus teachers can't help but 
emphasize tests. But this emphasis can cut short the aims 
of education, focusing attention prematurely on the result 
rather than the process, and frustrating those who make a 
lot of effort.

A CASE IN POINT: MY STUDENT SUNG-EUN

One example of this I have personally observed was 
my own student, Sung-eun, who studied hard and steadily. 
Although I gave her considerable praise, she found that she 
couldn't consistently get good marks. After several tests, I 
noticed her not concentrating on her studying any more and 
pointed out her idleness. To my embarrassment, she burst 
into tears, for her parents had similarly scolded her with 
similar results.
But perhaps the reason she did badly lies in the way 
she studies. And the way she studies may well result from 
the way she thinks about tests. Perhaps Sung-eun is 
trapped in a vicious circle: short-term 
motivation---intensive but narrow and inefficient 
studying---poor results---even more intense and even 
more short term motivation---another bad 
results---scolding from outside---dislike of studying 
English......etc. 

CLASS DISCUSSION: FOR LESS COMPETITIVE AND 
MORE PRODUCTIVE FORMS OF ASSESSMENT

Sung-eun was not alone in her frustration. After 
final-term of second semester, I held a class discussion 
about tests with my students. A number of students made 
it clear that they thought the discrete point test was a far 
from accurate measure of productive ability. They argued 
that it was not appropriate to evaluate the students 
according to indirect and passive tests in English, or, for 
that matter in music or art.
Some students mentioned that they preferred direct 
performance measurement in a pair-work or group-work 
writing task to the discrete point mid-term and final 
examinations, which were usually grammar-focused. For 
example, during the class, first, teachers would show 
students the topic or situation, about which the students 
could think and then cooperate to make good text. Group 
work or pair work would make the activity and the result 
better organized. 
 Other learners expressed reluctance to cooperate with 
dull or different 'level' students. They preferred to have 
tests like writing by themselves and for themselves from 
time to time during the class under the topic which 
teachers suggested previous time. It emerged that they 
thought writing was the most adequate method to evaluate 
their productive skill, involving the recognition of 
vocabulary or grammar and moreover discourse organization 
at a high level. Translation was also suggested as a 
possible test. 
During this class discussion, students suggested many 
kinds of test format, but they felt sorry that these formats 
were not likely to adopted in the real world. I wondered if 
their suggestions and there preferences were reflective of 
underlying anxieties and attitudes which might also be 
reflected in their test scores. So in addition to a number of 
suggestions, I took away from this discussion three 
constructs I wanted to examine in the light of test scores: 
attitudes towards tests, attitudes towards English study, 
and writing proficiency.

TEST ANXIETY, LANGUAGE ATTITUDE, AND 
PRODUCTIVE SKILLS: A BRIEF LOOK AT LITERATURE

As a practicing teacher doing "action research", I was 
unable to do a thorough literature survey to prepare the 
ground for the following study. But I did notice that the 
research on anxiety and motivation has generally dealt with 
the topic generally and not focused on the specific stimulus 
of testing. (Ellis 1994: 481-482, Chaudron 1988: 101-104).
Gardner and Lambert, for example, took their chief 
variables from the ESL situation in Canada, distinguishing 
between people who were "instrumentally motivated" or 
interested in assimilating to the host culture. These are 
clearly not appropriate categories for analyzing the attitudes 
and motivations of Korean middle school learners in an EFL 
situation (See Ellis 1994: 207-211). Dornyei and Csizer, 
working in an EFL situation in Hungary, give categories of 
classroom motivation that are much more appropriate to 
Korea. They found that interim goal setting by teachers for 
learners was the most underutilized strategy of all of their 
motivational strategies. (Dornyei and Csizer, 1998: 220). 
Teachers often appear to simply rely on mid-term and 
end-term tests to motivate. 
But test-based motivation may be too extrinsic to 
learners and too abstracted from their day-to-day interests 
to work consistently. Ellis tells us:
"Whereas leaners' beliefs about language learning are 
likely to be fairly stable, their affective states tend to be 
volatile, affecting not only overall progress but responses to 
particular learning activities on a day-by-day and even 
moment-by-moment basis." (Ellis 1994: 483)
This suggests, as my learners suggested in our 
discussion, a day-by-day or even moment-by-moment form 
of assessment--one that is part of the process rather than 
simply an evaluation of the product. Chambers, examining 
various classroom activities which motivate or fail to 
motivate learners, found that many learners enjoyed project 
work, pairwork, and groupwork, although he also found that 
these were rarely used (Chambers, 1998: 238). Similarly, a 
recent study by Ko Kyounghee showed that many learners 
believe that less competitive ways of developing and 
assessing language skills lesson emotional stress and may 
well be helpful in, for example, listening tests (Ko 1998: 
22).
Here in Korea, there is quite a bit of recent evidence 
that our current forms of assessment favor learners who 
work by themselves over those who enjoy group oriented 
activities. Kim Jee-in, working with elementary children, 
found some evidence that more introverted and intuitive 
learners tended to do better in English than outgoing, 
sensitive ones (Kim 1998: 104), and Lee Eui-kap argues 
that high school learners who are introverted tend to do 
better if their writing work was analytically assessed, 
rather than holistically (Lee 1998: 122). We may be 
unknowingly punishing a particular kind of learning style, 
and turning a valid attitude to language learning into an 
ineffective one.
 In the following exploratory study, I want to examine 
the test scores in the light of three constructs: test attitude, 
language learning motivation, and writing proficiency. 
There are of course many disadvantages to looking at 
complex and very subjective elements like attitude and 
motivation, particularly in a naturalistic classroom setting. 
Even though there may be lots of uncontrolled but 
interesting variables in getting good marks or bad marks on 
the test (that is, the difference of students' diligence, 
hakwan attendance, teacher enthusiasm, cheating......etc), I'll 
neglect these factors because it is beyond my ability to 
control all of these things and they are very difficult to 
prove or measure. This, of course, weakens the internal 
validity of my study, but it increases relevance to 
classroom conditions, and thus offers external validity.
In addition to examining test attitude and study 
motivation, I will examine the relationship between test 
scores and productive ability. At school, we make students 
have mostly reading and listening mid-term and final-term 
exams which, I think, are not strongly based on the context 
or real situation and have limitations as measurements of 
productive skill like writing and speaking. Kellogg 
demonstrated a very weak relationship between exam scores 
and productive ability in English in a study among Korean 
middle school learners in 1998, but his study was restricted 
to a comparison of oral output and test scores. Here, I will 
extend Kellogg's finding to written output (Kellogg 1998, 
Kellogg 1999; Seo et al, 1998).

SEVEN HYPOTHESES

In order to look at the relationship between test scores, 
learner motivation, and written output, I formulated the 
following seven hypotheses, which are provable and 
disprovable through my data, which consists of 
questionnaire results, test scores, and output on a writing 
task.

H1. Students tend to have test-centered motivation in 
studying English.

H2. Learners who are high scorers on listening and 
reading tests tend to have future oriented aims (career) or 
pleasure and interest in English. The students who don't 
study English voluntarily, that is, those who have 
short-term motivation in comparison with those who have 
long-term motivation get low scores on tests.

H3. Learners who are high scorers on listening and 
reading tests tend to prefer discourse questions to grammar 
questions or vocabulary recognition questions.

H4. Low scoring students on listening and reading tests 
prefer vocabulary recognition questions to grammar 
questions or discourse questions.

H5. Students prefer different test formats which need 
productive skills to discrete point tests like formal 
mid-term or final-term exam.

H6. High scorers on the tests prefer direct test such as 
writing or translation to indirect mid-term or final-term 
exams.

H7. Passive skills do not accurately predict actual 
production. That is, learners who score highly on listening 
tests do not necessarily do well on writing tasks.

SUBJECTS

The subjects were 42 girl students of second grade 
who are attending at Kwan-eum middle school in Taegu. 
They have been taught English for one year by me. Some 
of them have learned from me since entering middle school. 

QUESTIONNAIRE

I used a questionnaire to gather data on attitudes and 
motivation. gathered data from one class which is typical of 
classes of second grade in Kwan-eum girls' middle school. 
The questionnaire, which was thoroughly explained before it 
was administered, ran as follows:

1. What kind of question do you like in the discrete 
point test?
     ¨Í discourse questions
     ¨Î grammar questions
     ¨Ï vocabulary recognition questions

I will show you an example of these questions.
¢¿ Here is the example of discourse question, which is 
based on the discourse and has coherence between the 
sentences.
¢¿ Choose the most suitable answer and fill in the 
blank.
Jane and Mary were very good friends. Usually they 
did their homework and played together after school. On the 
weekend they often went to the park near their houses and 
had a good time. But now they                     . Their 
parents don't know why, and they worry about it.
¨ç don't even talk to each other  ¨è are close friends
¨é do their homework             ¨ê want to study 
       more
¨ë like to go to the park.

¢¿ This is an example of grammar question, which is 
focused on a grammar point.
¢¿ Choose the clumsy one which has a different 
meaning. 
 ¨ç I am surprised that he didn't come.
   = I am surprised at him not coming.
 ¨è I am sure that you'll pass the exam.
   = I am sure to pass the exam.
 ¨é He couldn't come because of the rain.
   = He couldn't come because it rained.
 ¨ê Let me introduce you to my brother.
   = Why don't you meet my brother?
 ¨ë How about taking a walk?
   =Let's take a walk. 

¢¿ An example of vocabulary recognition questions
¢¿ Choose one which does not belong with the others.
¨ç father   ¨è sister   ¨é friend   ¨ê brother 
 ¨ë mother

2. What makes you study English?
   ¨Í influence (pressure) of teachers or parents
   ¨Î fun and pleasure or interests
   ¨Ï good test scores
   ¨Ð future oriented aims

3. What kind of assessment do you prefer? (These 
choices are based on the opinion drawn from the class 
discussion. Therefore you can make reference to the class 
discussion.)
   ¨Í discrete point test (mid-term or final-term 
listening reading exams)
   ¨Î group activities
   ¨Ï interview
   ¨Ð memorizing of contents in the textbooks
   ¨Ñ translating
   ¨Ó writing 

TEST SCORES

The test scores consist of the outcomes of their 
mid-terms and final terms. These are divided into written 
tests (which, contrary to their name, actually focus on the 
reading skill) and listening tests. The tests were written by 
me.  I didn't want the students to memorize whole words or 
grammar parts but check how much they comprehend the 
context. So the discrete point tests have lots of discourse 
questions.
 
 

WRITING TASK

In addition, I made the students do a writing task, 
which was written about the plan of their winter vacation 
for twenty minutes during the class. Their writing tasks 
were evaluated by four separate teachers using criteria such 
as structure, the number of correct or comprehensible 
sentences and the length of whole writing task. Each 
teacher gave one to five points to each writing task. This 
means that the highest score on the writing taks is 20 and 
the lowest is 4 points.

ANALYSIS AND DISCUSSION

Below, I will discuss my results by presenting the data 
in chart form against each hypothesis and deciding whether 
the hypothesis stands or falls. Let me therefore explain the 
conventions used in the charts.
The scores given in the charts are labelled as follows:
'oneltest' means mid-term listening test scores of 
second semester.
'twoltest' means final-term listening test scores of 
second semester.
'onewtest' means scores of mid-term discrete point 
written tests of second semester, which are focused on 
reading ability.
'twowtest' means scores of final-term discrete point 
written tests of second semester, which are based on 
reading skill.
'writing' means scores of students' writing tasks 
which were done about the plan of their winter vacation 
during the class for 20 minutes.
 For ease of analysis, I use pie charts, boxplots and 
scatterplots, as well as statistical tests (Pearson product 
moment correlations). 
In the box plot, the black line indicates the mean. The 
large shaded "box" holds fifty percent of the scores nearest 
the mean. The extended "arms" of the box indicate the fifty 
percent of the scores furthest from the mean, and any 
extreme outliers are given as discrete points, with the 
student number next to the dot. At the bottom of each box, 
the number of students in the group is given; note that the 
groups are very different in size.
In each scatterplot, each student score is plotted along 
two axes, so that a strong relationship causes the scores to 
appear in a line, while a weak relationship shows them 
scattered in a "cloud". The strength of the relationship is 
also indicated by the Pearson correlations in the tables that 
follow hypothesis 7.

RESULTS BY HYPOTHESIS

H1.Students tend to have test-centered motivation in 
studying English.
 
 

Analysis of data relevant to H1.

This confirms that test scores are a big part of 
students' lives. If we add the negative motivation of 
"pressure" (presumably pressure to improve a test score), 
we get a majority of the students. This means that the 
majority of our students are extrinsically, not intrinsically, 
motivated, and that motivation is short-term, and not 
long-term in nature.

H2. Learners who are high scorers on listening and 
reading tests tend to have future oriented aims(career) or 
pleasure and interest in English; that is, they are 
intrinsically and not extrinsically motivated. The students 
who don't study English voluntarily, that is, those who 
have extrinsic motivation, tend to get low scores on tests.
This hypothesis is generally confirmed by the data. 
Although the mean for those motivated by test scores is 
only slightly lower than that for those motivated by career 
interest, the much smaller group which is motivated by 
intrinsic interest in English has a far higher mean than the 
small group negatively motivated by external pressure.

<Boxplots of test scores and motivation>
 
 
 
 
 

Analysis of data relevant to H2

Interests and pleasure may cause learners to pay more 
attention to the class or the teachers and to participate in 
English related class activities autonomously and creatively. 
Conversely, we can predict that students who study 
English because of the pressure from their parents, teachers 
or others instead of inherent interests or career aims don't 
do well on the test. The pressure may cause the learners to 
dislike English and they are not likely to pay attention to 
English if they are not in the presence of observers or 
watchmen. It appears that students who have intrinsic 
motivation are likely to do better on tests than those who 
have extrinsic motivation. This is a fairly consistent trend 
across all of the tests, reading and listening.

H3. Learners who are high scorers on listening and 
reading tests tend to prefer discourse questions to grammar 
questions or vocabulary recognition questions.
This hypothesis is generally confirmed, although there 
does not appear to be much difference in level between 
learners who like questions about "discourse" and those 
who like questions about "grammar", and the distinction, 
which is a new one in testing in Korea, may be unclear to 
most students.

Boxplots of listening scores and question style 
preference (42 girl students)
 

Box plots of scores of discrete point tests and the 
preference of question style.
 

Analysis of data associated with H3

In general, high scorers on the tests appear comfortable 
with sentences and situations, and not simply words. 
Students who prefer discourse questions are likely to read 
the text or books which have large vocabulary, grammar 
parts and discourse organization, trying to figure out 
context. Perhaps this makes students have ability to guess 
or comprehend. When they contact and acquire lots of text 
with pleasure, they are likely to learn lots of language. As 
a result, they can get good scores on the tests.

H4. Low scoring students on listening and reading tests 
prefer vocabulary recognition questions to grammar 
questions or discourse questions.
This hypothesis is amply confirmed by the boxplots 
above.

Analysis of data relevant to H4

Low scorers on the tests appear to prefer vocabulary 
recognition questions. We can hypothesize that they are 
uncomfortable with ambiguity, text, links between the 
words such as cohesion or coherence, and they just put 
their focus on small bits of familiar language. Therefore, 
they may learn only small bits of language.

H5. Students prefer different test formats which need 
productive skills to discrete point tests like formal 
mid-term or final-term exam.
 This hypothesis is completely disconfirmed, as a look 
at the pie chart will show.
 
 

Analysis of data related to Hypothesis 5

During the class discussion, most of the students 
expressed skeptical attitude to the traditional discrete point 
tests as evaluation. However, to my surprise, this pie chart 
shows us that a large number of students selected discrete 
point tests among the  methods of evaluation recommended 
by themselves. One obvious explanation for this result is 
that learners are profoundly conservative--they want what 
they are used to, or what authority has suggested, at any 
rate. 

H6. High scorers on the tests prefer direct test such as 
writing or translation to indirect discrete point mid-term or 
final-term exams. 
This hypothesis is strikingly confirmed by the boxplot, 
where those who preferred translation and writing tests to 
the discrete point exams got far higher mean scores.

Boxplots of listening scores and preference of test format

One class of second year girls (42 learners)
 

Box plots of discrete point test scores 
and preference of test style(42 learners)
 

Analysis of data associated with Hypothesis 6

High scorers tend to prefer writing or translating as 
tests. There are some possible explanations for this 
inclination. Writing and translating demand the ability to 
deal with vocabulary, grammar and discourse organization. 
Students who know lots of language may well feel 
confident in writing and translating, and choose test format 
which make distinctive classification between scores.
There was only one learner who preferred group 
activities, and it was a relatively high scorer. This single 
example does not of course disprove the findings of Lee 
and Kim that our current assessment methods favor 
introverted students.

H7. Passive skills do not accurately predict actual 
production. That is, learners who score highly on listening 
tests do not necessarily do well on writing tasks. 
This hypothesis was proven so far as listening is 
concerned. However, the "written" tests, which were 
actually discrete point tests of reading, have a fairly strong 
relationship with the score on the writing task, as the 
correlations and scatterplots that follow will show.
 

Analysis of data relevant to H7

These correlations and scatter plots show us the 
relatively strong relationship of discrete point reading tests 
and the integrative, productive skill of writing. This is in 
contrast to the findings of Seo et al and Kellogg, who 
found correlations of only .468 to .603 between standardized 
discrete point tests and a very coarse measurement of 
spoken output. All the correlations are statistically 
significant at the p < .01 level. 
It seems that students who have the ability to produce 
more language than others can get good marks on the 
reading test which have vocabulary, grammar, discourse 
questions. As we saw in the scatter plots and the 
correlation chart, we can predict reading test scores with 
writing scores. In addition, writing high scorers are usually 
listening high scorers because writing needs usage of lots 
of vocabulary, grammar and also discourse organization 
than other skills. Perhaps, if the tests are well designed, it 
is possible to take advantage of discrete point tests instead 
of writing in Korean schools where writing is so neglected 
because of classroom size constraints.
However, it seems that it is difficult to predict writing 
scores with listening scores; the correlations of listening 
scores and writing task scores are relatively weak. In fact, 
the correlations are very similar to those found by Seo et al 
and Kellogg in 1998 and 1999 between listening tests and 
oral production.

CONCLUSIONS AND IMPLICATIONS

Naturally, there are lots of uncontrolled variables at 
work, as with any study which uses actual learners in 
classroom conditions. Nevertheless, the factors we have 
looked at--attitudes towards tests, attitudes towards 
English, and writing proficiency--do account for some 
interesting features of the test results.
Learners who are high scorers on  tests tend to have 
future oriented aims (career) or fun as motivation in 
studying in English. Intrinsic motivation is apparently more 
characteristic of high scorers than extrinsic motivation. In 
addition, learners who are high scorers on written tests of 
mid or final term tend to prefer discourse questions or 
grammar questions to vocabulary recognition. When they 
study English, they read the text or books, trying to figure 
out the context the books give them rather than checking 
and learning by heart only the words they don't know. 
They have fun in the process of doing like this and they 
can get ability to guess and comprehend. When they like 
lots of text, they are likely to get lots of information and 
learn a lots of language. As a result, they can get good 
scores on the test.
On the other hand, low scoring students on tests prefer 
vocabulary recognition questions to discourse questions or 
grammar questions. They are uncomfortable with ambiguity, 
texts and links between the words such as cohesion or 
coherence. They just put focuses on small bits of familar 
language. Therefore, they learn small bits of language. It 
makes the students get low scores in the vicious circle. We 
can assume that the ability to deal with input affect output.
Before the survey, I expected that a large number of 
students wouldn't like to do the discrete point test. 
However, to my surprise, the result was different. This 
perhaps indicates that those who don't feel confident in 
English are worried about the burden of a different 
evaluation method.
I discovered that productive writing tasks as an 
evaluation method are relevant to current testing methods. 
Thus students who got high scores on writing did well on 
listening tests and discrete point tests based on reading. 
However, it's difficult to predict that students who are 
good at listening will do well on reading or writing. 
Perhaps, as the students argued in our initial class 
discussion, this is because writing logically implies other 
skills. Another explanation is that writing is so neglected in 
Korean schools that only the best students or most 
autonomous ones go on to develop it.
In Korea, recently the circumstance of evaluation is 
likely to change into performance assessment. It is desirable 
trend but it may be possible in small size class. And it has 
lots of problems like objectivity or burden to teachers who 
are charge of large class (over 40 students a class) and 
have lots of classes (more than 20 classes a week) and 
non-teaching related jobs. If the test questions are well 
designed and consider various possibilities and not simply 
one possible right answer, perhaps it will reflect students' 
performance ability, as the close relationship between the 
test scores and writing appears to imply.
But perhaps it is necessary to take a more radical view 
of the necessity of testing. During this study, I have been 
preoccupied with the thought that the very emphasis on 
testing may be cheating our students of good test results. I 
have found that students who have interest in studying 
English and deal with large amount of English get good 
result on the tests. If English were not a subject of 
entrance exams, students would have less pressure in 
English. They would feel relaxation in dealing with large 
amount of English and would be able to approach to 
English through reading books, watching films, listening to 
music, talking with friends in English in stead of analyzing 
of grammar points and memorizing of vocabulary. And they 
won't get afraid of making contact with English or English 
speakers.
 Tests strongly affect students' lives as well as 
teachers'. We can't reproach students just because they 
don't do well on tests; such reproach, and such pressure 
may in itself cause them to do badly. And after all, test 
scores are a very small part of English ability. Even the 
lowest scorers can memorize the whole lyric lines of their 
favorite singers. Both Chambers and Dornyei and Csizer 
have pinpointed the teacher as being the single most 
important factor in learner motivation. Thus it is the 
teachers' role to make students have interest in English and 
do their best.
 

REFERENCES:

Chambers, G.N., "Pupils' perceptions of the foreign 
language learning experience", Language Teaching Research, 
Volume 2, Number 3, 1998, pp. 231-261.

Dornyei, Zoltan and Kata Csizer, "Ten commandments 
for motivating language learners: the results of an empirical 
study", Language Teaching Research, Volume 2, Number 3, 
1998, pp. 203-231.

Chaudron, C., Second Language Classrooms: Research 
on teaching and learning, Cambridge University Press, 
Cambridge: 1988

Ellis, R., The Study of Second Language Acquistion, 
Oxford University Press, Oxford: 1994

Kellogg, David, "Pairs work: a comparison of student 
output in pairwork and teacher-fronted interviews", West 
Taegu English Language Teaching Research Association, 
Taegu: August 1998.

Kellogg, David, "HOGS and LOGS: Developing 
materials for different levels of output generation", 
unpublished manuscript, 1999.

Kim Jee-in, "Personality Variables and EFL 
Proficiency", English Teaching, Vol. 53, No. 4, 1998, pp. 
93-106.

Ko Kyounghee, "Cooperative Learning in Noninteractive 
Listening Tasks", English Teaching, Vol. 53, No. 4, 1998, 
pp. 3-30.

Lee Eui-kap, "The Effect of Temperament Types, 
Feedback Types, and Writing Modes on EFL Students' 
Task-based Writing", English Teaching, Vol. 53, No. 4, 
1998, pp. 107-134.

Martin, N., "Scope for intentions", in Learning to write: 
first language/second language, Freedman & Pringle eds., 
Longman, Harlow: 1983

Seo et al, "Pairwork Áß ½É ÀÇ ¼ö ÁØ º° ±³ Áµ °³ ¹ß ¹× Àû 
¿ë", West Taegu English Language Teaching Research 
Association, Taegu: December 1998.