On Test Task Design[下学期]
文档属性
| 名称 | On Test Task Design[下学期] |  | |
| 格式 | rar | ||
| 文件大小 | 245.4KB | ||
| 资源类型 | 教案 | ||
| 版本资源 | 通用版 | ||
| 科目 | 英语 | ||
| 更新时间 | 2007-04-08 23:05:00 | ||
图片预览
文档简介
课件88张PPT。On Test Task Design Shaoqian (Sheila) Luo
The Chinese University of Hong Kong
Beijing Normal University
Nanjing, Dec 2006The tester visits his folks after “The Tester’s Lament” by Richard Young“So, what do you do?” my parents ask.
“I measure,” I reply.
My father wants me to measure the house,
So he c’n be sure he’s not been cheated by the realty louse.
My mother wants me to measure the windows,
So she c’n fit them with curtains in the color of rainbows.
My brother wants me to measure his height,
To prove that he’s ready to take big brother on for a fight.
My sister wants me to measure her waist,
To show that she c’n now dress in real good taste.“Sorry,” I say. “I don’t measure for play.
I measure what’s in you that enables you
To say what you say,
And do what you do.”
Alas, they conclude that I’m a fool,
And throw me into the swimming pool.
Quote from International Conference on Language Testing on Research Methodology in Language Testing
Guangdong University of Foreign Studies, Guangzhou, China December 8-9, 2006
表1 2005年秋季考试试卷结构（五级）表2 2005年秋季考试试卷结构（二级）Easy vs. difficultFamiliarity
Background knowledge
Task type
Linguistic knowledge
There is a need to
reinterpret cultural factorsResearch QuestionIs there a relationship between the cultural factors and the students’ performance in task-based assessment?
If there is, what are the factors?
How these factors affect task design and task selection in task-based assessment?
Participants60 Grade 9 students in ordinary middle schools（March ~ April 2004）
34 Grade 9 students in ordinary middle schools (April 2006)
26 experienced middle school teachers (March ~ May 2004)TasksPlanning the weekend
Shopping in a supermarket
Radio weather information
Complete the “Customer Satisfaction Form”
Finding your lost bag
Reserving a table
Getting directions to the party
You did a survey on “Your daily life” in your class (50 students) and the table below is the results. Based on the results, write a letter (about 80 words) of suggestion to the school headmaster: the theme is “Reduce the students’ workload”. In the letter you are to list the phenomena and consequences(后果).You and a group of friends recently went to your favorite fast food restaurant for a meal. However, you found that in some ways the restaurant was not as good as the last time you went. These are some of the comments you made.Complete the ‘Customer Satisfaction Form’.
Dear Customer: Thank you for eating at McGregor’s Restaurant. If you think there is any way we can improve, we would love to hear from you. Please write your suggestions on the form below and put it in the box at the counter. Many thanks.
What is your opinion of the food at McGregor’s?
Do you think we provided comfortable surroundings for your meal?
Were our staff polite and friendly?4) What can we do to make McGregor’s even better?Task 5 Write a TV weather report Prompt: Loot at a map of weather information, including marks of wind, air temperature, rainfall, cloud cover, etc. in different parts of a particular place or of the world. Write a TV weather report and act it out in groups.
Realia/Materials: A map of weather information, including marks of wind, air temperature, rainfall, cloud cover, etc.
Time: 10 min.
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Task 6 Write an e-mail Prompt: One of your friends from another city is going to visit you on May Day holiday. You have just received an e-mail from him/her asking about what the weather is like in your city. You are supposed t write an e-mail to describe the weather. If necessary, you are also expected to give some suggestions concerning the dressing for the weather.
Realia/Materials: Show what an e-mail account looks like and how to write an e-mail.
Time: 10 min
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Task 12 Do you feel the same? Prompt: Write a paragraph to describe one kind of your feelings (happy, sad, angry, upset, pleased, proud, lonely, worried, nervous, afraid, etc.). However, in this paragraph you cannot use the word that directly tells your feeling – for example, if you are describing the feeling of being happy, you cannot use the word happy – you need to describe a particular occasion that embodies the word. For instance, you might describe a lucky thing that you once experienced so that the reader would understand that you are saying you are HAPPY. Read your paragraph aloud in class, and see if the class recognize what you have described.
Realia/Materials: A sample paragraph to show how to do the writing task.
Time: 10 minTask Planning the weekend Several friends are talking about what they are doing Sunday. Listen to their discussion (twice) and complete their plan. Time: 10 min
A trip to Lotus Mountain on Sunday
Meet at:__________________, at the Grand Theatre.
Bus leaves at: 10 a.m
Bus will return home at about ________
Food and drink to bring:
5 bags of potato chips
________________________________________
10 cans of soft drink
_______________________________________Task 6 ShoppingTask: ( Read and write)
You plan to do shopping with some of your friends, Sandy, Tom, Jane and Peter. After the shopping, your friends and you are going to watch a movie. The movie theater is inside the shopping mall near the Exit. The shopping mall is very big and you do not want to waste time. So you ask your friends to tell you the things they want to buy and make a plan. You want to spend the shortest time in the shopping mall together with all your friends.
Time: 5min
Data collection and Findings Questionnaires and focus group interview
20 questions related to culture and cognitive familiarity
24 out of 34 students have never listened to weather reports in English
13 students consider “fast food customer satisfaction form” the most difficult
23 students think that “restroom” is a place for taking a rest.Analysis –task difficultyPersonal experience
Background knowledge
Topic familiarity and predictability
Regional differences
Complexity of cognitive familiarity1 personal experience Teachers primarily make the judgment based on their students’ life experience
Task: Finding your lost bag
T1… they have no such experiences as going aboard …
T2…most teachers were at a loss when they met with such a thing in Australia…though the topic is familiar…
2 background knowledge Teachers believe a task would be difficult if students have no relevant background knowledge
T: …they have no such background knowledge, ..they would not read the illustrations in English… 3 topic familiarity and predictability If students have the knowledge of the task type or relevant knowledge, it is easy. They could make inferences
Task: Getting directions to the party , Asking the way, Showing the way
There are such topics in their textbooks, even in the Chinese textbook, and student are very familiar with it…
4 Regional differences (1) Teachers from less developed areas put more emphasis on cultural differences
T: …I look at the task topics first to decide the difficulty…
T: …the weather report is difficult for the students… because it is a bit far from their daily life..
T: …transportation is difficult… such as subways, …they have no idea what that is…4 Regional differences (2)Rural areas: no idea about fast food restaurant or modern transportation, computer or internet,
T …they do not know what broccoli is
T … what we call a TV is called a computer in the city…
Teachers with MA degree in big cities
In some rural areas, few primary school teachers of English have received professional training
4 Regional differences (3) Students in cities are very familiar with the “western culture”,… especially popular culture
pop stars, NBA, fast food restaurant, western festivals & holidays, parties…
They have very positive attitude about TBA
S1…I hope there will be more content on language use..
S2…We learn English for communication, …
S3…these (tasks) can help develop our thinking
skills...
5 Complexity of cognitive familiarity” Even if they are familiar with the topics and task type, they may still make mistakes…
1 29 students out of 60 chose fast food restaurant to buy food for the next week, but not supermarket.
2 2 chose restroom to buy food and 2 for handbag for his/her mother
3 And where to put shoes is also problematic because in most of the North countryside areas, shoes are put on the floor.
Complexity of “cognitive familiarity” A weather report in Britain
28 out of 34 students listen to weather
reports regularly in Chinese,
but 24 never listen to weather report
in English
No single student recognized map of UK
only 1 student “guessed it is Britain as London was mentioned in the audio tape.”
English for toilet, WC, washroom, bathroom, John。
23 -- a place to rest
4 -- no idea
1 – gym /1-- hotel/ 1– sitting room Implications and ConclusionsA task is closely related with a specific context.
They feel difficult if they are not familiar with the task type, even if students are familiar with the topics.
They may have difficulties if they do not know certain vocabulary, even if they are familiar with both topics and task type.
Students in different areas may have different cultural knowledge about English speaking countries. It may not be fair to test all students with the same cultural background knowledge
It may affect students performance if a particular word or piece of information is used in the test.Content fair? ---do not advantage specific groups of test-takers…
…to what extent is it possible to separate the test-takers’ use of language from their real-world knowledge expertise…?
Ellis (2003),p.309
‘ordinary situations’ --familiar to test takers Morrow (1979) in Ellis测试内容的决定具体化 (identification): 内容以何种具体形式体现
选择 (selection):内容的选定以什么为依据
说明 (specification): 选定的内容如何描写和限定. 内容说明是课程和考试设计的基础, 是教学或考试大纲里的主要部分. 明晰性 (explicitness) 尤其重要.测试内容的三种形式语言点: 关于目标语的一项一项的知识. 考的是平时教的语言点. 无客观的科学依据, 凭编书人教师及命题人的经验和主观判断.
掌握有限数量的搭配力最强, 使用率最高, 概括性最广的基本形式结构. 重点在句型上. 测试的明晰性提高. (语言本身: 语言学)
学习者需求分析: 调查受试者应该能用英语做些什么事, 才能通过考试, 得出的目标行为就直接作为考试内容. (社会语言学)Tasks 1 类型要求：真实，多样
2 语言、文体要求：难易度适合 （343）构成难度的多种因素，尤其未能从中国人的角度考虑难易。目前选用的（阅读）测试用资料，对语言程度的高低，主要还是凭命题人的主观判断。（我们）需要更科学地给材料的难度定级。课文体自成一体，跟课文外的现实生活中能见到的任何文体都不像。是专门为放在课本里写的、（几乎）完全人工的、不真实的文章。这种类型的材料，不但充塞中学考试和模拟考试，在很正规的大型公共考试中也常有出现。这种课本问题文章/材料有两个语言特点：
词句都经过不自然的人为简化。1）不是说考试不能用文字及结构简单的材料。但是自然简单和人为的简化是两回事。2）不是说对材料不能加工改写。事实上初、中级靠使用的材料很多都不可避免地要经改写。改写必须改得自然尔不露痕迹，并保留所属篇章类型的特征。
叙述啰嗦重复。只让学生看一切都解释得清楚得不能再清楚的文章，遇到稍要动脑筋才能看懂的文章（正常的文章），又怎能期待他们看得懂？ 写、说和多技能运用试题最大的优点是效度高。是现代语言测试可能取得最高效度的试题。其效度从三方面得以体现：
产生性运用试题，能同时测试接受性运用。听懂不一定能说出来，读懂不一定能写出来。能说出来自然能听懂，能写出来自然能读懂。（一般情况是这样。有时有单一的写作。）
产生性运用试题，能同时测试语言的各个层次和各个因素范畴。
产生性运用试题，能同时测试准确性、流利性和得体性。
语言的产生性运用是一种创造性行为。产生性运用试题的设计，就是为了让受试者把他对目标语的创造力发挥出来。这创造力，自然是真实情景制约下的创作力。准确性指符合语言形式系统自身的规律。这属于交际能力中的语言能力。
流利性指语言运用过程中的顺利程度，即运用中的效率。这不是语言形式系统自身静止的属性，而是他在运用中的属性。运用，是在情景中的运用，所以自然要牵涉到语篇能力和语用能力。
得体性，指语言的运用是否符合语境情景的要求。这就是语篇能力和语用能力的范畴了。Task samplesSituation: You have invited your colleague Maria to dinner. She hasn’t been to your house before so you go to her office to tell her where you live. She isn’t there so you decide to leave her a message.
Instruction/Task: Write a short note to Maria giving her a clear set of instructions so that she can get from work to your house for dinner.Task samplesSituation: You are the editor of an English medium magazine. One of your readers has sent you a letter. This letter is printed below. The writer is still a beginner of English and the letter is not very well written.
Instruction/Task: Without changing the ideas, rewrite the letter on the answer sheet provided so that it will be well written when it appears in your magazine. An Example of a Pedagogic TaskFour students – each has one picture and describes it to the rest of the class.
Students from the rest of the class ask the four students questions about their pictures.
One student from the class tries to tell the story.
If necessary Steps 2 and 3 are repeated.A Real-World Task Look at the e-mail message below. Listen to Mr. Pointer’s instructions on the tape. Make notes if you want to. Then write a suitable reply to Leo.
Dear Mr. Pointer,
Please send flight number, date and time of arrival
and I will arrange for someone to meet you at the
airport.
LeoListening Jeff is talking to Steve about his health/Jeff thinks that he isn’t very healthy. Listen to their conversation. Number the activities in the order you hear them.
walk to school _____ eat junk food _______
go to the doctor _____ exercise ______
eat fruit and vegetables _____ play tennis _______Listen again. How often does Jeff do these things?
sometimes twice a week hardly ever every day two or three times a year
Jeff ... How often?
1. exercises Sometimes
2. walks to school _________________________
3. eats fruit and vegetables _____________________
4. goes to the doctor __________________________
5. plays tennis ______________________________
6. eats junk food ______________________________Linda isn’t feeling very well and Kate is giving her advice. Listen to the conversation again. What advice does Kate give?
Problems Advice
fever and a stomachache drink lots of water
go to the doctor
sore arm feels terrible lie down
take Chinese herbs
feels tired toothache don't eat anything
drink tea with honey
headache sore throat don’t talk Alan is talking about things his family can do. Fill in the chart with the sports that Alan, Paul, and Helen can play
My name is Alan. I have a brother and a sister. My brother's name is Paul and my sister's name is Helen. I can sing and dance, but I can't play the guitar. I can play basketball but I can't play soccer. Paul can sing but he can't dance. He can play the guitar. He can play soccer play basketball. Helen can dance but she can't sing. She can't play the guitar but she can play the trumpet. She can play but she can't play soccer.Name Sports they can play
Alan ___________________________
Paul ___________________________
Helen ___________________________Vera wants to join John’s music club. Listen to their conversation. Fill in the card.
Music Club
Name: _________ __________ Age: ____
E-mail address: ____________________
What can you play?
________________________________________________________________________
Why do you want to join the music club?
___________________________________________________VocabularyThe teacher asks Bob and Helen to work in a group with other students. They must each choose two students, one boy and one girl. Are these girls' names or boys' names? Complete the table. Check () the correct answers.
Look at John's family tree. Complete the chart then write sentences to show the relationship between the people. Goals and Techniques for Teaching Grammar Students need overt instruction that connects grammar points with larger communication contexts.
Students do not need to master every aspect of each grammar point, only those that are relevant to the immediate communication task.
Error correction is not always the instructor's first responsibility. Overt Grammar Instruction Teach the grammar point in the target language or the students' first language or both. The goal is to facilitate understanding.
Limit the time you devote to grammar explanations to 10 minutes, especially for lower level students whose ability to sustain attention can be limited.
Present grammar points in written and oral ways to address the needs of students with different learning styles.
two basic principles:
Be sure the examples are accurate and appropriate. They must present the language appropriately, be culturally appropriate for the setting in which they are used, and be to the point of the lesson.
Use the examples as teaching tools. Focus examples on a particular theme or topic so that students have more contact with specific information and vocabulary.
Teaching Grammar: Error Correction1 When students are doing structured output activities that focus on development of new language skills, use error correction to guide them.
Example: Student (in class): I buy a new car yesterday. Teacher: You bought a new car yesterday. Remember, the past tense of buy is bought.
Teaching Grammar: Error Correction2 When students are engaged in communicative activities, correct errors only if they interfere with comprehensibility. Respond using correct forms, but without stressing them.
Example: Student (greeting teacher) : I buy a new car yesterday! Teacher: You bought a new car? That's exciting! What kind? Testing grammarFive factors that contribute to the “communicative” nature of a grammar test:
1 语境化（the contextualization of test items:
不应该有脱离语境的句子;
2 测试活动的交际目的（The identification of a communicative purpose for the test activity）;
3 交流对象/观众（The identification of an audience to whom the communication is addressed）;
4 注重意义（Instructions to the studnets that focus on meaning rather than on form）:流利、准确
5 学生创造和得体运用语言的机会(The opportunity for students to create his/her own message and to produce grammatical responses as appropriate to a given context：得体性
P. Dickins (1991)Reading
The students at a language school want to phone each other during the holiday. Alice and Harry are making a list of their phone numbers. Read the dialog. Complete the phone list.
Writing
Write instructions in five steps to tell a friend how to do or make something. E.g. make a phone call, cook something, or do today's homework.INTEGRATING SKILLSRead the information and write messages to put on the lost and found bulletin board.
1. Mark lost his gold watch. His phone number is seven-two-eight, oh-four-nine-nine.
2. Bill lost his baseball. It has his name on it. His phone number is five-two-four, one-four-oh-nine.
3. Mike found a small, blue pencil case on the school bus on Tuesday. His phone number is five-three-three, six-two-one-one.试题设计试题应以信息表达为测试目的
产生性运用从意义出发，还是从形式出发？
试题的情景，应该对受试群是真实的情景 (relevant to them)
测试的层次应是语篇层次：语篇的定义是语境和情景的语言，是具有平面的维度（dimensions）也有立体的维度的语言。试题要求的应该是互动性行为（interaction）
语言的制约应（自然）来自交际情景，而不是来自人为的对语言形式的主观选定：传统的从语言形式出发的试题，根据要靠的词汇和语法去设计试题。产生性运用试题不以课本里教过的词汇语法为出发点去设计试题，而从意义出发设计试题。并且完全可以按不同的程度去设计。所要求的词汇语法，可以按不同的程度加以控制。在真实生活里，任何人要说什么写什么，从来都不是先有了某些词汇某些语法想要用，然后才决定去说或者写某些话。二是倒过来，先有想要表达的意思，然后才去选择语法词汇，决定说的写的话的形式。而这想要表达的意思，却又受决定于交际情景。（举例：丢了朋友的书，道歉。）情景决定要表达的意思，要表达的意思再决定要说的话的形式。
具体设计三要素：情景、馈入、和输出。输入要将输出的内容需要具体化；输出的内容需要限制。语言教学和测试切合学语言用语言的真实的心理活动：应该根据科学对语言学习和运用的认识去设计语言学习和测试，让它尽可能地接近学语言用语言的真实心理状态。参考书目:
李筱菊 (1997). 语言测试科学与艺术. 湖南教育出版社.
刘 润 清 (2006). 语言测试理论的发展趋势. 济南语言测试研讨会。
Munby, J. L. (1978). Communicative syllabus design. Cambridge: CUP.
Oller, J. W. Jr. (1976). Evidence for a general language proficiency factor: An expectancy grammar. Die Neueren Sprachen. 75. pp. 165-174
Go For It Review (2006). PEP & Thomson Learning
试卷的标准（1）：效度（validity)：考到了要考的内容
内容效度(content validity)
编制效度(construct validity)
实证效度(empirical validity)
卷面效度(face validity)效度分类内在效度：
内容效度：测试是否考试了考试大纲规定要考的。内容效度的保证，主要靠试题设计和实体生产（命题和审题）时严格遵守大纲要求。验证主要靠考试前对试题的覆盖分析。验证的依据是大纲（它本身是否有效度呢？）。
结构效度：是考试的原则理论的效度。这里结构不是指试题或试卷的结构，而是指整个考试的意志为基础的理论结构。测试是否有结构效度，指其是否有效的语言观（包括语言运用观和语言学习观）为根据。结构效度决定了整个考试的性质。一个考试的最终价值，不在于它能用多少证据去证明，而在于它能给人带来美好的、快乐的效果，有助于使人变得更完美（李，1990）。
如果说英语测试的目的只是测试英语知识（不管运用的、狭义的知识），那么这套试题是效度颇高的。问题时，现代语言测试理论认为，仅仅是语言知识，远远不足以使一个人能够运用语言。因此语言测试如果只是测试语言知识，那就从根本上是无效度的。试卷的标准（2）：信度(reliability)
又称一致性(consistency)
稳定性信度(stability reliability)
又称一测再测信度(test-retest reliability)
等值信度(equivalent-form reliability)试卷的标准（3）：区分度(discriminative power )
实用性(practicality)
反拨作用(backwash effect)
外在的错误来源；内在的错误来源一套试卷的总体设计语言水平的定义
应试群体的特征
实施条件和所需设备
考试大纲（几种题型、各占分数、测试
内容、难易程度、词汇范围等）命题步骤培训命题人员（语言水平高，懂的测试理论）
设计题目（最好每人设计一种题型）
在与应试群体相同的人群中预测(pretest)
根据预测结果修改
再次预测
再次修改，直到满意为止任务类型以及任务设计 任务类型以及任务设计的意义Learning can be promoted through the use of tasks in four major ways.
1 learners can develop their accuracy in using the language.
2 learners can develop their capacity to
deploy a range of structures and a range of
vocabulary items and idioms within the context
3 learners can develop their fluency in doing what they do, by adjusting their patterns of fluent runs, uses of repetition and of rephrasing, and their utterance structure (Edmondson, 1981)
4 learners need to align their resources in the expression of their own individuality.
Two argumentation tasks 1 Give students a list of options to rank-order for an out-of-school weekend class activity;
2 Give students a list of options for an additional subject to be added to the school curriculum.
两个步骤：
每个学生独自决定自己的选择
与同伴讨论决定
准备阶段
一系列需要定夺的活动和科目。Narrative tasks Each student gets a different picture to prepare and then tell a short story to his or her partner.
准备阶段
学生有图作为输入以准备故事。
在图片中引入一些复杂的因素， 如角色、细节、更多的图片等。
Argumentative tasks 有激活大脑的作用（brainstorming quality）: 句子短；回应答语（responses tend to be echoic）; 支持或反对观点重复;所得列单也有重复.
逐渐发展一个稳定的立场由于必须全班报告使学生不得不使用难句并排练；
使学生参与并运用复杂语言.
E.g. 增加一些 引诱的或不可思议的观点
（tempting but potentially outrageous items in
the list of options）. Conclusion 不同类型的人物驱使不同形式的语言运用(different types of task set up different patterns of language use).
运用任务引导学生数出多样化的语言(to use tasks as a way of leading learners to produce differentiated types of language).
设计不同的任务是学生达到语言的流利、准确、复杂和得体。Critical thinking— 22 comprehension strategiesDrawing conclusions
Understanding cause and effect
Making connections
Comparing view points
Evaluating ideas
Reading between the lines
Making judgments
22 comprehension strategiesMaking comparisons
Interpreting meaning
Making inferences
Comparing characters
Reading a map
Interpreting graphs and charts
Making calculations
Examining reasons
22 comprehension strategiesSeparating facts from opinion
Identifying the main idea
Organizing information
Sequencing
Summarizing
Recalling details
Synthesizing
Brainstorm, either by yourself or with a group and create a list of 6-8 real-world tasks. Take one of these and think how it might be transformed into a pedagogical task, e. g. as a role play, simulation or problem-solving task. Then indicate how your task meets the key characteristics identified above. (Skehan, 1998; CDC Syllabus for English Language, Secondary 1-5, 1999: 43.)Making judgmentsHighlight the three pieces of information
in the text that you think everyone should know.
Most important_________________
Second most important___________
Least important__________________Developing alternative solutionsWith a group, brainstorm three different solutions to ..’s problem.
A __________________
B __________________
C ___________________
When you brainstorm, everyone contribute ideas. Write down every idea. ….Examining reasonsWhy do you think it’s necessary in this experiment to collect…?
____________________________
How do you think the author says to use…?
_____________________________
Why do you think…?
____________________________Making comparisonsSimilarities
____________________________
____________________________
Differences
____________________________Forming opinionsHow do you think the author of this article feel about…?
___________________________
Do you think these.. are fair or unfair? In your notebook, write a paragraph explaining your opinion about what happens to…
_____________________________
_____________________________Organizing informationIn the text, the author talks about three kinds of problems: environmental, animal and human. For each statement below, label the kind of problem being described
___________________________ __________________________ __________________________Contrasting and comparing selections When you contrast selections you think about how they are different.
When you compare selections you think about how they are the same.summarizingTo summarize is to make a brief statement giving the main points of a text.
Check off the three sentences below that you think best summarize “…”.Synthesizing （综合） When you synthesize information, you combine bits of information into a complex whole.
The first step in synthesizing is to identify your topic.
The second step is to find information about your topic in a text
The third step is to think about all the information you have found.
Ask yourself: what does this mean? Why is it important
Writers craftUsing antonyms
I try to be polite and kind because I know that I don’t like it when people are rude and cruel to meWriters craftAnalysing sensory language ( words that describe how something looks, sounds,feels, smells,or tastes )
Underline words and phrases that help you visualize what is happening
Sight____________________________
Sound___________________________
Taste____________________________
Smell____________________________
Touch____________________________
The Chinese University of Hong Kong
Beijing Normal University
Nanjing, Dec 2006The tester visits his folks after “The Tester’s Lament” by Richard Young“So, what do you do?” my parents ask.
“I measure,” I reply.
My father wants me to measure the house,
So he c’n be sure he’s not been cheated by the realty louse.
My mother wants me to measure the windows,
So she c’n fit them with curtains in the color of rainbows.
My brother wants me to measure his height,
To prove that he’s ready to take big brother on for a fight.
My sister wants me to measure her waist,
To show that she c’n now dress in real good taste.“Sorry,” I say. “I don’t measure for play.
I measure what’s in you that enables you
To say what you say,
And do what you do.”
Alas, they conclude that I’m a fool,
And throw me into the swimming pool.
Quote from International Conference on Language Testing on Research Methodology in Language Testing
Guangdong University of Foreign Studies, Guangzhou, China December 8-9, 2006
表1 2005年秋季考试试卷结构（五级）表2 2005年秋季考试试卷结构（二级）Easy vs. difficultFamiliarity
Background knowledge
Task type
Linguistic knowledge
There is a need to
reinterpret cultural factorsResearch QuestionIs there a relationship between the cultural factors and the students’ performance in task-based assessment?
If there is, what are the factors?
How these factors affect task design and task selection in task-based assessment?
Participants60 Grade 9 students in ordinary middle schools（March ~ April 2004）
34 Grade 9 students in ordinary middle schools (April 2006)
26 experienced middle school teachers (March ~ May 2004)TasksPlanning the weekend
Shopping in a supermarket
Radio weather information
Complete the “Customer Satisfaction Form”
Finding your lost bag
Reserving a table
Getting directions to the party
You did a survey on “Your daily life” in your class (50 students) and the table below is the results. Based on the results, write a letter (about 80 words) of suggestion to the school headmaster: the theme is “Reduce the students’ workload”. In the letter you are to list the phenomena and consequences(后果).You and a group of friends recently went to your favorite fast food restaurant for a meal. However, you found that in some ways the restaurant was not as good as the last time you went. These are some of the comments you made.Complete the ‘Customer Satisfaction Form’.
Dear Customer: Thank you for eating at McGregor’s Restaurant. If you think there is any way we can improve, we would love to hear from you. Please write your suggestions on the form below and put it in the box at the counter. Many thanks.
What is your opinion of the food at McGregor’s?
Do you think we provided comfortable surroundings for your meal?
Were our staff polite and friendly?4) What can we do to make McGregor’s even better?Task 5 Write a TV weather report Prompt: Loot at a map of weather information, including marks of wind, air temperature, rainfall, cloud cover, etc. in different parts of a particular place or of the world. Write a TV weather report and act it out in groups.
Realia/Materials: A map of weather information, including marks of wind, air temperature, rainfall, cloud cover, etc.
Time: 10 min.
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________Task 6 Write an e-mail Prompt: One of your friends from another city is going to visit you on May Day holiday. You have just received an e-mail from him/her asking about what the weather is like in your city. You are supposed t write an e-mail to describe the weather. If necessary, you are also expected to give some suggestions concerning the dressing for the weather.
Realia/Materials: Show what an e-mail account looks like and how to write an e-mail.
Time: 10 min
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Task 12 Do you feel the same? Prompt: Write a paragraph to describe one kind of your feelings (happy, sad, angry, upset, pleased, proud, lonely, worried, nervous, afraid, etc.). However, in this paragraph you cannot use the word that directly tells your feeling – for example, if you are describing the feeling of being happy, you cannot use the word happy – you need to describe a particular occasion that embodies the word. For instance, you might describe a lucky thing that you once experienced so that the reader would understand that you are saying you are HAPPY. Read your paragraph aloud in class, and see if the class recognize what you have described.
Realia/Materials: A sample paragraph to show how to do the writing task.
Time: 10 minTask Planning the weekend Several friends are talking about what they are doing Sunday. Listen to their discussion (twice) and complete their plan. Time: 10 min
A trip to Lotus Mountain on Sunday
Meet at:__________________, at the Grand Theatre.
Bus leaves at: 10 a.m
Bus will return home at about ________
Food and drink to bring:
5 bags of potato chips
________________________________________
10 cans of soft drink
_______________________________________Task 6 ShoppingTask: ( Read and write)
You plan to do shopping with some of your friends, Sandy, Tom, Jane and Peter. After the shopping, your friends and you are going to watch a movie. The movie theater is inside the shopping mall near the Exit. The shopping mall is very big and you do not want to waste time. So you ask your friends to tell you the things they want to buy and make a plan. You want to spend the shortest time in the shopping mall together with all your friends.
Time: 5min
Data collection and Findings Questionnaires and focus group interview
20 questions related to culture and cognitive familiarity
24 out of 34 students have never listened to weather reports in English
13 students consider “fast food customer satisfaction form” the most difficult
23 students think that “restroom” is a place for taking a rest.Analysis –task difficultyPersonal experience
Background knowledge
Topic familiarity and predictability
Regional differences
Complexity of cognitive familiarity1 personal experience Teachers primarily make the judgment based on their students’ life experience
Task: Finding your lost bag
T1… they have no such experiences as going aboard …
T2…most teachers were at a loss when they met with such a thing in Australia…though the topic is familiar…
2 background knowledge Teachers believe a task would be difficult if students have no relevant background knowledge
T: …they have no such background knowledge, ..they would not read the illustrations in English… 3 topic familiarity and predictability If students have the knowledge of the task type or relevant knowledge, it is easy. They could make inferences
Task: Getting directions to the party , Asking the way, Showing the way
There are such topics in their textbooks, even in the Chinese textbook, and student are very familiar with it…
4 Regional differences (1) Teachers from less developed areas put more emphasis on cultural differences
T: …I look at the task topics first to decide the difficulty…
T: …the weather report is difficult for the students… because it is a bit far from their daily life..
T: …transportation is difficult… such as subways, …they have no idea what that is…4 Regional differences (2)Rural areas: no idea about fast food restaurant or modern transportation, computer or internet,
T …they do not know what broccoli is
T … what we call a TV is called a computer in the city…
Teachers with MA degree in big cities
In some rural areas, few primary school teachers of English have received professional training
4 Regional differences (3) Students in cities are very familiar with the “western culture”,… especially popular culture
pop stars, NBA, fast food restaurant, western festivals & holidays, parties…
They have very positive attitude about TBA
S1…I hope there will be more content on language use..
S2…We learn English for communication, …
S3…these (tasks) can help develop our thinking
skills...
5 Complexity of cognitive familiarity” Even if they are familiar with the topics and task type, they may still make mistakes…
1 29 students out of 60 chose fast food restaurant to buy food for the next week, but not supermarket.
2 2 chose restroom to buy food and 2 for handbag for his/her mother
3 And where to put shoes is also problematic because in most of the North countryside areas, shoes are put on the floor.
Complexity of “cognitive familiarity” A weather report in Britain
28 out of 34 students listen to weather
reports regularly in Chinese,
but 24 never listen to weather report
in English
No single student recognized map of UK
only 1 student “guessed it is Britain as London was mentioned in the audio tape.”
English for toilet, WC, washroom, bathroom, John。
23 -- a place to rest
4 -- no idea
1 – gym /1-- hotel/ 1– sitting room Implications and ConclusionsA task is closely related with a specific context.
They feel difficult if they are not familiar with the task type, even if students are familiar with the topics.
They may have difficulties if they do not know certain vocabulary, even if they are familiar with both topics and task type.
Students in different areas may have different cultural knowledge about English speaking countries. It may not be fair to test all students with the same cultural background knowledge
It may affect students performance if a particular word or piece of information is used in the test.Content fair? ---do not advantage specific groups of test-takers…
…to what extent is it possible to separate the test-takers’ use of language from their real-world knowledge expertise…?
Ellis (2003),p.309
‘ordinary situations’ --familiar to test takers Morrow (1979) in Ellis测试内容的决定具体化 (identification): 内容以何种具体形式体现
选择 (selection):内容的选定以什么为依据
说明 (specification): 选定的内容如何描写和限定. 内容说明是课程和考试设计的基础, 是教学或考试大纲里的主要部分. 明晰性 (explicitness) 尤其重要.测试内容的三种形式语言点: 关于目标语的一项一项的知识. 考的是平时教的语言点. 无客观的科学依据, 凭编书人教师及命题人的经验和主观判断.
掌握有限数量的搭配力最强, 使用率最高, 概括性最广的基本形式结构. 重点在句型上. 测试的明晰性提高. (语言本身: 语言学)
学习者需求分析: 调查受试者应该能用英语做些什么事, 才能通过考试, 得出的目标行为就直接作为考试内容. (社会语言学)Tasks 1 类型要求：真实，多样
2 语言、文体要求：难易度适合 （343）构成难度的多种因素，尤其未能从中国人的角度考虑难易。目前选用的（阅读）测试用资料，对语言程度的高低，主要还是凭命题人的主观判断。（我们）需要更科学地给材料的难度定级。课文体自成一体，跟课文外的现实生活中能见到的任何文体都不像。是专门为放在课本里写的、（几乎）完全人工的、不真实的文章。这种类型的材料，不但充塞中学考试和模拟考试，在很正规的大型公共考试中也常有出现。这种课本问题文章/材料有两个语言特点：
词句都经过不自然的人为简化。1）不是说考试不能用文字及结构简单的材料。但是自然简单和人为的简化是两回事。2）不是说对材料不能加工改写。事实上初、中级靠使用的材料很多都不可避免地要经改写。改写必须改得自然尔不露痕迹，并保留所属篇章类型的特征。
叙述啰嗦重复。只让学生看一切都解释得清楚得不能再清楚的文章，遇到稍要动脑筋才能看懂的文章（正常的文章），又怎能期待他们看得懂？ 写、说和多技能运用试题最大的优点是效度高。是现代语言测试可能取得最高效度的试题。其效度从三方面得以体现：
产生性运用试题，能同时测试接受性运用。听懂不一定能说出来，读懂不一定能写出来。能说出来自然能听懂，能写出来自然能读懂。（一般情况是这样。有时有单一的写作。）
产生性运用试题，能同时测试语言的各个层次和各个因素范畴。
产生性运用试题，能同时测试准确性、流利性和得体性。
语言的产生性运用是一种创造性行为。产生性运用试题的设计，就是为了让受试者把他对目标语的创造力发挥出来。这创造力，自然是真实情景制约下的创作力。准确性指符合语言形式系统自身的规律。这属于交际能力中的语言能力。
流利性指语言运用过程中的顺利程度，即运用中的效率。这不是语言形式系统自身静止的属性，而是他在运用中的属性。运用，是在情景中的运用，所以自然要牵涉到语篇能力和语用能力。
得体性，指语言的运用是否符合语境情景的要求。这就是语篇能力和语用能力的范畴了。Task samplesSituation: You have invited your colleague Maria to dinner. She hasn’t been to your house before so you go to her office to tell her where you live. She isn’t there so you decide to leave her a message.
Instruction/Task: Write a short note to Maria giving her a clear set of instructions so that she can get from work to your house for dinner.Task samplesSituation: You are the editor of an English medium magazine. One of your readers has sent you a letter. This letter is printed below. The writer is still a beginner of English and the letter is not very well written.
Instruction/Task: Without changing the ideas, rewrite the letter on the answer sheet provided so that it will be well written when it appears in your magazine. An Example of a Pedagogic TaskFour students – each has one picture and describes it to the rest of the class.
Students from the rest of the class ask the four students questions about their pictures.
One student from the class tries to tell the story.
If necessary Steps 2 and 3 are repeated.A Real-World Task Look at the e-mail message below. Listen to Mr. Pointer’s instructions on the tape. Make notes if you want to. Then write a suitable reply to Leo.
Dear Mr. Pointer,
Please send flight number, date and time of arrival
and I will arrange for someone to meet you at the
airport.
LeoListening Jeff is talking to Steve about his health/Jeff thinks that he isn’t very healthy. Listen to their conversation. Number the activities in the order you hear them.
walk to school _____ eat junk food _______
go to the doctor _____ exercise ______
eat fruit and vegetables _____ play tennis _______Listen again. How often does Jeff do these things?
sometimes twice a week hardly ever every day two or three times a year
Jeff ... How often?
1. exercises Sometimes
2. walks to school _________________________
3. eats fruit and vegetables _____________________
4. goes to the doctor __________________________
5. plays tennis ______________________________
6. eats junk food ______________________________Linda isn’t feeling very well and Kate is giving her advice. Listen to the conversation again. What advice does Kate give?
Problems Advice
fever and a stomachache drink lots of water
go to the doctor
sore arm feels terrible lie down
take Chinese herbs
feels tired toothache don't eat anything
drink tea with honey
headache sore throat don’t talk Alan is talking about things his family can do. Fill in the chart with the sports that Alan, Paul, and Helen can play
My name is Alan. I have a brother and a sister. My brother's name is Paul and my sister's name is Helen. I can sing and dance, but I can't play the guitar. I can play basketball but I can't play soccer. Paul can sing but he can't dance. He can play the guitar. He can play soccer play basketball. Helen can dance but she can't sing. She can't play the guitar but she can play the trumpet. She can play but she can't play soccer.Name Sports they can play
Alan ___________________________
Paul ___________________________
Helen ___________________________Vera wants to join John’s music club. Listen to their conversation. Fill in the card.
Music Club
Name: _________ __________ Age: ____
E-mail address: ____________________
What can you play?
________________________________________________________________________
Why do you want to join the music club?
___________________________________________________VocabularyThe teacher asks Bob and Helen to work in a group with other students. They must each choose two students, one boy and one girl. Are these girls' names or boys' names? Complete the table. Check () the correct answers.
Look at John's family tree. Complete the chart then write sentences to show the relationship between the people. Goals and Techniques for Teaching Grammar Students need overt instruction that connects grammar points with larger communication contexts.
Students do not need to master every aspect of each grammar point, only those that are relevant to the immediate communication task.
Error correction is not always the instructor's first responsibility. Overt Grammar Instruction Teach the grammar point in the target language or the students' first language or both. The goal is to facilitate understanding.
Limit the time you devote to grammar explanations to 10 minutes, especially for lower level students whose ability to sustain attention can be limited.
Present grammar points in written and oral ways to address the needs of students with different learning styles.
two basic principles:
Be sure the examples are accurate and appropriate. They must present the language appropriately, be culturally appropriate for the setting in which they are used, and be to the point of the lesson.
Use the examples as teaching tools. Focus examples on a particular theme or topic so that students have more contact with specific information and vocabulary.
Teaching Grammar: Error Correction1 When students are doing structured output activities that focus on development of new language skills, use error correction to guide them.
Example: Student (in class): I buy a new car yesterday. Teacher: You bought a new car yesterday. Remember, the past tense of buy is bought.
Teaching Grammar: Error Correction2 When students are engaged in communicative activities, correct errors only if they interfere with comprehensibility. Respond using correct forms, but without stressing them.
Example: Student (greeting teacher) : I buy a new car yesterday! Teacher: You bought a new car? That's exciting! What kind? Testing grammarFive factors that contribute to the “communicative” nature of a grammar test:
1 语境化（the contextualization of test items:
不应该有脱离语境的句子;
2 测试活动的交际目的（The identification of a communicative purpose for the test activity）;
3 交流对象/观众（The identification of an audience to whom the communication is addressed）;
4 注重意义（Instructions to the studnets that focus on meaning rather than on form）:流利、准确
5 学生创造和得体运用语言的机会(The opportunity for students to create his/her own message and to produce grammatical responses as appropriate to a given context：得体性
P. Dickins (1991)Reading
The students at a language school want to phone each other during the holiday. Alice and Harry are making a list of their phone numbers. Read the dialog. Complete the phone list.
Writing
Write instructions in five steps to tell a friend how to do or make something. E.g. make a phone call, cook something, or do today's homework.INTEGRATING SKILLSRead the information and write messages to put on the lost and found bulletin board.
1. Mark lost his gold watch. His phone number is seven-two-eight, oh-four-nine-nine.
2. Bill lost his baseball. It has his name on it. His phone number is five-two-four, one-four-oh-nine.
3. Mike found a small, blue pencil case on the school bus on Tuesday. His phone number is five-three-three, six-two-one-one.试题设计试题应以信息表达为测试目的
产生性运用从意义出发，还是从形式出发？
试题的情景，应该对受试群是真实的情景 (relevant to them)
测试的层次应是语篇层次：语篇的定义是语境和情景的语言，是具有平面的维度（dimensions）也有立体的维度的语言。试题要求的应该是互动性行为（interaction）
语言的制约应（自然）来自交际情景，而不是来自人为的对语言形式的主观选定：传统的从语言形式出发的试题，根据要靠的词汇和语法去设计试题。产生性运用试题不以课本里教过的词汇语法为出发点去设计试题，而从意义出发设计试题。并且完全可以按不同的程度去设计。所要求的词汇语法，可以按不同的程度加以控制。在真实生活里，任何人要说什么写什么，从来都不是先有了某些词汇某些语法想要用，然后才决定去说或者写某些话。二是倒过来，先有想要表达的意思，然后才去选择语法词汇，决定说的写的话的形式。而这想要表达的意思，却又受决定于交际情景。（举例：丢了朋友的书，道歉。）情景决定要表达的意思，要表达的意思再决定要说的话的形式。
具体设计三要素：情景、馈入、和输出。输入要将输出的内容需要具体化；输出的内容需要限制。语言教学和测试切合学语言用语言的真实的心理活动：应该根据科学对语言学习和运用的认识去设计语言学习和测试，让它尽可能地接近学语言用语言的真实心理状态。参考书目:
李筱菊 (1997). 语言测试科学与艺术. 湖南教育出版社.
刘 润 清 (2006). 语言测试理论的发展趋势. 济南语言测试研讨会。
Munby, J. L. (1978). Communicative syllabus design. Cambridge: CUP.
Oller, J. W. Jr. (1976). Evidence for a general language proficiency factor: An expectancy grammar. Die Neueren Sprachen. 75. pp. 165-174
Go For It Review (2006). PEP & Thomson Learning
试卷的标准（1）：效度（validity)：考到了要考的内容
内容效度(content validity)
编制效度(construct validity)
实证效度(empirical validity)
卷面效度(face validity)效度分类内在效度：
内容效度：测试是否考试了考试大纲规定要考的。内容效度的保证，主要靠试题设计和实体生产（命题和审题）时严格遵守大纲要求。验证主要靠考试前对试题的覆盖分析。验证的依据是大纲（它本身是否有效度呢？）。
结构效度：是考试的原则理论的效度。这里结构不是指试题或试卷的结构，而是指整个考试的意志为基础的理论结构。测试是否有结构效度，指其是否有效的语言观（包括语言运用观和语言学习观）为根据。结构效度决定了整个考试的性质。一个考试的最终价值，不在于它能用多少证据去证明，而在于它能给人带来美好的、快乐的效果，有助于使人变得更完美（李，1990）。
如果说英语测试的目的只是测试英语知识（不管运用的、狭义的知识），那么这套试题是效度颇高的。问题时，现代语言测试理论认为，仅仅是语言知识，远远不足以使一个人能够运用语言。因此语言测试如果只是测试语言知识，那就从根本上是无效度的。试卷的标准（2）：信度(reliability)
又称一致性(consistency)
稳定性信度(stability reliability)
又称一测再测信度(test-retest reliability)
等值信度(equivalent-form reliability)试卷的标准（3）：区分度(discriminative power )
实用性(practicality)
反拨作用(backwash effect)
外在的错误来源；内在的错误来源一套试卷的总体设计语言水平的定义
应试群体的特征
实施条件和所需设备
考试大纲（几种题型、各占分数、测试
内容、难易程度、词汇范围等）命题步骤培训命题人员（语言水平高，懂的测试理论）
设计题目（最好每人设计一种题型）
在与应试群体相同的人群中预测(pretest)
根据预测结果修改
再次预测
再次修改，直到满意为止任务类型以及任务设计 任务类型以及任务设计的意义Learning can be promoted through the use of tasks in four major ways.
1 learners can develop their accuracy in using the language.
2 learners can develop their capacity to
deploy a range of structures and a range of
vocabulary items and idioms within the context
3 learners can develop their fluency in doing what they do, by adjusting their patterns of fluent runs, uses of repetition and of rephrasing, and their utterance structure (Edmondson, 1981)
4 learners need to align their resources in the expression of their own individuality.
Two argumentation tasks 1 Give students a list of options to rank-order for an out-of-school weekend class activity;
2 Give students a list of options for an additional subject to be added to the school curriculum.
两个步骤：
每个学生独自决定自己的选择
与同伴讨论决定
准备阶段
一系列需要定夺的活动和科目。Narrative tasks Each student gets a different picture to prepare and then tell a short story to his or her partner.
准备阶段
学生有图作为输入以准备故事。
在图片中引入一些复杂的因素， 如角色、细节、更多的图片等。
Argumentative tasks 有激活大脑的作用（brainstorming quality）: 句子短；回应答语（responses tend to be echoic）; 支持或反对观点重复;所得列单也有重复.
逐渐发展一个稳定的立场由于必须全班报告使学生不得不使用难句并排练；
使学生参与并运用复杂语言.
E.g. 增加一些 引诱的或不可思议的观点
（tempting but potentially outrageous items in
the list of options）. Conclusion 不同类型的人物驱使不同形式的语言运用(different types of task set up different patterns of language use).
运用任务引导学生数出多样化的语言(to use tasks as a way of leading learners to produce differentiated types of language).
设计不同的任务是学生达到语言的流利、准确、复杂和得体。Critical thinking— 22 comprehension strategiesDrawing conclusions
Understanding cause and effect
Making connections
Comparing view points
Evaluating ideas
Reading between the lines
Making judgments
22 comprehension strategiesMaking comparisons
Interpreting meaning
Making inferences
Comparing characters
Reading a map
Interpreting graphs and charts
Making calculations
Examining reasons
22 comprehension strategiesSeparating facts from opinion
Identifying the main idea
Organizing information
Sequencing
Summarizing
Recalling details
Synthesizing
Brainstorm, either by yourself or with a group and create a list of 6-8 real-world tasks. Take one of these and think how it might be transformed into a pedagogical task, e. g. as a role play, simulation or problem-solving task. Then indicate how your task meets the key characteristics identified above. (Skehan, 1998; CDC Syllabus for English Language, Secondary 1-5, 1999: 43.)Making judgmentsHighlight the three pieces of information
in the text that you think everyone should know.
Most important_________________
Second most important___________
Least important__________________Developing alternative solutionsWith a group, brainstorm three different solutions to ..’s problem.
A __________________
B __________________
C ___________________
When you brainstorm, everyone contribute ideas. Write down every idea. ….Examining reasonsWhy do you think it’s necessary in this experiment to collect…?
____________________________
How do you think the author says to use…?
_____________________________
Why do you think…?
____________________________Making comparisonsSimilarities
____________________________
____________________________
Differences
____________________________Forming opinionsHow do you think the author of this article feel about…?
___________________________
Do you think these.. are fair or unfair? In your notebook, write a paragraph explaining your opinion about what happens to…
_____________________________
_____________________________Organizing informationIn the text, the author talks about three kinds of problems: environmental, animal and human. For each statement below, label the kind of problem being described
___________________________ __________________________ __________________________Contrasting and comparing selections When you contrast selections you think about how they are different.
When you compare selections you think about how they are the same.summarizingTo summarize is to make a brief statement giving the main points of a text.
Check off the three sentences below that you think best summarize “…”.Synthesizing （综合） When you synthesize information, you combine bits of information into a complex whole.
The first step in synthesizing is to identify your topic.
The second step is to find information about your topic in a text
The third step is to think about all the information you have found.
Ask yourself: what does this mean? Why is it important
Writers craftUsing antonyms
I try to be polite and kind because I know that I don’t like it when people are rude and cruel to meWriters craftAnalysing sensory language ( words that describe how something looks, sounds,feels, smells,or tastes )
Underline words and phrases that help you visualize what is happening
Sight____________________________
Sound___________________________
Taste____________________________
Smell____________________________
Touch____________________________