Lecture 1 What is research Research is the process of going up alleys to see if they are blind Marston Bates, American writer Research is what I am doing when I don’t know what I am doing Von Braun, American engineer If you steal from one author, it’s plagiarism; if you steal from many, it’s research. Wilson Mizner, American dramatist
Lecture 1 Compulsory Readings Nunan. D. 1992. Research Methods in Language Learning. Cambridge: CUP (P. 1 - 23) 23) Brown, J.D. & Rodgers, T.S., 2002, Doing Second Language Research, Oxford: OUP (P 3-18) Wisker, G. 2001, The Postgraduate Handbook, New York: Palgrave (P. 113 - 126) McDonough, J. & S. McDonough, 1997, Research Methods for English Language Teachers, London: London: Arnold (P. 37 37 - 55)
What is research? Research is a systematic process of inquiry consisting of three elements or components: A question, a problem or a hypothesis Data Analysis and interpretation of data (Nunan 1992)
Research process A process of formulating questions, problems, or hypothesis, collecting data or evidence relevant to these questions, problems or hypothesis and analyzing or interpreting data (Nunan 1992). The basic process of research problem / experience / observation hypothesis investigation and experimentation to test the hypothesis
data gathering data analysis and interpretation confirmation or disapproval of the hypothesis (Wisker 2001)
1.Truth and research Two views: External truths independent of the researcher/observer Truths are questionable, subjective
Two main approaches: Deductivism and Inductivism Deductivism Begin with a hypothesis/theory Search for evidence to - support the hypothesis/theory - refute the hypothesis/theory Inductivism Seek to derive general principles/theories/ ‘truths’ from an investigation and documentation of single instances
Research traditions Psychometric Interaction analysis Discourse analysis Ethnography
Vanlier: two parameters Interventionalist Seclectivity
Brown: Primary vs secondary Primary: basing on primary sources of information
Secondary: basing on the secondary sources of information
Questions for Tutorial 1 What is research? What is a research carried out for? Find a research report in an applied linguisti linguistics cs journal (such as TESOL Quarterly, Language Learning) and point out the objectives and how these objectives are achieved. Present briefly some traditions in applied linguistics research with illustrations illustrati ons from real research reports from applied linguistics linguistics linguistics journals. What are Deductivism and Inductivism in applied linguistics research? Give some examples from real research studies in applied linguistics. linguistics. Explain briefly some of the following terms with illustrations illustrat ions from illustrations applied linguistics research: Deductivism Deductivism Inductivism Reliability Systematic Validity Logical Construct Tangible Variable Variable Replicable Longitudinal Reductive Qualitative Quantitative Quantifiable Psychometric Primary data Secondary data
Types of research papers in Applied Linguistics Term paper MA thesis Prelim (qualifying paper) Dissertation proposal Grant proposal Dissertation Review of literature Conference abstract Conference paper Squib Book review Journal article Monograph, book
Textbooks Qualitative research (Non-numeral research ) All knowledge is relative Subjective element to knowledge Procedures: - observation - interview - field work - open response questionnaires
- transcripts
Quantitative research (Numeral research) Obtrusive Controlled Objective Generalisable Outcome-oriented Procedures: - tests - interview & self-reports - closed response questionnaires - numbers, figures
Reliability The consistency consiste ncy obtai ned from a piece consisten cy of the results obtained obtained of research Internal reliability: consistency of data d ata collection, data analysis and interpretation External reliability: similar results can be reproduced by independent researcher
Validity The extent to which piece p iece of research actually piece investigates what the researcher purports to investigate Internal validity: the interpretability of a research External validity: the extent to which the results can be generalised from samples to population Construct A psychological quality that cannot be observed but can only be assumed to exit and can be explained through observable behaviours (intelligence, proficiency, motivation, aptitude etc.) Construct validity: the construct under investigation must be defined accessible to the outside observer, the construct characteristics must be identifiable to reader
Variable Something that may vary, differ over time Some variables differ over over time ti me and and among am ong time among individuals (language proficiency, motivation motivation,, selfesteem etc.)
Variable vs construct Variables represent the underlying constructs Variable: what can be observed or quantified of the human characteristics or abilities involved Construct: the actual characteristic or ability that it represents in human being Systematic Clear structure with definite procedural rules Easy to read, interpret and evaluate Logical Step-by-step progression Straightforward, logical pattern with underlying rule and procedures Tangible Based on the collection and manipulation of data from the real world Data are well-defined quantity, rank or category Replicable The possibility of doing the research again under the same conditions by an independent researcher r esearcher researcher Replicating the research may yield similar results or information to evaluate it
Reductive The possibility of establishing patterns and relationships among individual variables, facts, and observable phenomena in a research
Questions for Tutorial 1 What is research? What is a research carried out for? Find a research report in an applied linguistics journal (such as TESOL Quarterly, Language Learning) and point out the objectives and how these objectives are achieved.
Present briefly different views of the nature of social sciences and social reality. Give some some examples from real research studies in applied linguistics. Present briefly a model of research process in applied linguistics with illustrations from real research reports from applied linguistics journals. 4. Explain briefly some of the following terms with illustrations from applied linguistics research: Deductivism Inductivism Reliability Systematic Validity Logical Construct Tangible Variable Replicable Longitudinal Reductive Qualitative Quantitative Quantifiable Psychometric Primary data Secondary data Social realty
Lecture 2 Compulsory Readings Nunan. D. 1992. Research Methods in Language Learning. Cambridge: CUP (P. 24 - 50) Selinger, H.W., and E. Shohamy. 1989. Second Language Research Methods. Oxford: OUP (P. 135 - 152) Brown, J.D. 1988. Understanding Research in Second Language Learning : A Teacher’s Guide to Statistics and Research Design. New York: CUP Hatch, E. and A. Lazaraton, 1991. The Research Mannual: Design and Statistics for Applied Linguistics. New York: Newburry House
What is an experimental research? Experimental research is carried out to explore explore the strength of relationship between variables
2. Type of scale Nominal scale Ordinal scale Interval scale Ratio scale Inferential statistics: The statistics which may be inferred from sample to population T-test: the test to compare two means (2 groups) F-test: the test to compare more than two means (more than 2 groups) ANOVA: Analysis Of Variance, a procedure to compare more than two means
Chi-square: calculate the different between the expected frequencies and compare them with the actual frequencies d egree Correlation: a statistical test to estimate the degree (strength) of association between two two variables Control group: the group under normal situation experimental group: the group under experimental condition True-experiment: - Randomly assigning the subjects into control and experimental groups - Administering both pre- and post-treatme post-treatment ntt tests post-treatmen (Highly controlled, highly selective) Quasi-experiment: - No random arrangement of subjects Pre-experiment - Only pre-treatment test
The Logic of Statistical Inference Mean
X
The average of a set of scores The central tendency of the scores
The Logic of Statistical Inference Statistical Inference Mode: The score that occurs most frequently in a set of scores. Median: the middle point in a distribution, the midway between the two middle scores The central tendency of the scores
Standard Deviation SD Important measure of dispersion Estimating: 1) deducting the mean from each individual score 2) Squaring the resulting figures to get rid of minus signs 3) adding these together and dividing by the total scores minus one ﾁ¨ Variance 4) Obtaining the square root of the figure ﾁ¨ SD
Standard Error
SE
The standard deviation of sample means: placing a single sample mean in relation to the population mean Estimating: Dividing SD of a single sample by the square square root of the number of observations in the sample (N)
SE = SD : ﾁ ã N
Questions for Tutorial 2 Find a Find a report of an experimental study in applied linguistics and give Question: your comments on the following checklist: I. Introduction: Is the framework for the study clear? A. Literature review : can you tell where the study fits in? 1. Is the background or rationale provided? 2. Is the relationship to previous research clear? B. Statement of purpose: can you tell where the study is heading? Are any of the following included? 1. Purpose 2. Research question 3. Research hypothesis II. Method : Is the study replicable? A. Subjects 1. Is the description of participants adequate? 2. Is the method of selection clear? B. Materials 1. Is there a description of tests, questionnaires, rating scales, and so forth? 2. Do the variables represent reasonable operational definitions of the underlying constructs or characteristics involved? 3. Is there a description of any equipment (when applicable)? C. Procedures 1. Is there a description of the preparation of material, administration, scoring and so on? 2. Is there a description of the conditions during study? D. Analyses 1. Is there a description of the arrangement and grouping of the data? 2. Are the statistical tests listed in order of use? III. Results A. Are all the statistical tests previously listed represented as results ? B. Is there a prose explanation (optional)? IV. Discussion/Conclusion A. Is the original research question, or questions, answered ? B. Is there an explanation of why the results were as they were? 1. If the conclusion is based on previous research, is it well supported and reasoned? 2. If the conclusion is speculative, is it qualified as such and well reasoned? C. Are suggestions for further research provided? V. References, Notes, or Footnotes A. Are all the references cited in the text included ? B. Are nay pertinent references missing? VI. Appendixes A. Are they necessary? B. Are they complete?
Lecture 3 ETHNOGRAPHIC RESEARCH Compulsory Readings
Nunan. D. 1992. Research Methods in Language Learning . Cambridge: CUP (P. 52 - 73) Brown, J.D& Rodgers T.S. Doing Second Language Research . OUP. 2002 (P.53-112)
Coffey, A. & Atkinson, P. 1996, Making Sense of Qualitative Data, Complementary Research Strategies, CA: SAGE Publications. May, T. (ed) 2002, Qualitative Research in Action , London: Sage Publications. McMillan, j. & S.Schumacher, Research in Education, A Conceptual Introduction , Harper Collins College Publishers. Wajnryb, R. 1992, Classroom Observation Tasks , Cambridge: CUP. Newman, P. & Ratliff, M. 2001, Linguistics Fieldwork , Cambridge: CUP What is ethnography?
Ethnography, which seeks to understand and represent the points of view of the members of a particular culture, is a primary approach to data collection and analysis in anthropology and sociology.
Ethnographic research Ethnographic research is one form of qualitative research which concerns with studying human behavior within the context in which that behavior would occur naturally and in which the role of the researcher would not affect the normal behavior of the subjects. Ethnographic research Ethnography research requires: - much training, skill and dedication - a great store on the collection and - question and hypothesis emerge during rather than beforehand
interpretation of data the course of investigation,
What is ethnography research
•
Ethnographic research uses naturalistic observation observation to record systematically the behaviour of the subject under study in its own settings. Its root is in ethnography: anthropology and sociology and the focus is on cultural aspects of behaviour
Characteristics of ethnographic research
•
Contextual: carried out out in the context in which the subjects normally normally live and
work
• • •
Unobtrusive: the researcher does not try to control or handle the phenomenon Longitudinal: the research takes a relative long time Collaborative: the research involves the interaction between participants
•
Interpretative: interpretative analysis of data
•
Organic: ethnographers may obtain data which do not support their original
questions or hypothesis but are suggestive of others. Procedures for conducting ethnographic research:
•
Define the phenomenon to be described.
• • • •
Use qualitative methods to gather data. Look for patterns in the data. Validate initial conclusions by returning to the data or collecting more data. Recycle through the process or the data.
Principles of ethnography ethnography Two sets of hypotheses about human behaviours:
•
naturalistic-ecological
•
qualitative – phynomenological
Naturalistic-ecological
•
Context has significant influence on behaviour
•
Investigating natural contexts: field research, real life situation, real world
Qualitative-phenomenological
• •
Belief: there is no objective reality independent of subjective perceptions
• •
No objective observer
Human behaviour must be investigated with subjective perception and belief systems of researcher Cultural meanings are revealed by the behaviours of the subject under study
Principles of ethnography ethnography
• • •
The use of participant and non-participant observation A focus on natural settings Use of subjective views and belief systems of the participant in the research
process
• •
Not to manipulate the study variables Holism and thick explanation
Low Inference Descriptors Vs High Inference Descriptors
• •
Low Inference Descriptors Descriptors (LID): Behaviours easily observed and agreed High Inference Descriptors (HID): Behaviours requires much inference
External reliability
•
To what extent can the findings from a study carried out in a particular site be generalized to other sites?
•
Threat: Based on detailed description of a particular context/ situation to conduct another research
→
difficult for outsiders
To overcome this: Explicit about 5 key aspects •
Status of researcher
•
Choice of informants
•
Social situation & conditions
•
Constructs & premises
Method of data collection & analysis Questions to be asked: • Is the status of the researcher made explicit? Does the researcher provide a detailed description of subjects? • Does the research provide a detailed description of the context and conditions • under which the research was carried out? Are constructs and premises explicitly defined? • • Are data collection and analysis methods presented in detail? •
Internal reliability • Can an independent researcher come to the same conclusion as the original investigators when he/ she analyzes the same primary data? •
Threat
Ethnographer rarely uses standardized instruments researcher to reanalyze data themselves
→
difficult for independent
To overcome this:
• • • •
Use low inference descriptors Multiple researchers/ participant researchers Peer examination Use mechanically recorded data
Questions to be asked: Does the research use low inference descriptors? •
• • •
Does it employ more than one researcher/ collaborator? Does the researcher invite peer examination or cross-site corroboration? Are data mechanically recorded?
Internal validity
To what extent the research is measuring what it purports to measure? To overcome this: Employ data collection and analysis techniques: + ethnographer as a participant + informants interviewing + participant observation + ethnographic analysis Questions to be asked:
•
Is it likely that maturational changes occurring during the course of the research will affect outcomes? • Is there bias in the selection of informants? Is the growth or attrition of informants over time likely to affect outcomes? • • Have alternative explanations for the phenomena been rigorously examined and excluded? External validity
To what extent can research outcomes be extended to other groups? To overcome this:
Describe phenomena explicitly so that they can be compared with other studies/ carry out multiple-site investigation Questions to be asked: • Are some phenomena unique to a particular group or site and therefore noncomparable? Are outcomes due in part to the presence of the research? • • Are cross-group comparisons invalidated by unique historical e xperiences of particular groups? • To what extent are abstract terms and constructs shared across different groups and research sites? Criteria for assessing ethnographic ethnographic reports
• • • • •
What are the goals of ethnography? What is the research problem? In what contexts was the research conducted? What is the group or case under study? What conceptual and theoretical frame works inform the study?
What field techniques are used? For how much time? In what contexts? What was the role of the ethnographer?
6. What analysis strategies were developed and used? What levels and types of context attended to in interpretation? 7. What recurrent patterns are described? 8. What cultural interpretation is provided? 9. What is the contribution of the study study to our knowledge of socio-cultural socio-cultural factors involved in schooling in second language and culture? 10. What are the stated implications for teaching? Questions for Tutorial 3 What is ethnographic research? ( State the difference between an ethnographic • research and a psychometric research and give example from applied linguistic studies.
2. Find a report report of an ethnographic research in applied linguistics and give your comments on the following points: - The research question - The contexts the research was conducted - What is group or case under study? - What conceptual and theoretical frame works inform the study? - What field techniques were used? For how much time? In what contexts? What were the roles of the ethnographer?
•
Find a report of a ethnographic research in applied linguistics and give your comments on: - What field techniques were used? For how much time? In what contexts? What were the roles of the ethnographer? - What analysis strategies were developed and used? what levels and types of context were attended to in interpretation? - What recurrent patterns are described? - What cultural interpretation is provided? - What are the stated implications for teaching?
