Cambridge Baseline Study 2013

Cambridge Baseline 2013 Introduction

1

2


1. INTRODUCTION Project background and aims

3

Cambridge English expertise

3

Conceptual framework

3

Key questions

6

Structure of the report

7

2. METHODOLOGY Research design: Convergent parallel mixed-method design

8

Project sample

9

Data collection instruments

13

Data analysis

13

Project timeline

14

Limitations

14

3. STUDENT ENGLISH LANGUAGE PROFICIENCY Overall national profile

17

Summary: National profile

36

National profile: Comparison with other countries

38

State profiles

40

Urban/rural/remote profiles

46

School type profiles Gender profiles

55 62

Class specialisation profiles

68

High performing versus low performing learners

71

Summary

76

4. TEACHER ENGLISH LANGUAGE PROFICIENCY, TEACHING KNOWLEDGE AND TEACHING PRACTICE Language proficiency: Teachers

79

Teaching Knowledge

88

Teaching practice

93

The role of assessment in learning and teaching

106

5. CURRICULA, LEARNING MATERIALS AND NATIONAL EXAMINATIONS Curricula Learning materials

111 115

National exams

118

6. RECOMMENDATIONS AND NEXT STEPS

120

REFERENCES

124


3

1. Project background and aims Malaysia has embarked upon a visionary English language education reform programme which will fundamentally transform the existing system, providing the young people of Malaysia with the knowledge, skills, attitudes and beliefs to enable them to become global citizens of the 21st century.

In October 2011, the Ministry of Education launched a review of the education system in order to develop a new National Education Blueprint – the Malaysia Education Blueprint 2013–2025 (referred to as the Education Blueprint in the rest of this document). This document recognises the increasing importance of English as a global language and the fact that the English proficiency of the population of a country is linked to its economic development:

‘Education is a major contributor to the development of our social and economic capital’ Prime Minister of Malaysia: Dato’Sri Mohd Najib bin Tun Haji Abdul Razak, Education Blueprint

Therefore, in 2013 the Ministry commissioned Cambridge English Language Assessment to undertake a comprehensive evaluation of the learning, teaching and assessment of English language in Malaysian schools from Pre-school to pre-university. The resulting evidence-based 2013 baseline gives the Ministry a clear picture of how the Malaysian English language education system is currently performing against internationally recognised standards. The findings and recommendations will act as the basis for discussion with the Ministry so that together Cambridge English Language Assessment and the Ministry can move to the next phase of collaboration.

Cambridge English expertise Cambridge English is uniquely positioned to deliver these services to the Ministry of Education given our expertise in educational reform, especially where English is concerned. We deliver over 4 million language assessments every year and have worked with governments and organisations around the world on similar projects. The effective and timely delivery of the baseline project involved a unique team of Cambridge English staff and external consultants with extensive expertise and experience in the fields of English language assessment, curricula development, teacher training and development, primary and secondary education, sampling, research methodology, data analysis, operational delivery and processing, and educational reform.

Conceptual framework Central to the design of this project was the construct of communicative language competence, which has become widely accepted as the goal of language education and as central to good classroom practice (Canale and Swain 1980; Bachman and Palmer 1982). Communicative language competence comprises linguistic competence, as well as the ability to functionally use that competence in language activities which involve oral and/or written reception, production and interaction in different domains. An approach driven by a communicative view entailed the inclusion of the four language skills of Reading, Listening, Writing and Speaking in the investigation of the

4


student and teacher English language baseline. Measuring just the receptive skills of Reading and Listening, albeit the most straightforward from a test administration perspective, would have given only a partial picture of the level of English proficiency of students and teachers. It was felt important, therefore, to include Speaking and Writing in the project as well, in order to gain a more comprehensive and accurate view of the language proficiency baseline. A further conceptual premise underlying the project was the belief in the importance in triangulating the investigation of English language proficiency with a complementary investigation of lesson observations and teacher subject and pedagogic knowledge. Such an approach allowed us to develop a more in-depth view of Malaysia English teachers’ profiles and their impact on student English language proficiency. Teachers and students do not exist in a vacuum, but are part of an ‘e cological system’ which also includes documents such as curricula, textbooks and examinations, which often determine policy, classroom practices and educational impact. As such, one strand of the project included a review of current primary and secondary school curricula, national textbooks and examinations. Taking into consideration the views of stakeholders is a further important conceptual premise which was fundamental to the project. The context of and stakeholder attitudes to English language learning and teaching can provide insight into factors that may be influencing learning outcomes. Therefore, background and attitudinal questionnaires for teachers, students and Heads of Panel/Head Teachers, as well as interviews with teachers, Heads of Panel/Head Teachers and policy planners from the Ministry of Education, comprised a further strand in the project. The Common European Framework of Reference (CEFR, Council of Europe 2001) provides a useful common basis for the investigation of many of the questions of interest in the project, such as language proficiency, curricula, examinations and textbooks. The CEFR: ‘describes in a comprehensive way what language learners have to learn to do in order to use language for communication and what knowledge and skills they have to develop so as to be able to act effectively. … The Framework also defines levels of proficiency which allow learners’ progress to be measured at each stage of learning and on a life-long basis’ (Council of Europe 2001:1). The CEFR, therefore, provided a useful tool for description and comparison, which also allowed the findings of the project to be considered against a broader international context where the CEFR is used. The CEFR describes a set of six broad common reference levels, which cover the language learning continuum from a basic ‘Breakthrough’ level to an advanced ‘Mastery’ level. The CEFR is not Europe-specific and has also been used beyond Europe, either in its srcinal form or adapted to different contexts, e.g. the CEFR-J is an adaptation of the srcinal CEFR to language teaching, learning and assessment in Japanese contexts (Negishi, Takada and Tono 2013). The CEFR common reference levels and the ‘Can Do’ statements which characterise the levels can be seen in Table 1.1. The ‘Can Do’ statements for each level will be useful reference points for readers of this report, since they specify in broad terms what the baselines established for the different cohorts in the project actually mean.

Cambridge Baseline 2013 Introduction Table 1.1 CEFR Common reference levels: Global scale (Council of Europe 2001:33)

C2

Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in the most complex situations.

C1

Can understand a wide range of demanding, longer texts, and recognise

Proficient

implicit meaning.

User

Can express ideas fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.

B2

Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of

Independent

various options.

User B1

Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise while travelling in an area where the language is spoken. Can produce simple connected text on topics that are familiar or of personal interest. Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans.

A2

Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate

Basic User

A1

environment and matters in areas of immediate need. Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

5

6


Key questions The project was guided by the following set of key questions:

Students: Language proficiency 1. How do students at different school stages in the Malaysian states/federal territories perform on a set of Cambridge English language tests overall and by skill (Reading, Writing, Listening, Speaking) against the CEFR? 2. How does overall and by skill student performance at different school stages compare according to: a. state/federal territory (16 states/federal territories) b. location (urban/rural/remote) c.

type of school (National, Chinese, Tamil at Primary; National and Religious at Secondary)

d. gender (male/female) e. class specialisation at Form 5 and 6 (Arts, Science, Religious, Vocational/Technology)?

Teachers: Language proficiency, teaching knowledge and teaching practice Language proficiency 3. How do teachers teaching at different school stages in Malaysian states/federal territories perform on a set of Cambridge English language tests overall and by skill (Reading, Writing, Listening, Speaking) against the CEFR? 4. How does overall and by skill teacher performance compare according to: a. school stage (primary/secondary) b. location (urban/rural/remote)?

Teaching knowledge 5. How do teachers teaching at different school stages in the Malaysian states/federal territories perform overall on a test of teaching knowledge? 6. How does teaching knowledge compare according to: a. school stage (primary/secondary) b. location (urban/rural/remote)?

Teaching practice 7. How do teachers teaching at different school stages in the Malaysian states/federal territories perform overall in terms of teaching practice based on classroom practice evaluations and observations and postobservation discussions? 8. How does teaching practice compare according to: a. school stage (primary/secondary) b. location (urban/rural/remote)?

Curricula, national textbooks and examinations 9. What are the features of currently used learning, teaching and assessment materials (e.g. curricula, textbooks, national tests) according to international standards, e.g. the CEFR, and according to current trends in teaching practice?


7

Recommendations 10. What recommendations can be made, based on the benchmarking of teachers/students and the evaluation of teaching/assessment materials, to enable the achievement of the envisioned education transformation by the Malaysia Ministry of Education?

Structure of the report The report is structured following the key aims and questions which underlie it. After the introduction (Section 1), an overview of the methodology will be given (Section 2), outlining how the project was carried out in terms of research design, data collection, analysis and participants. The findings on student English language proficiency then follow (Section 3), with a focus on the overall and by-skill performance of the five school grades of interest and the attitudinal and background factors which play a role in English language achievement. The performance of the cohorts overall is followed by an investigation of performance based on key variables and comparisons between them, such as: states/federal territories; urban, rural and remote location; school types; gender; class specialisation. The next section (Section 4) presents findings on teacher English language proficiency, teaching

knowledge and teaching practice. In each case, performance overall is given, followed by comparisons based on key variables, such as urban, rural and remote location, and primary/secondary school. The test performance findings are integrated with findings about attitudinal and background variables which play a role in teacher attainment, open-ended comments from the teacher questionnaire, interviews with Heads of Panel/Head Teachers and Ministry of Education officials, and extended feedback from the classroom observers. The review of key policy-setting documents which shape the learning, teaching and assessment in classrooms follows (Section 5), with a discussion of current curricula, learning materials and examinations. Finally, the report ends with a set of

recommendations which emerge as a result of the qualitative and quantitative findings, and suggestions for ways forward (Section 6). More detailed information on the project, including sampling, project participants, instrument development, data analysis procedures and significance testing output can be found in the Cambridge Baseline 2013 Technical Report.

8

Cambridge Baseline 2013 Methodology

2. The large-scale scope of the project and its aim, i.e. to build up a comprehensive profile of English language education in Malaysia in order to provide the Ministry of Education with data-driven recommendations for improving English language standards in Malaysia, necessitated a research design which allowed the gathering of different types of information through the use of a range of tools. A premise recognised in educational reform is that ‘a key characteristic of the educational process is that student learning is influenced by many small factors rather than a few large ones’ (Chapman, Weidman, Cohen and Mercer 2005:526); therefore, any recommendations made in this project need to be based on an in-depth understanding of all aspects of the educational system in order to ensure they are achievable and reduce the chances of any negative unintended consequences. As a result, the project focused not only on measuring English language levels of students and teachers, but also on investigating the context of learning, the availability and quality of resources, and stakeholder perceptions. A mixed-method approach formed the basis of the study and a convergent parallel design (Creswell 2009) was chosen due to its value in collecting qualitative and quantitative data strands in a parallel fashion and in relatively short timeframes.

Research design: Convergent parallel mixed-method design A fundamental feature of the convergent parallel mixed-method design is that it consists of two data strands – a qualitative and a quantitative one. These occur independently and concurrently, are analysed separately and findings are then converged and integrated to inform the final overall findings. The key assumption of this design is that a complex project, such as for example a national survey, can best be understood by gathering and investigating different types of information which provide insights into different aspects of the project. A typical aim, therefore, is to use large-scale statistical quantitative findings and detailed qualitative insights to develop an in-depth understanding of a situation or an event through integrating these two types of information. The use of two data collection strands in the Cambridge Malaysia Baseline Project – one quantitative and one qualitative – allowed the project to maximise the value of two different research methodologies which are grounded in different theoretical paradigms and interpret data through different lenses. The quantitative paradigm underlies research which is based on large data sets, representative samples and statistical methods. A quantitative strand was, therefore, well suited to the investigation of student/teacher language ability, teacher pedagogic knowledge and stakeholder views through a suite of tests and questionnaires. The qualitative paradigm draws on discovery and description; it is inductive and uses small purposefully chosen samples. It was, as such, well suited to observations of classroom practices, interviews with stakeholders, comments written in response to open-ended questions by stakeholders, and curricula, textbooks and examination reviews. This approach allowed the project to build a rich picture of the current situation with regards to learning, teaching and assessment in Malaysia and enhance the validity of the findings. Figure 2.1 presents an overview of the data collection and data analysis procedures which formed the backbone of the project.


9

Figure 2.1 Research design

QUANTITATIVE DATA ANALYSIS QUANTITATIVE DATA COLLECTIO N

Benchmarking tests for students and teachers Questionnaires for students, teachers and education leaders Classroom observations – teaching practice assessment

Descriptive statistics of test scores and questionnaire responses (mean and standard deviation; frequency %, mode) Mapping onto CEFR levels (Rasch analysis and ability estimates) Linear and logistic regression Multilevel modelling

INTEGRATION AND INTERPRETATION QUALITATIVE DATA COLLECTION

Comments in questionnaires for students, teachers and leaders Semi-structured interviews with policy planners Classroom observations – observer comments and post-observation discussions Review of curricula, examinations, learning materials

QUALITATIVE DATA ANALYSIS

Thematic analysis of questionnaire comments, interviews, classroom observations notes and document review trends

Project sample The participants included students at five different age ranges, teachers, Heads of Panel/Head Teachers and policy planners. Specific information on their distribution within the sample can be seen in Table 2.1. The selected schools were chosen as part of a two-part stratified sampling methodology (i.e. schools were selected first based on a range of variables and then students were selected within each school). This methodology resulted in a sample which was intended to be representative of the overall target population.

10


Table 2.1 Participants: by data collection instruments Participants

Tests Reading/ Listening

Students

20402

Writing 9921

Speaking

Teaching knowledge

Classroom observations (Number of classes)

Interviews

1372

Q’res

17104

Pre-school

3430

2127

187

12

Year 6

4795

1328

206

14

4956

Form 3

4858

2422

331

16

4740

Form 5 Form 6

5458 1861

2941 1103

431 217

23 13

5495 1913

Teachers

424

266

Primary

115

71

13

42

196

600

26

78

558

1290

Secondary

287

188

29

352

52

732

Unknown

22

7

0

52

Heads of Panel/Head Teachers

41

31

Primary

14

9

Secondary

27

22

Ministry of Education

4

Figure 2.2 shows the distribution of students by school grade. As can be seen, the sample covered all school grades of interest, with Year 6, Forms 3 and 5 having a roughly similar proportion, and Form 6 having the smallest proportion of students in the sample due to the smaller proportion of Form 6 in the target population. Figure 2.2 Distribution of students by school grade (N=20,402)

Form 6 9%

Pre-school 17%

Form 5 27% Year 6 23%

Form 3 24%


11

The distribution of teachers in the sample can be seen in Figure 2.3, which shows that approximately one quarter of the teachers in the survey came from the primary school level, and approximately two thirds came from secondary schools. Figure 2.3 Distribution of teachers by primary/secondary school (N=424) Unknown 5%

Primary 27%

Secondary 68%

In addition to the language tests for teachers and students, questionnaires for students, teachers and Head Teachers/Heads of Panel were also distributed in order to allow the collection of attitudinal and background data. The distribution of completed questionnaires across the different school grades is given in Figure 2.4. Pre-school children were not included in the questionnaire data, due to concerns with the reliability and validity of questionnaire data supplied by young children (Borgers, Leeuw and Hox 2000). Figure 2.4 Distribution of student questionnaire responses by school grade (N=17,104)

Form 6 11% Year 6 29%

Form 5 32% Form 3 28%

12


The distribution of teacher questionnaire responses by primary/secondary school is shown in Figure 2.5. Figure 2.5 Distribution of teacher questionnaire responses by primary/secondary school (N=1,290)

Primary 43% Secondary 57%


13

Data collection instruments A range of instruments was used to allow the collection of both quantitative and qualitative data. They comprised:

Benchmarking English language tests for students and teachers: aimed at providing information on language proficiency, in terms of Reading, Listening, Writing and Speaking as measured against the CEFR

Benchmarking teaching knowledge test for teachers : intended to provide a measure of knowledge of and familiarity with teaching knowledge concepts in an objectively-scored test

Student, teacher and Head of Panel/Head Teacher questionnaires: aimed to gather stakeholder perceptions of and attitudes toward English language learning, teaching and assessment Classroom observations and post-observation discussions: intended to gather in-depth information on teaching competence and performance for a smaller sub-set of the selected sample

Semi-structured interviews with policy planners and senior school administrators: focused on exploring perceptions of the review project and expected outcomes, as well as views on curriculum, textbooks, examinations, teaching practice

Curricula, textbooks and examinations review : intended to investigate issues such as the relationship between standards, curricula, textbooks and examinations and the CEFR; information on the extent to which the different documents complement each other and reflect latest trends in learning, teaching and assessment, e.g. student-centred learning and teaching, learning-oriented assessment, communicative-ability assessment.

Data analysis As noted above (Figure 2.1), the mixed-method research design underlying this project involved both qualitative and quantitative analyses, which comprised:

CEFR level mapping: Rasch analysis and ability estimates Descriptive statistics in the quantitative strand: aimed to provide an overall picture of CEFR language level, teaching knowledge and stakeholder perceptions, as well as the amount of variability within each group. The analysis focused on the cohort as a whole (e.g. all Form 5 students) and on specific va riables within the cohort (e.g. Form 5 boys and girls; Form 5 urban, rural and remote students)

Linear and logistic r egression: aimed to investigate which background and attitudinal factors play a role in high- and low-achievers

Multilevel modelling: aimed to explore and confirm whether any attitudinal and background variables (e.g. student motivation, use of the internet, etc.) played a significant role in predicting the language level of students

Chi-square test of independence: aimed to investigate whether the different variables of interest (e.g. state, location, gender, etc.) were related to questionnaire responses. Standardised residuals were also computed to identify which responses were contributing to the test of significance

ANOVA and t-tests: aimed to explore whether there was any variance in the teacher group means for questionnaire composite measures. Questionnaire statements on similar topics (e.g. assessment practices, use of English in the classroom, etc.) were grouped together to determine whether teacher variables (i.e. experience, education, school type, etc.) influenced responses

Thematic analysis in the qualitative strand: focused on bringing the wealth of collected in-depth observational, questionnaire, interview and descriptive data into general thematic categories which indicated major issues brought up by the different stakeholders participating in the project.

14


The final analysis stage involved an integration of the quantitative and qualitative findings. This was essentially an expert judgement approach which involved key members of the project team identifying relationships between the different quantitative and qualitative findings and providing evidence for these relationships. Ethical guidelines from the University of Cambridge, the British Association of Applied Linguistics and the British Educational Research Association were followed during all data collection and data analysis phases of this project.

Project timeline Activity

Date

Cambridge English proposal for the baseline project submitted to the Ministry of Education Gathering of relevant data from the Ministry of Education Development of assessment and questionnaire instruments Review and analysis of curricula, teaching documentation, examination data, etc. Visits to schools for classroom/teacher observations, Speaking tests, interviews with staff Administration of paper-based benchmarking tests in schools

3 February 2013

February – September 2013

February – April 2013

February – October 2013

18 October – 1 November 2013

29 October – 5 November 2013

Receipt and processing of results

November – December 2013

Analysis of data and production of reports

January 2014 – March 2014

Presentation of findings and recommendations

1 – 4 April 2014

Limitations There were several issues that arose during the dispatch, administration and return of the exam materials which resulted in fewer candidates taking the tests and completing questionnaires than expected or in some data being removed from the analysis. Exam/classroom observation administration: The administration of the Cambridge English exams coincided with the end of term, which resulted in fewer students in attendance than expected. For the classes who had not yet taken their exams, the lessons observed tended to focus on exam preparation, which may not have been indicative of a typical English lesson.


15

Missing/incorrect ID data: Pre-school learners comprise a smaller proportion in the sample than srcinally planned, due to the high incidence of missing or incorrect student data in the data files. ID and name information for Preschool children was not provided. The data entered by the children and/or their teachers showed frequent irregularities which led to some data being discarded. The proportion of primary and secondary school teachers was more similar in the srcinal sample (41% primary school classes/teachers and 59% secondary school classes/teachers). However, due to large numbers of incorrectly completed test sheets for teachers and the absence of such data from the Ministry of Education for cross-referencing, it could not be determined which school stage some teachers taught in and as such their data was excluded from the data set. The number of teachers who completed the language tests (N=424) and teaching knowledge test (N=600) was smaller than the number expected (N=934). This suggests that some teachers may have opted out of taking the tests and that the teachers who did complete the tests may have been selfselected (See Section 4 for more information). Exam malpractice: During the marking of Writing scripts, examiners noticed many instances of malpractice in that either pairs of students had the same/similar answer or in some cases the whole class had the same/similar answer. This was particularly evident for Pre-school classes. Questionnaires: The learner questionnaire was intended to be administered to all secondary students and Year 6 students only; however, it was inadvertently included in the test materials sent to Pre-school classes. Response data from Pre-school students was not included in the analysis because reliability and accuracy could not be assured. The proportion of questionnaires returned from each state was also unequal, with Selangor accounting for 20% of all teacher questionnaires and all learner questionnaires returned. Perlis and WP Putrajaya accounted for fewer than 1% of all teacher questionnaires returned. WP Labuan and WP Putrajaya accounted for fewer than 1% of all learner questionnaires returned. Examination review: Cambridge English did not receive important statistical data about item and test performance for the national exam sample question papers. This information would have allowed a far more in-depth analysis to be carried out.

Cambridge Baseline 2013

16

Student English language proficiency

3.

Key findings The key findings provide a snapshot of the established learner proficiency baseline and also highlight the most salient and meaningful differences in proficiency levels which emerged across certain variables as statistically significant, such as location, gender and class specialisation. In addition, the key findings note the main attitudinal and background factors which were found to be associated with high performing learners.

Pre-school On average below CEFR level A1 78% below A1, 22% at A1/A2 Writing and Speaking emerge as weaker skills than Listening and Reading

Year 6 On average at CEFR level A1 32% below A1, 56% at A1/A2, 13% at B1/B2 Similar performance observed across Listening, Reading and Speaking; Writing emerges as the strongest skill Students in remote and rural areas perform significantly worse than in urban areas

Form 3 On average at CEFR level A2 69% at A1/A2 and below; 30% at B1/B2; 1% at C1/C2 Speaking emerges as the weakest skill Students in remote and rural areas perform significantly worse than in urban areas Female students perform significantly better than male learners

Form 5 On average at CEFR level A2 55% at A1/A2 and below; 43% at B1/B2; 2% at C1/C2 Speaking emerges as the weakest skill Students in remote and rural areas perform significantly worse than in urban areas Male students perform significantly worse than female students Students in Science classes perform significantly better than students in other specialisation classes

Form 6 On average at CEFR level A2/B1 41% at A1/A2 and below; 53% at B1/B2; 6% at C1/C2 Listening and Speaking emerge as the weakest skills Students in remote and rural areas perform significantly worse than in urban areas Students in Science classes perform significantly better than students in other specialisations

Cambridge Baseline 2013 Student English language proficiency

17

Role of attitudinal and background variables Learners at all levels recognise the importance of learning English for improving their employment and educational opportunities in the future Learners have very little exposure to English or the opportunity to use E nglish either within the learning environment or outside it Learners whose parents speak English are more likely to be performing better in school Learners in rural and remote schools are less likely to be highly motivated or have parents who speak English when compared to their urban counterparts Note: Percentages may not always add up to 100% due to rounding.

As noted in the Methodology section in this report, the language tests used to benchmark the students (and teachers) tapped into both receptive language skills – Listening and Reading – and productive skills – Writing and Speaking. The use of a suite of tests which are based on a communicative construct of language ability was an important aim of the project, in order to allow a more comprehensive and valid profile of language proficiency to be built. The need to gather a broad range of data about language proficiency, instead of just relying on easy-toadminister reading and listening tests, is also emphasised in the CfBT Education Trust Commentary on the Malaysia Education Blueprint 2013-2025 (McAleavy, Hamilton and Latham no date:12). We now move on to an overview of the national baseline at the school grades of interest and supplement the test findings with the key insights obtained from the questionnaires, interviews and classroom observations.

Overall national profile Question 1: How do students at different school stages in the Malaysian states/federal territories perform on a set of Cambridge English language tests overall and by skill (Reading, Writing, Listening, Speaking) against the CEFR?

Language proficiency: Primary school Pre-school Overall, Pre-school learners in Malaysia achieved a CEFR level below A1. In terms of the language skills of Reading, Writing, Listening and Speaking, all four skills were found to be below CEFR level A1, as can be seen in Table 3.1. The CEFR levels in the table are based on the mean scores of the Pre-school cohort, and for Listening and Reading, the Pre-school children achieved a mean score just below the A1 level boundary, indicating that their productive skills of Speaking and Writing in English lag behind their receptive Listening and Reading skills. This could partly be a reflection of the slower development of literacy in young learners, who often lack the skills to handle the cognitively complex processes of writing in both their first and additional languages (Bangert-Drowns, Hurley and Wilkinson 2004). It is also in line with the trend found in many learners for receptive skills to be stronger than productive ones. Table 3.1 Average CEFR proficiency levels: Pre -school Language skill

Listening

Reading

Writing

Speaking

CEFR level

Below A1

Below A1

Below A1

Below A1


18


The distribution of Pre-school learners across the different CEFR levels covered in their tests (Table 3.2 and Figure 3.1) ranged from A2 to below A1. In all four skills, the largest proportion of Pre -school children were below level A1, but as noted above, Speaking and Writing appear to be weaker than Reading and Listening: in the case of Listening and Reading, approximately a third to a half achieved either level A1 or A2 (32% for Listening and 44% for Reading), in the case of Speaking and Writing that proportion was much smaller (12% for Writing and 3% for Speaking). Table 3.2 Distribution of CEFR levels: Pre-school Language skill

Listening

Reading

Writing

Speaking

Overall

A2 A1

9% 23%

11% 33%

4% 8%

0% 3%

6% 16%

Below A1

68%

56%

89%

97%

78%

Note: ‘Overall’ percentage is based on the equal distribution o f all four skills. Percentages may not always add up to 100% due to rounding.

Figure 3.1 Distribution of overall CEFR levels: P re-school 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Below A1

A1

A2

Year 6 Overall, Year 6 learners in Malaysia achieved a CEFR level A1. Listening, Reading and Speaking were observed to be at A1, with Writing performance at level A2 (Table 3.3). However, the mean score which the CEFR Writing level is based on corresponds to a low A2 level, making the four skills generally comparable in development. Table 3.3 Average CEFR proficiency levels: Year 6 Language skill CEFR level

Listening

Reading

Writing

Speaking

A1

A1

A2

A1

The range of CEFR levels across Year 6 can be seen in Table 3.4 and Figure 3.2, which indicate that the performance of most Year 6 students spans CEFR levels B1 to below A1. Compared to Pre-school learners, Year 6 learners show a smaller proportion at level A1 and below (94% at Pre-school, 66% at Year 6), and a larger proportion at the higher A2 level, with some achieving B1 as well. The gap between the high and low achievers in Year 6 is worth noting, as it signals that a variation in learning gains is already present at the end of primary school, and will increase further in Form 3 and Form 5, as will be seen later.


19

Table 3.4 Distribution of CEFR levels: Year 6 CEFR level

Listening

Reading

Writing

Speaking

B2

0%

0%

2%

0%

Overall 1%

B1

10%

0%

31%

5%

12%

A2

20%

14%

33%

20%

22%

A1

28%

29%

19%

59%

34%

Below A1

42%

57%

14%

16%

32%

Note: Percentages may not always add up to 100% due to rounding.

Figure 3.2 Distribution of Overall CEFR levels: Year 6 40% 35% 30% 25% 20% 15% 10% 5% 0% Below A1

A1

A2

B1

B2

Attitudinal and background factors: Primary school In order to better understand the context of learning and teaching in Malaysia, students were asked to complete a questionnaire. The focus of the questionnaire was on learners’ attitudes toward learning and background factors which may play a role in learner motivation, with the aim being to identify key factors which may be associated with high achievement in English language learning. Due to the young age of the Pre-school learners, it was not deemed appropriate to ask them to complete a questionnaire. There is widespread consensus that questionnaires are not generally given to children under the age of 8 because they are still at an early stage of their linguistic and cognitive development, which can make it difficult to ensure the validity and reliability of their responses (Borgers, Leeuw and Hox 2000). As such, this section focuses on Year 6 only. Questionnaires were completed by 4,956 Year 6 students (29% of the returned student questionnaires) and by 443 teachers who indicated that they have the most experience teaching Year 6 learners (35% of teacher questionnaires). The key findings from the questionnaires which emerged as significant and meaningful are summarised below. The data analysis procedures for the questionnaires can be found in the Cambridge Baseline 2013 Technical Report.

Attitudes towards learning English The academic literature on second/foreign language acquisition has indicated that learners’ attitudes towards learning a language and the extent to which they perceive the language to be useful can influence learner behaviour, both in terms of the amount of effort exerted on language learning and the extent to which they persist with learning it (Csizér and Dörnyei 2005; Gardner 1985; Oxford and Ehrman 1993). This was, as such, an important construct captured in parts of the questionnaire in this project.


20


Fifty percent of Year 6 learners strongly agreed that learning English is important to them; however, this figure was lower than all other grades (see Figure 3.3), possibly because unlike secondary school learners they have less experience of the world and are further away from making choices about further study or work. Figure 3.3 also shows that primary school students were more likely than secondary school students to strongly agree when asked if they enjoy their English lessons at school. This may be linked to the level of difficulty of their lessons as the lower level of proficiency of Year 6 students means that their lessons likely focus on simple grammatical structures and vocabulary, whereas the older leaners are facing more cognitively demanding lessons as their language ability increases. The secondary school students are also preparing for exams which have a more direct impact on their future plans, perhaps influencing their level of enjoyment.

Figure 3.3 Importance and enjoyment of English by grade (percentage who strongly agree) 70% 60% 50% 40% 30% 20% 10% 0% Year 6 Form 3 Learning English is important

Form 5

Form 6 I enjoy English lessons

Several statements in the questionnaire were designed to shed further light on the motivational factors that may be influencing learners’ language learning behaviour such as instrumentality or milieu (Gardner 1985). Instrumental motivation refers to the utilitarian benefit or incentives associated with learning a language such as getting a job, a place in university, travelling or using the internet, whereas the motivational dimension of milieu refers to the influence of learners’ immediate social environment (i.e. parents, family and friends), excluding teachers, in shaping their attitudes to learning. Learners who perceive family support for language learning are more likely to persist with it and more willing to work harder at it (Colletta, Clément and Edwards 1983; Gardner 1985). Figure 3.4 shows that Year 6 learners did recognise the functional purpose of English, as 73% of these learners reported that it is very important for them to learn English as it will help them get into a good university and 69% reported that it is very important as it will help them get a good job. These learners were also more likely than the students in all other grades to say that it is very important for them to learn English to please their parents (53%) (Form 3: 36%; Form 5: 34%; Form 6: 22%).


21

Figure 3.4 Reasons for learning English: Year 6

How important are these reasons for you? Learning English will... …help me get into a good university. …help me get a good job. Very important …make it easier for me to travel to other countries.

Important Not very important

…please my parents.

Not at all important

…make it easier for me to talk to other people.

Not sure

….help me use the internet to get information. 0%

20%

40%

60%

80% 100%

These findings suggest that Year 6 learners are primarily motivated to learn English for instrumental reasons, and to a lesser extent but likely a very important reason, to please their parents. Although neither motivational dimension is causally linked to learner outcomes because other variables such as instructional quality, learning opportunities and learner ability also play a crucial role, they do provide an indication of how much energy or attention learners are willing to expend on learning. However, despite these indications that learners are motivated to learn English, only 11% of teachers strongly agreed that they thought their Year 6 students enjoyed their English lessons and several teachers commented in the open-response questions that they thought students were unmotivated and lacked interest in learning English: ‘Students have no background in English and have no interest’ (Rural primary school teacher, Sarawak) This point was also mentioned by Heads of Panel/Head Teachers: ‘The students are not interested in learning English’ (Urban primary school teacher, Melaka) During post-observation interviews, some teachers felt that parents could do more to support and motivate their children. Teachers were asked in the questionnaire whether their students’ parents participate actively in their education and 45% of primary school teachers either disagreed or strongly disagreed with this question. Teachers did recognise though that perhaps parents, particularly those in rural areas, may not have the skills or knowledge to provide this support and that the Ministry may need to take on the responsibility of helping parents in this area: ‘Support and encouragement from parents can help. Education Ministry should have a series of motivational courses for parents from rural areas to expose the importance of English to them’ (Rural primary school teacher, Melaka) ‘Have seminars for parents to create awareness about the importance of mastering the language’ (Urban primary school teacher, Sarawak)

22


Teachers see the value of involving parents in their children’s education; however, ensuring parents have the information and skills to do this may be a necessary first step.

Exposure to English It is a widely accepted premise both in the theoretical and practical language learning domains that exposure to a foreign language within the learning environment and/or the home environment plays a positive role in learning. The European Survey on Language Competences (ESLC) found that greater use of English, by both students and teachers, in the classroom was positively related to language ability (Jones 2013). The ESLC also found that parents’ knowledge of the foreign language being studied and leaners’ exposure to it in the home or community was positively related to learner outcomes (Jones 2013). Therefore, in the questionnaire, we investigated the extent to which leaners are exposed to English either in their home environment or at school. Approximately 50% of Year 6 students reported that their parents speak English either very well or moderately well and 38% take English lessons outside school, which is the highest percentage for all grades (Form 3: 26%; Form 5: 21%; Form 6: 10%). However, despite this, a majority of these students said that they either never speak English or do not speak English very often outside school, which includes with family or friends (56%) or in the wider community (74%). Figure 3.5 shows the frequency of English language usage for different activities that young people are likely to engage in. Figure 3.5 Frequency of English language usage for differ ent activities: Year 6 Outside of school…

…I use the internet in English including playing games. …I watch TV of films in English.

Very often Sometimes

…I read books or comic books in English.

Not very often

…I speak English with my family and/or friends.

Never Not sure

…I speak English to people in my village/town/city. 0%

20%

40%

60%

80%

100%

As learners do not appear to be using English very often in their home environment, the use of English within the learning environment becomes even more important to their language development. Only 55%, the lowest percentage of all grades, said that their teacher speaks English very often during lessons (Form 3: 75%; Form 5: 79%; Form 6: 90%). Comments in the teacher questionnaire suggest that too much instruction may be taking place in the learners’ first language: ‘The headmaster should ensure that teaching and learning English must be done in English. Sometimes when people visit it's in English, otherwise they teach in their mother t ongue … There should be more focus on using English during teaching and learning in class’ (Rural primary school teacher, Perak) ‘English teachers should speak English during lessons’ (Urban primary school teacher, Kedah) This finding could relate to teacher language proficiency, as 13% of primary school teachers as compared to 4% of secondary school teachers reported that they speak the local language a lot during lessons because they do not


23

feel confident teaching in English. The teacher test data, to be discussed in Section 4, will show that primary teachers’ language ability is lower than that of secondary school teachers. However, the relatively low percentage of English usage in the classroom could also be linked to teachers’ perceptions of their learners’ language ability. Some teachers admit that they do not speak English during lessons because they do not believe learners are able to cope with instruction which is completely in English: ‘Lessons cannot be fully conducted in English as students don’t understand’ (Rural primary school teacher, Perlis) This finding is worrisome because it suggests that teachers may not be fully aware of how to grade their language so that it is comprehensible for their learners. Observation findings indicate that this is an issue for some teachers (see Section 4). Turning to the learners’ use of English, they reported that they do not use English that often during English lessons either, whether it is with their teacher or with their classmates (see Figure 3.6). Figure 3.6 Use of English in the classroom: Year 6 During English lessons...

…my teacher speaks English.

Very often Sometimes

…I speak to the teacher in English.

Not very often

…I speak to other students in English.

Never Not sure 0%

20%

40%

60%

80%

100%

The limited use of English in the classroom could be related to the type of learning activities chosen by teachers, and mentioned by observers, such as drilling, reading aloud, etc., which do not give learners opportunities to use English communicatively (see Section 4). The observers also noted a propensity for teacher-dominated lessons which also tends to reduce the amount of English produced by learners. Beyond the classroom, the school environment can also influence students’ motivation to learn English by providing opportunities to practise English outside the classroom and by showing support for English learning. Thirty-five percent of primary school teachers strongly agreed that their school has created an English environment outside the English classroom, but they did comment that the school could do more: ‘Expose students to more English, e.g. through English charts and labels around the school compound’ (Urban primary school teacher, Kelantan) Despite the fact that Year 6 learners do not appear to have much exposure to English either within or outside the classroom, 40% of them strongly agreed that they would like to learn other subjects in English, such as Maths or Science. This point was also made by several teachers, in response to the open-ended questions; they feel that learners would benefit from Content and Language Integrated Learning (CLIL): ‘Teaching and learning in English for subjects like Science and Mathematics can have a positive impact. This can help the students improve their understanding of the English language’ (Urban primary school teacher, Johor)

24


Other suggestions frequently made by teachers to improve English learning in their school related to reducing the number of students in each class and increasing the number of contact hours in English: ‘Reduce the number of students in the classroom to about 20 or 25’ (Urban primary school teacher, Pahang) ‘The time allotted for English is insufficient’ (Urban primary school teacher, Melaka) Research suggests that large class sizes can reduce the likelihood that teachers will actively encourage student participation, because there is a tendency for them to be more concerned with controlling student behaviour (Fuller and Kapakasa 1991). Therefore, to ensure that students are given opportunities to use English in the classroom, it may be necessary to further investigate optimal class sizes. The suggestion to increase the number of English contact hours could also be explored further. A review of the Malaysian educational system by UNESCO found that the curriculum was ‘overcrowded’ and that many ‘student competencies may not be fully developed’ as a result (2013:7). It may be worth considering whether it is feasible for schools that are struggling to meet English language targets to reduce their curricular offerings in order to increase lesson time for core subjects like English until they reach the expected learning outcomes. This would require the national curriculum to be structured into a core component and an optional component in order to allow schools some flexibility to focus attention on the needs of their students. This has been found to lead to improvements in learning (Chapman et al 2005 and Wagner et al 2006). However, it is important to point out that both of these suggestions will only lead to improved learner outcomes if other factors, such as school leadership, instructional quality, the curriculum, etc. are improved at the same time.

Learner perceptions of their language ability Learners’ beliefs about their own capacity to learn, often referred to as self-efficacy, have been found to be positively associated with academic outcomes (Mills, Pajares and Herron 2006; Multon, Brown and Lent 1991). Self-efficacy centres on the belief that one is capable of learning. In order to investigate this construct, learners were asked about the perceived difficulty of their lessons and their strengths and weaknesses in English, as these can both be an indication of linguistic self-confidence (Clément, Gardner and Smythe 1977; Wesely 2012). Sixty-four percent of Year 6 students reported that their English lessons were at the right level of difficulty but 15% said they were too easy and 11% said they were too hard. This suggests that learners generally feel that they are able to cope with their lessons and succeed at the tasks they are given. When asked about their strengths and weaknesses in English, Year 6 students believed that Reading was their strongest skill followed by Listening, whereas they indicated that Speaking was their weakest skill (see Table 3.5). When teachers were also asked about their learners’ strengths and weaknesses they agreed with their students’ self-assessment (see Table 3.5). However, the test data suggests that learners may be performing slightly better in Writing than they think (performance in Writing was at level A2, with the other skills at A1). Year 6 students overwhelmingly want to improve their Speaking skills (40%).


25

Table 3.5 Learners’ strengths and weaknesses in English: Learner and teacher perceptions Skill/system

Weaknesses

Strengths Learners: Year 6

Teachers: Year 6

Learners: Year 6

Teachers: Year 6

Listening

21%

26%

7%

5%

Reading

29%

54%

10%

2%

Writing

13%

10%

23%

33%

Speaking

7%

3%

27%

42%

Vocabulary

2%

4%

10%

10%

Grammar

4%

3%

9%

8%

Not sure

24%

0%

14%

0%


An interesting finding related to learners’ perceptions of their strengths and weakness is the number who selected ‘Not sure’ when asked about their strength (24%). The frequent selection of ‘Not sure’ could indicate an issue with either their language learning awareness, their confidence as language learners or it could be linked to a sense of modesty, related to cultural values associated with a collectivist society. A combination of these factors may be the likeliest explanation because when students were asked about their worst skill in English, only 14% of them selected ‘Not sure’ which could indicate that they are more willing to state their weaknesses or are more focused on them than their strengths. However, the percentage who did select ‘Not sure’ for their weaknesses is still quite high when compared to other questions. It may be that Year 6 learners lack some awareness of their English language ability. This could be expected of primary school learners who may have limited knowledge about language.

Discussion English language education in primary school is critical not only because it is where learners are taught the basics of English (i.e. grammar and vocabulary) but also because it is during this school stage that children are going to decide whether they like English and are capable of learning it. If children have negative learning experiences, it could impact their future learning. Many secondary school and Year 6 teachers complained in the questionnaires that they felt students were not being adequately prepared in the lower grades, which they associated with the number of non-optionist teachers in primary school. As two teachers put it: ‘Obtain an English optionist teacher to teach the Pre-school level to ensure they’re taught the basics correctly’ (Urban primary school teacher, Perak) ‘Students should have a strong foundation in English in primary school before moving to secondary school’ (Rural secondary school teacher, Sabah) Questionnaire findings do also suggest that non-optionist teachers are less confident than optionist teachers in a number of areas, such as planning English lessons, using English in the classroom and assessing their students, (see Section 4 for more detail). However, teacher shortages may not make it feasible to hire only optionist teachers. Therefore, upskilling of current teachers by providing specialist training becomes even more essential. Ministry policy planners had expressed the hope that by the end of primary school, students would have achieved level B1. Although 13% of Year 6 learners are at B1 or above, the majority of students are A1 and below. The desire to achieve B1 by the end of primary is perhaps ambitious when one considers that many European countries target either A1 or A2 as the expected minimum by the end of primary school (Eurydice 2012). The conditions for learning English are somewhat more conducive in Europe because of the presence of the language in many aspects of daily culture, yet the ESLC results show that only in Sweden, Malta and Belgium do at least 80% of learners achieve B1 or above by the end of lower secondary (Jones 2013). It is clear from the questionnaire data

26


that Malaysian learners in primary school are facing challenges particularly related to their exposure to English, both within the learning environment but also in the home environment which their European counterparts are facing to a lesser extent. The variation in performance across Year 6 indicates that a phased approach may be necessary to achieve higher levels of English language ability at the end of primary school, which takes into consideration realistic targets achieved over several phases. More detailed recommendations will be discussed in Section 6.


27

Language proficiency: Secondary school Form 3 Overall, Form 3 learners in Malaysia achieved a CEFR level A2, as seen in Table 3.6. As observed with Pre-school and Year 6 learners, Speaking performance was weaker, at CEFR level A1. Table 3.6 Average CEFR proficiency levels: Form 3 Language skill CEFR level

Listening

Reading

Writing

Speaking

A2

A2

A2

A1

The range of CEFR levels achieved by the majority of Form 3 learners (Table 3.7 and Figure 3.7) spans CEFR levels B2 to below A1. Speaking presents the weakest profile, with relatively more students at the lowest A1/A2 levels and below than any of the other skills (84% at Speaking, 67% at Reading, 64% at Listening, 64% at Writing). The range in CEFR levels observed at Form 3 is a striking finding, since this range in CEFR levels represents learners who are very basic beginners to independent upper intermediate learners, and signals that the differentiation in language proficiency, which started to become apparent at the end of primary school, where the span of levels for Year 6 was B1 to below A1, has increased by one CEFR level to span levels B2 to below A1. The achievement gap is, therefore, widening in secondary school. This may be because less equitable exposure to factors which contribute to language learning (e.g. exposure to the language, the internet, books in English) is starting to make a stronger impact among those secondary school learners who have access outside school to such an enabling environment than among those learners who do not. The questionnaire data supports this supposition, and a full discussion of the attitudinal and background variables for all secondary school students will be presented at the end of this section. Table 3.7 Distribution of CEFR levels: Form 3 Language skill

Listening

Reading

Writing

Speaking

Overall

C2

0%

0%

0%

1%

0%

C1

0%

0%

1%

1%

1%

B2

21%

13%

14%

4%

13%

B1

16%

20%

21%

11%

17%

A2

23%

45%

27%

21%

29%

A1

30%

16%

10%

56%

28%

Below A1

11%

6%

27%

7%

12%



28


Figure 3.7 Distribution of Overall CEFR levels: Form 3 35% 30% 25% 20% 15% 10% 5% 0% Below A1

A1

A2

B1

B2

C1

C2

Form 5 On average, Form 5 learners in Malaysia achieved a CEFR level A2, as seen in Table 3.8. Despite achieving the same CEFR level across all skills, the mean scores the CEFR levels are based on indicate that learners are stronger in Listening and Reading compared to Writing and Speaking. Table 3.8 Average CEFR proficiency levels: Form 5 Language skill CEFR level

Listening A2

Reading A2

Writing A2

Speaking A2

Form 3 learners were also found to be on average at CEFR level A2 (Table 3.6); however, taking mean scores and distribution of CEFR levels into account, Form 5 students displayed higher proficiency as a cohort compared to their Form 3 counterparts and the learners at the lowest levels of language proficiency (A1 and below) are smaller in proportion in Form 5. The range of overall CEFR levels achieved by Form 5 students (Table 3.9 and Figure 3.8) spans below A1 to C1. Similar to the performance of other school grades reported so far, speaking presents a slightly weaker profile, with relatively more students at the lowest A1/A2 levels and below than any of the other skills (65% for Speaking, 57% for Reading, 55% for Listening; 45% for Writing). Table 3.9 Distribution of CEFR levels: Form 5 Language skill

Listening

Reading

Writing

Speaking

Overall

C2

0%

0%

0%

1%

0%

C1

0%

0%

2%

4%

2%

B2

26%

20%

9%

11%

17%

B1

18%

23%

45%

19%

26%

A2

24%

41%

27%

24%

29%

A1

25%

13%

Below A1

6%

3%

18%

31% 10%

27%

Note: The percentages observed for Writing are combined at levels A1 and below because the Writing test which was given to Form 5 cannot reliably distinguish between these two levels and they are, therefore, reported together. Percentages may not always add up to 100% due to rounding.


29

Figure 3.8 Distribution of Overall CEFR levels: Form 5 35% 30% 25% 20% 15% 10% 5% 0% A1 and Below A1

A2

B1

B2

C1

C2

Form 6 On average the majority of Form 6 learners in Malaysia achieved a level spanning A2 and B1, as seen in Table 3.10. Compared to Form 5, they performed similarly in Listening, and better in Reading, Writing and Speaking in terms of average score. Table 3.10 Average CEFR proficiency levels: Form 6 Language skill CEFR level

Listening

Reading

Writing

Speaking

A2

B1

B1

A2

Table 3.11 and Figure 3.9 show that the range of CEFR levels achieved by the majority of Form 6 students (80%) spans levels A2 and B2, with Speaking showing a weaker profile (31% were found to be at level A1 and below for Speaking, compared to 17% for Listening, 5% for Writing and 2% for Reading), which is in line with findings for the other school grades. It is worth noting that the variation in language proficiency found for Forms 3 and 5 is not as pronounced at Form 6, possibly due to the fact that Form 6 students are self-selecting and more motivated to succeed academically, including in English, and therefore present a more homogenous group. The smaller differentiation in achievement could also be as a result of the previous national policy to teach Science and Maths in English, which is no longer in effect, but is likely to have contributed to the language proficiency of Form 6 learners. Table 3.11 Distribution of CEFR levels: Form 6 Language skill

Listening

Reading

Writing

Speaking

Overall

C2

4%

1%

2%

0%

2%

C1

6%

1%

5%

5%

4%

B2

13%

16%

43%

13%

21%

B1

18%

44%

39%

27%

32%

A2

43%

35%

8%

24%

27%

A1

13%

2%

Below A1

4%

0%

5%

15% 16%

14%

Note: The percentages observed for Writing are combined at levels A1 and below because the Writing test which was given to Form 6 cannot reliably distinguish between these two levels and they are, therefore, reported together. Percentages may not always add up to 100% due to rounding.


30


Figure 3.9 Distribution of Overall CEFR levels: Form 6 35% 30% 25% 20% 15% 10% 5% 0% A1 and Below A1

A2

B1

B2

C1

C2

Attitudinal and background factors: Secondary school Questionnaires were completed by 4,740 Form 3 learners (28% of all student questionnaires), 5,495 Form 5 learners (32%), 1,913 Form 6 learners (11%) and by 738 secondary school teachers (57% of all teacher questionnaires). The grades in which teachers indicated they have the most experience teaching are as follows: Form 3 – 305, Form 5 – 330 and Form 6 – 87. The key findings which emerged as significant and meaningful are summarised below for all secondary students.

Attitudes towards learning English As discussed in the Year 6 section, learners’ attitudes towards learning a language and their reasons for learning it can shed light on their motivation and ultimately provide insight into the level of effort they are willing to expend on language learning (Csizér and Dörnyei 2005; Gardner 1985; Oxford and Ehrman 1993). The secondary school leaners recognise the importance of learning English and the majority of them strongly agreed with the statement ‘Learning English is important to me’ (Form 3: 54%; Form 5: 61%; Form 6: 64%). They also appear to be very aware of the importance of English for employment, university entrance and travel (see Figure 3.10). The secondary school learners were less likely than Year 6 students to report that it was very important for them to learn the language to please their parents (Year 6: 53%, Form 3: 36%, Form 5: 34%, Form 6: 22%) suggesting that parental pressure eases off as learners become older. However, teachers noted that parents can still play an important role in their children’s education, but that they may need support in order to do this successfully:

‘Parents should be counselled and taught easy teaching techniques to help their children at home. Most have no guidance at home and cannot focus in class’ (Urban secondary school teacher, Selangor)


31

Figure 3.10 Reasons for learning English: Secondary school students

How important is each of these reasons for you? Learning English will... …help me get into a good university. …help me get a good job. Very important …make it easier for me to travel.

Important

…make it easier for me to talk to other people.

Not very important Not at all important

….help me use the internet to get information.

Not sure

…please my parents. 0%

20%

40%

60%

80%

100%

The findings presented above suggest that secondary school learners are primarily motivated to learn English for instrumental reasons such as getting a good job, a place in a good university or for travel. However, we also find that secondary students are showing a slightly stronger tendency than primary school students to report that they are interested in learning English out of cultural interest or interest in English media. Clément, Dörnyei, Noels (1994) and Csizér and Dörnyei (2005) describe this dimension of motivation as relating to learners’ interest in the cultural products or media associated with the second language. Secondary school learners are more likely than primary school students to report that it is very important to learn English to use the internet (53%) and teachers also reported that they are interested in English music and TV: ‘Students enjoy watching English movies and listening to English songs’ (Urban secondary school teacher, Pahang) Although these learners seem to be motivated to learn, they were less likely than Year 6 learners to strongly agree that they enjoy their English lessons at school (Secondary: 31%, Year 6: 43%). This may be attributable to the increasing difficulty of their lessons as they progress from one grade to the next and/or to a more prominent focus on passing exams. Some teachers also raised a concern about the number of courses students are required to take in secondary school: ‘Students are confused as there is too much content and too many subjects for them to learn. An overambitious syllabus confuses students leading to a haywire situation’ (Urban secondary school teacher, Terengganu) When secondary school teachers were given the statement ‘My learners like learning English’, only 5% of them strongly agreed while 29% either disagreed or strongly disagreed. Secondary school teachers, like their primary counterparts, consistently commented in the questionnaire open fields and during post-observation interviews that they thought learners were unmotivated and do not perceive English as important: ‘My students lack motivation because English is not a necessity in their lives’ (Rural secondary school teacher, Perak) ‘Students are not interested in English because they haven’t realised its importance’ (Urban secondary teacher, WP Kuala Lumpur)

32


There appears to be a disconnect between what learners are reporting and their teachers’ perceptions of their motivation. One possible consequence of this is that teachers may not be taking advantage of what is actually motivating their students when they plan their lessons. Learning how to take advantage of learner motivation may be a useful feature in any future teacher upskilling programmes.

Exposure to English Learners’ exposure to English in school and outside school plays an important role in learning. When teachers were asked whether their school provides learners opportunities to practise English outside their English classes, only 28% strongly agreed. Many teachers made comments that the school could do more: ‘School authorities should inculcate a culture of speaking English in school by having more English activities’ (Rural secondary school teacher, Negeri Sembilan) ‘[We need] to provide an English-friendly environment in schools: notice boards in English, speaking in English, English speaking day, debates, public speaking, quizzes, choir, reduce ‘teacher talk’ in the classroom and encourage students to speak more’ (Urban secondary school teacher, Sabah) ‘Create an English environment in school where it is compulsory for all students and teachers to speak in English’ (Urban secondary school teacher, Kedah) Showing support for English language learning within the school can also contribute to learner motivation, and is thus an issue that should not be ignored. Turning to the English classroom, secondary school learners are more likely (between 75% and 90%) than their Year 6 counterparts (55%) to report that their teacher speaks English very often during lessons. They are also more likely to report that they speak to their teacher very often in English (between 21% and 35%) compared to Year 6 learners (12%). However, all learners whether in primary or secondary school report that they either do not speak English very often with other students during lessons (between 36% and 40%) or they never do this (between 6% and 15%). These findings support the observations made by observers that English lessons may be too teacherdominated, which is limiting learners’ opportunities to use English communicatively, since the focus is likely to be on accuracy in answering the teacher’s questions, and not on communicating with each other. Secondary school teachers, like their primary counterparts, also point out that the current amount of time devoted to English lessons is inadequate: ‘English knowledge does not happen overni ght. Students cannot grasp English fluency in 5 periods a week’ (Urban secondary school teacher, Selangor) As mentioned earlier when discussing this issue for primary school students, there may be an opportunity to reduce the curricular offerings in other subjects in order to increase the number of contact hours in English. This may be particularly important for those schools that are not achieving the desired learning outcomes. Although research shows that giving schools this type of flexibility has resulted in improved outcomes, it is only feasible if a number of conditions are met: firstly, the school administration need to have the skills necessary to make these decisions and understand the associated trade-offs (Chapman et al 2005); secondly, other aspects of the educational system need to be improved at the same time (e.g. instructional quality, the curriculum, assessment practices, etc.) (UNESCO 2013).


33

Despite the fact that secondary school students are less likely to enjoy their English lessons than Year 6 students, large numbers did strongly agree that they would like to learn other subjects in English (between 37% and 40%). Their teachers also mentioned CLIL as a possible approach to improve learning outcomes: ‘Retain English in the teaching of Maths and Science’ (Urban secondary school teacher, Melaka) Secondary school students are more likely to have had some experience with CLIL classes than their primary school counterparts so their level of agreement is an interesting finding which may require further investigation. Although secondary school students are less motivated to learn English to please their parents than primary school students, their parents’ knowledge of English can influence their own language development as well as opportunities to use the language outside school. Between 32% and 49% of secondary school students reported that their mother speaks English either very well or moderately well and this increases to between 42% and 53% for their father. Similar to the trends found with Year 6 students, secondary students do not seem to use English outside school - between 49% and 55% reported that they either never speak English with family and friends or do not do this very often, while approximately 77% either never speak English with people in their community or do not do this very often. Test data indicates that they have more capacity to speak in English than primary school students (31%, 45% and 59% of Form 3, 5 and 6 learners respectively are at CEFR level B1 and above, compared to 13% for Year 6 students). However, it may be that they either do not have opportunities or are unwilling to take advantage of these opportunities. Teacher comments seem to suggest that the former is more likely the case: ‘[Students] don’t speak English at home’ (Urban secondary school teacher from Selangor) ‘Lack of exposure, limited use of English outside the school (family background, friends and environment) do not help much in learning English’ (Urban secondary school teacher, Terengganu) Although secondary school students do not seem to use English in face-to-face interactions, they do appear to be exposed to English when engaging with technology or media. Secondary school students are more likely to watch TV or films very often in English (41%) and use the internet very often in English (65%) compared to primary school students (23% and 46% respectively). Their higher proficiency may account for these findings. They might be more willing and able to engage in these activities because they would find them easier.

Learner perceptions of their language ability Linguistic self-confidence or self-efficacy is another factor that can positively impact learner motivation (Clément, Gardner and Smythe 1977). The majority of learners in secondary school reported that their English lessons are at the right level of difficulty (between 69% and 77%) with between 6% and 10% saying that their classes are too easy and the same proportion saying that they are too hard. This indicates that, in general, learners believe they are able to cope with their lessons. Turning to their perceptions of their strengths and weaknesses, secondary students unanimously selected Reading as their strongest skill followed by Listening (Table 3.12). They selected Writing as their weakest skill followed by Speaking. Again, as seen with Year 6 students, secondary students are also focused on Writing as a weak skill. This could be linked to the content and focus of the exams they are taking, which emphasise Reading and Writing. Not unlike the Year 6 learners, a large percentage of learners selected ‘Not sure’ when asked about their strengths (between 26% and 31%). As mentioned previously, this could be linked to cultural factors of modesty. It could also be an indication that learners are not receiving sufficient teacher feedback in their lessons, and/or may suggest that they lack confidence in their abilities. However, unlike the Year 6 learners, secondary school students should have more meta-cognitive awareness about their language ability and as a result more capacity to self-assess. The large number who selected ‘Not sure’ indicates that they may need to engage in more awareness-raising activities

34


in their lessons, which will support the development of their linguistic self-confidence and make it easier for them to set appropriate language learning goals.

Table 3.12 Strengths and weaknesses in English: Learner perceptions Strengths Skill/system Learners: Learners: Learners: Learners: Form 3 Form 5 Form 6 Form 3

Weaknesses Learners: Form 5

Learners: Form 6 10%

Listening

25%

26%

15%

5%

5%

Reading

33%

36%

43%

4%

3%

1%

Writing

5%

4%

4%

35%

34%

27%

Speaking

7%

6%

7%

20%

18%

23%

Vocabulary

2%

1%

0%

10%

12%

11%

Grammar

2%

1%

1%

13%

16%

15%

Not sure

26%

26%

31%

13%

12%

13%


When teachers were asked about their learners’ strengths and weaknesses in the questionnaire,they agreed that Reading was their strongest skill and Writing their weakest. Secondary school students, like their primary counterparts, overwhelmingly selected Speaking as the skill they most want to improve (between 43% and 58%).

Discussion During the interviews, one of the Ministry officials noted that in lower secondary school it was hoped learners would have achieved B2, and by the end of Upper Form 6, C1. Clearly the data shows that learners in secondary school are still a long way off these targets with only 14% in Form 3 achieving B2 or above and 6% of Form 6 students achieving C1 or above. Again, the majority of European countries are targeting B2 as the expected exit level for secondary students (Eurydice 2012) and only in the northern European countries are these targets close to being achieved (Jones 2013). Attitudinal and background factors affecting the achievement of B2 or C1 in secondary school primarily relate to the variation in learning environments (including instructional quality which will be discussed in Section 4), exposure to English in the home and school environment and motivation. The following summary will shed light on language progression from Pre-school to Form 6 and help in identifying more realistic targets for the short, medium and long-term.


Summary: National profile So far the language proficiency of each grade has been discussed separately, with only limited comparisons with other grades. In addition to such a comparison within grades, it is also useful to explore an across-grades comparison, so that progression from one school grade to another can be clearly seen. The progression (shown also in Figure 3.11) was found to be as follows:

Pre-school

On average below CEFR level A1 78% below A1, 22% at A1/A2

Year 6

On average at CEFR level A1 32% below A1, 56% at A1/A2, 13% at B1/B2

Form 3

On average at CEFR level A2 12% below A1, 57% at A1/A2, 30% at B1/B2, 1% at C1/C2

Form 5

On average at CEFR level A2 55% at A1/A2 and below, 43% at B1/B2, 2% at C1/C2

Form 6

On average at CEFR level A2/B1 41% at A1/A2 and below, 53% at B1/B2, 6% at C1/C2

Figure 3.11 shows a progression from one grade to the next, with the proportion of learners at CEFR A levels (in blue in the chart) decreasing, as learners move from one grade to the next and achieve the higher B and C levels. As noted above, however, the proportion of learners with basic or no proficiency in English (levels A2 and below) is relatively high at the end of secondary school. Figure 3.11 Overall CEFR level: Comparison of school grades

Form 6 Below A1 Form 5

A1 A2

Form 3 B1 B2 Year 6

C1 C2

Pre-school 0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Note: Levels A1 and Below A1 are amalgamated for Forms 5 and 6, but in the Figure proportions had to be plotted equally against level A1 and below B1 for technical reasons. This simplification was felt to be warranted for the sake of visual illustration.

35


36


The CEFR levels can roughly be seen as representing three important thresholds for success in a bilingual context:

level B1 represents low intermediate learners who are independent users of the language, but in a limited range of familiar contexts; level B2 is widely recognised as the threshold for high intermediate, independent users of a language who can function in a foreign language in a range of familiar and unfamiliar contexts; level C1 is seen as the level at which advanced language proficiency is displayed. Taking these three thresholds into account, the baseline project findings indicated that at Form 5 – the key school stage when a large proportion of learners leave school – 45% have B1 level proficiency in English or above, 19% have B2 proficiency or above and 2% display advanced C1 proficiency or above. At Form 6, the proportion of learners at these levels has increased, with 59% at B1 and above, 27% at B2 and above and 6% at C1 and above (Figure 3.12). Figure 3.12 Distribution of CEFR levels B1 and above across school grades 70% 60% 59% 50% 40%

45%

30% 31%

27%

20% 19% 10% 0% 0% 0%

13%

14% 1% 0%

1%

2%

6%

0% Pre-school

Year6

B1 and above

Form3 B2 and above

Form5

Form6

C1 and above

In terms of distribution of learners at the lower CEFR levels – A1 and A2, which represent basic language proficiency – we can see that in the two top school grades (Form 5 and Form 6) the proportion is still relatively high, with 27% of Form 5 learners and 14% of Form 6 learners being at the lowest A1 level and below (Figure 3.13). The high proportion of learners who make very limited progress in English at the end of secondary school is a cause for concern, and may indicate that some aspirations in the Education Blueprint are highly ambitious.

Figure 3.13 Distribution of CEFR levels A2 and below across school grades 100% 90%

100% 94% 88%

80% 70% 60%

66%

70%

50%

56%

40% 41%

30%

41%

20%

27%

10%

14%

0% Pre-school

Year6 A2 and below

Form3

Form5

A1 and below

Form6


37

The distribution of CEFR levels according to the four skills is given in Figures 3.14–3.17. Speaking (Figure 3.17) is the weakest skill, with the smallest proportion of learners achieving levels B1 and above. The weaker development in Speaking could be partly attributed to the teacher-dominated lessons which were frequently observed, where opportunities for free practice and meaningful communication between students were missed. It could also be due to the impact of national exams, which were found to be narrowly focused on Reading and Writing at the expense of Speaking and Listening.

Figure 3.14 Listening CEFR level: Comparison between school grades


A1 A2

Form 3

B1 B2

Year 6

C1 C2

Pre-school

0%

20%

40%

60%

80%

100%

Figure 3.15 Reading CEFR level: Comparison between school grades


A1 A2

Form 3

B1 B2

Year 6

C1 C2

Pre-school 0%

20%

40%

60%

80%

100%

38


Figure 3.16 Writing CEFR level: Comparison between school grades


A1 A2

Form 3

B1 B2 C1

Year 6

C2 Pre-school 0%

20%

40%

60%

80%

100%

Figure 3.17 Speaking CEFR level: Comparison between school grades


A1 A2

Form 3 B1 B2 Year 6

C1 C2

Pre-school 0%

20%

40%

60%

80%

100%

National profile: Comparison with other countries This project focused on establishing a baseline of students’ and teachers’ language proficiency within Malaysia with a suite of benchmarking tests. It would also be useful to compare the performance of Malaysian learners on a different set of tests – the Cambridge English suite of General English tests – against a number of countries of interest. The selected countries chosen for comparison were: India, Indonesia, Myanmar, Thailand and Brazil. The first four countries on the list were chosen due to their geographic proximity with Malaysia and historical, political and economic ties. Brazil was selected as a useful comparison due to its implementation of an ambitious educational reform programme, which has allowed it to demonstrate learner achievement, and to move up in ranking within the PISA survey. The data were based on exams taken in 2012 and 2013. These comparisons have only indirect relevance for the present survey, since the exams they are based on are not necessarily aimed at the state-school sector, and the test takers are self-selecting. This is, of course, different, from the methodology and cohort in the current survey. Albeit an indirect piece of information, this was nonetheless felt useful, as it may provide guidance for future language learning targets for Malaysia.


39

If we take CEFR level B2 as the minimal threshold of being operational in a language, then we can see (Figure 3.18) that Malaysian test takers taking the Cambridge English General English exams perform better than all the countries in their geographic region: the proportion of learners at level B2 and above for Malaysia is 25%, Thailand 10%, India and Indonesia 9%, Myanmar 8%. Brazilian learners have shown stronger performance than Malaysia, with 49% achieving B2 or higher. Figure 3.18 Comparison across selected countries: Cambridge English exams

Overall CEFR level across countries (based on Cambridge English General English exams) Brazil Below A1 Malaysia

A1

Thailand

A2

Myanmar

B1 B2

Indonesia

C1

India

C2 0%

20%

40%

60%

80%

100%

40


State profiles Question 2a: How does overall and by skill student performance at different school stages compare according to state/federal territory?

State profiles: Primary school Pre-school In addition to looking at performance of Malaysian Pre-school children at national level, a range of variables identified in the Education Blueprint (2012:E7) as linked to achievement gaps (e.g. location, school type and gender) were also explored in the analysis. One of these variables relates to performance across the 16 states/federal territories. This was identified as an important variable in the Education Blueprint, which noted that states with more rural and remote schools (e.g. Sabah and Sarawak) showed a weaker performance than states with fewer rural schools. Test performance by state at Pre-school level showed little variation, as can be seen in Table 3.13, with most states performing below CEFR level A1 and a few achieving level A1 in Listening, Reading and Writing (Speaking is not included in the state-level comparison due to the small number of students who were assessed for Speaking in each state). The states which showed a stronger than average performance in terms of CEFR level and/or average score in all three skills were found to be Johor and WP Kuala Lumpur. The stronger findings for these two states are most likely due to higher proportion of urban schools (e.g. Kuala Lumpur) and to the successful implementation of a state-wide school improvement programme (e.g. Johor, as noted in the Education Blueprint 2012:E7). Other states which performed higher than the national average in two skills were Pahang, Perlis, Terengganu and WP Putrajaya. An analysis of top-performing classes (operationalised as classes whose average scores were 1.5 standard deviations above the national mean) indicated that the top-performing Pre-school classes by state came from Johor (National and Chinese schools), Pahang (National and Tamil schools) and Perak (National and Tamil schools). It is important to note that in all states, including the ones performing above the mean for Listening and Reading, Writing emerged as a weaker skill, which would be expected due to children’s delayed development in literacy in both first and additional languages.


41

Table 3.13 Overall CEFR proficiency levels by State/Federal Territory: Pre-school Malaysia overall

Listening

Reading

Writing

Below A1

Below A1

Below A1

JOHOR

A1

A1

Below A1

KEDAH

Below A1

Below A1

Below A1

KELANTAN

Below A1

Below A1

Below A1

MELAKA

Below A1

Below A1

Below A1

NEGERI SEMBILAN

Below A1

Below A1

Below A1

PAHANG

Below A1

A1

Below A1

PERAK

Below A1

Below A1

Below A1

PERLIS

A1

A1

Below A1

PULAU PENANG

Below A1

Below A1

Below A1

SABAH

Below A1

Below A1

Below A1

SARAWAK

Below A1

Below A1

Below A1

SELANGOR

Below A1

Below A1

Below A1

TERENGGANU

Below A1

A1

Below A1

A1

A1

Below A1

Below A1

Below A1

Below A1

A1

A1

-

WP KUALA LUMPUR WP LABUAN WP PUTRAJAYA

Note: The missing data for Writing is due to late scripts, which were not included in the analysis; Speaking is not reported by state due to the small size of the Speaking sample in a number of states.

Year 6 Test performance by state at Year 6 showed some variation, as can be seen in Table 3.14. The states which showed stronger than average performance in all three skills were found to be: WP Kuala Lumpur (which also performed above the mean at Pre-school) and Pahang. Negeri Sembilan and Selangor performed above the national average in two of the skills. WP Kuala Lumpur, Pahang and Selangor also had some of the top -performing Year 6 classes in terms of Reading and Listening. Perlis presents an interesting case in Writing – at Pre-school and Year 6 it achieved an average score above the national mean (in fact, the highest score across all states for Year 6). At secondary school, as will be seen later in the report, it was found to be one of the lowest achieving states. Such a ‘jagged’ profile at primary and secondary level in this state is most probably due to the small number of students who took the test (N=178 at primary school, N=120 at secondary school). Sabah and Sarawak showed performance below the national average CEFR level in all three skills, most likely due to the large proportion of rural and remote schools in those two federal territories, which have been shown to perform less well than urban schools (Education Blueprint 2012:E9).

42


Table 3.14 Overall CEFR proficiency levels by State/Federal Territory: Year 6 Listening

Reading

Writing

Malaysia overall

A1

A1

A2

JOHOR

A1

A1

A2

KEDAH

A1

A1

A2

KELANTAN

A1

A1

-

MELAKA

A1

A1

A2

NEGERI SEMBILAN

A1

A1

A2

PAHANG

A1

A1

A2

PERAK

A1

A1

A2

PERLIS

A1

A1

A2

PULAU PENANG

A1

A1

A2

SABAH

A1

A1

A2

SARAWAK

A1

A1

A2

SELANGOR

A1

A1

A2

TERENGGANU

A1

A1

A2

WP KUALA LUMPUR

A2

A2

A2

WP LABUAN

A1

Below A1

-

WP PUTRAJAYA

A1

Below A1

-

Note: The missing data for Writing is due to late scripts, which were not included in the analysis; Speaking is not reported by state due to the small size of the Speaking sample in many states.

Attitudinal and background factors by state: Primary school The questionnaire responses indicated that there were variations by state to many of the questions. The largest differences by state for Year 6 learners were found in the area of learners’ exposure to English, both inside and outside the classroom. Outside the classroom, only 16% of learners from WP Kuala Lumpur reported that they never speak English with friends or family while 38% of learners from Sarawak reported the same. Similarly, when asked if they use English to talk to people in their neighbourhood or town, 37% of learners from Kuala Lumpur said that they never do this, while 64% of learners from Sarawak said the same. The learners from Sarawak, Sabah and Perak are more likely to report that their mother or father cannot speak English (between 16% and 22%) than learners from WP Kuala Lumpur and WP Putrajaya (between 4% and 10%). This is an important finding which could partly explain the difference in achievement across states, since having and making use of opportunities to practice a second/foreign language has a beneficial effect on language learning. Within the English classroom, there were also differences between states in the amount of English being used. Between 33% and 34% of Year 6 students from Negeri Sembilan, Sabah and Sarawak reported that they never speak English to other students during English lessons, whereas only 13% to 18% of learners from WP Kuala Lumpur, Pahang and Selangor said the same. This finding also relates to students’ perception of the difficulty of their English lessons. Learners from Sabah and Sarawak were more likely to find their English lessons too hard (between 16% and 18%) than learners from states that have fewer remote and rural schools such as WP Kuala Lumpur, Selangor and Johor, who tend to indicate more frequently that their lessons are too easy (between 19% and 25%).


43

State profiles: Secondary school Form 3 Test performance by state at Form 3 showed variation (Table 3.15) and Selangor, WP Kuala Lumpur, WP Labuan, and WP Putrajaya showed stronger than average performance, with learners from those states/federal territories achieving CEFR B1 level in Listening and Reading or a high average score within A2. Selangor also had four out of the eight top-performing Form 3 classes. This finding is most probably due to the high proportion of urban schools in those territories, which tend to perform better. Perlis showed performance below the national average in all three skills, with Listening being particularly weak. Table 3.15 Average CEFR proficiency levels by State/Federal Territory: Form 3 Listening

Reading

Writing

Malaysia overall

A2

A2

A2

JOHOR

A2

A2

A2

KEDAH

A1

A2

A1

KELANTAN

A1

A2

A2

MELAKA

A2

A2

A2

NEGERI SEMBILAN

A1

A2

A1

PAHANG

A1

A2

A2 B1

PERAK

A2

A2

PERLIS

Below A1

A1

A1

PULAU PENANG

A1

A2

A2

SABAH

A2

A2

A2

SARAWAK SELANGOR

A2 A2

A2 A2

A2 A2

TERENGGANU

A1

A2

A1

WP KUALA LUMPUR

B1

B1

A2

WP LABUAN

A2

B1

-

WP PUTRAJAYA

B1

B1

-

Note: The missing data for Writing is due to late scripts, which were not included in the analysis; Speaking is not reported due to the small Speaking sample in many states.

Form 5 As Table 3.16 indicates, test performance by state at Form 5 showed some variation. Similar to the findings observed for Year 6 and Form 3, WP Kuala Lumpur showed stronger than average performance compared to the national average of A2. In this case it was in all three skills, with Form 5 WP Kuala Lumpur learners achieving CEFR level B1. The analysis of top-performing Form 5 classes also indicated that four (out of 10) of the top performers were in WP Kuala Lumpur. The stronger performance in Writing of Negeri Sembilan and Pulau Pinang is also worth noting, as is the fact that two of the top-performing Form 5 class were in Pulau Pinang. Perak also showed B1 level performance in Writing; however, this is based on a mean score just over the B1 level boundary. In terms of weaker performance, Perlis Form 5 students achieved a lower CEFR level and/or average score than the national average in all skills. The relatively weaker performance for Perlis was also observed at Form 3, and could be due to the relatively small proportion of students from this state in the sample.

Table 3.16 Overall CEFR proficiency levels by State/Federal Territory: Form 5 Malaysia overall

Listening

Reading

Writing

A2

A2

A2

44


JOHOR

A2

A2

A2

KEDAH

A2

A2

A2

KELANTAN

A2

A2

A2

MELAKA

A2

A2

A2

NEGERI SEMBILAN

A2

A2

B1

PAHANG

A2

A2

A2

PERAK

A2

A2

B1

PERLIS

A1

A2

A2

PULAU PENANG SABAH

A2 A2

A2 A2

B1 A2

SARAWAK

A2

A2

A2

SELANGOR

A2

A2

A2

TERENGGANU

A2

A2

A2

WP KUALA LUMPUR

B1

B1

B1

WP LABUAN

A2

A2

A2

WP PUTRAJAYA

A2

A2

A2

Note: The missing data for Writing is due to late scripts, which were not included in the analysis; Speaking is not reported due to the small speaking sample in many states.

Form 6 As was found in all other school grades, test performance by state showed some variation at Form 6 (Table 3.17) and several states showed performance higher than average in at least two out of the three skills: Melaka, Pulau Penang, Selangor, WP Kuala Lumpur and WP Labuan. Top performing Form 6 classes were found in WP Kuala Lumpur, Selangor and Sabah. In terms of mean scores, Perlis showed the weakest performance, which was also observed with Form 3 and Form 5.


45

Table 3.17 Overall CEFR proficiency levels by State/Federal Territory: Form 6 Listening

Reading

Writing

Malaysia overall

A2

B1

B1

JOHOR

A2

B1

B1

KEDAH

A2

B1

B2

KELANTAN

A2

B1

B1

MELAKA

B1

B1

B2

NEGERI SEMBILAN

A2

B1

B2

PAHANG

A2

B1

B1

PERAK

A2

B1

B1

PERLIS

A2

B1

A2

PULAU PENANG

B1

B1

B2

SABAH

A2

B1

B1

SARAWAK

A2

B1

B2

SELANGOR

B1

B1

B1

TERENGGANU

A2

B1

B1

WP KUALA LUMPUR

B1

B1

B1

WP LABUAN

B1

B2

B1

WP PUTRAJAYA

A2

B1

B1

Note: Speaking is not reported due to the smaller speaking sample in a number of states.

Attitudinal and background factors by state: Secondary school Although there is variation in learner responses by state, the majority of differences are not meaningful. Where the variation in responses is meaningful, the pattern is very similar to those found in Year 6, namely exposure to English within and outside the learning environment. As the findings by state are strongly linked to the number of rural/remote schools within a particular state, a discussion of secondary school student differences is provided by school location on the following pages.

46


Urban/rural/remote profiles Question 2b: How does overall and by skill student performance at different school stages compare according to rural/urban/remote location?

Location profiles: Primary school Pre-school The analysis also focused on the distribution of CEFR levels by location, i.e. urban, rural and remote. Table 3.18 presents the relevant findings and shows that in general there was no difference in CEFR level performance across the different locations, with the exception of Reading, where urban Pre-school learners performed slightly better, with a mean score just above the A1 level boundary, than their rural and remote counterparts. A multilevel regression analysis indicated that there was no statistically significant difference in performance base d on location at Pre-school. (Descriptive statistics and multilevel regression analysis findings for this variable and all subsequent variables discussed in the report can be found in the Cambridge Baseline 2013 Technical Report.) Table 3.18 Average CEFR proficiency levels by location: Pre-school Listening

CEFR Level

Urban Below A1

Rural Below A1

Reading Remote Below A1

Urban A1

Rural Below A1

Writing

Speaking

Remote Urban Rural Remote Urban Rural Remote Below Below Below Below Below Below Below A1 A1 A1 A1 A1 A1 A1

Table 3.19 and Figure 3.19 present the distribution of CEFR levels of students in the three location types across all skills and overall. In the case of Writing and Speaking, Pre-school students in all three locations performed in a very similar fashion, with the vast majority below level A1. With Listening and Reading, urban and rural students achieved a more even spread across A2, A1 and below A1, whereas students in remote schools were found to be mostly at level A1 and below. This finding supports previous school achievement results reported for Malaysia in the Education Blueprint (2012:E9). Interestingly, in terms of top performing classes, the analysis indicated that out of seven classes, six came from Pre-school classes in rural schools. Table 3.19 Distribution of CEFR levels by location: Pre-school Listening

Reading

Writing

Speaking

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

A2

21%

33%

7%

11%

12%

10%

4%

4%

0%

0%

0%

0%

A1

29%

36%

20%

35%

31%

18%

8%

7%

0%

6%

0%

0%

Below A1

51%

32%

73%

53%

57%

72%

88%

89%

100%

94%

100%

100%

Overall Urban

Rural

A2

9%

12%

Remote 4%

A1

20%

18%

10%

Below A1

72%

70%

86%



47


Figure 3.19 Distribution of overall CEFR levels by location: Pre-school 100% 90% 80% 70% 60%

Below A1

50%

A1

40% A2

30% 20% 10% 0% Urban

Rural

Remote

Year 6 Table 3.20 presents the relevant findings by urban/rural/remote location of schools and shows that consistently Year 6 classes in remote and rural schools performed weaker than urban classes. A multilevel regression analysis indicated that there was indeed a statistically significant difference between Year 6 students in urban areas, as compared with rural and remote areas. Performance was the lowest in remote areas and the difference is statistically significant. The difference between rural and urban Year 6 classes is also statistically significant for Listening, Reading and Writing, but the effect is smaller than the one found for remote schools. This finding supports previous school achievement results reported for Malaysia in the Education Blueprint (2012:E9), where an achievement gap was noted between urban and rural/remote schools, in favour of urban schools.

Table 3.20 Average CEFR proficiency levels by location: Year 6 Listening Urban CEFR Level

A1

Reading

Rural

Remote

A1

Below A1

Urban A1

Writing

Rural

Remote

A1

Below A1

Urban A2

Speaking

Rural

Remote

Urban

Rural

Remote

A2

Below A1

A1

A1

Below A1

Note: The number of students from remote locations for Writing and Speaking is very small (N=10 Writing, N=16 Speaking).

Table 3.21 and Figure 3.20 present the distribution of CEFR levels of students in the three location types. The urban schools showed the largest proportion of Year 6 students at the higher A2 and B1 levels, and remote schools showed the lowest (39% overall for urban, 30% for rural, 3% for remote). The vast majority of students (98%) in remote locations achieved either level A1 or below. However, the much smaller number of students in remote locations compared to rural and urban locations must be kept in mind, since the performance of a few individual students in remote schools could have a large effect on the percent reported.


48


Table 3.21 Distribution of CEFR levels by location: Year 6 Listening

Reading

Writing

Speaking

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

C1

0%

0%

0%

0%

0%

0%

0%

0%

0%

1%

0%

0%

B2

0%

0%

0%

0%

0%

0%

3%

1%

0%

0%

0%

0%

B1

16%

5%

2%

0%

0%

0%

38%

26%

0%

5%

6%

0%

A2

24%

18%

6%

21%

8%

2%

33%

34%

0%

19%

23%

0%

A1

30%

28%

19%

34%

27%

11%

18%

21%

0%

72%

59%

0%

30%

49%

72%

45%

65%

86%

8%

18%

100%

3%

12%

100%

Below A1

Overall Urban

Rural

Remote

C1

0%

0%

0%

B2

1%

1%

0%

B1

15%

9%

1%

A2

24%

21%

2%

A1

38%

34%

8%

Below A1

22%

36%

90%


Figure 3.20 Distribution of CEFR levels by location: Year 6 100% 90% 80% 70%

Below A1

60%

A1

50%

A2

40%

B1

30%

B2

20% 10% 0% Urban

Rural

Remote

Attitudinal and background factors by location: Primary school The questionnaire findings suggest that learners from remote schools are facing more challenges to learning English than their counterparts in urban and rural schools. Forty-seven percent of primary school students from remote schools reported that their first language was something other than Malay, Chinese or Tamil. Only 5% of urban students and 6% of rural students report having a first language other than Malay, Chinese or Tamil. This indicates that learners from remote schools are likely learning English as their third or fourth language, which could be one reason why they are performing less well than students in urban and rural areas. One teacher highlighted this issue:


49


‘Many students learn 3 languages and are usually influenced by their native tongue’ (Rural primary school teacher, Sarawak) Learners who study in remote schools are also more likely to report that their English lessons are too hard (20%) than students from rural schools (13%) and urban schools (9%). These findings may again be linked to the lack of exposure to English in their daily lives. Learners from remote schools were more likely to indicate that neither their mother nor father can speak English (38% and 34% respectively) compared to rural school students (14% for both parents) and urban school students (9% for both parents). Learners from remote areas are also less likely to use English in the classroom. Twenty-five percent reported that they never speak to their teacher in English, while 17% of students from rural schools and 10% o f students from urban schools said the same. Not surprisingly, there also appear to be fewer opportunities for students from remote areas to use English in the wider community. Thirty-six percent of students never speak to family or friends in English and 63% never speak English in their communities (rural students: 28% and 54% respectively; urban students: 20% and 44% respectively).

Location profiles: Secondary school Form 3 Table 3.22 presents the relevant findings by urban/rural/remote location of schools and shows that Form 3 classes in rural and remote schools generally performed less well than urban classes in all skills, achieving a lower CEFR level and/or a lower average. A multilevel regression analysis indicated that there was indeed a statistically significant difference between Form 3 students in urban areas, as compared with rural and remote areas. The effect was significant for all three skills and was most pronounced for Writing. This finding is in line with the gap between urban and rural/remote students found at Year 6 and with the findings noted earlier from the Education Blueprint (2012:E9), where an achievement gap was mentioned between urban and rural schools, in favour of urban schools. Table 3.22 Average CEFR proficiency levels by location: Form 3 Listening Location CEFR Level

Urban

A2

Rural

A1

Reading Remote

A1

Urban

A2

Rural

A2

Writing Remote

A2

Urban

A2

Rural

A2

Speaking Remote

A1

A2

Urban

A1

Rural

Remote

A1

Table 3.23 and Figure 3.21 present the distribution of CEFR levels of Form 3 students in the three location types. As was found with Year 6, the urban schools showed the largest proportion of Form 3 students at the higher B1 and B2 levels, and remote schools showed the lowest (37% urban, 20% rural, 13% remote), with the majority of students in remote locations (86%) achieving either level A2, A1 or below.


50


Table 3.23 Distribution of CEFR levels by location: Form 3 Listening Location

Reading

Writing

Speaking

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

C2

0%

0%

0%

0%

0%

0%

0%

0%

0%

2%

0%

0%

C1

0%

0%

0%

0%

0%

0%

2%

0%

0%

2%

0%

0%

B2

29%

10%

6%

19%

5%

0%

18%

7%

0%

6%

2%

0%

B1

18%

13%

17%

24%

15%

31%

24%

16%

0%

10%

14%

0%

A2

23%

21%

33%

40%

51%

42%

23%

34%

21%

19%

18%

41%

A1

21%

43%

25%

11%

22%

28%

6%

16%

14%

57%

56%

53%

Below A1

8%

14%

19%

5%

7%

0%

26%

27%

64%

5%

9%

6%

Location

Urban

Rural

Remote

C2

1%

0%

0%

C1

1%

0%

0%

B2

18%

6%

1%

B1

19%

14%

12%

A2

26%

31%

34%

A1

23%

34%

30%

Below A1

11%

14%

22%

Overall


Figure 3.21 Distribution of overall CEFR levels by location: Form 3 40% 35% Below A1

30%

A1

25%

A2 20%

B1

15%

B2

10%

C1 C2

5% 0% Urban

Rural

Remote

Form 5 Table 3.24 presents the relevant findings by urban/rural location of school. (‘Remote’ is not included as a category due to the small number of students in remote Form 5 classes – only 82 students from remote locations in Sabah took the Form 5 test). The table shows a difference in CEFR level and/or average score in all skills, with learners in rural schools consistently performing less well. The multilevel regression analysis indicated a statistically significant difference between urban and rural students. The effect was found to be significant for Listening, Reading and Writing; there was no conclusive evidence regarding Speaking, possibly due to the small sample size


51

for Speaking. This finding is in line with the gap between urban and rural/remote students found at Year 6 and Form 3. Table 3.24 Average CEFR proficiency levels by location: Form 5 Listening

Reading

Writing

Speaking

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

A2

A2

A2

A2

B1

A2

A2

A2

CEFR Level

Note: ‘Remote’ is not included in the analysis d ue to the small number of Form 5 students from remote schools in the sample (N=24 for Listening, Reading, Speaking, N=10 for Writing, N=82 total).

Table 3.25 and Figure 3.22 present the distribution of CEFR levels of Form 5 students in the different location types. As was found with Year 6 and Form 3, the urban schools showed the largest proportion of Form 5 students at the higher B1 and above levels (53% urban, 34% rural). Similar to Form 3, where approximately half of the learners were either at the A1 and below or B2 ends of the proficiency spectrum, a wide variation in achievement is also observed for a large proportion of Form 5 urban students (20% at level A1 and below, 24% at level B2 and above). Table 3.25 Distribution of CEFR levels by location: Form 5 Listening

Reading

Writing

Speaking

Overall

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

C2

0%

0%

0%

0%

0%

0%

1%

0%

0%

0%

C1

0%

0%

0%

0%

2%

1%

7%

1%

2%

1%

B2

34%

16%

26%

13%

11%

6%

16%

7%

22%

10%

B1

19%

16%

24%

21%

48%

40%

24%

14%

29%

23%

A2

22%

27%

38%

45%

25%

29%

23%

25%

27%

32%

A1

20%

32%

9%

18%

31%

31%

Below A1

4%

9%

3%

4%

0%

21%

20%

35%

14%

24%

Note: ‘Remote’ is not included as a separate category in t he analysis due to the small number of Form 5 students from remote schools in the sample (N=24 for Listening, Reading, Speaking, N=10 for Writing , N=82 total). Percentages may not always add up to 100% due to rounding.

Figure 3.22 Distribution of overall CEFR levels by location: Form 5 40% 35% 30% Below A2

25%

A2 20% B1 15%

B2

10%

C1

5% 0% Urban

Rural


52


Form 6 Table 3.26 presents the relevant findings by urban/rural location of school. (‘Remote’ is not included as a category due to the small number of remote Form 6 students (N=84)). The Table shows a difference in CEFR level and/or mean score in all skills. In line with the significance testing results noted for Year 6, Forms 3 and 5, the multilevel regression analysis indicated a statistically significant difference between urban and rural students at Form 6. The effect was found to be strongest for Speaking. Table 3.26 Overall CEFR proficiency levels by location: Form 6 Listening

CEFR Level

Urban B1

Reading

Rural A2

Urban B1

Writing

Rural B1

Urban B2

Speaking

Rural B1

Urban B1

Rural A2

Note: ‘Remote’ is not included in the analysis due to the small number of Form 6 students from remote schools in the sample (N=23 for Listening, Reading, Speaking, N=15 for Writing, N=84 total).

Table 3.27 and Figure 3.23 present the distribution of CEFR levels of Form 6 students in the different location types and indicate that approximately one third (36%) of urban students were found to be at CEFR levels B2 and above overall, and therefore having operational proficiency of English. The corresponding proportion for rural students was 19%. Table 3.27 Distribution of CEFR levels by location: Form 6 Listening

Reading

Writing

Speaking

Overall

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

C2

6%

1%

1%

0%

1%

2%

1%

0%

2%

1%

C1 B2

8% 17%

2% 9%

2% 21%

0% 11%

6% 53%

2% 29%

10% 18%

3% 13%

7% 27%

2% 16%

B1

21%

14%

45%

44%

32%

49%

39%

19%

34%

32%

A2

38%

50%

29%

41%

5%

12%

26%

18%

24%

30%

A1

8%

19%

1%

3%

6%

16%

Below A1

2%

5%

0%

0%

0%

31%

5%

20%

3%

6%

Note: ‘Remote’ is not included in the analysis due to the small number of Form 6 students from remote schools in the sample (N=23 for Listening, Reading, Speaking, N=15 for Writing, N=84 total). Percentages may not always add up to 100% due to rounding up and down.

Figure 3.23 Distribution of overall CEFR levels by location: Form 6 40% 35% 30%

Below A2

25%

A2

20%

B1 B2

15%

C1 10%

C2

5% 0% Urban

Rural


53

Attitudinal and background factors by location: Secondary school Due to the low numbers of secondary school learners from remote schools who completed the questionnaire (N=79), they have been grouped together with rural school students for this analysis. Learners from rural schools were found to be less positive about learning English and either have fewer opportunities to use English inside or outside school or are less likely to use English when given the opportunity. Unlike the primary school findings, there is more language variation in urban secondary schools. Between 84% and 89% of rural/remote students indicated that Malay is their first language whereas between 56% and 67% of urban students said the same. Secondary school learners who study in rural schools are slightly less likely to strongly agree with the statement ‘Learning English is important for me’ (between 50% and 57%) than their counterparts in urban schools (between 57% and 69%). Although both groups recognise the functional purpose of English, urban school students are more likely to take English lessons outside school (between 12% and 31%) than students from rural/remote schools (between 7% and 19%). Form 6 learners from rural/remote schools, when asked if they enjoy their English lessons, are less likely to strongly agree (22%) than their urban counterparts (32%). Secondary school students from urban schools are more likely to strongly agree that they would like to learn other subjects in English (between 46% and 48%) compared to students from rural schools (between 24% and 29%). These findings suggest that rural secondary school students may be less engaged with learning English and although they recognise its importance are facing challenges to learning it. When looking at the questions that focus on the use of English within the learning environment and home environment, the main difference between urban and rural secondary students had to do with the extent to which their parents speak English. We found that secondary students from rural areas are less likely to report that their parents speak English very well or moderately well (between 24% and 39% for their mothers and between 35% and 43% for their fathers) whereas urban secondary students report that between 38% and 56% of their mothers and between 55% and 61% of their fathers speak it very well or moderately well. As mentioned previously, parents’ knowledge of a language can positively influence their children’s language learning behaviour and their attitude towards learning a language. Students studying in rural locations appear to be facing more disadvantages in terms of exposure to English and parental support for learning than their urban counterparts, which could be negatively affecting their attitude and motivation to learn. Teachers’ comments supported the viewpoints mentioned above: ‘My students come from nearby rural villages. They are not exposed to English and hence dislike it’ (Urban secondary school teacher, Pahang) ‘Students are mainly from rural villages. They come from different primary schools and most speak their own local dialect. They don’t speak English at all’ (Rural secondary school teacher, Selangor) ‘My students are all from rural areas. Family background influences their disinterest’ (Rural secondary school teacher, Pulau Pinang) In order to reduce the achievement gap between urban and rural students, it may be necessary to consider ways of addressing these differences. In particular, those differences associated with opportunities to use English both within the English classroom and the school, parental engagement in English language learning and student motivation.

54


School type profiles Question 2c: How does overall and by skill student performance at different school stages compare according to school type?

School type profiles: Primary school Pre-school Another variable the analysis focused on was the performance of Pre-school children in different school types: National, Chinese and Tamil schools. Table 3.28 suggests that there was some variation, and a multilevel regression analysis indicated a statistically significant difference in the case of Tamil schools, with students in Tamil schools performing slightly better on the Listening, Reading and Speaking tests than students in National schools. It is also worth noting that an analysis of top performing classes at Pre-school indicated that classes from all three school types were represented in the top group. This finding is in contrast with results reported in the Education Blueprint (2012:E9), which noted an achievement gap between National and National-type Chinese and Tamil primary schools in the Malaysia primary school system. At Pre-school level at least, we can see relative equity in student outcomes in English language proficiency. Table 3.28 Overall CEFR proficiency levels by school type: Pre-school Listening

CEFR level

N

C

Below A1

Below A1

Reading T

N

A1

Below A1

C A1

Writing

Speaking

T

N

C

T

N

A1

Below A1

Below A1

Below A1

Below A1

C

T

-

Below A1

Note: N=National school, C=Chinese school, T=Tamil school. No National Chinese Pre-school classes were represented in the Speaking sample.

Table 3.29 and Figure 3.24 present the distribution of CEFR levels of Pre-school students in the three school types. In the case of Writing and Speaking, learners in the three school types performed in a very similar fashion, with the vast majority below level A1. The highest proportion of children in levels A1 and A2 came from Tamil schools (41% from Tamil schools, 26% from Chinese schools, 21% from National schools). Table 3.29 Distribution of CEFR levels by school type: Pre -school Listening

Reading

Writing

Speaking

N

C

T

N

C

T

N

C

T

N

C

T

A2

6%

10%

33%

10%

11%

25%

3%

4%

7%

0%

-

0%

A1

23%

12%

33%

31%

35%

45%

7%

8%

17%

2%

-

5%

Below A1

70%

78%

33%

59%

55%

30%

90%

88%

76%

98%

-

95%

Overall

A2

N 5%

C 8%

T 16%

A1

16%

18%

25%

Below A1

80%

74%

59%



55


Figure 3.24 Distribution of overall CEFR levels by school type: Pre-school 90% 80% 70% 60% 50%

Below A1

40%

A1

30%

A2

20% 10% 0% N

C

T

Year 6 The findings for Year 6 students based on school type can be seen in Table 3.30. They indicate some variation in performance, e.g. Chinese schools performed better in Speaking, but that difference was not found to be statistically significant. Table 3.30 Average CEFR proficiency levels by school type: Year 6 Listening

CEFR level

Reading

Writing

Speaking

N

C

T

N

C

T

N

C

T

N

C

T

A1

A1

A1

A1

A1

A1

A2

A2

A2

A1

A2

A1

Table 3.31 and Figure 3.25 present the distribution against CEFR levels of students in the three school types. There is some variation in the proportion of learners at different CEFR levels, such as, the fact that the smallest overall proportion of students below A1 came from Tamil schools (33% National, 31% Chinese, 20% Tamil) and the largest proportion at B1 and B2 was found in Chinese schools (12% National, 26% Chinese, 8% Tamil); however, as noted above, the multilevel regression analysis indicated that there was no statistically significant difference in performance by school type. Table 3.31 Distribution of CEFR levels by school type: Year 6 Listening

C1

Reading

Writing

Speaking

N

C

T

N

C

T

N

C

T

N

C

T

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

3% 0%

B2

0%

0%

0%

0%

0%

0%

1%

8%

1%

0%

0%

B1

10%

11%

6%

0%

0%

0%

31%

36%

27%

5%

50%

0%

A2

19%

20%

33%

14%

15%

11%

35%

21%

36%

18%

0%

30%

A1

28%

26%

37%

28%

27%

46%

20%

15%

25%

58%

50%

65%

Below A1

43%

43%

24%

58%

59%

43%

13%

20%

11%

19%

0%

3%

Overall


56


C1

N

C

T

0%

0%

1% 0%

B2

0%

2%

B1

12%

24%

8%

A2

21%

14%

27%

A1

33%

29%

43%

Below 33% 31% 20% A1 Note: Percentages may not always add up to 100% due to rounding.

Figure 3.25 Distribution of CEFR levels by school type: Year 6 50% 45% 40% 35%

Below A1

30%

A1

25%

A2

20%

B1

15%

B2

10%

C1

5% 0% N

C

T

Attitudinal and background factors by school type: Primary school Although there were no statistically significant differences in performance by school type, the questionnaire responses do indicate differences in attitude to English language learning. Year 6 learners from Tamil schools have, in general, a much more positive view of English language learning than students from National schools, who are more positive than students from Chinese schools. Learners from Tamil schools are also more likely to have a mother and/or father who speak English very well than students from National schools and students from Chinese schools (see Figure 3.26).


57

Figure 3.26 Parents’ ability to speak English by school type (percentage for ‘very well’)

'My mother/father speaks English very well' 40% 35% 30% 25% 20%

Mother

15%

Father

10% 5% 0% Tamil school

National school

Chinese school

Tamil school students are also more likely to report that they speak English very often with family and friends (24%) and with people in their communities (17%) than National school students (5% and 3% respectively) and Chinese school students (5% and 5% respectively). However, this is likely linked to the previous findings that their parents are more likely able to speak English. Learners from Tamil schools are more likely to strongly agree that they enjoy their English lessons (68%) when compared to students from National schools (44%) and students from Chinese schools (23%). These students are also more likely to report that their English lessons are too easy (34%) compared to students from National and Chinese schools (13%).

School type profiles: Secondary school Form 3 The findings for Form 3 students based on school type – National or Religious – can be seen in Table 3.32. They indicate some variation in performance, e.g. Religious secondary schools performed better in Writing and Speaking, but that difference was not found to be statistically significant since the difference in CEFR levels for Writing and Speaking is based on minimal variation in mean scores. Table 3.32 Average CEFR proficiency levels by school type: Form 3 Listening School type

Reading

Writing

Speaking

N

R

N

R

N

R

N

R

A2

A2

A2

A2

A2

B1

A1

B1

CEFR level Note: N=National school type, R=Religious school type

Table 3.33 and Figure 3.26 present the distribution of CEFR levels of students in the two school types. There is some variation in the proportion of learners at different CEFR levels (e.g. Religious schools had higher proportions of students at the higher B1 and above levels: 30% National, 50% Religious); however, as noted above, this variation is relatively small and not statistically significant.


58


Table 3.33 Distribution of CEFR levels by school type: Form 3 Listening C2

Reading

Writing

Speaking

N

R

N

R

N

R

N

R

0%

0%

0%

0%

0%

0%

1%

0% 0%

C1

0%

0%

0%

0%

1%

3%

1%

B2

21%

25%

13%

12%

13%

20%

3%

7%

B1

16%

21%

20%

30%

20%

33%

7%

50%

A2

22%

27%

45%

48%

27%

32%

19%

40%

A1

30%

23%

16%

9%

10%

9%

61%

3%

Below A1

11%

4%

6%

1%

28%

4%

8%

0%

Overall C2

N

R

0%

0%

C1

1%

1%

B2

13%

16%

B1

16%

33%

A2

28%

37%

A1

29%

11%

Below A1

13%

2%


Figure 3.27 Distribution of overall CEFR levels by school type: Form 3 40% 35% 30%

Below A1

25%

A1

20%

A2 B1

15%

B2 10%

C1

5% 0% N

R


59

Form 5 The findings for Form 5 students based on school type – National or Religious - can be seen in Table 3.34. Similar to the findings for Form 3, Religious secondary schools performed better, in this case in Reading and Writing, but that difference was not found to be statistically significant since the difference in CEFR levels is based on minimum variation in mean scores. Table 3.34 Average CEFR proficiency levels by school type: Form 5 Listening

Reading

Writing

Speaking

School type

N

R

N

R

N

R

N

R

CEFR level

A2

A2

A2

B1

A2

B1

A2

-

Note: No speaking tests were carried out in Form 5 religious classes.

Table 3.35 and Figure 3.28 present the distribution of CEFR levels of students in the two school types. There is some variation in proportion of learners at different CEFR levels; however, as noted above, this variation is not statistically significant.

Table 3.35 Distribution of CEFR levels by school type: Form 5 Listening

Reading

Writing

Speaking

N

R

N

R

N

R

N

R

C2

0%

0%

0%

0%

0%

0%

1%

-

C1

0%

0%

0%

0%

2%

0%

4%

-

B2

26%

26%

20%

29%

9%

7%

11%

-

B1

18%

20%

22%

29%

44%

63%

19%

-

A2

24%

21%

41%

32%

27%

25%

24%

-

A1

25%

20%

13%

9%

Below A1

6%

13%

3%

1%

19%

4%

31%

-

10%

-

Overall C2

N

R

0%

0%

C1

1%

0%

B2

18%

21%

B1

28%

37%

A2

31%

26%

22%

16%

A1 Below A1

Note: Overall estimate is based on Listening, Reading and Writing only. Percentages may not always add up to 100% due to rounding.


60


Figure 3.28 Distribution of overall CEFR levels by school type: Form 5 40% 35% 30% Below A2

25%

A2 20%

B1

15% B2 C1

10% 5% 0% N

R

Form 6 School type is not included in the analysis due to the small number of Religious schools at Form 6 in the sample.

Attitudinal and background factors by school type: Secondary school Despite the fact that 51% of the Religious secondary schools in the sample are located in rural areas compared to only 41% of National secondary schools, there is very little variation between the two groups in the questionnaire responses. More of the National school students take lessons outside school (22%) than Religious school students (16%). Religious school students are also less likely to report that they speak to their teacher very often during lessons (16%) compared to National school students (25%).


61

Gender profiles Question 2d: How does overall and by-skill student performance at different school stages compare according to gender?

Gender profiles: Primary school Pre-school The performance of Pre-school children by gender can be seen in Table 3.36. The performance of boys and girls is similar and any differences in mean scores were not found to be statistically significant. Table 3.36 Average CEFR proficiency levels by gender: Pre-school Listening

CEFR level

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

Below A1

Below A1

Below A1

A1

Below A1

Below A1

Below A1

Below A1

Table 3.37 and Figure 3.29 show the distribution of boys and girls in each of the CEFR levels covered by their test. The findings indicate a very similar distribution in Reading and Listening, with approximately half to two thirds of boys and girls below A1 and the remaining children at levels A1 and A2. Performance in Writing and Speaking is also similar, with the overwhelming majority of both boys and girls below level A1. Table 3.37 Distribution of CEFR levels by gender: Pre-school Listening

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

A2

9%

9%

11%

12%

3%

4%

0%

0%

A1

23%

23%

33%

32%

7%

7%

1%

5%

Below A1

68%

68%

55%

56%

90%

89%

99%

95%

Overall Male

Female

A2

6%

6%

A1

16%

17%

Below A1

78%

77%



62


Figure 3.29 Distribution of overall CEFR levels by gender: Pre-school 90% 80% 70% 60% 50%

Below A1

40%

A1

30%

A2

20% 10% 0% Male

Female

Year 6 The performance of Year 6 learners by gender (Table 3.38) is similar, except for the better performance of girls in Writing, which is statistically significant. Table 3.38 Average CEFR proficiency levels by gender: Year 6 Listening

CEFR level

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

A1

A1

A1

A1

A2

A2

A1

A1

Table 3.39 and Figure 3.30 show the distribution of boys and girls in each of the CEFR levels covered by their test. It shows a very similar spread of levels, with the majority of both boys and girls mainly achieving between B1 and below A1, except in the case of Writing, where a larger proportion of girls achieved the higher B1/B2 levels. Table 3.39 Distribution of CEFR levels by gender: Year 6 Listening

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

C1

0%

0%

0%

0%

0%

0%

0%

1%

B2

0%

0%

0%

0%

1%

3%

0%

0%

B1

10%

10%

0%

0%

24%

38%

4%

7%

A2

18%

21%

14%

13%

35%

30%

14%

23%

A1

28%

28%

27%

31%

22%

16%

71%

50%

Below A1

43%

41%

58%

56%

17%

12%

11%

19%

Overall Male

Female

C1

0%

0%

B2

0%

1%

B1

10%

14%

A2

21%

22%

A1

37%

32%

Below A1

32%

32%



63

Figure 3.30 Distribution of overall CEFR levels by gender: Year 6 40% 35% 30% Below A1

25%

A1 20%

A2

15% B1 B2

10% 5% 0% Male

Female

Attitudinal and background factors by gender: Primary school The questionnaire data suggests that girls in Year 6 are, in general, more positive about English than boys. They are more likely to strongly agree that learning English is important for them (58%) compared to boys (43%) and more likely to strongly agree that they enjoy their English lessons (50%) than boys (36%). Girls are also less likely to say that their English lessons are too hard (9%) than boys (14%).

Gender profiles: Secondary school Form 3 The performance of Form 3 learners by gender (Table 3.40) shows some differences, and the results of a multilevel regression analysis showed that Form 3 boys score significantly lower than girls in all four skills. The gender gap was found to be the greatest in the Writing exam. Table 3.40 Overall CEFR proficiency levels by gender: Form 3 Listening Gender CEFR level

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

A2

A2

A2

A2

A2

A2

A1

A2

Table 3.41 and Figure 3.31 present the distribution of Form 3 boys and girls in each of the CEFR levels covered by their test. It shows that more girls than boys achieved the higher B1 and above levels (26% boys, 35% girls).

Table 3.41 Distribution of CEFR levels by gender: Form 3 Listening Reading Gender

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

C2

0%

0%

0%

0%

0%

0%

1%

1%

C1

0%

0%

0%

0%

1%

2%

0%

2%

B2

21%

21%

12%

14%

10%

17%

4%

3%

B1

14%

18%

17%

23%

15%

26%

9%

12%

A2

22%

23%

44%

46%

27%

27%

23%

19%

A1

30%

30%

19%

13%

11%

9%

50%

61%

Below A1

13%

9%

8%

4%

36%

19%

13%

2%


64


Overall Male

Female

C2

0%

0%

C1

0%

1%

B2

12%

14%

B1

14%

20%

A2

29%

29%

A1

28%

28%

Below A1

18%

8%

Gender


Figure 3.31 Distribution of overall CEFR levels by gender: Form 3 35% 30% 25%

Below A1 A1

20%

A2 15%

B1

10%

B2 C1

5% 0% Male

Female

Form 5 The performance of Form 5 learners by gender (Table 3.42) indicates that boys and girls achieved similar CEFR levels, except for the better performance of girls in Writing, which was also observed in Year 6 and Form 3. In addition, the mean scores by gender were found to be consistently higher for girls for all skills, even though the CEFR levels are the same, and a multilevel regression analysis indicated that girls in Form 5 scored significantly higher than boys in all four language skills and not just in Writing. Table 3.42 Overall CEFR proficiency levels by gender: Form 5 Listening Gender CEFR level

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

A2

A2

A2

A2

A2

B1

A2

A2

Table 3.43 and Figure 3.32 show the distribution of Form 5 boys and girls in each of the CEFR levels covered by their test. It shows that more girls than boys achieved the higher B1 and above levels (39% boys, 49% girls). Interestingly, at B2 and above, the gender-specific proportions are similar.


65

Table 3.43 Distribution of CEFR levels by gender: Form 5 Listening

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

C2

0%

0%

0%

0%

0%

0%

0%

1%

C1

0%

0%

0%

0%

1%

2%

5%

4%

B2

27%

26%

20%

21%

6%

11%

10%

12%

B1

16%

19%

20%

25%

39%

49%

15%

22%

A2

23%

25%

41%

41%

29%

24%

20%

28%

A1

27%

23%

15%

11%

36%

26%

Below A1

7%

5%

4%

3%

13%

6%

24%

13%

Overall Male

Female

C2

0%

0%

C1

1%

2%

B2

16%

18%

B1

22%

29%

A2

28%

29%

32%

22%

A1 Below A1


Figure 3.32 Distribution of overall CEFR levels by gender: Form 5 35% 30% 25% Below A2 20%

A2 B1

15%

B2 10%

C1

5% 0% Male

Female

Form 6 The performance of Form 6 students by gender can be seen in Table 3.44. In contrast to findings at the other school grades, male students were either on a par with female students, or slightly better, as in the case with Listening. A multilevel regression analysis indicated no statistically significant difference between male and female Form 6 students. This finding is in line with other findings about Form 6, which indicated that differences within Form 6 are not significant, possibly due to the more homogenous nature of the Form 6 cohort due to selfselection.


66


Table 3.44 Average CEFR proficiency levels by gender: Form 6 Listening

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

B1

A2

B1

B1

B1

B1

A2

A2

CEFR level

Table 3.45 and Figure 3.33 show the distribution of Form 6 male and female students in each of the CEFR levels covered by their test. As can be seen, the distribution is very similar across all CEFR levels. Table 3.45 Distribution of CEFR levels by gender: Form 6 Listening

Reading

Writing

Speaking

Male

Female

Male

Female

Male

Female

Male

Female

C2

5%

3%

1%

1%

2%

1%

1%

0%

C1

8%

5%

1%

1%

6%

4%

5%

5%

B2

16%

12%

19%

15%

39%

45%

19%

10%

B1

19%

17%

42%

46%

39%

38%

19%

32%

A2

40%

44%

34%

35%

9%

7%

13%

29%

A1

9%

15%

2%

2%

13%

16%

Below A1

2%

4%

0%

0%

29%

8%

5%

4%

Overall Male

Female

C2

2%

1%

C1

5%

4%

B2

23%

20%

B1

30%

33%

A2

24%

29%

15%

12%

A1 Below A1


Figure 3.33 Distribution of overall CEFR levels by gender: Form 6 35% 30% 25%

Below A2 A2

20%

B1 15%

B2

10%

C1 C2

5% 0% Male

Female


67


Attitudinal and background factors by gender: Secondary school Although there were significant differences in attitudes towards learning English between boys and girls in primary school, these differences are less pronounced at secondary school. There are a few differences at Form 3 and 5 in that girls still tend to be more positive about learning English and see it as more important to their future. However, these differences gradually disappear as students reach Form 6, which is not surprising as these students are generally self-selecting.

Class specialisation profiles Question 2e: How does overall and by-skill student performance at different school stages compare according to class specialisation?

Class specialisation profiles: Secondary school Form 5 The specialisation of Form 5 students into Arts, Science, Vocational and Religious classes presented an opportunity to investigate the effect of this variable, which was not possible with the Form 3 classes. The distribution of CEFR levels by class specialisation is given in Table 3.46. Students in Science classes consistently emerged as having higher language proficiency – a difference which was shown to be statistically significant. Table 3.46 Overall CEFR proficiency levels by class specialisation: Form 5 Listening Reading

Writing

Speaking

Specialisation

A

S

V

R

A

S

V

R

A

S

V

R

A

S

V

R

CEFR level

A2

B1

A2

A1

A2

B1

A2

A2

A2

B1

A2

A2

A2

A2

B1

A1

Note: A=Arts, S=Science, V=Vocational, R=Religious. Speaking for ‘Vocational’ and ‘Religious’ is based on a small number of learners (N=29 Vocational, N=49 Religious)

The distribution of CEFR levels across class specialisation is given in Table 3.47 and Figure 3.34, and the findings indicate that a much larger proportion of Form 5 learners in Science classes achieved the higher B2 and above levels, compared with learners from the other specialisations: 32% of Form 5 students in the Science classes were found to be at CEFR level B2 and above, which can be considered the level at which operational proficiency is displayed, compared to between 4% and 10% of learners at this level in the other three specialisations. Table 3.47 Distribution of CEFR levels by class specialisation: Form 5 Listening Reading

Writing

Speaking

A

S

V

R

A

S

V

R

A

S

V

R

A

S

V

R

C2

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

0%

1%

1%

0%

0%

C1

0%

0%

0%

0%

0%

0%

0%

0%

0%

4%

1%

0%

7%

4%

0%

0%

B2

12%

47%

12%

7%

6%

40%

7%

6%

3%

18%

3%

3%

9%

15%

10%

0%

B1

13%

23%

17%

13%

15%

33%

18%

16%

29%

65%

33%

51%

14%

21%

67%

2%

A2

25%

21%

31%

27%

51%

24%

56%

59%

34%

13%

37%

39%

27%

20%

24%

29%

A1

39%

9%

31%

40%

22%

2%

16%

16%

24%

31%

0%

69%

Below A1

10%

1%

8%

13%

5%

1%

4%

3%

34%

1%

27%

7%

17%

8%

0%

0%


68


Overall A

S

V

R

C2

0%

0%

0%

0%

C1

2%

2%

0%

0%

B2

8%

30%

8%

4%

B1

18%

35%

34%

20%

A2

34%

20%

37%

38%

38%

13%

21%

37%

A1 Below A1

Note: Speaking for ‘Vocational’ and ‘Religious’ is based on a small number of learners (N=29 Vocational, N=49 Religious).Percentages may not always add up to 100% due to rounding.

Figure 3.34 Distribution of overall CEFR levels by class specialisation: Form 5 40% 35% 30% A1 and below

25%

A2 20%

B1

15%

B2

10%

C1

5% 0% Arts

Science

Vocational

Religious

Form 6 Class specialisation at Form 6 was investigated in terms of Arts/Science. The distribution of CEFR levels by class specialisation (Table 3.48) indicates a marked difference in performance, with Science classes performing better than Arts classes. This difference was found to be statistically significant and is in line with the findings observed at Form 5. Table 3.48 Overall CEFR proficiency levels by class specialisation: Form 6 Listening

CEFR level

Arts A2

Science B2

Reading Arts B1

Science B1

Writing Arts B1

Science B2

Speaking Arts A2

Science B2

The distribution of CEFR levels across class specialisation at Form 6 is given in Table 3.49 and Figure 3.35, and the findings indicate that a much larger proportion of learners in Science classes have achieved the higher levels, compared with Arts students: 60% of Science students are at level B2 and above, compared to 19% of Arts students. This is a striking difference, which signals very different learning gains for students in different specialisations.


69

Table 3.49 Distribution of CEFR levels by c lass specialisation: Form 6 Listening

Reading

Writing

Speaking

Arts

Science

Arts

Science

Arts

Science

Arts

C2

1%

14%

0%

2%

0%

6%

0%

2%

C1

4%

16%

1%

3%

2%

13%

0%

24%

B2

11%

26%

13%

32%

37%

64%

8%

39%

B1

17%

24%

43%

46%

45%

16%

22%

35%

A2

48%

19%

39%

16%

10%

1%

27%

0%

A1

15%

1%

2%

1%

4%

0%

0%

0%

Below A1

6%

1%

Science

20%

0%

24%

0%

Overall Arts

Science

C2

0%

6%

C1

2%

14%

B2

17%

40%

B1

32%

30%

A2

31%

9%

18%

1%

A1 Below A1


Figure 3.35 Distribution of overall CEFR levels by class specialisation: Form 6 45% 40% 35% A1 and below

30%

A2 25%

B1

20%

B2

15%

C1

10%

C2

5% 0% Arts

Science

Attitudinal and background factors by specialisation: Secondary school When questionnaire responses were analysed according to specialisation, the students who specialised in Science were found to be much more positive about learning English and much more likely to use English both inside and outside the learning environment than all other groups. When asked whether learning English is important to them, 73% (Form 5) and 76% (Form 6) of students in Science classes said it was very important whereas 50% (Form 5) and 62% (Form 6) of students in Arts classes, 55% of Form 5 students in Vocational classes and between 34% and 49% of Form 5 and 6 students in Religious classes said

70


the same. Similarly, the parents of Science students are more likely to speak English very well (between 13% and 22%) compared to all other groups (between 5% and 14%). Large numbers of students in Science classes reported watching TV or films in English very often (48% –54%) compared to students in Arts classes (35%–47%), Vocational classes (40%) and Religious classes (32%–35%). Between 20% and 28% of students in Form 5 and 6 Science classes reported reading books in English very often while only 2% to 10% of the other groups indicated the same. The students in Science classes are also more likely to use the internet in English very often (72%–76%) compared to students in Arts classes (59%–70%), Vocational classes (61%) and Religious classes (54%–59%). Students in Form 5 and 6 Science classes are less likely to report that their classes are too hard (2–3%) than students in Arts classes (7–12%), Vocational classes (11%) and Religious classes (6–11%). Students in Science classes also tend to use English much more often in class. Although only 35% of students in Form 5 Science classes reported speaking to their teacher in English very often, this jumps to 77% for Form 6 students. The students who specialise in the other subjects who reported speaking very often to their teacher in English ranged between 4% (Religious classes) to 27% (Arts classes). Students in Science classes are more likely to strongly agree that they would like to study other subjects in English (Form 5: 56% and Form 6: 80%) than students in Arts and Vocational classes (29%) and students in Religious classes (13%). This is a striking difference but as these students are more proficient than the other groups, it is not that surprising.

High performing versus low performing learners Finally, possible background and attitudinal factors which could account for the differences between high performing students and low performing students in primary and secondary schools were investigated. High performing students were defined as those who scored 1.5 standard deviations above the mean of each cohort in the Listening and Reading tests; low performing students were defined as those who scored 1.5 standard deviations below the mean of each cohort in both exams. Students were classified into two groups, high performers and low performers, according to the above criteria, and the characteristics which make students more likely to belong to the high performing group were investigated. (See theCambridge Baseline 2013 Technical Report for a detailed description of the analysis). For Year 6 only, gender is linked to performance (see Figure 3.36). Females in Year 6 are more likely to be high achievers in English than male students. As noted earlier, we found that girls tended to be more positive towards English language learning in general which could account for the increased likelihood that they fall into the highachiever category. However, it is important to note that these findings do not suggest a cause-effect relationship. That is, girls may be more positive about language learning because they are doing well or receiving positive feedback on their performance or they may be more likely to be found in the high-achiever category because of better literacy skills in their first language. Research suggests that girls tend to outperform boys in acquiring their first language (Maccoby and Jacklin 1974) and these skills can support second language development.


71

Figure 3.36 High performing versus low performing Year 6 learners by gender

'What is your gender?' 100% 80% 60%

Lowest

40%

Highest

20% 0%

Male

Female

Parents’ knowledge of English was also found to be a factor that is associated with high achievement. However, there are differences by grade and parent. Year 6 and Form 3 learners who reported that their mothers speak English very well or moderately well were more likely to be high achievers (see Figure 3.37). Interestingly, for secondary school students it was their father’s English language ability that was found to be statistically associated with achievement rather than their mother’s (see Figure 3.38). Although this difference is puzzling, it could be related to parental roles within the household that change as the child gets older. Figure 3.37 High performing versus low performing Year 6 and Form 3 learners by mother’s English ability

'My mother is able to speak English…' 50% 40% 30% 20%

Lowest

10%

Highest

0% very w ell

moderately well

not very well

my mother can't speak English

not sure

Figure 3.38 High performing versus low performing secondary school learners by father’s English ability

'My father is able to speak English…' 50% 40% 30% 20%

Lowest

10% 0%

Highest very w ell

moderately well

not very well

my father can't speak English

not sure

The use of the internet in English was also associated with high-performing students but only for those in Year 6 and Form 5. Figures 3.39 and 3.40 show that learners who reported that they use the internet very often in English were more likely to be in the high-achiever group. The role of technology in widening access to both information and people could be seen as an important factor for learners who may not otherwise have opportunities to use English in their immediate home environment. However, access to or use of the internet may be linked to socioeconomic status, which other international studies have found is positively associated with learning outcomes.


72


Using the internet to play video games has also been linked to cognitive development which could positively impact language learning (Blumberg and Fisch (Eds) 2013). Figure 3.39 High performing versus low performing Year 6 learners by internet use

'I use the internet in English.' 100% 80% 60% 40%

Lowest Highest

20% 0% Very often Sometimes

Not very often

Never

Not sure

Figure 3.40 High performing versus low performing Form 5 learners by use of the internet in English

'I use the internet in English.' 100% 80% 60% 40%

Lowest

20%

Highest

0% Very often Sometimes

Not very often

Sometimes

Not sure

Learners in all secondary grades who reported that they very often read books in English are more likely to be high achievers (see Figure 3.41). Figure 3.41 High performing versus low performing secondary school learners by frequency of reading in English

'I read books or c omic books in English.' 100% 80% 60% 40%

Lowest

20%

Highest

0% Very often Sometimes

Not very often

Sometimes

Not sure

Although not statistically significant, there was some variation for Year 6 learners in terms of reading frequency in English (see Figure 3.42) and those in the high-performing category were more likely to read often in English. This association should be interpreted cautiously, as it is likely that low performing learners are not reading in English because they do not have the language ability to do so, rather than because they choose not to.


73

Figure 3.42 High performing versus low performing Year 6 learners by frequency of reading in English

'I read books or comic books in English.' 100% 80% 60% Lowest 40%

Highest

20% 0% Very Often

Never

The majority of the findings presented above reinforce the positive relationship between exposure to English, opportunities to use English and learner outcomes. It is clear that schools need to maximise learners’ exposure to English, their opportunities to use English and to better structure work done in and out of the class. These findings also suggest that it may be beneficial to engage parents in English language learning, as their influence appears to be quite significant.

Summary Achievement gaps In the previous part of this section we reported on the national profile of each school grade of interest and provided a baseline as follows:

Pre-school

On average below CEFR level A1 78% below A1, 22% at A1/A2

Year 6

On average at CEFR level A1 32% below A1, 56% at A1/A2, 12% at B1/B2

Form 3

On average at CEFR level A2 12% below A1, 57% at A1/A2, 30% at B1/B2, 1% at C1/C2

Form 5

On average at CEFR level A2 55% at A1/A2 and below, 43% at B1/B2, 2% at C1/C2

Form 6

On average at CEFR level A2/B1 41% at A1/A2 and below, 53% at B1/B2, 6% at C1/C2.


74


Location Further to this baseline, one of the most striking findings to emerge from the analysis was the achievement gap across learners at the same school grade. The achievement gap was most pronounced in terms of location of the school, with students in remote and rural areas and in states with a higher proportion of rural and remote schools consistently performing significantly worse than their urban counterparts. This urban-rural divide was found in all school grades, with the exception of Pre-school. Figures 3.43 and 3.44 provide an illustration of this gap for Form 5, chosen as an illustration here. These graphs clearly show the higher percentages of learners at levels B1 and above in urban Form 5 classes and the lower percentage at levels A2 and below. Several factors were identified from the questionnaire responses that could be contributing to this achievement gap. In particular, rural school students tend to be less motivated to learn English which may be related to their limited exposure to the language both within the school and at home, and parents who are less able to help them with their English language education. Other factors such as instructional quality are also important to consider, and these will be presented in the next section. Figure 3.43 Distribution of CEFR levels B1 and above by location: Form 5 60% 50%

53%

40% 30%

34%

20%

24%

10%

11%

2%

1%

Urban

Rural

0% Urban

Rural

B1andabove

Urban

Rural

B2andabove

C1andabove

Figure 3.44 Distribution of CEFR levels A2 and below by location: Form 5 80% 70% 67%

60% 50% 40%

47%

30%

35%

20% 20% 10% 0% Urban A2andbelow

Rural

Urban A1andbelow

Rural


75

Gender A further gap in achievement, which was especially pronounced in Forms 3 and 5, was based on gender, with female students performing significantly better than their male classmates: in Form 3 26% of boys were found to be independent users of English (i.e. at CEFR level B1 and above), compared to 35% of girls; in Form 5 39% of boys were independent users, compared to 49% of girls. The gender difference for Form 5 male and female students can be seen in Figures 3.45 and 3.46. The questionnaire findings suggest that girls tend to be more positive about learning English than boys; however, this difference in attitude narrows as learners get older. Helping boys from an early age to see the value of learning English and develop linguistic self-confidence may be necessary in order to reduce this achievement gap.

Figure 3.45 Distribution of CEFR levels B1 and above by gender: Form 5 60% 50% 49% 40% 39% 30% 20%

0%

20%

17%

10%

Male

Female

B1andabove

Male

Female

B2andabove

1%

2%

Male

Female

C1andabove

Figure 3.46 Distribution of CEFR levels A2 and below by gender: Form 5 70% 60% 60% 50%

51%

40% 30%

32%

20%

22%

10% 0% Male

Female A2andbelow

Male A1andbelow

Female


76


Class specialisation A third factor contributing to an achievement gap was found to be class specialisation. In Forms 5 and 6 learners in Science specialisation classes were found to perform significantly better than their classmates in Arts, Vocational and Religious class specialisations. In Form 5, 67% of Science specialisation learners were found to be independent users of English (at level B1 and above), compared to 28% in Arts classes, 42% in Vocational and 24% in Religious; in Form 6 90% of Science learners were independent users, compared to 51% learners in Arts classes. The difference by class specialisation for Form 5 Arts and Science students can be seen in Figures 3.47 and 3.48. The questionnaire responses of learners in the different specialisations highlighted some striking differences. Not only are learners specialising in Science much more positive about learning English than learners specialising in other subjects, they appear to be using English much more frequently both within the learning and home environment. Learners in Science classes seem to be much more prepared to study through the medium of English, which is likely to be the next step in their education path. Figure 3.47 Distribution of CEFR levels B1 and above by class specialisation: Form 5 80% 70% 67%

60% 50% 40% 30% 20%

32%

28%

10% 10%

2%

2%

Arts

Science

0% Arts

Science

B1andabove

Arts

Science

B2andabove

C1andabove

Figure 3.48 Distribution of CEFR levels B1 and above by class specialisation: Form 5 80% 70%

72%

60% 50% 40% 38%

30%

33%

20% 10%

13%

0% Arts

Science A2andbelow

Arts A1andbelow

Science

Cambridge Baseline 2013 Teacher English language proficiency, teaching knowledge and teaching practice

4.

Key findings The key findings provide a snapshot of the established teacher language proficiency, teaching knowledge and teaching practice baseline and also highlight the most salient and meaningful differences in performance which emerged across certain variables as statistically significant, such as school type and location. In addition, the main interview, observation and document review findings are summarised, as well as relevant attitudinal and background factors found to be associated with teacher performance in terms of proficiency, teaching knowledge and teaching practice.

Teacher language proficiency On average at CEFR level B2 Majority (84%) range between CEFR level B2 and C2; 2% are below B1, 45% are at B1/B2, 52% are at C1/C2 Teachers in Urban schools (vs. Rural and Remote) perform significantly better: average CEFR level for teachers in Urban schools is C1, compared to B2 for teachers in Rural and Remote schools Teachers in secondary schools (versus primary) perform better: average CEFR level is B2/C1, compared to B2 for primary school teachers

Teaching knowledge Average at Band 3 (on the Teaching Knowledge Test 1-4 band scale), which represents teachers with generally comprehensive knowledge of teaching concepts, terminology, practices and processes

Teaching practice Average at Band 2 (on the Teaching Knowledge Test: Practical 1-4 band scale), which represents teachers who are aware of and can plan and implement a range of appropriate teaching procedures and techniques, but need to do so more consistently and effectively to maximise learner engagement and participation in the learning process Observers found teachers to be very good at establishing rapport with students, but they tended to dominate lessons and there were few opportunities for learners to engage in meaningful communication In some cases, teachers’ language proficiency limited their classroom effectiveness and they did not provide learners with a good language model

Role of attitudinal and background variables Although overall teachers enjoy teaching, new and non-optionist teachers were less positive about the profession New teachers and non-optionist teachers are less likely to be confident when adapting lessons or assessing their learners’ progress

Document review: curriculum, textbooks and exams The primary and secondary school curricula were found not to reflect international standards Teachers report that the textbooks are outdated and that they tend to rely on additional resources that they have prepared themselves National exams are too narrowly focused on Reading and Writing at the expense of Listening and Speaking, resulting in negative washback In their current form, tests are not likely to be a valid and reliable indicator of student performance

77

78


Having presented student language proficiency across the grades, we now turn our attention to investigating teacher language proficiency, teaching knowledge and teaching practice. Developing a wide-ranging profile of teachers’ language proficiency, knowledge and practice is an important strand that will complement the student data in identifying areas in need of further attention. The following chapter will provide an overview of overall teacher language proficiency by skill against the CEFR and according to school stage (i.e. primary/secondary), state/federal territory and location (i.e. urban, rural, remote). We will then provide the quantitative findings from the teaching knowledge test and teaching practice assessment scale. Each section within this chapter will include a discussion of the relevant information from the qualitative data strands (e.g. questionnaires, lesson observation reports, curricula/textbooks/examinations review, etc.) that can shed light on the factors that may be influencing the outcomes in language proficiency, teaching knowledge and practice.

Language proficiency: Teachers National profile: Teachers Research Question 3: How do teachers perform on a set of Cambridge English language tests overall and by skill (Listening, Reading, Writing and Speaking) against the CEFR?

Overall, teachers in the baseline project achieved a CEFR level B2, as seen in Table 4.1. It is worth noting that the average scores for Listening and Reading were found to be just below the C1 level boundary, making the performance in those skills a high B2 performance. Writing was found to be the strongest skill, with teachers achieving C1. Table 4.1 Average CEFR proficiency levels: Teachers CEFR level

Listening

Reading

Writing

Speaking

B2

B2

C1

B2

The range of CEFR levels achieved by the majority of teachers (84%) spans CEFR levels B2 to C2, representing independent to operational proficiency in a language (Table 4.2 and Figure 4.1). Amongst those teachers, 52% can be considered to have operational proficiency (CEFR levels C1 and C2) and 32% can be considered to be independent speakers of English (CEFR level B2). Writing emerged as the strongest skill, and Speaking as the weakest. The strong performance in Writing and on the test as a whole could potentially be a reflection of the high education and experience level of the teachers who took the test (to be discussed later in this Section). The weakest performance in Speaking is worth noting, as it is in line with the findings for students, who were also least strong in Speaking. Table 4.2 Distribution of CEFR levels: Teachers Listening

Reading

Writing

Speaking

Overall

C2 C1

29% 27%

18% 26%

28% 48%

7% 24%

21% 31%

B2

25%

43%

18%

43%

32%

B1

12%

12%

6%

24%

13%

Below B1

6%

1%

0%

2%

3%



79

Figure 4.1 Distribution of Overall CEFR level: Teachers 35% 30% 25% 20% 15% 10% 5% 0% Below B1

B1

B2

C1

C2

Understanding teacher language proficiency As mentioned in the previous section, 52% of teachers achieved an overall CEFR level of C1 or above (see Figure 4.1 and Table 4.2). This finding was somewhat surprising because observers, Heads of Panel and even teachers themselves raised concerns about the language proficiency of teachers. The following quotes are typical: ‘A lot of teachers have not mastered the language , thus not able to deliver lessons confidently and accurately’ (Urban secondary school, Head of Panel, Melaka) ‘The teacher is held back by her language skills. Her poor grammar and vocabulary and control of prosodic features lead to inaccurate examples and modelling’ (Urban secondary school, Form 3, Pahang, Observer JL) ‘[We need] a teacher who is proficient in the language and can teach properly’ (Rural secondary school teacher, Selangor) It would appear that this divergence in findings could be explained when we consider the profile and number of teachers who completed the test. In many cases, information on the selected teachers in the sample was not available, so test forms with teacher names and IDs could not be pre-printed, and therefore it is was not possible to ensure that the teacher whose class was selected was the same teacher taking the test. It may be that many self-selected teachers took the test, and not the teachers in the sample. This conjecture is based on the fact that within the cohort of teachers who took the test, 40% reported to have a Bachelor of Education degree and 10% a Master’s in Education. This is a considerably different profile from the education background of Malaysian teachers in general, only 10% of whom have a Bachelor’s degree and only 1% a Master’s degree, as indicated in data supplied by the Ministry. In addition, the teachers reported having a significant amount of experience, as 60% have been teaching for 11 years or more and the proportion of optionist teachers is slightly higher than in the overall teaching population as reported in the Malaysia Educational Statistics: Quick Facts 2012 document. Finally, we expected 943 teachers to take the test but in fact only 424 did so. These lower test numbers, when compared to the 1,290 teachers who completed the questionnaire, do suggest that there was some deliberate nonparticipation in the test sessions. Therefore, we could conclude that the teachers who were selected/self-selected to take the test were in fact the most proficient in English in the school or for that grade. However, as will be

80


explained in the next section, teacher proficiency varied significantly when one considers the other variables of interest (i.e. state, location and school type).

State profiles: Teachers Research Question 4: How do teachers perform on a set of Cambridge English language tests overall and by skill (Listening, Reading, Writing and Speaking) according to state?

Test performance by state for teachers showed some variation, as can be seen in Table 4.3 (which does not report Speaking performance due to the small size of the Speaking sample by state). Teachers from Pulau Penang, WP Kuala Lumpur and WP Putrajaya showed stronger than average performance and achieved C1/C2 levels in two or three of the skills. Teachers from Sabah and Sarawak performed below the national average in Listening and Reading (no Writing data was available).

Table 4.3 Average CEFR proficiency levels by State/Federal Territory: Teachers Language skill

Listening

Reading

Writing

CEFR level

CEFR level

CEFR level

Malaysia overall

B2

B2

C1

JOHOR

B2

B2

C1

KEDAH

B2

B2

C1

-

-

-

MELAKA

B2

B2

C1

NEGERI SEMBILAN

B2

B2

C1

PAHANG

B2

B2

B2

PERAK

C1

B2

C1

PERLIS

C1

B2

-

PULAU PENANG

C1

C1

C1

SABAH

B2

B2

-

SARAWAK

B2

B2

-

SELANGOR

C1

B2

C1

TERENGGANU

B2

B2

C1

WP KUALA LUMPUR

C1

C1

C1

-

-

-

C2

C1

-

KELANTAN

WP LABUAN WP PUTRAJAYA

Note: The missing data is due to late tests, which were not included in the analysis; Speaking is not reported by state due to the smaller size of the Speaking sample in each state.

Location profiles: Teachers

Table 4.4 presents the findings by urban/rural/remote location of school and shows that in all four skills teachers in rural and remote locations on average performed more weakly than their urban colleagues. A multilevel regression analysis indicated that teachers from rural and remote areas performed significantly worse than teachers from urban areas.


81

Table 4.4 Average CEFR proficiency levels by location: Teachers Listening

Reading

Writing

Speaking

Location

Urban

Rural

Urban

Rural

Urban

Rural

Urban

Rural

CEFR Level

C1

B2

C1

B2

C1

C1

C1

B2

Note: Teachers from remote locations are not included in the table due to the very small numbers (N=4 for Listening and Reading, N=0 for Writing and N=2 for Speaking in remote areas).

Table 4.5 and Figure 4.2 present the distribution of CEFR levels of teachers in the two location types. As can be seen, teachers from urban schools performed consistently better in all four skills, compared to their colleagues in rural and remote locations, by showing a higher proportion at the higher levels: overall, 63% of teachers in urban areas were at CEFR levels C1/C2, compared to 42% of their rural colleagues achieving the same levels. A relatively small proportion of teachers in both location types can be found at the low end of the scale (CEFR levels B1 and below): 13% for urban locations and 18% for rural ones, which in essence means that 1 in 7 teachers and 1 in 5 teachers in urban and rural areas have basic or minimally operational proficiency in English. Table 4.5 Distribution of CEFR levels by location: Teachers Language skill Location

Listening

Reading

Writing

Speaking

Overall

Urban

Rural

Urban

Rural

Urban

15%

37%

19%

15%

0%

27%

14%

21%

49%

56%

40%

10%

36%

28%

38%

48%

12%

17%

20%

65%

23%

40%

11%

13%

2%

7%

25%

20%

12%

14%

0%

3%

0%

1%

0%

5%

1%

4%

Urban

Rural

Urban

Rural

C2

36%

21%

21%

C1

26%

27%

29%

B2

23%

29%

B1

9%

16%

Below B1

6%

7%

Rural

Note: Teachers from remote locations are not included in the table due to the very small numbers (N=4 for Listening and Reading, N=0 for Writing and N=2 for Speaking in remote areas). Percentages may not always add up to 100% due to rounding.

Figure 4.2 Distribution of overall CEFR levels by location: Teachers 45% 40% 35% 30%

Below B1

25%

B1

20%

B2 C1

15%

C2

10% 5% 0% Urban

Rural

Exploring teacher language proficiency by state and location When considering teacher language proficiency by state and location, we found that rural schools and the states/federal territories with more rural schools face challenges in attracting teachers who not only have strong

82


English language skills but are also highly trained. As one Head of Panel noted when asked how the teaching of English could be improved in their school: ‘More excellent teachers should come and help pupils who live in rural areas like this school’ (Rural primary school, Head of Panel, Selangor) This sentiment was echoed by teachers in the questionnaire and during post-observation interviews. There is a shortage of teachers trained to teach English and many generalist teachers get co-opted into teaching English. This particularly seems to be the case in rural/remote schools as 25% of these teachers are non-optionists compared to 15% of urban school teachers. A concern frequently raised is that generalist teachers often do not have the requisite training: ‘Have sufficient trained teachers and not teachers of other subjects conveniently reassigned to teach English. Competent teachers are important as they provide knowledge’ (Urban secondary school teacher, Terengganu) Teachers and Heads of Panel were at pains to emphasise that this is not a criticism of generalist teachers as they stressed that these teachers were working hard to do a good job, but that generally non-optionist teachers are not as proficient in English or knowledgeable about second language learning as they could be: ‘At this time, there are only 2 English teachers with English teaching degrees/certificates. To have a better result in performance, there is an urgent need for the right people doing the right job. In such cases, teachers with the right specialisation will contribute their knowledge, experience as well as expertise effectively and efficiently’ (Rural secondary school teacher, Kelantan)

The questionnaire findings do seem to suggest that there are differences between optionist and non-optionist teachers in terms of frequency of language use and confidence in preparing lessons. Non-optionist teachers were less likely to report in the questionnaire that they teach in English very often (52%) than optionist teachers (80%). This finding may be attributable to their language level as 25% of non-optionist teachers agreed or strongly agreed that they use the local language because they do not feel confident teaching in English, whereas only 4% of optionist teachers said the same. Similarly, non-optionist teachers appear to be less confident in planning lessons with only 15% strongly agreeing that they are confident in their lesson planning ability compared to 44% of optionist teachers who indicated the same. It would appear that the differences between optionist and nonoptionist teachers may be impacting lesson quality. However, teacher shortages may not make it feasible to hire only optionist teachers particularly in rural areas. Therefore, upskilling of current teachers by providing specialist training becomes even more essential.

Primary/secondary school profiles: Teachers The findings for teachers based on primary and secondary school (Table 4.6) indicate that secondary school teachers (S) performed better than their primary school colleagues (P) in all four skills. The difference was only found to be statistically significant for Listening and Reading, and not Speaking and Writing, possibly due to the small number of observations for the latter two tests. Table 4.6 Average CEFR proficiency levels by school type: Teachers Language skill

Listening

Reading

Writing

Speaking

School type

P

S

P

S

P

S

P

S

CEFR level

B2

C1

B2

B2

B2

C1

B2

B2


83

Table 4.7 and Figure 4.3 present the distribution of CEFR levels of teachers by primary and secondary school and show that the majority of primary school teachers span CEFR levels B1 to C1, whereas for secondary school teachers that range is B2 to C2. This is an important finding, since it indicates that in primary schools a relatively large proportion of teachers (in this case, 29%) are below level B2, which is typically considered to be the minimum threshold for operational proficiency (Council of Europe 2001). This drops to 11% for secondary schools.

Table 4.7 Distribution of CEFR levels by school type: Teacher s Language skill

Listening

Reading

Writing

School type C2

P 16%

S 34%

P 11%

S 21%

P 10%

S 35%

C1

21%

29%

20%

28%

35%

B2

28%

25%

49%

40%

41%

B1

24%

7%

17%

10%

Below B1

11%

4%

3%

0%

Speaking

Overall

P 0%

S 10%

P 9%

S 25%

53%

8%

31%

21%

35%

10%

46%

41%

41%

29%

14%

3%

46%

14%

25%

8%

0%

1%

0%

3%

3%

3%


Figure 4.3 Distribution of overall CEFR levels by school type: Teachers 45% 40% 35% 30%

Below B1

25%

B1

20%

B2 C1

15%

C2

10% 5% 0% Primary

Secondary

Exploring teacher language proficiency by primary/secondary school Analysis of the questionnaire data showed that primary school teachers were more likely to have less teaching experience than their secondary colleagues. Twenty percent of primary school teachers have 3 years or less experience (referred to as ‘new’ teachers in the remainder of this document) whereas only 8% of secondary school teachers reported the same. Primary school teachers were also more likely to be non-optionist/generalist teachers (31%) than secondary school teachers (11%). Primary school teachers appear to be educated to a lesser extent than the secondary school teachers: 32% of primary teachers compared to 66% of secondary teachers have a Bachelor’s degree in Education or higher. One could conclude from the educational profiles and optionist status of primary school teachers that they may have received less English language training, and that this could account for their lower performance on the language proficiency tests. A teacher’s language proficiency may influence the amount and quality of English used during lessons (Altinyelken 2010). When asked how often they teach in English, primary school teachers were less likely to indicate that they teach in English very often (69%) than secondary teachers (80%). They were also more likely to report that they speak the local language in class because their students cannot follow the lesson if it is taught in English (49% strongly agree or agree) compared to secondary school teachers (37%). The use of the local language is not, of


84

Teacher English language proficiency, teaching knowledge and teaching practice course, taboo, and depends on whether the local language is used when seen as necessary, e.g. to make sure all instructions are understood, or whether it is used when not justified. However, the questionnaire findings do suggest that primary school teachers’ language proficiency may be negatively affecting their language use. That is, primary school teachers were more likely to strongly agree or agree that they teach in the local language because they do not feel confident teaching in English (13%) as compared to their secondary school counterparts (4%). This is an important finding because research indicates that exposure to a foreign language within the learning environment is positively associated with achievement (Jones 2013). The language proficiency of primary school teachers could be a contributing factor to the variation in language levels found in Year 6.

Summary Language proficiency The range of CEFR levels achieved by 98% of teachers was found to span CEFR levels B1 to C2, representing lower intermediate to advanced proficiency. The stronger than expected language performance observed in the teacher sample was also inconsistent with the findings from the Cambridge English Placement Test (CEPT) and may be due to the low rate of participating teachers (N=424), which represented less than half the expected participation, and the possibility that more proficient and experienced teachers self-selected to do the test. This conjecture is based on the fact that the cohort of teachers who took the test reported in the questionnaire that they were significantly better qualified and experienced than the overall teaching population.

Language skills profile Similar to the student findings, Speaking emerged as the weakest skill for teachers, with 31% achieving C1/C2 levels in Speaking, compared to 44% for Reading, 56% for Listening and 76% for Writing (Figure 4.4). Figure 4.4 Distribution of CEFR levels by skill: Teachers 80% 70%

76%

60% 56%

50% 40%

44%

30% 31% 20% 10% 0% Writing

Listening

Reading

Speaking

C1 and above

Achievement gaps Location Teachers from urban schools performed consistently better in all four skills, as compared to their colleagues in rural and remote locations. The strongest distinguishing difference between rural and urban teachers is the proportion at the highest C levels: overall, 63% of teachers in urban areas were at CEFR levels C1/C2, compared to 42% of their rural colleagues achieving the same levels (Figure 4.5). A relatively small proportion of teachers in both locations were found at the low end of the scale (below level B1).


85

Figure 4.5 Distribution of overall CEFR levels by location: Teachers 100% 98%

95%

80%

86%

82%

60%

63%

40%

42%

20% 0% Urban

Rural

B1andabove

Urban

Rural

B2andabove

Urban

Rural

C1andabove

School stage Teachers in secondary schools performed better than their primary school colleagues: the majority of primary school teachers were found to span CEFR levels B1 to C1, whereas for secondary school teachers that range was B2 to C2. In primary schools 71% of teachers were found to be at B2, typically considered to be the minimum threshold for operational proficiency in a work or academic environment, compared to 89% of secondary school teachers. That difference was more pronounced at the higher C1 and above levels (Figure 4.6). This is perhaps not surprising as the number of non-optionist teachers in primary schools is larger than in secondary schools. Nonoptionist teachers generally have less English language training, which was raised as a concern by Heads of Panel and teachers who feel that primary school students are not being adequately prepared to teach English because of this lack of training and language proficiency.

Figure 4.6 Distribution of overall CEFR levels by school type: Teachers 100% 97%

96% 89%

80%

71%

60%

60% 40% 30%

20%

0% SPSPSP B1andabove S=Secondary; P=Primary

B2andabove

C1andabove

86


Discussion The aspiration of the Ministry policy planners is for all teachers to achieve a C2 level. The language test data shows that almost 80% of teachers have not achieved this, with 47% of teachers at least 2 CEFR levels below target. Although the language ability of teachers can have a direct impact on the quality of instruction (see ‘Teaching practice’) and ultimately influence learner outcomes, it may not be feasible to expect all teachers to reach the C2 level. Heads of Panel and teachers consistently mentioned teacher shortages as a persistent problem which could make it difficult to find and retain teachers at this CEFR level, particularly for rural/remote schools. Research conducted in other countries which face challenges in recruiting and retaining qualified teachers in remote areas point out that learning outcomes can be negatively impacted by high teacher turnover, as this is often linked to lower quality instruction (Mueller, Carr-Stewart, Steeves and Marshall 2011). Therefore, it may be more justified to consider a varied approach to teacher language proficiency, as opposed to one universal language proficiency requirement for all teachers in all contexts. This would require a multi-tiered approach in that teachers who do not have the desired English language proficiency are given additional support (i.e. English language assistants, teaching mentors, intensive language provision, etc.) to ensure that learning outcomes are achieved.


87

Teaching knowledge Question 5: How do teachers teaching at different school stages perform overall on a test of teaching knowledge?

As mentioned in the Methodology section, part of the project focused on benchmarking the teaching knowledge of the teachers in the sample. This was done with a multiple choice test of teaching knowledge, which focused on concepts and terminology for describing language and language learning and teaching, lesson planning and use of resources, and managing the teaching and learning process. Performance was reported on a four-band scale, where Band 4 represents comprehensive and accurate knowledge of concepts, terminology, practices and processes, and Band 1 represents restricted knowledge. The average performance on the test by the teachers in the sample was Band 3, and the proportion of teachers in the four bands can be seen in Table 4.8. As can be seen, approximately two thirds of the teachers (65%) were at that band, with 31% in the band below.

Table 4.8 Overall Teaching Knowledge Test (TKT) level: Teachers Band

Overall

4

1%

3

65%

2

31%

1

3%

The overall national-level performance of Malaysian teachers in this baseline project can also be compared to performance on a Cambridge English test which focuses on teaching knowledge (‘Teaching Knowledge Test: TKT’) and is taken by teachers worldwide. The two tests are not identical, but the concepts and test format are very similar, and the TKT population has a large proportion of state-school teachers, making a comparison justified. The results can be seen in Figure 4.7, which shows that Malaysia is on a par with teachers from around the world: in both groups the largest proportion of teachers achieved Band 3. Fewer Malaysian teachers, however, achieved the top Band 4, compared with teachers from around the world. Figure 4.7 Average Teaching Knowledge Test (TKT) level: Comparison between Malaysia and TKT population 70% 60% 50% 40% Malaysia 30% Rest of the World 20% 10% 0% 1234

88


Question 6: How do teachers teaching at different school stages compare according to location and primary/secondary school on a test of teaching knowledge?

In terms of the remote/rural/urban distribution, teachers in both areas achieved an average of Band 3. The distribution across the four bands (Table 4.9) shows that the vast majority of teachers in the different locations were at Bands 2 and 3, with urban teachers showing a slightly larger proportion at the higher level. Table 4.9 Overall Teaching Knowledge Test (TKT) level by loc ation: Teachers Band

Urban

Rural

Remote

4 3

0% 71%

1% 59%

0% 67%

2

27%

34%

33%

1

2%

5%

0%

Note: ‘Remote’ is based on a small sample (N=52). Percentages may not always add up to 100% due to rounding.

A difference was found in the performance of teachers in primary and secondary schools (Table 4.10), where primary school teachers achieved on average Band 2, and secondary school teachers a Band 3. This difference can be seen in Table 4.10, which shows that a larger proportion of secondary school teachers achieved the higher Band 3 (75% for secondary, compared to 49% for primary). This finding is in line with the teacher language proficiency findings, which showed that secondary school teachers performed better. In English language teaching, teacher language proficiency is both the medium of teaching and the subject of teaching, so a higher level in one supports a higher level in the other. Table 4.10 Overall Teaching Knowledge Test (TKT) level by Primar y/Secondary school: Teachers Band

Primary

Secondary

4

1%

1%

3

49%

75%

2

43%

24%

1

7%

1%


Teacher knowledge: The role of curriculum, teacher training and teacher beliefs The success of any educational reform initiative will depend greatly on teachers’ willingness and ability to implement or mediate change (Fullan 2001). Teachers’ current knowledge of how languages are learned and their understanding of good practice in language learning is the foundation on which the Ministry will have to build in order to achieve the intended changes in behaviours, beliefs and practices which could lead to improved learning outcomes. This point was raised by several teachers in the questionnaire: ‘To improve teaching and learning in my school the most important factor is the role of the teacher. A knowledgeable English teacher equipped with teaching skills would help in improving the students’ performance in English’ (Rural primary school teacher, Perak) In this section, we will start with a brief look at the suitability of the curriculum and teacher training and professional development programme in terms of their role in influencing teacher knowledge. Related findings from the questionnaire will also be presented. Then we will discuss findings which emerged during the observations and post-observation discussions concerning teachers’ beliefs about learning and teaching, as they provide insight into their knowledge and understanding of good practice.


89

The curriculum and lesson planning In many school environments, the curriculum is an important document that communicates the educational aims, describes the learning outcomes and identifies the pedagogical approaches that are valued. Our review of the primary and secondary curricula indicated that they lack clarity and adequate exemplification of these expected outcomes or the desired pedagogic practices to achieve them (see Section 5 for the full review of the curricula). The quality and user-friendliness of the curricula is crucial as 50% of teachers strongly agreed with the statement that they refer to them when preparing their English lessons. If the curricula are vague and imprecise in their targeting of language and skills focus, it is not surprising that this may possibly be influencing the quality of instruction. The quality of the curriculum may also contribute to the finding that only 38% of teachers strongly agreed that they feel confident in their ability to plan appropriate English lessons. Not surprisingly though, new teachers (less than 3 years’ experience) and non-optionist teachers were less likely to strongly agree (18% and 15% respectively) with this statement than their more experienced and optionist counterparts (27% for teachers with 4-10 years’ experience and 47% for teachers with 11 or more years’ teaching experience and 44% for optionist teachers). Teachers responded least positively when asked whether they find it easy to adapt lessons to cater for different student abilities, with only 25% strongly agreeing. However, when we look at responses from non-optionist teachers, only 16% strongly agreed with this statement. This is an important finding because many teachers commented that their classes are comprised of learners with a range of language levels (and the test data tends to support this). When faced with this situation, being able to modify lessons for stronger and weaker students within the same class is essential. ‘Teaching becomes difficult when too many students with different levels of proficiency are grouped together in a class’ (Urban secondary school teacher, Perak) ‘My students are comprised of some who have exceeded the required level of English for Primary levels, some who are not fluent and some who are weak’ (Urban primary school teacher, Selangor) If teachers are unable to cater for all levels within the class, they could be negatively impacting learner motivation, confidence and outcomes. It would appear that only the secondary curricula for Grades 3 to 5 address the issue of differentiated levels of achievement and provide guidance. Curricula at all levels would benefit from much greater exemplification of differentiated outcomes in the different skills areas (see Section 5 for a full review of the curricula).

Training and professional development Teacher training and professional development activities are one way of disseminating good practice and ensuring that teachers’ knowledge about language learning is current. Teacher training can also be an effective tool in broadening teacher knowledge and encouraging them to try new approaches to teaching. Encouragingly, 75% of teachers have reported attending training activities in the last 2 years. Teachers reported participating most often in training activities that have primarily centred on teaching the skills Writing (12%), Reading (10%) and Speaking (10%). When asked about the impact of the training activities on their practice or knowledge, 22% strongly agreed that the training activities helped them improve their English language ability, 18% strongly agreed that they discovered new ways of teaching as a result of the activities and 17% strongly agreed that the training improved their knowledge of how foreign languages are learned (see Figure 4.8). However, 31% either agreed or strongly agreed that they have found it difficult to use the ideas from the training activities in their lessons. This may suggest that there could be a mismatch between teacher profiles and the type or content of training they have been given.

90


Figure 4.8 Impact of training on teacher practice and knowledge

As a result of participating in training, ... …I have improved my English language ability. Strongly Agree

…I have discovered new ways of teaching English.

Agree Disagree

…I have improved my knowledge of how foreign languages are learned.

Strongly disagree Don't know

I have found it difficult to use the ideas from training activities in my lessons. 0%

20%

40%

60%

80%

100%

The Heads of Panels were also asked about training and professional development in their schools and they indicated that they greatly value continuing professional development opportunities in general and in-service training in particular which focuses on practical teaching techniques. Some Heads of Panel and Head teachers expressed an interest in sharing expertise and learning from peers. This was mentioned in two of the observers’ reports, and is well illustrated by this remark: ‘Half of the Heads… as well as some teachers talked at some length about sharing expertise, within the school and more importantly with other (preferably better) schools. They would like to see a lot more visits by expert teachers and sharing sessions between experienced teachers from different kinds of schools’ (Observer DB) However, despite the generally positive attitudes toward training, Heads of Panel also pointed out some problems with the current system concerning the quality, focus and scope of training and/or professional development: ‘Due to time constraints, the training and professional development is carried out not as professionally and adequately as it should’ (Urban secondary school, Head of Panel, Melaka) ‘[There is] too much training which is not related to English language subjects’ (Urban secondary school, Head of Panel, WP Putrajaya) ‘I think that not only the English teachers should be given training development but we should include all the administrators and other teachers as well as to show support to English language’ (Urban secondary school, Head of Panel, Negeri Sembilan) A concern raised by a policy planner when discussing training and professional development decisions was that it is ‘the state and the district English language officer who identifies the programme for the teacher rather than the teacher identifying their own needs and asking for a programme’. When Heads of Panel were asked a similar question, this issue became clear. Very few Heads of Panel mentioned that teachers were involved in training decisions which may explain why some felt that the benefits of training are not always visible and the aims of training are not always being achieved: ‘There are a lot of courses offered for English teachers but the impact is quite slow’ (Rural school, Head of Panel, Negeri Sembilan)


91

‘Some teachers have the attitude of not applying what they have learned after going on courses’ (Rural primary school teacher, Perak) The limited satisfaction with the current teacher training provision could partially be addressed by a more bottomup approach to professional development, where schools share good practice. In the questionnaire teachers expressed such a desire to share ideas about teaching and learning with other teachers: ‘Educational trips to other schools to see and share our experience of teaching and learning English [to improve teaching and learning within the school]’ (Rural primary school teacher, Pulau Pinang) ‘A teacher exchange programme between nearby schools for a week, for example, would introduce new techniques of learning and answering [i.e. responding] to the students. Maybe good teaching practices from other schools can be implemented in our school and vice versa’ (Rural primary school teacher, Kedah) School exchanges may be more difficult to organise in rural and remote areas, but it seems to be a feasible and useful possibility for consideration for ongoing teacher development. As noted earlier, the success of any educational reform is directly dependent on the effectiveness of teachers in implementing or mediating the proposed change, and professional development can play an integral part in encouraging changes in behaviour and beliefs. The Ministry may need to review their training/professional development programme in order to focus the offerings around the approaches to instructions that they are trying to impact, as well as involve teachers more in training decisions.

Beliefs about learning and teaching The observations and post-observation interviews provided an opportunity to better understand teachers’ beliefs about learning and teaching. The predominant view of learning in the lessons observed appeared to be that of a transmissive input-output model. In this model, teachers are the owners of knowledge, and explain what they know. The learners’ role is largely to receive, file, and store the information they receive. While it is true that some learning can occur in this way, and that views of learning may be culturally mediated and dependent on context (Bax 2003, Holliday 1994), the view of learning apparent from lesson observations does not reflect current views of learning, which stress that learners construct meaning through supported social interaction, and supported problem-solving (Maybin et al 1992, Fisher 2005). This view of learning often results in teacher-dominated lessons which were observed during the observations: ‘Lesson was completely teacher -led, no pairs or groups. It was teacher-dominated, formal, no rapport.’ (Rural secondary school, Form 5, Johor, Observer RC) The preference for teacher-dominated lessons may also be linked to teachers’ language proficiency as this approach allows teachers with limited proficiency to avoid being pushed out of their linguistic comfort zone (Altinyelken 2010). A related finding from discussions with teachers concerning regular teaching practices is that teachers generally appeared to have a limited understanding of the concept of teacher and learner roles. A common view was that the teacher’s role is to facilitate by explaining, with the learner’s role being to understand and/or learn from their mistakes. This view of teaching and learning tallies with the classroom practices observers noticed when observing lessons. In most instances where educational reform fails, there is a lack of coordination between the intentions of the curriculum, the focus of teacher training and instructional realities (Orafi and Borg 2009). Any modifications to the

92


curricula will need to carefully take into consideration teachers’ current knowledge and practice in order to increase adoption of new teaching approaches as well as the beliefs that are associated with these approaches.

Teaching practice Research Question 7: How do teachers teaching at different school stages perform in terms of teaching practice?

A further strand in the project involved classroom observations. The observations gathered both qualitative and quantitative data on teaching practices. The quantitative data was based on a four-band assessment scale of various aspects of teaching practice (e.g. learning atmosphere, focus on language and systems), with Band 4 representing a teacher who demonstrates a good range of procedures and techniques supporting the delivery of a very effective lesson, which provides for learner interaction and challenges and engages the learners. Band 1 represents a teacher who displays awareness of some appropriate procedures and techniques but is unable to implement them consistently, and there is little evidence of learner engagement and participation in the learning process to achieve learning outcomes. The qualitative data was derived through open-ended observer comments about the classroom observations, teacher questionnaire feedback and interviews with teachers and Heads of Panel/Head Teachers. The observed teachers were mostly distributed in the Band 2 and 3 range, as can be seen in Table 4.11. This distribution was seen overall and throughout the specific observation criteria, with the exception of ‘Learning atmosphere’, where the highest proportion of teachers achieved a Band 4 (30%, compared to 12%–19% for the other assessment categories). The higher performance in terms of learning atmosphere is supported by the qualitative data from the classroom observations, which indicated that teachers were generally seen by the observers to try to provide a learning environment where they want learners to be successful and where the teacher-learner relationships are friendly and respectful: ‘The teachers almost all viewed their strengths as their connection to or relationship with the students. They viewed their ability to relate to the students as friends or parent figures as very important’ (Observer DL) Table 4.11 Teaching Practice level Bands: Teachers

Use of English

Monitoring, Feedback and Correction

Overall

19%

16%

18%

33%

33%

32%

35%

46%

40%

40%

44%

42%

10%

9%

8%

8%

5%

Learning atmosphere

Language and Systems

Classroom management

4

30%

12%

18%

3

39%

32%

2

29%

1

3%

Band


The classroom performance of Malaysian teachers can also be compared to worldwide teacher performance on the Cambridge English ‘TKT: Practical’ test, which focuses on practical teaching and is taken by teachers worldwide. The conditions of the Malaysia observations and the above-mentioned test were not identical, but the scale used to assess teaching practice was identical in both cases, making a comparison possible. The results can be seen in Figure 4.9, which indicates that Malaysian teachers appear to be slightly weaker in terms of teaching practice in comparison with teachers from around the world, as seen in the higher proportion of Malaysian teachers achieving Band 2 and a smaller proportion than their worldwide colleagues achieving Band 3 and 4. Such


93

a comparison is useful, as it provides a broader international metric. However, as it is based on a small sample of 78 Malaysian teachers only, this comparison should be viewed with caution. Figure 4.9 Overall Teaching Practice level: Comparison between Malaysia and TKT Practical population 60% 50% 40% 30%

Malaysia Rest of the World

20% 10% 0% 1234

Research Question 8: How do teachers teaching at different school stages perform in terms of teaching practice according to location and primary/secondary school? It is also useful to look at the distribution of bands in the teacher cohort by urban/rural/remote location and by primary/secondary school, although the small sample size has to be kept in mind. Table 4.12 presents a summary of the findings by location and shows that in all categories, teachers in urban schools performed better, with a larger proportion achieving Band 4, compared to rural and remote teachers. However, it is important to note that excellent, effective classes were observed in all three locations, as seen in the Band 4 marks for all three. Table 4.12 Teaching Practice Bands by location: Teachers Learning atmosphere Band

Language and systems


Urban

Rural

Remote

Urban

Rural

Remote

Urban

Rural

Remote

4

37%

24%

25%

17%

9%

0%

23%

14%

13%

3

40%

38%

38%

40%

20%

50%

34%

29%

50%

2

20%

35%

38%

37%

60%

25%

31%

51%

25%

1

3%

3%

0%

6%

11%

25%

11%

6%

13%

Use of English

Band 4

Monitoring, feedback and correction

Overall

Urban 23%

Rural 14%

Remote 25%

Urban 18%

Rural 14%

Remote 13%

Urban 23%

Rural 14%

Remote 13%

3

29%

40%

25%

41%

29%

13%

43%

26%

38%

2

46%

37%

25%

38%

49%

50%

31%

54%

38%

1

3%

9%

25%

3%

9%

25%

3%

6%

13%


The performance of primary and secondary school teachers is given in Table 4.13, and the results indicate that overall, primary and secondary school teachers were comparable in terms of classroom practice. Some interesting differences do emerge, such as the higher performance of secondary school teachers in terms of ‘Use of English’,

94


i.e. the teacher’s appropriate and effective use of English throughout the lesson: 61% of secondary school teachers achieved Bands 3 and 4, compared to 36% for primary school teachers. Table 4.13 Teaching Practice Bands by Primary/Secondary school: Teachers Learning atmosphere

Language and systems


Primary

Secondary

Primary

Secondary

Primary

Secondary

4

24%

33%

16%

9%

24%

15%

3

40%

38%

32%

32%

28%

36%

2

36%

25%

40%

49%

36%

42%

1

0%

4%

12%

9%

12%

8%

Band

Use of English Band

Monitoring, feedback and correction

Overall

Primary

Secondary

Primary

Secondary

Primary

Secondary

4

8%

25%

8%

19%

16%

19%

3

28%

36%

48%

25%

36%

34%

2

48%

36%

36%

48%

40%

43%

1

16%

4%

8%

8%

8%

4%


Exploring teaching practice The section will provide a more detailed discussion of the qualitative findings from the teacher observations, including observer/teacher post-observation discussions, interviews with Heads of Panel and teacher questionnaire responses. Observer comments given to explain the observation score awarded will illustrate some of the excellent classroom practices which were observed, and will also provide insights into areas which some teachers find challenging and which can provide guidance for any future educational transformational programme.

Learning atmosphere This construct relates to teachers’ ability to create a positive learning atmosphere by establishing rapport with learners and ensuring they are fully engaged in activities. The majority of teachers performed well on this aspect of teaching practice as 39% achieved a Band 3 and 30% a Band 4. Many teachers stressed the importance of rapport with learners. Observers noted that most teachers clearly want learners to be successful, and they make every effort to create an inclusive and secure classroom atmosphere as a result. The quote below tallies with the tenor and content of feedback in most of the observers’ reports: ‘Without exception, teachers claimed to love the lesson and love their students. This seemed entirely genuine’ (Observer RC) The questionnaire findings support these observations. In general, teachers are positive about teaching English, as 56% of them strongly agreed that they like teaching English (see Figure 4.10). However, further analysis of the data showed that teachers responded differently to the statement‘I like teaching English’ depending on their teaching experience, the grade they primarily teach, their level of education and their optionist status. New teachers (i.e. with 3 years or less experience) were less likely to strongly agree with this statement (37%) than the other two groups (47% – teacher with between 4 and 10 years’ experience – referred to as teachers with ‘some’ experience in the remainder of this document, and 64% – teachers with 11 or more years’ experience – referred to as ‘experienced’ teachers from this point forward). Teachers who teach Pre-school (who are more likely to be


95

newer teachers) were also less likely to strongly agree (17%) than teachers who teach the other grades (between 53% and 74%). Teachers with a Master’s degree in Education were much more likely to strongly agree (72%) that they like teaching English than the rest (Bachelors of Education: 63%; Diploma in Education: 47% and Certificate in Teaching: 43%). Optionist teachers were more likely to strongly agree that they like teaching English (64%) than non-optionist teachers (24%). Figure 4.10 Teachers’ attitude toward teaching by experience

I like teaching English.

Experienced teachers Strongly Agree Some experience

Agree Disagree

New teachers

Strongly disagree Don't know

All teachers 0%

20%

40%

60%

80%

100%

There seems to be an association between teachers’ experience and education, including specialist training, and their enjoyment of teaching, which could influence the extent to which a positive learning environment is created. It may be necessary to investigate further why new teachers and non-optionist teachers are less positive about teaching English, in order to ensure that they are given the support they may need to be successful and remain in the profession if appropriate. Despite the findings that a positive learning atmosphere was created there were examples of teachers who failed to involve learners fully in the lesson, even though they exhibited a good rapport with learners, and thereby limited opportunities to learn: ‘Teacher's presence was reasonable, bit too dominant. Learner engagement was off and on. Unequal attention to individual learners’ (Rural primary school, Pre-school, Sarawak, Observer JL) ‘Decent rapport, but too dominant throughout. Asked many questions but generally answered them herself. Students were given time to complete tasks on their own, but no time was allowed for practising language in context or real sentences. No pair or group work’ (Rural secondary school, Form 5, Pahang, Observer JL) Clearly, establishing good rapport is only one element of supporting learning and more attention may need to be paid in teacher development programmes to learner engagement. This is particularly important because teachers consistently commented that they did not think learners were motivated: ‘The students are not interested in learning English’ (Urban primary school teacher, Melaka) This viewpoint, when considered alongside other comments by the observers and by teachers relating to teachers’ beliefs in their students’ capacity to learn is somewhat worrisome. Observers noted that there were examples of teachers who may have been limiting their students’ learning opportunities because they had already determined that students were not able to accomplish particular tasks:

96

Cambridge Baseline 2013 Teacher English language proficiency, teaching knowledge and teaching practice ‘In some cases, teachers had already decided that students were not capable of doing well in some sections of the upcoming exams so they focused only on the sections which they could gain marks in’ (Observer DB)

Research has shown that teachers who believe their students are unable to learn a subject may inadvertently adjust teaching behaviour, resulting in this expectation being met (Jussim and Harber 2005). If teachers do not believe their students are capable of learning, it is possible that this belief could affect learners’ linguistic selfconfidence. Another related issue that emerged from the data is that some teachers left comments in the questionnaire that seem to suggest they may believe that students bear sole responsibility for ensuring they have a positive attitude and are motivated to learn English: ‘To improve teaching and learning, students need to show a positive attitude in class’ (Urban secondary school teacher, Selangor) ‘Students don’t make an effort to remember the meaning of words which affects understanding and interest in this subject’ (Rural, secondary school teacher, Pahang) ‘They [students] are lazy to try and always dependent on teachers to help them’ (Secondary school teacher, Melaka) ‘Students choose to remain silent during lessons’ (Rural, secondary school teacher from Sabah) These attitudes could indicate that teachers do not fully appreciate their role in motivating students. This characteristic is often linked to a teacher’s low sense of self-efficacy. Self-efficacy refers to a belief in one’s own ability to achieve a desired outcome (Bandura 1977). In the case of teachers, self-efficacy often manifests itself in the belief that one can change students’ behaviours and help them learn. Teachers with a low sense of selfefficacy are less likely to accept responsibility for motivating learners and may attribute learner failure to faults with the learner rather than their own teaching methods (Dembo and Gibson 1985). As a result, they are less likely to adjust their teaching practices and may be less willing to change their behaviours in response to educational initiatives. Identifying and addressing this issue will be an important aspect of any reform initiative.

Physical features of classrooms The physical features of the classrooms observed are also related to the learning atmosphere. Observers commented that the primary and secondary classrooms which they visited were generally large, light, and in good condition. All had boards, while some had laptops and projectors. In many cases, the laptop in class was the teacher’s own. Owing to the heat and humidity, doors and windows were open in order to provide ventilation. This meant that noise from traffic, the school corridor or in some cases fans sometimes made it difficult to hear the voices of individual learners or a quiet teacher in plenary interaction. One observer wondered if the tendency for teachers to elicit choral responses from learners was a direct consequence of the prevalence of ambient noise. Observers remarked that each class generally has its own designated classroom, and so for each lesson subject teachers move to the classroom of the learners they are going to teach. Desks were generally in rows. Some observers commented on the possibility of using a different classroom layout from the traditional format of desks in rows. This would facilitate more pair/group interaction but may require classroom furniture that is lightweight and allows easy switching between more traditional rows and nests of desks. All observers noticed the absence of visual display (e.g. examples of learners’ work, classroom English phrases) on most classroom walls. Some observers felt that there might be scope to put up posters in English and samples of learners’ work on classroom walls to support learning and motivation. In contrast, observers noticed a marked difference in Pre-school classrooms. One observer commented:


97

‘The walls were universally covered in children’s artwork and colourful child-friendly displays, usually including quite a bit of English’ (Observer RC) Another observer remarked of Pre-schools classrooms: ‘Their dedicated rooms and resources were really impressive’ (Observer BO) The difference with Pre-school classrooms in terms of classroom displays is worth noting, since such a multisensory approach to the physical features of classrooms could possibly be carried over into other primary and secondary classes. This may, of course, be difficult if teachers do not have dedicated classrooms, but is worth considering in any future transformational programmes. In general, the observations revealed a harmonious learning atmosphere in which teachers generally want learners to be successful. However, teachers need to ensure that they are utilising techniques to engage all of their learners and not just those that are already motivated to learn. The physical features of the classroom are also an area that may need further consideration, as the ability to display students’ English work can help learner motivation. In addition, the ability to move the desks could encourage the use of different interactional patterns and increase the amount of communicative interaction within the classroom (see ‘Classroom management’ for more on this point).

Focus on language and/or skills The discussion now turns to a focus on language in terms of form, meaning and phonology and on the development of language skills. The language or skills focused on should be at an appropriate level for learners and linked back to curricular aims. The majority of teachers (42%) achieved Band 2 for this aspect which may be accounted for by the influence of the exams and limited competence in providing learners with appropriate activities that give them sufficient opportunity for communicative practice. A consistent comment by observers and the teachers in the questionnaires was the role of exams in determining the lesson focus. Observers noted that in Form 5 and 6, exams had a negative washback effect on learning, as lesson time was devoted more to passing the exam than to learning English: ‘…most teachers reported they stopped using course books 3-6 months before the exam. In Form 6 there is no curriculum/course book so exam prep work seems to predominate’ (Observer BO) ‘Nearly half the lessons observed were dominated by formal exams. Teachers reported that schools, parents and students expected this exam focus, which reduced the time they could devote to fluency activities’ (Observer NU) Clearly, learners want and need to do well in exams. The strong focus on exams is pragmatic and understandable but it is of very limited educational benefit, as it reduces learners’ opportunities to use and develop skills for English. This is particularly relevant when the exams are not based on a communicative approach to assessment and/or do not include all four skills. In such a case, the skill(s) not included in the exam will likely not receive much attention during lessons, as this comment highlights: ‘One teacher told me that of late she has been required to do more speaking and listening in her lessons, but that the students don’t need it in the exam, “So I’m frightened to have students who can chatter but don’t know anything”’ (Observer RC) Teachers’ comments supported the observers’ impression about the central role played by exams during the interviews and in the questionnaires:

98

Cambridge Baseline 2013 Teacher English language proficiency, teaching knowledge and teaching practice ‘Teachers would prefer a more communicative approach to teaching and learning, but they understand the conundrum that if they don’t prepare the students adequately for the state exams they would be doing the students a disservice, despite the fact that it [sic] is ineffective in helping the students master the language’ (Observer JL) ‘The system of teaching and learning only focuses on students to do well for the exam. The assessment only looks at the percentage of passes and good results. It does not take into consideration whether students really understand it or not. Teachers are forced to teach students how to answer the questions rather than ensuring they really understand it’ (Rural primary school teacher, Perlis)

It would appear that current exams are narrowing the language focus in the classroom which is a concern that was raised by policy planners during interviews. Policy planners pointed out that teaching to the test is very widespread, which affects the quality of teaching and learning. Because all the current exams, with the exception of MUET, focus only on Reading and Writing, the other skills (i.e. Listening and Speaking) are not receiving the attention they deserve during class time (see Section 5 for the full review of exams). The format of these exams, which rely heavily on multiple-choice items, may also limit the extent to which teachers focus on interactive language instruction. Not surprisingly, the test data confirms that Speaking is the learners’ weakest skill, as discussed in Section 3. (It is worth noting that the language exam ELSA, due for release in late 2014, does include all four language skills and is likely to be a positive influence on the development of Speaking skills.) Turning now to the opportunities given to students to practise the language focus of the lesson, only 20% of teachers reported that students speak to them in English very often and even fewer indicated that students speak to each other very often in English (5%). In the questionnaire, teachers commented on some of the reasons for this, which primarily relate to confidence and lack of exposure to English outside the classroom: ‘Students in rural areas are shy in speaking English with their friends and teachers. Their confidence needs to be built up’ (Rural, primary school teacher, Selangor) ‘Students are afraid to speak in fear of being laughed at by peers’ (Urban secondary teacher, Selangor) ‘Students can understand teachers’ instructions but are reluctant to respond in fear of making mistakes’ (Urban secondary teacher, Johor) ‘The exposure to English is confined to the four walls of the classroom’ (Rural secondary teacher, Selangor) However, as students have very little exposure or opportunity to use English outside the classroom, it is important that teachers create activities that both encourage learners to use English in a meaningful way and ensure that the input is at the appropriate level. Unfortunately, observers pointed out that some teachers were not necessarily selecting a language focus at an appropriate difficulty for learners. Some observers noticed that there was an absence of challenge for learners in the classroom learning tasks: they felt these were well within learners’ capabilities for the most part. One observer commented: ‘Of the lessons I saw, none were too difficult and many were too easy for the mid-level of the class’ (Observer NU) In other instances, learners were kept busy throughout the lesson but the observers questioned the pedagogical value of the tasks, as they felt that they were insufficiently challenging. Interestingly, during some interviews, teachers themselves seemed to agree as they were not always sure about the learning value of the tasks they provided learners with. Some of their comments suggested that they are aware intellectually of the need for differentiation of tasks according to learners’ level, but aren’t sure about how to do this in class. These comments


99

link back to the finding that some teachers do not feel confident in adapting materials to cater for different learner abilities.

Classroom management Classroom management refers to a teachers’ ability to set up, manage and pace whole-class and individual, pair or group activities including the appropriate use of materials, so that the lesson aims are achieved. This is another important aspect of teaching practice. Most teachers felt that they were able to keep learners busy in class and to get learners to engage with content but as mentioned previously, the classroom observers noted that the value of some of the learning tasks was questionable. Most teachers found it hard to identify lesson aims, with a tendency to view them as being synonymous with completion of lesson activities. Observers reported that some teachers measured success in a lesson by quantity of output from learners rather than quality of output. This analysis is consistent with the absence of supportive teacher talk such as reformulation of learners’ responses or corrective feedback (see ‘Monitoring, feedback and correction’ in this Section). Some observers commented that lessons were dominated by the teacher: ‘Lesson is 98% Teacher language. Lesson is basically a complete recitation of about 20 rules relating to use of English articles. Explanations for too disjointed, numerous and complex and go completely over students’ heads’ (Rural secondary school, Form 3, Ke dah, Observer BO) And, in general, observers noted that teachers used a limited range of techniques: ‘Lesson had promise but opport unities to provide useful practice were missed as language produced in lengthy elicitation sequence not focused on, practised, corrected or consolidated to any discernible purpose’ (Urban primary school in Perak, Year 6, Observer BO) ‘Lesson included some useful activities, but was at times ineffective. Learner engagement and participation in the learning process was minimal. Opportunities were missed to include all learners, elicit, encourage and praise learners’ (Urban secondary school, Form 5, Pahang, Observer JL) ‘Students got controlled practice in the grammar rules but no chance to produce full sentences themselves, either written or spoken’ (Rural secondary school, Form 3, Selangor, Observer DB) ‘Much higher levels of student engagement and participation could be generated by working context and exploring student answers and probing their experiences’ (Urban secondary school, Form 6, Kedah, Observer BO) Being comfortable with using a bigger toolkit of techniques is important, since it allows greater opportunities for language to be used in meaningful and interactional tasks in the classroom which go beyond a limited focus on language form and controlled practice. Such useful practice of the language would increase learner engagement and support learning.

Use of English Findings from the attitudinal questionnaires suggest that learners have little exposure to English outside the classroom, which implies that teachers are the main language model. Therefore, the language used in lessons needs to be both accurate and appropriate.

100


When teachers were asked how often they teach in English, 75% said they do this very often and observers found this to be case. However, observers noted that some teachers’ English was not accurate or fluent, and that some had unclear pronunciation at times. It was also noted that teachers generally did not use audio recordings to provide learners with an accurate model of dialogues or other input from the textbooks. The following comments reflect this point: ‘Teacher does not always model words clearly and repeat sufficiently to ensure all learners are producing correctly. Teacher explanations and corrections sometimes above learner level …’ (Remote primary school, Pre-school, Perak, Observer BO) ‘The teacher used some nice visuals, strong drilling and matching to teach the words – market, hospital, garden, police station – meaning, pronunciation and written form. But the lesson stopped here with very little teacher language apart from single words and short phrases such as ‘ok good’, ‘this one’, ‘you’, ‘up your hand’…’ (Rural primary school, Pre-school, Selangor, Observer DB) ‘The teacher is held back by her language skills. Her poor grammar and vocabulary and control of prosodic features lead to inaccurate examples and modelling’ (Urban secondary school, Form 3, Pahang, Observer JL)

Teachers’ command of English is an important aspect of their competence, alongside their pedagogic knowledge, since English is both the medium of instruction and the subject of instruction in English lessons. A good teacher does not necessarily need advanced proficiency in English, but the appropriate use of English in the classroom is essential, since learners learn from good language models. It is important, therefore, to ensure not just suitable English language proficiency of teachers, but also to provide training which raises teachers’ skill in and awareness of the importance of grading their language in the classroom so that it is comprehensible for learners.

Monitoring, feedback and correction Research suggests that it is important for learners to receive appropriate and comprehensible feedback as it helps to increase noticing, which can lead to self-correction (Lyster and Ranta 1997). In other words, if learners do not notice aspects of a language, they will be less likely to learn them. Knowing when to intervene in a learning activity to provide feedback and the use of a variety of correction techniques are important characteristics of good practice. In many primary and secondary classes supportive teacher talk was absent. For instance, teachers often did not provide learners with feedback on their contributions, or correct learner utterances when appropriate. Most observers commented on the premium placed on avoiding errors at all costs in class. Though an interest in success orientation is laudable, this view of error is also not congruent with current views of learning which stress that learners generally achieve linguistic accuracy at the end of a process which may involve trial and error rather than at once (Lightbown and Spada 2013). The following comment from an observer highlights this issue: ‘Teacher uses a limited range of correction techniques, preferring to jump in and correct rather than elicit a corrected response’ (Rural primary school, Pre-school, Perak, Observer BO) In addition, some teachers mentioned a fear of mistakes in their lessons, due to concerns about learner selfesteem and the potential effect on the atmosphere of the class. ‘One teacher said that much of what she does in the classroom is coloured by the fear that her students will make mistakes. When this occurs she feels that she had let her students down badly and exposed them to a negative learning outcome and the possibility of humiliation’ (Smith 2013)


101

In general, teachers were missing opportunities to provide feedback and ultimately missing opportunities for learning to take place: ‘More feedback needed on the language produced by the students, both during group work and during moments of individual production’ (Urban secondary school, Form 3, Pahang, Observer JL) Teacher explanations too lengthy and opportunities are missed to question, probe, elicit and to write key language on the board (Urban secondary school, Form 6, Kedah, Observer BO) The observation findings suggest that teachers lack skill in and awareness of supportive teacher talk. A teachers’ ability to monitor learner language as well as provide appropriate and timely feedback is important for the linguistic development of their learners. Not only does the learner who receives the feedback learn, but also the other learners in the class. The ability to use a range of corrective techniques was also found to be lacking. If teachers are able to develop their repertoire of corrective techniques, this will help learners to notice errors and to begin self-correcting, which could improve learning.

Monitoring instructional quality One issue that directly influences the extent to which changes in teaching practices are likely to occur relates to how schools evaluate instructional quality. This issue was discussed both with teachers and Heads of Panel. Some interviewees claimed that lesson observation for professional development within school is generally carried out by a senior administrator rather than a teacher. This suggests that lesson observations may be viewed more as a duty which complies with administrative procedures than as a potential means of professional learning. More troubling is that in some schools the only measure of teacher quality is student performance in exams. This may be exacerbating the finding that teachers appear to be overemphasising exam preparation. This, in turn, may be leading to potentially negative washback on classroom teaching practice: ‘I monitor English teaching quality based on the students’ performance in examinations’ (Rural secondary school, Head of Panel, Melaka) ‘Good English teaching quality can be monitored by the achievement of examination results’ (Rural secondary school, Head of Panel, Kelantan) These comments suggest that the current approach to monitoring instructional quality may need further attention to ensure that a range of activities is used to evaluate teaching performance and that a well-designed observational programme which focuses on supporting learning is included. However, one challenge to implementing an observation programme is that most teachers found it difficult to reflect on their lessons. Observers commented that in part this seemed due to unfamiliarity with the concept of reflection, the unusual circumstances of discussing a lesson with an unknown observer, and to low levels of English when an interpreter was not available. There was a tendency instead for teachers to talk more about their general teaching practices. Reflection is considered an important feature of successful professional development programmes (Richards and Lockhart 1994); therefore, this professional skill will need to be developed and nurtured in training programmes in order to achieve the transformative change that is described in the Education Blueprint.

Challenges to improving teaching Although teachers were very positive about teaching, there was a great deal of concern raised by them in the questionnaire and during post-observation interviews about large class sizes, the range of levels within a class and the fact that they are feeling overburdened by administrative paperwork, all of which have an impact on their lesson planning:

102


Large class sizes ‘Our classes have too many students, averaging 35-38 students’ (Urban secondary school teacher, Kelantan)

Range of language levels in classes ‘The weaker students must be helped; the better ones should be given opportunities to excel’ (Rural secondary school teacher, Kedah) ‘My students are comprised of some who have exceeded the required level of English for Primary levels, some who are not fluent and some who are weak’ (Urban primary school teacher, Selangor)

Non-teaching responsibilities ‘Give teachers less paperwork so they can focus on teaching English properly’ (Urban secondary school teacher, Terengganu) ‘Teachers are trying their best to improve the quality of teaching and learning but their workload is increasing and it is affecting the effectiveness of teaching and learning in class’ (Urban secondary school teacher, Johor) ‘Reduce teachers’ workload so that they can focus on preparing lesson plans… Preparation of materials is important. Under pressure, the teacher might choose unsuitable/inappropriate materials. Materials from professionals would very much assist teachers’ (Rural secondary school teacher, Kelantan) ‘Reduce non-English related workload of English teachers so they can focus more on preparing and teaching students effectively’ (Urban primary school teacher, Selangor) During times of educational reform, teachers need to be given the opportunity to reflect on changes so that they have a thorough understanding of the principles and practices underpinning those changes (Richards and Lockhart 1994). Large class sizes which also contain a wide range of language levels within them can make it challenging to teach appropriate lessons for all students in the class. However, this situation is not unusual and can be overcome if teachers have the necessary skills to prepare differentiated lessons and the time to design these lessons. The last challenge mentioned above related to non-teaching responsibilities may be the one area in which the Ministry can affect the most change. If the amount of administrative paperwork is reduced for teachers then they can focus on improving their lessons and the quality of instruction.

Examples of teaching excellence Having presented the findings concerning teacher practice, we would now like to end this section by showing examples of teachers performing to a high level in terms of international standards in communicative language teaching. Teachers at all five school grades of interest achieved high marks, and they can be seen as excellent models of teaching and learning. Indeed, a similar point was also made in the Education Blueprint regarding education in Malaysia in general, which noted that ‘excellent examples exist across the system’ (2012:E7). Some of the observer comments are reproduced below:


Pre-school ‘This was a lively class in which students were very

‘This was an action-packed lesson that had a hyper

motivated, enjoyed reading together, increased

group of five-year-olds happily and productively

their range of vocab (relating to Deepavali), revised

performing a range of tasks in English. Effective

some other common words and phrases. The

learning was in evidence and learners performed a

teacher managed the class very well’ (Urban school

range of 'Starters' type tasks [a Cambridge English

in Selangor, Pre-school, Observer DB)

test for young learners] with good results’ (Rural school in Pulau Penang, Pre-school, Observer BO)

Year 6 ‘Teacher monitored well enough, although the

‘Excellent learning atmosphere, use of humour,

noisiness of the class made this hard. She used some

good rapport. Good and varied eliciting. Close

good correction techniques, highlighting the wrong

monitoring and immediate feedback to students.’

word or the kind of error made, in order to get the

(Urban school in Melaka, Year 6, Observer RC)

students or a classmate to correct themselves’ (Urban school in Selangor, Year 6, Observer DB)

Form 3 ‘Effective range of procedures and techniques used

‘Language was well-graded. Difficult concepts were

to deliver a lesson which engaged and challenged

simply explained using examples directly related to

the learners. Clearly a well-planned lesson’ (Urban

students. Frequent random comprehension checks.

secondary school, Form 3, Sarawak, Observer JL)

Continual feedback provided on language, concepts, material, understanding. Attention given to individual students. Very encouraging [teacher]’ (Rural secondary school, Form 3, Pahang, Observer JL)

Form 5 ‘Teacher language was clear, simple and

‘This was a very useful lesson for students to

enthusiastic. Teacher spent a lot of time checking,

prepare them fully for an exam question (and

eliciting, modelling, joking. Clear model and diction,

related questions) which is very likely to come up.

well-modulated. Teacher monitored and interacted

Not only this, but the students gained valuable

selectively as well as correcting during pair work.

practice in presentation skills, fielding questions,

Engaged and debated with students; tried to draw

interacting and discussing controversial subjects.

them out rather than just correct or give answers’

The teacher encouraged them in a natural and

(Urban secondary school, Form 5, Kelantan,

appropriate way, allowing all to shine… The teacher

Observer NU)

stood back to allow a free open class discussion, with just the right amount of intervention to encourage quiet students to speak up, stop verbose ones, and generate discussion on key points related to tolerance and acceptance’ (Urban secondary school, Form 5, WP Kuala Lumpur, Observer DB)

103

104


Form 6 ‘Teacher and students thoroughly engaged

‘Excellent working atmosphere, teaching well -

throughout. Teacher is a very positive presence and

geared to learners' ability and students fully

gives appropriate attention to group/individual

engaged. Teacher monitors constantly and provides

learners at all times. A good range of activities and

appropriate feedback and correction, including self-

techniques for an effective lesson. Learners are

and peer-correction’ (Rural secondary school, Form

engaged and challenged throughout. Effective

6, Terengganu, Observer NU)

learning and language practice achieved competently’ (Urban secondary school, Form 6, Negeri Sembilan, Observer JL)

Discussion Although teacher knowledge and practice were comparable to the rest of the world and there were examples of excellent practice, the observations highlighted some areas that are in need of support and further development. In particular, teachers’ beliefs about learning could be influencing their choice of a teacher-dominated approach to their lessons. The curricula and training provided may not adequately encourage teachers to reconsider these beliefs or equip them with the skills or strategies to try other approaches. The lesson observations provided an opportunity to better understand teaching practices in order to identify both their strengths and weaknesses. Teachers were uniformly strong in establishing good rapport with students but they were generally much weaker in planning, managing and monitoring learning. In some cases, their limited language ability and/or limited skill in using graded language suitable for their learners was negatively impacting their effectiveness. In general, teachers used a limited range of techniques and tended to keep learners busy without considering the value of the activities used. This finding is directly relevant to the finding that in general teachers lack the ability to reflect on their practice and to the general lack of observation for formative purposes. Teachers’ attitudes to training are very positive and they do seem to want to improve their knowledge and practice. The Ministry can capitalise on this desire for training and sharing good practice by facilitating and encouraging schools to engage in collaborative activities with other schools. They can also support the introduction of a more bottom-up approach to sharing good practice where schools feel empowered to collaborate in teacher development.


105

The role of assessment in learning and teaching It is widely accepted that teachers play an important role in learner achievement through the provision of quality instruction (Mashburn et al 2008). However, as seen in the previous sections, many factors such as curricula, school culture, teacher training and professional development, teaching resources and examinations can shape the learning environment which in turn can influence both the quality and focus of instruction. In this section, we will investigate the role of assessment in learning and teaching by presenting the relevant findings from interviews, observations and questionnaires.

Assessment Assessment has multiple functions in education, including the measurement of achievement, public accountability and providing feedback to learners, and it also tells learners what we value or what they should pay attention to (Boud 2000). Therefore, assessment plays an important role in education. Teachers’ attitudes towards assessment, confidence in assessing learners and their assessment practices were investigated in the questionnaire and during post-observation interviews. Teachers recognise the importance and need for assessment as they frequently made comments about this in the questionnaire:

‘Assessment is important to test the command of the language’ (Rural primary school teacher, Sabah) ‘Without exams, students are not motivated to study English’ (Rural secondary school teacher, Pahang) ‘I can identify the strengths and weaknesses of my students from their exam results’ (Urban primary school teacher, Sarawak) During interviews they indicated that they design their own informal tests regularly. Some teachers reported carrying out formative assessment, although not using that terminology. One Cambridge observer ( Observer NU) wondered whether this may partly account for the systematic use he saw of reading aloud and feeding back on discussion or written tasks, so that teachers could collect an adequate sample of each learner’s language for the purpose of informal assessment. This observer went on to explain, however, that the extensive collection of information from classroom data formed the basis for only generalised judgements of class performance against lesson aims. In most instances, he claims, this information was not then translated into specific formative assessment feedback for individual learners. It is not surprising that the observers found teachers’ informal assessment practices were limited, because when asked in the questionnaire about their confidence in assessing their learners, teachers did not indicate strongly that they were confident (see Table 4.14). Differences in confidence were also based on teaching experience and school location. New teachers are much less confident assessing their students English language ability across all skills and systems than the most experienced teachers (see Table 4.14). This difference in confidence level was also found when comparing teachers who work in urban schools versus those in rural/remote schools. Teachers from urban schools reported being more confident assessing their students’ English (average of 27% across all skills and systems for strongly agree) than their colleagues who work in rural/remote schools (average of 17% across all skills and systems). Both findings are statistically significant.

106


Table 4.14 Teachers’ confidence in assessing learners by teaching experience Confidence in assessing learners

Years of experience

Strongly Agree

Agree

Disagree

Strongly disagree

Don't know

I feel confident in my ability to develop an appropriate classroom test for my students.

3 or less

9%

76%

12%

0%

3%

4-10

13%

76%

11%

0%

1%

11 or more

18%

75%

7%

0%

0%

I feel confident assessing my students' reading ability.

3 or less

10%

83%

5%

0%

1%

4-10

18%

79%

3%

0%

0%

11 or more

28%

71%

1%

0%

0%

3 or less 4-10

12% 17%

77% 76%

8% 7%

2% 0%

1% 0%

11 or more

28%

70%

1%

0%

0%

3 or less

12%

78%

7%

1%

2%

4-10

18%

74%

7%

0%

0%

11 or more

26%

71%

3%

0%

0%

3 or less

13%

70%

14%

1%

1%

4-10

16%

75%

9%

0%

0%

11 or more

26%

71%

3%

0%

0%

3 or less

14%

71%

13%

1%

1%

4-10

16%

76%

7%

0%

0%

11 or more

26%

71%

3%

0%

0%

3 or less

13%

72%

13%

1%

2%

4-10

15%

74%

10%

0%

0%

11 or more

27%

70%

3%

0%

0%

Istudents' feel confident assessing writing ability. my

I feel confident assessing my students' listening ability.

I feel confident assessing my students' speaking ability.

I feel confident assessing my students' vocabulary knowledge.

I feel confident assessing my students' grammatical knowledge.


Learning oriented assessment Learning oriented assessment (LOA) represents an approach that recognises that the main function of assessment, whether formative or summative in nature, is that it should improve learning (Carless 2009). LOA involves ‘the collection and interpretation of evidence about performance so that judgements can be made about further language development’ (Purpura 2004:236). This approach to assessment requires learners to be involved in assessment through self/peer-assessment as well as by using the feedback they receive from different sources (e.g. teachers, peers, tests) to decide on what they need to do next. LOA practices have the potential to increase learner autonomy, motivation and engagement, and as such the use of LOA practices was investigated in the questionnaire. Table 4.15 shows the extent to which primary and secondary school teachers engage in LOA practices. Although teachers frequently indicated that they believe learners are responsible for their own learning, they are less likely to report engaging in practices that will equip their learners to actively take responsibility for their learning. For example, learners will find it difficult to set appropriate language learning goals if they do not fully understand the learning objectives or the criteria on which they will be evaluated. Similarly, learners need appropriate feedback in order to understand their learning needs and be more autonomous. The observation data indicated that teachers often did not take advantage of opportunities to provide correction and the corrective techniques that they did use tended towards explicit correction.

Cambridge Baseline 2013 Teacher English language proficiency, teaching knowledge and teaching practice Table 4.15 Learning oriented assessment practices reported by teachers Learning oriented assessment Strongly School level Agree practices Agree

Disagree

Strongly disagree

Don't know

I believe students have a responsibility for their own learning. I discuss learning objectives and/or learning outcomes with my students. I tell my students about the criteria on which they will be evaluated.

Primary

41%

52%

6%

0%

0%

Secondary

54%

43%

4%

0%

0%

Primary

16%

69%

15%

0%

0%

Secondary

17%

71%

11%

1%

0%

Primary

28%

65%

6%

0%

1%

Secondary

38%

61%

1%

0%

1%

I collect information about my students' progress in English from their work (e.g. exercise books, homework, projects, etc.) I assess my students using methods other than tests.

Primary

32%

66%

2%

0%

1%

Secondary

29%

69%

2%

0%

0%

Primary

18%

73%

8%

0%

1%

Secondary

13%

80%

7%

0%

0%

After my students take a test, I give them feedback on their strengths and weaknesses.

Primary

34%

61%

4%

0%

1%

Secondary

40%

58%

1%

0%

0%

107


Finally, LOA recognises that teachers can monitor learner progress from both informal and formal assessment activities. In addition, learners can use the feedback from both to decide, in collaboration with their teacher, what learning activities they should pursue next. Teacher and Head of Panel comments indicated that in some cases, students do not receive their results from exams. This prevents assessments, which have the potential to be important learning events, from feeding back into learning: ‘Results from the assessment should be given to the students so that they can improve their weaknesses’ (Urban secondary school teacher, Selangor) ‘Parents complain that their children do not have tests that they can see the score’ (Urban school, Head of Panel, Selangor)

Assessment practices Table 4.16 shows teachers’ responses to statements related to their assessment practices and their attitudes towards national/international tests. Students were also asked similar questions and their responses indicate that despite teacher reports that the education system is heavily exam focused, they do not have negative views about assessment. Only 20% of students strongly agreed that they worry about taking tests while 34% strongly agreed that they work harder in class when preparing for a test. In addition, 51% of students strongly agreed that tests help them understand their strengths and weaknesses. In fact, 25% of students strongly agreed that they like taking tests while 30% strongly agreed that they would like to take an international test of English. These findings suggest that tests could be used to motivate and support learning, as students are not reporting high levels of anxiety and appear to recognise their value as a source of feedback.

108


Table 4.16 Teachers’ attitudes to assessment by school level Strongly Agree

Agree

Disagree

Strongly disagree

Don't know

Primary

19%

66%

14%

1%

0%

Secondary

13%

68%

20%

0%

0%

Primary

6%

19%

65%

11%

0%

Secondary

4%

28%

61%

8%

1%

Primary

18%

64%

16%

1%

1%

Secondary

24%

61%

13%

1%

1%

Primary

12%

55%

27%

3%

3%

Secondary

23%

58%

16%

1%

2%

Primary

21%

70%

6%

1%

2%

Secondary

20%

69%

11%

0%

0%

Primary

18%

51%

26%

3%

2%

Secondary

24%

61%

14%

0%

1%

Attitudes to assessment

School level

I regularly give my students tests to monitor their progress in English (e.g. weekly/monthly). I only test my students' English at the end of term. When my students are preparing for a test, I have noticed that they work harder in class. My students worry about taking English tests. The national English tests my students take (e.g. UPSR, PMR, SPM) are at an appropriate level of difficulty for my students. International tests are important to show that my students have achieved an internationally recognised level of English.


When teachers were given the opportunity to comment on their own assessment practices or school-based assessment in the questionnaire, the majority of their comments focused on the following four points:

1. The importance of continuous assessment: ‘Assessment needs to be done from time to time so that students are ready at all times’ (Urban secondary school teacher, Selangor) ‘Continuous assessment is needed to measure students’ progress consistently’ (Urban secondary school teacher, Johor) ‘Assessment should be done regularly to assess ongoing progress of students’ (Rural primary school teacher, Sabah)

2. Challenges to assessing learners – time: ‘A good and effective test instrument needs adequate time to prepare. Teachers nowadays are burdened with too much other unnecessary paperwork’ (Rural primary school teacher, Sarawak) ‘Written and oral testing takes too much time’ (Urban secondary school teacher, Pahang)

3. Challenges to assessing learners – school-based assessment: ‘PBS assessment is not accurate in determining the students’ abilities and is very burdensome for the teachers’ (Urban secondary school teacher, Pulau Pinang) ‘PBS online is very troublesome for teachers’ (Rural primary school teacher, Selangor)


109

4. Negative washback of exams: ‘Teaching and learning is too exam-focused’ (Urban primary school teacher from Pahang) ‘The system of teaching and learning only focuses on students to do well for the exam. The assessment only looks at the percentage of passes and good results. It does not take into consideration whether students really understand it or not. Teachers are forced to teach students how to answer the questions rather than ensuring they really understand it’ (Rural primary school teacher, Perlis) Although teachers understand the role of assessment, they face challenges in terms of time, resources and confidence in their own assessment practices. They did also make comments concerning the validity, reliability and fairness of the national exams which will be discussed in Section 5.

Discussion Assessment appears to have a prominent role in Malaysian classrooms. Despite the fact that the current exams seem to be narrowing the curriculum, both teachers and learners recognise the importance of assessment in learning. Students appear to be motivated by taking exams and understand that they can use their results to help them understand their current level. However, teachers, especially new teachers and non-optionist teachers, do not feel overly confident in assessing their students and feel overburdened by the current school-based assessment programme. The observations found that although teachers value formative assessment, they do not seem to be adept at implementing it or using the information collected to provide learners with feedback. If exams are based on a communicative approach, then it is likely that they would have a positive washback effect on classroom practices. Teachers, students, and it would appear, parents are already focused on exam results, so providing better exams may be a first step in creating the changes in behaviour outlined in the Education Blueprint.

110

Cambridge Baseline 2013 Curricula, learning materials and national examinations

5.

Question 9: What are the features of currently used learning, teaching and assessment materials (e.g. curricula, textbooks, national tests) according to international standards, e.g. the Common European Framework of Reference, and according to current trends in teaching practice?

Key findings Curricula Some good recent improvements but uneven design across all curricula Poor alignment with international standards A need for delineation between learning outcomes and more precise targeting of language and skills focuses Speaking and Listening need more emphasis

Learning materials Poor grading and targeting of grammatical and functional focuses Little opportunity for guided discovery Limited opportunities for integrated task and project work

Assessments Poor alignment with international standards An emphasis on Reading and Writing over Speaking and Listening Many design problems Current exams are having a negative impact on teaching and learning

Curricula The curriculum defines the key pedagogical principles that are meant to inform teaching practice. As such, the curriculum fundamentally tells teachers what is important and what they should focus on. It can help shape their thinking about language learning and pedagogy. Research also suggests that learner achievement is linked to the quality of the curriculum in terms of exemplifying a clear progression of instruction and materials from one grade to the next (Kennedy, Mullis, Martin and Trong 2007). Therefore, a key strand of this project was to review the primary and secondary curricula to determine to what extent they are fit for purpose. In addition, the review focused on whether the curricula provided teachers with the necessary information and guidance they need to be effective teachers of English. Cambridge English reviewed curricula for Pre-school, Primary Levels 1 and 2 and Secondary school Forms 1 to 5. The review also included Teacher Guides produced for primary Years 1 to 3.

Primary curricula Teachers in Malaysian primary schools said during post-observation interviews and in the questionnaire, that they make regular reference to curricula documents for planning purposes and rely more on such documents than


111

textbooks for an overview of the teaching content for the year. However, our reviews showed that in general, the primary curricula lacked clarity and precision. For example, the Pre-school curriculum lacks a clear overview section and Primary Level 1 and 2 curricula do not successfully set out the key pedagogic principles that inform the delivery of English instruction for these grades. However, there were some positive features identified in the review. These included a generally accessible layout allowing some teachers to do some cross-referencing, and topics that were on the whole educational and age-appropriate for learners.

Reflecting international standards Generally, the curricula are not strongly consistent with international standards and they do not seem to have a clear perspective on fundamental issues in primary second language pedagogy. These issues include: the role of teacher talk the relationship between first and second language literacy development the extent to which the instructional focus should be balanced between meaning and accuracy-focused work approaches to teaching grammar in the early years course programmes methods of reading instruction the integration of digital forms of literacy.

Learning outcomes The curricula do not seem to have a very clear delineation between different learning outcomes within and across each school year. In addition, there does not seem to be a clear specification of the communicative purpose and content of the learning objectives. Early primary learning appears to be based largely on ‘look and say’ ‘say it right’, ‘say-spell-say’ and chanting activities with limited opportunity for meaning-based focuses related to re al language use. Within each school year learning objectives are not clearly situated in relation to age-appropriate, cross curricular, meaning-focused forms of activity. The general pitching of literacy objectives seem more appropriate to an L1 rather than an L2 learning context. In addition, the purposes of Listening and Reading comprehension are not always made clear in the curricula. For example, comprehension tasks could cater for the development of Higher Order Thinking Skills but this is not sufficiently exploited in the existing curricula.

Progression Across levels within these curricula there are some instances of progression indicators. However, this approach is not a consistent one. There are considerable areas of overlap when comparing one curriculum to another. Some learning outcomes also seem to be introduced randomly in a particular school year rather than evolve out of or into a related outcome in previous or subsequent school years. One example of this is in Year 5, where learners are expected to identify idioms. This does not seem to link clearly to Year 4 or Year 6 learning outcomes. In other cases, the introduction of objectives seems rather late in the learning cycle. For example, in the Level 2 grammar module, children are expected to construct imperative sentences correctly, and use present continuous verbs correctly and appropriately, yet the Pre-school and Level 1 curricula are based around activity contexts in which the use of such structures is implicit.

Transparency and user-friendliness The review found that the overall organisation of the curricula would benefit from a clearer, shared organisation. For example the Level 1 and Level 2 curricula do not have a clear reference progression from the objectives of the previous curriculum document as their starting point. Similarly, ‘Content Standards’ and ‘Learning Standards’ do not reference learning outcomes within the same time frame, which means teachers cannot see the objectives in mind for the short and medium as well as the longer term.

112


The documents are not as user-friendly as they could be because they were found to lack detail in the specification of learning objectives, the exemplification of teacher input and the expected learner language. Finally the wordlists for Pre-school and Primary levels lacked a coherent, unifying organisation.

Teacher guides Cambridge English also reviewed teacher guides designed to support the implementation of the curriculum for Years 1, 2 and 3. The findings of this review can be broadly summarised as follows:

While providing clear support on planning processes and sample lesson plans, there is very little guidance on the ways teachers might enact the curriculum through exemplification of classroom processes and activity types relating to creative activities, bringing the real world into the classroom and developing children’s thinking skills. There is currently no section relating to the rationale of the wordlists and how reference to such appendices might usefully inform classroom practice, the selection of course material and the design of formative and summative assessments. While there is quite a clear structure in the guides, the user-friendliness of these documents could be improved, with improvements including more signposting, a more direct authorial voice, and improved proofreading.

Secondary curricula: Forms 1 and 2 It is difficult to assess the curricula for Forms 1–2 and Forms 3–5 as a whole as they were clearly developed at very different times and from very different perspectives as to what a curriculum document should encompass. For this reason, we present our findings in terms of Form 1 and 2 curricula and Forms 3 to 5 curricula.

Reflecting international standards The Forms 1 and 2 secondary school curricula appear to predate the CEFR and, as such, do not reflect international standards and are outdated in terms of current beliefs about learning and teaching. They also do not provide any meaningful exemplification of learning outcomes, standards or pedagogical practices in the classroom.

Learning outcomes The Forms 1 and 2 curricula lack meaningful exemplification of learning outcomes and have a rather narrow thematic focus for the age group for a whole year, e.g. Community, Town and Village. This does not reflect the desire in the Education Blueprint for English teaching to provide a window on the world. In addition, they have content focuses which are dated and not age-appropriate, e.g. fill in banking and order forms. There is also a lack of specification of comprehension purposes when engaging with different types of Listening and Reading texts and the related, differentiated skills development outcomes. There is insufficient scope for meaningful and challenging integrated task and project work. This is important because this type of work can support the cross-curricular attainment, leadership and ICT goals targeted in the Education Blueprint. The fact that these curricula have not been recently reviewed means that they make no reference to the development of digital literacy in and through English, nor do they reference the integration of digital media into learning processes.


113

Progression As with the primary curricula, there is a clear absence of links with adjacent grades and there is no clear indication of learning progression. There are a number of instances of duplication from previous curricula.

Transparency and user-friendliness The curricula for Forms 1 and 2 are not teacher-friendly because they consist of extensive lists. These make it cumbersome to cross-reference indicators within a particular curriculum and with previous (Level 2) and subsequent (Form 3) curricula in the school system. Finally, there are no teacher guides to assist teachers with using the curricula as a planning tool.

Secondary curricula: Forms 3 to 5 Reflecting international standards The curricula for Forms 3 to 5 on the other hand, are consistent in some respects with good pedagogic practice and international standards in that they focus on areas of language use, specify learning outcomes, state differentiated levels of achievement and provide exemplification of activities to be worked on in relation to broad topic domains. However, the absence of skills increments for Reading, Writing, Listening and Speaking means that descriptions of progression across levels and across years are somewhat arbitrary.

Learning outcomes Learning outcomes are divided into three areas of language use: the interpersonal, the informational, and the aesthetic. This gives learning outcomes a more communicative and purposeful feel. This is clearly reflected in examples of teaching activities given within the curricula and reflected in the organisation and activities in textbooks provided for evaluation. Within the Form 3 to 5 curricula, the learning outcomes are expressed as a mixture of broad functional, topic, activity and skill focuses but level attainment specifications are very hard to discern. Thus, level specifications (1, 2, 3) relating to the attainment of particular learning outcomes often appear random in how they are set across levels. For example, in the Form 3 curriculum the taking part in social interaction specifications are given as Level 1: expressing hope, Level 2: writing notes, Level 3: discussing plans. At other times, level attainment specifications do not provide specific standards for differentiation. In the Form 4 curriculum the specification for obtain information for different purposes for all 3 levels is simply given as Listening to and understanding a variety of texts.

Progression There are a number of instances of overlap between these and preceding curricula. As mentioned above, the arbitrary nature of level specifications relating to attainment of outcomes leads to a lack of clear progression, evident within and across the curricula. In addition, it is difficult to perceive progression because some learning outcomes are worded more like descriptions of activities rather than statements of what students will have learned.

Transparency and user-friendliness These curricula share a more accessible format with an introduction which sets out the objectives and contexts for teaching. The body of each curriculum is divided up into Learning Outcomes and Specifications for the curriculum. The format is, therefore, far more user-friendly than curricula in earlier school years. However, there are no teacher guides to assist teachers with using the curricula as a planning tool.

114


Teacher perceptions of the curriculum In Section 4 we looked at teachers’ use and perceptions of the curricula through their questionnaire responses. We found that 50% of teachers strongly agreed with the statement that they refer to the curriculum when preparing their English lessons. However, only 38% of them strongly agreed that they feel confident in their ability to plan appropriate English lessons. Therefore, the quality of the curricula is crucial. Teachers left comments in the questionnaire that they felt the curricula needed to be improved: ‘Teachers need a syllabus which is suitable and of an appropriate length’ (Urban secondary school teacher, Kelantan) ‘Syllabi are “one-size-fits-all”, therefore weaker students … do not benefit from the teaching and learning process because the material is too difficult’ (Rural secondary school teacher, Sarawak) The test data indicates that there is a large amount of variation in learner ability in each grade and class (see Section 3), so it is important that the curriculum provides information on differentiation strategies. This is particularly important because only 25% of teachers strongly agreed that they feel confident in their ability to adapt lessons to cater for different student abilities. With the exception of the curricula for Forms 3 to 5, the curricula do not seem to help teachers address the issue of differentiated levels of achievement within a class.

Learning materials In many language classrooms, the learning resources may determine the focus and structure of lessons (Allwright 1990). Textbooks are the most common resources used in language classrooms and they may provide the main source of language input for learners in many contexts. New teachers often rely on the textbook to guide lesson planning as they use it to determine the grammatical structures, skills or activities to focus on (Richards 2014). Therefore, it is crucial that textbooks and other learning resources accurately and coherently reflect the curricular aims, otherwise teachers will need to find or create materials to bridge this gap, which takes skill as well as time. An important strand of this project was to review the learning resources produced by the Ministry such as textbooks in order to determine their fitness for purpose. Although 87% of teachers indicated in the questionnaires that they use an English textbook in lessons, only 13% strongly agreed that it is the main focus of most lessons. 41% of primary school teachers and 21% of secondary school teachers strongly agreed that the textbook was useful. However, only 19% of primary teachers and 4% of secondary teachers strongly agreed that students like the textbook. Students were given the same question and were slightly more positive: 36% of Year 6 students strongly agreed that they like the textbook and 15% of secondary school students said the same. When asked about the level of difficulty of the textbook, 71% of primary school teachers said it was at the right level for their students but only 56% of secondary school teachers said the same, with 24% of them saying that the material is too difficult for their students. Although some teachers made positive comments about the textbooks, the majority of comments focused on textbooks being outdated, too difficult and lacking appropriate activities/exercises: ‘Some of the contents in the textbooks are outdated especially reading comprehension and not up to the level of students nowadays’ (Urban secondary school teacher, Johor) ‘The contents of XXX textbook is too challenging. It is not applicable to the students’ level’ (Rural secondary school teacher, Selangor) ‘The textbook does not contain enough exercises’ (Primary school teacher, Pahang)


115

‘The textbook is not comprehensive because it does not have exercises that are in line with the exam’ (Rural secondary school teacher, Pulau Pinang) Similar comments were made during the post-observation interviews: ‘The teachers generally dislike the state textbooks and use them sparingly’ (Observer JL) In order to understand the strengths and weakness of the textbooks, several of them were reviewed. In general, the textbooks reflected good features of organisation and activity focus; however, overall there is considerable misalignment of appropriately graded skills work and language development focuses across years.

Primary textbooks The Cambridge English review of primary course books found the materials to be: characterised by the use and modelling of inauthentic language populated predominantly by lower order types of activity such as reciting, chanting, matching and specific information comprehension tasks based heavily on narrow phonic approaches which limit learner engagement with texts context-poor in that contexts chosen to explore language largely ignore the immediate environment, what children know of the world from other school subjects and learner input imbalanced in that they target accuracy-focused outcomes at the expense of more meaning-focused outcomes.

Secondary textbooks The secondary textbooks reviewed exhibit some good features in the following areas: well-staged reading and listening work a variety of focus in vocabulary work well-integrated project work features writing skills work guided by process approaches some opportunities for pair and group work a good degree of learner input into activities the emergence of guided discovery approaches to grammar from Form 4 some effective exploitation of theme and context. However, they were generally weaker in the following areas: the inauthentic nature of the texts and dialogues in a lot of the material which lends the discourse a stilted feel the simplistic treatment of some topics e.g. ‘technology’ where far more creative and challenging forms of learner engagement could be targeted poor grading and targeting of grammatical and functional focuses, particularly in Forms 1 to 3 materials, where there is little opportunity for guided discovery or discussion about forms and meanings, and almost no tasks requiring creative use of the forms which have been studied a limited range of topics at each school grade leading to a lack of challenge. The perception that textbooks do not meet the needs of teachers and students was reflected in questionnaire responses. Nearly 50% of teachers strongly agreed that they use additional materials to supplement the textbook for most lessons and many teachers echoed this in the post-observation interviews. Teachers clearly see the need for more up-to-date, relevant and learner friendly material. However, only 33% strongly agreed in the

116


questionnaire that they feel confident creating these materials themselves. This suggests that they may need more support in this area. In fact, teachers strongly agreed that they would find it easier to plan lessons if they had a teaching guide (53%). The need for more resources was something that was raised by teachers in their comments: they do not feel they have time to create materials and would like to be provided with more materials and resources: ‘I am confident the teaching of English will be more effective with the help of teaching aids that are interesting and suitable’ (Urban primary school teacher, Sarawak) ‘Relevant teaching aids need to be provided to English teachers. For teachers who teach 5 different levels, looking for materials is not easy and is time-consuming. Reference books, additional questions and activities should be provided’ (Rural primary school teacher, Melaka) ‘Teachers need additional modules other than textbooks’ (Urban secondary school teacher, Selangor) In addition to teaching aids, teachers expressed a desire for improved access to technology in the questionnaire and during interviews: ‘Teaching and learning of English can be made more interesting, efficient and effective if schools are given the budget to build more language labs specifically for this language alone. Right now, day schools like ours have only 2 computer labs that have to be shared with about 30 classes which, is clearly not enough to go around. A language lab is very important to the teaching and learning of a foreign language because students nowadays remember and understand better via ICT tools’ (Urban secondary school teacher, Kelantan) ‘If the school has the infrastructure and ICT tools that are sufficient and in good condition, it will help English teachers in their teaching by showing online content to the students to improve their English. A proper language lab (not a makeshift computer lab) will help English teachers’ (Rural secondary school teacher, Sarawak) The consistency of feedback from teachers on the need for more resources and improved textbooks suggests that this is a high priority issue. This finding represents both a threat in the short-term, and an opportunity in the longer term. The threat relates clearly to the fact that many teachers see the textbooks as not being fit for purpose. Scrutiny of the data observers have co llected does suggest that the learning materials do not reflect international standards, and so may restrict learning. This in turn has implications for the quality, equity, unity and efficiency aspirations set out in the Education Blueprint. The opportunity is represented by the fact that the Education Blueprint sets out new aspirations for the educational system and for young Malaysians. New English language learning materials could contribute significantly towards motivating learners and teachers to achieve these aspirations. It is clear that a review is needed to align materials at all levels further to good practice in terms of learning focus and task selection, and international standards in terms of learning outcome, in line with the aspirations of the Education Blueprint.

National exams As discussed in previous sections, English language instruction in Malaysia appears to be exam-oriented in that the content and structure of the national exams are often determining the content of lessons and influencing teaching


117

practices (see Section 4). Although national standardised exams may be necessary to ensure that curricular aims are being achieved across a disparate educational landscape, it is important to investigate the extent to which these tests are fit for purpose (Cambridge English 2013). That is, are the tests consistent with international standards such as the CEFR and do they exhibit the essential test qualities of validity, reliability, and fairness as well as promoting a positive impact on learning and teaching? Therefore, the national exams were reviewed in order to evaluate their fitness for purpose. The review involved an analysis of past papers supplied by the Ministry of Education of the UPSR, PMR, SPM and MUET national exams. The review was limited to analysing sample question papers provided by the Ministry of Education. Cambridge English did not receive important statistical data about item and test performance which would have allowed a far more in-depth analysis to be carried out. The review found that there is generally a high degree of congruence between text and topic focus in the tests and the aspirations that are laid out in the Education Blueprint: knowledge, ethics and spirituality, leadership skills and in MUET, thinking skills. However, with the exception of MUET, Listening and Speaking skills are not assessed. There may be good practical reasons for this, but the absence of a Listening or Speaking assessment reduces the overall validity of assessment as it relates to the wider range of skills covered in the English curriculum. Moreover, as many teachers pointed out in the questionnaire and during post-observation interviews, they believe that the national exams need to test all skills: ‘Oral skills should be tested formally. Students should not be tested solely on writing’ (Urban secondary school teacher, Melaka) As the national tests primarily focused on Writing and Reading, we will now look at how each of these constructs are tested. The Writing test constructs, which vary across tests, generally reflect the range of skills detailed in the curricula, whereas the exclusive reliance on multiple-choice items in the Reading test construct in all exams limits their content validity and fitness for purpose. On closer inspection of the Reading papers, there are a significant number of technical problems and inconsistencies which would directly affect the validity and reliability of the tests. These included double keys, the targeting of above-level vocabulary, answers that could be deduced from world knowledge or word spotting, and a number of editing errors in the tests. Overall the review of the sample tests makes it clear that they generally lack the following key features: reliability and validity assessment based on communicative achievement clear alignment to international standards opportunities for accurate monitoring of learner language outcomes.

118


Teacher perceptions of the national exams Teachers were given the opportunity to comment on the current national exams and assessment in general. As mentioned in Section 4, teachers recognise the importance of assessment; however, only 20% of the teachers strongly agreed that the national exams were at an appropriate level for their students. They also left comments raising concerns about the effectiveness of these exams: ‘Year-end exams do not fully reflect students’ abilities. A more holistic assessment is needed’ (Rural secondary school teacher, Selangor) ‘Assessments do not prepare students to use English in their daily life’ (Urban secondary school teacher, Pahang) ‘The same assessment is given to students of all levels. It is like the same test for the monkey and elephant. And the task is to climb the tree. Of course the monkey can do it. The elephant can’t’ (Rural primary school teacher, Pahang) ‘Some aspects of exam assessment are unfair to students from rural areas’ (Urban secondary school teacher, Negeri Sembilan) Teachers feel that the current approach to assessment needs to be more systematic and standardised to maximise the benefit to students: ‘Validity and reliability type of assessment needs to be carried out’ (Rural secondary school teacher, Pulau Pinang) ‘The passing mark should be standardised and not changed on a whim’ (Rural secondary school teacher, Negeri Sembilan) ‘There should be more standardised tests in all levels to have a clearer picture of all levels’ (Urban secondary school teacher, Pulau Pinang) ‘Assessment should be set at an international standard’ (Urban secondary school teacher, Selangor) ‘The Education Ministry especially the Language Unit should prepare standardised tests for all schools so teachers can judge students’ performance in other schools and districts (city vs rural)’ (Rural primary school teacher, Melaka) All of these comments, coupled with the Cambridge English findings, make it clear that a revision of all the national exams is required. The revision process must ensure that a robustly designed suite of tests is produced. These tests can ensure that teachers and the Ministry of Education can accurately determine the extent to which students are able to meet targets that are set and to achieve the broader aspirations of the Education Blueprint. They must also play a positive role in the learning process.

Cambridge Baseline 2013 Recommendations and next steps

119

6. The findings and recommendations connected to the three areas of the project – students, teachers and curricula, textbooks and exams – are inevitably linked in a common ‘ecological system’, where changes to one affect the others. It is, therefore, not always easy to separate recommendations into discrete categories. We have endeavoured to do this, for the sake of clarity for the readers and users of the report, and have presented the recommendations for improving student success and for upskilling teachers separately from the recommendations based on our review of curricula, examinations and textbook. Each set of recommendations has been linked to the most relevant transformational Shifts in the Education Blueprint. It must be kept in mind, however, that the different categories under which the recommendations are discussed are not mutually exclusive. As instructional quality has been shown to increase the likelihood of learner achievement (Anderson, Greene and Loewen 1988; Justice, Mashburn, Hamre and Pianta 2008), the following recommendations are made with the goal of improving teaching quality and ultimately creating the best possible conditions for learning to occur. Support for teachers and learners needs to be systematic and sustained rather than one-off. All new initiatives should be evaluated in the short term based on their impact on teaching behaviour and beliefs, and in the long term based on learning outcomes. It is important to keep in mind that changing teacher and/or learner behaviours and beliefs takes time and significant improvements in performance are unlikely to be witnessed immediately. In reality, learner achievement is likely to go down during the initial period of a new initiative that involves changing fundamental beliefs about teaching and/or learning. All the recommendations to follow have emerged as a result of the findings on teacher performance, and lesson observations as well as attitudinal and background factors. They address the final question guiding the project:

Question 10: What recommendations can be made, based on the benchmarking of teachers/students and the evaluation of teaching/assessment materials, to enable the achievement of the envisioned education transformation by the Malaysia Ministry of Education?

Shift 1: Provide equal access to quality in education of an international standard ‘Benchmark the learning of languages to international standards’

Set realistic language proficiency targets, which take into consideration the baselines established in this review project and the achievement gaps related to location, gender, school type and specialisation. Targets need to be benchmarked to international standards, i.e. the CEFR, staged and phased in a differentiated approach according to cohort characteristics and systematically monitored. Setting realistic goals which are evidence-based will lead to more fit-for-purpose solutions.

120


‘Launch new revised primary a nd secondary curriculum’

Link the curricula reliably to international standards. This will provide clearer pathways of progression and attainment in terms of specificity of target skills development, depth in language focus and challenge in topic and thinking skills treatment. Carry out a root-and-branch review of teaching and learning materials and the revision of in-service and pre-service teacher training and pedagogical support programmes that will reflect the shift in educational emphases in the curricula.

Define English learning outcomes clearly in order to set clear and attainable curriculum-defined standards and to enable teachers to utilise a range of appropriate teaching strategies in achieving differentiated goals. Update textbook materials for English to ensure they reflect the values and aspirations of the Education Blueprint, complement the aims of the new curricula and meet the needs of the students. Ongoing teacher support must be planned to accompany any introduction of new learning materials. Explore options to give teachers access to a much wider resource base with particular emphasis on audio-visual material that can be efficiently shared digitally.

‘Revamp examinations and assessments’

Redesign national English examinations to align to international standards and the CEFR in line with the aspirations and attributes highlighted in the Education Blueprint. Redesign English examinations to espouse a communicative approach, incorporate Speaking and maintain an equal focus on the four skills. Redesign the content and format of English examinations to deliver both greater reliability and validity. Valid, reliable exams based on a communicative approach are great motivators for learning, and it is therefore recommended to introduce exams which are based on a state-of-the-art view of teaching and assessment. Build on the progress made so far with school-based assessment (SBA) to focus on teachers’ understanding of principles underlying SBA. Effective SBA needs to allow for an appropriate administrative workload and the teachers’ ability to take action on what their assessments tell them about their students’ learning.

Shift 2: Ensure every child is proficient in Bahasa Malaysia and English language ‘Upskill English language teachers and expand opportunities for greater exposure to English language’

Provide suitable English language provision for teachers. As levels of English vary considerably within the sample of teachers observed, this provision should be targeted first at those teachers who have been identified as having the lowest levels of proficiency (CEFR levels B1 and below). Develop the pedagogic and practical teaching competence of teachers. Both the quantitative and qualitative data gathered from classroom observations suggest that many teachers are in need of


121

immediate professional development. Provide training for a differentiated approach to learning and assessment. The large variation in language proficiency within the same school grade, especially at Year 6, Forms 3 and 5 necessitates the need for differentiated strategies for teaching and assessment. Monitor and evaluate the effectiveness of upskilling programmes to ensure quality, transparency and accountability.

Reward teachers with internationally-recognised qualifications for both English language proficiency and teaching knowledge and practice.

‘Encourage every child to learn an additional language by 2025’

Use the new English curriculum as a model for mapping the content, progression in language and skills development and assessment processes when introducing additional languages into the curriculum.

Shift 3: Develop values-driven Malaysians Build on the existing focus on values through the inclusion of themes or topics in order to foster the learners’ value base through the revised curriculum in terms of its skills development framework.

Shift 4: Transform teaching into the profession of choice ‘Upgrade the quality and personalisation of CPD with greater emphasis on s chool-based

training’ Raise the entry bar to the teaching profession by making internationally-recognised Cambridge English assessments the required language qualifications for entry to teacher training programmes. Benchmark the teaching of languages to international standards. During the planning of the schoolbased professional development programme, ensure that content reflects the principles of the internationally recognised teaching qualifications for both language proficiency and teaching produced by Cambridge English. Develop a context-sensitive programme of school-based professional development with SMART objectives linked to both language needs (e.g. classroom language proficiency and intelligible pronunciation), and pedagogical needs (e.g. differentiation, supportive teacher talk, continuous assessment). As contexts and needs differ, variety is essential. This programme should therefore consist of a portfolio of opportunities including bottom-up CPD initiatives within or between schools. It should also contain a strong practical component so that teachers are able to apply ideas to their own contexts. Link the following processes into a unified yet flexible professional development programme for teachers, so that each element is not seen in isolation: teacher training, curricula and examination revision, the integration of ICT into day-to-day learning and tailed on the ground teacher coaching.

122

Cambridge Baseline 2013 Recommendations and next steps Develop the format and content of PRESET (pre-service training) and INSET (in-service training) education for English in particular and for other content subjects in general, as any change made is likely to need to be applicable across the curriculum. The efficiency and effectiveness of current training models, the coherence between course content, and the capacity and quality of training providers must be taken into account in the development process.

‘Develop a peer-led culture of excellence and certification process’

Identify master practitioners teaching in schools across Malaysia to act as coaches and mentors both within their own schools and within the broader teaching community. Their expertise should also be considered when revising the curriculum, textbooks and teaching resources. Fellow teachers will value the mentors’ familiarity with the realities of their classrooms and be open to adopting new approaches. Introduce a teacher observation programme that is holistic in nature and not just based on providing an assessment of teacher performance. Lesson observations are an effective method to encourage teacher reflection, identify areas in need of further development and share good practice. As such, they are an important component of any CPD programme. ‘Focus teachers on their core function of teaching by reducing administration burdens’

Provide practical teaching resources or guides that teachers can use on a day-to-day basis while they are adjusting to the new teaching and learning approach.

Shift 7: Leverage ICT to scale up quality learning across Malaysia Develop teacher support programmes which promote technology as a learning tool rather than as a simple solution to complex challenges in the education system. Place ICT and media modules involving ICT at the heart of English language teacher training and upskilling programmes. Creative ways should be explored to optimise the use of media within current classroom constraints. Expand the newly launched global teacher support web platform and use it as a vehicle for bottom-up teacher development initiatives, to share good practice as well as to provide a channel to share more media-rich material not currently exploited in classroom contexts. This platform could then be expanded to include a portal for students, to increase their engagement, and for parents, to increase their involvement in their children’s education.

Next steps As a critical next step, Cambridge English recommends collaborating closely with the Ministry to develop a detailed, integrated language policy that includes the setting of achievable, staged targets for both students and teachers and coordinates the end of secondary school language targets with the entrance requirements for tertiary education. The development of this language policy should result in a strategy plan specifying realistic timescales that tie in with the sequence of transformation outlined in the Education Blueprint. An implementation plan and an evaluation and impact analysis plan should then be created to ensure full achievement of the goals set out in the Education Blueprint.

Cambridge Baseline 2013 References

123

References Allwright, R L (1990) What do we want teaching materials for? In Rossner, R and Bolitho, R, (Eds), Currents in Language Teaching, Oxford: Oxford University Press. Altinyelken, H (2010) Curriculum change in Uganda: Teacher perspectives on the new thematic curriculum, International Journal of Educational Development 30, 151-161. Anderson, R, Greene, M and Loewen, P (1988) Relationship among teachers’ and students’ thinking skills, sense of efficacy and student achievement, Alberta Journal of Educational Research 34, 148-165. Bachman, L F and Palmer, A S (1982) The construct validation of some components of communicative proficiency. TESOL Quarterly 16 (4), 449-65. Bandura, A (1977) Self-efficacy: The Exercise of Control, New York: W H Freeman. Bangert-Drowns, R L, Hurley, M and Wilkinson, B (2004) The effects of school-based writing-to-learn interventions on academic achievement, Review of Educational Research 74, 29-58. Bax, S (2003) The end of CLT: A context approach to language teaching, ELT Journal 57 (3), 278-287. Blumberg, F C and Fisch, S M (Eds) (2013) Digital games: A context for cognitive development, New Directions for Child and Adolescent Development 139, 41-50. Borgers, N, Leeuw, E D and Hox, J (2000) Children as respondents in survey research: Cognitive development and response quality 1, Bulletin de Méthodologie Sociologique 66, 60-75. Boud (2000) Sustainable assessment: Rethinking assessment for the learning society, Studies in Continuing Education 22 (2), 151-167. Cambridge English Language Assessment (2013) Principles of Good Practice: Quality Management and Validation in Language Assessment, Cambridge: UCLES, available online: http://www.cambridgeenglish.org/images/22695-principles-of-good-practice.pdf Canale, M and Swain, M (1980) Theoretical bases of communicative approaches to second language teaching and testing, Applied Linguistics 1 (1), 1-47. Carless, D (2009) Trust, distrust and their impact on assessment reform, Assessment and Evaluation in Higher Education 34 (1), 79-89. Chapman, D W, Weidman, J, Cohen, M and Mercer, M (2005) The search for quality: A five country study of national strategies to improve educational quality in Central Asia, International Journal of Educational Development 25, 514-530. Clément, R, Dörnyei, Z and Noels, K (1994), Motivation, self-confidence and group cohesion in the foreign language classroom, Language Learning 44, 417-448. Clément, R, Gardner, R C and Smythe, P C (1977) Motivational variables in second language acquisition: A study of francophones learning English, Canadian Journal of Behavioral Science 9, 123-133. Colletta, S P, Clément, R and Edwards, H P (1983) Community and Parental Influence: Effects on Student Motivation and French Second Language Proficiency, Quebec: International Center for Research on Bilingualism.

124


Council of Europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge University Press. Creswell, J W (2009) Research Design: Qualitative, Quantitative and Mixed Methods Approaches (3rd Edition), Thousand Oaks: Sage. Csizér, K and Dörnyei, Z (2005) The internal structure of language learning motivation and its relationship with language choice and learning effort, The Modern Language Journal 89, 19-36. Dembo, M and Gibson, S (1985) Teachers’ sense of efficacy: An important factor in school achievement, Elementary School Journal 86, 173-184. Eurydice (2012) Key data on teaching languages at school in Europe, available online: http://bookshop.europa.eu/en/key-data-on-teaching-languages-at-school-in-europe-pbECXA12001/ Fisher R. (2005) Teaching Children to Think (2nd Edition), Cheltenham: Nelson Thornes. Fullan, M (2001) The New Meaning of Educational Change (3rd Edition), New York: Teachers College, Columbia University. Fuller, B and Kapakasa, A (1991) What factors shape teacher quality? Evidence from Malawi, International Journal of Educational Development 11 (2), 119-127. Gardner, R C (1985) Social Psychology and Second Language Learning: The Role of Attitudes and Motivation, Baltimore: Edward Arnold. Holliday, A (1994) Appropriate Methodology and Social Context, Cambridge: Cambridge University Press. Jones, N (2013) The European Survey on Language Competences and its significance for Cambridge English Language Assessment, Research Notes 52, 2-6. Jussim, L and Harber, K D (2005) Teacher Expectations and Self-Fulfilling Prophecies: Knowns and Unknowns, Resolved and Unresolved Controversies, Personality and Social Psychology Review 9 (2), 131-155. Justice, L M, Mashburn, A J, Hamre, B K and Pianta, R C (2008) Quality of language and literacy instruction in preschool classroom serving at-risk pupils, Early Childhood Research Quarterly 23, 51-58. Kennedy, A M, Mullis, I V S, Martin, M O, and Trong, K (Eds) (2007) PIRLS 2006 Encyclopedia: A Guide to Reading Education in the Forty PIRLS 2006 Countries, Chestnut Hill: TIMSS & PIRLS Inte rnational Study Center, Boston College. Lightbown, P M and Spada, N (2013) How Languages are Learned (4th Edition), Oxford: Oxford University Press. Lyster, R and Ranta, L (1997) Corrective feedback and learner uptake, Studies in Second Language Acquisition 19 (1), 37-66. Maccoby, E E and Jacklin, C N (1974) The Psychology of Sex Differences, Stanford: Stanford University Press. Mashburn, A J, Pianta, R C, Hamre, B K, Downer, J T, Barbarin, O A, Bryant, D, Burchinal, M, Early, D M and Howes, C (2008) Measures of classroom quality in prekindergarten and children’s development of academic, language, and social skills, Child Development 79, 732-749. Maybin J, Mercer N and Stierer B (1992) ‘Scaffolding’ learning in the classroom, in Norman, K (Ed.) Thinking Voices: The Work of the National Curriculum Project, London: Hodder and Stoughton, 186-195.


125

McAleavy, T, Hamilton, A, and Latham, M (no date) CfBT Education Trust commentary on the Malaysian Education Blueprint 2013-2025, available online: http://www.cfbt.com.my/file/CfBT_Blueprint_Report_Malaysia.pdf Mills, N, Pajares, F and Herron, C (2006) A reevaluation of the role of anxiety: Self-efficacy, anxiety and their relation to reading and listening proficiency, Foreign Language Annals 39, 276-295. Mueller, R, Carr-Stewart, S, Steeves, L and Marshall, J (2011) Teacher recruitment and retention in select First Nations schools, in education 17 (3), available online: http://ineducation.ca/ineducation/article/view/72/553 Multon, K D, Brown, S D, and Lent, R W (1991) Relation of self-efficacy beliefs to academic outcomes: A metaanalytic investigation, Journal of Counseling Psychology 18, 30–38. Negishi, M, Takada, T and Tono, Y (2013) A progress report on the development of the CEFR-J, in Galaczi E D and Weir, C J (Eds) Exploring Language Frameworks: Proceedings of the ALTE Kraków Conference, July 2011, Studies in Language Testing volume 36, Cambridge: UCLES/Cambridge University Press, 135-163. Orafi, S M S and Borg, S (2009) Intentions and realities in implementing communicative curriculum reform, System 37, 243-253. Oxford, R L and Ehrman, M E (1993) Second language research on individual differences. In Grabe, W (Ed.), Annual Review of Applied Linguistics XIII, Issues in Second Language Teaching and Learning, 188-205. Purpura, J E (2004) Assessing Grammar, Cambridge: Cambridge University Press. Richards, J C (2014) The Role of Textbooks in A Language Program, Cambridge: Cambridge University Press, available online: http://www.cambridge.org.br/authors-articles/articles?id=337 Richards, J C and Lockhart, C (1994) Reflective Teaching in Second Language Classroom, Cambridge: Cambridge University Press. UNESCO (2013) Malaysia Education Policy Review: Abridged Report, May 2013, available online: http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/ED/pdf/M_EPR-abridged-report-Annex24052013.pdf Wagner, T, Kegan, R, Lahey, L, Lemons, R W, Garnier, J, Helsing, D, Howell, A, Thurber Rasmussen, H (2006) Change Leadership: A Practical Guide to Transforming our Schools, San Francisco: John Wiley & So ns. Wesely, P (2012) Learner attitudes, perceptions and beliefs in language learning, Foreign Language Annals 45, 98-117.

126



127

Contact us Cambridge English Language Assessment 1 Hills Road Cambridge CB1 2EU United Kingdom [email protected] www.cambridgeenglish.org/consultancy

© UCLES 2014

CE/2437/4Y01/Ma 12

Cambridge Baseline Study 2013

Recommend Documents