MEASUREMENT & EVALUATION
Dr. Arbab Khan Afridi Dr. Arshad Ali Dr. Muhammad Rauf
In Collaboration With
MASTER COACHING ACADEMY (MCA) (IER) UNIVERSITY OF PESHAWAR All rights reserved with the Author
Authors:
Dr. Arbab Khan Afridi Dr. Arshad Ali Dr. Muhammad Rauf
Book:
Measurement & Evaluation
1st Edition:
March, 2015
Composer:
M. Nawaz Khan Abbasi 0333-9352585
Printers:
Ijaz Printers, Peshawar
Quantity:
1000
Price:
150/-
Available at MCA Academy and leading book shops
[email protected] Contact: 091-5843361 Cell: 0300-5930899
2
TABLE OF CONTENTS UNIT-1:..................................................................1 INTRODUCTION......................................................1 1.1 1.2 1.3 1.4 1.5 1.6
EVALUATION, ASSESSMENT, MEASUREMENT AND TEST:........1 THE PURPOSE OF TESTING..........................................30 GENERAL PRINCIPLES OF ASSESSMENT:..........................35 TYPE OF EVALUATION PROCEDURE................................37 NORM- REFERENCED AND CRITERION REFERENCED TEST:...43 EDUCATIONAL:........................................................45
UNIT-2:................................................................50 JUDGING THE QUALITY OF THE TEST.........................50 2.1 2.2 2.3 2.4 2.5
VALIDITY, METHODS OF DETERMINING VALIDITY:............51 FACTORS AFFECTING VALIDITY....................................54 RELIABILITY, AND METHODS OF DETERMINING RELIABILITY: 56 FACTORS AFFECTING RELIABILITY:...............................61 PRACTICALITY:........................................................64
UNIT-3:................................................................66 APPRAISING CLASSROOM TESTS (ITEMS ANALYSIS)......66 3.1 THE VALUE OF ITEM.................................................66 3.2 THE PROCEDURE/ PURPOSE OF ITEM ANALYSIS:................72 3.2 MAKING THE MOST OF EXAMS: PROCEDURES FOR ITEM ANALYSIS:.......................................................................73 3.3 ITEM DIFFICULTY:....................................................91 3.4 THE INDEX OF DISCRIMINATION...................................93 UNIT-4:................................................................98 INTERPRETING THE TEST SCORES............................98 4.1 4.2 4.3 4.4
THE PERCENTAGE CORRECT SCORE:..............................98 THE PERCENTILE RANKS:.........................................108 STANDARD SCORES:................................................113 PROFILE:..............................................................115 1
UNIT-5:..............................................................117 EVALUATING PRODUCT, PROCEDURES & PERFORMANCE ........................................................................117 5.1 5.2 5.3 5.4 5.5
EVOLUTION THEMES AND TERMS PAPERS:....................117 EVALUATING GROUP WORK & PERFORMANCE................127 EVALUATING DEMONSTRATION:..................................131 EVALUATION OF PHYSICAL MOVEMENTS AND MOTOR SKILLS: 138 EVALUATING ORAL PERFORMANCE:.............................144
UNIT-6:..............................................................148 PORTFOLIOS......................................................148 6.1 PURPOSE OF PORTFOLIOS:.........................................148 6.3 GUIDELINE AND STUDENTS ROLE IN SELECTION OF PORTFOLIO ENTRIES AND SELF-EVALUATION:.........................................156 6.4 USING PORTFOLIOS IN INSTRUCTION AND COMMUNICATION: 160 6.5 POTENTIAL STRENGTH AND WEAKNESSES OF PORTFOLIOS: 163 6.6 EVALUATION OF PORTFOLIO:......................................169 UNIT-7:..............................................................171 BASIC CONCEPTS OF INFERENTIAL STATISTS.............171 7.1 7.2 7.3 7.4 7.5 7.6 7.7
CONCEPT & PURPOSE OF INFERENTIAL STATISTICS:.........171 SAMPLING ERROR:..................................................173 NULL HYPOTHESIS:.................................................175 TESTS OF SIGNIFICANCE:..........................................177 LEVELS OF SIGNIFICANCE:........................................180 TYPE-I AND TYPE-II ERRORS: REMAINING:....................182 DEGREES OF FREEDOM:............................................186
UNIT-8:..............................................................191 SELECTED TESTS OF SIGNIFICANCE........................191 8.1 8.2 8.3
T-TEST:...............................................................191 CHI-SQUARE (X2):..................................................194 REGRESSION:........................................................199
2
FOREWORD Knowledge is the main distinctive characteristic of human, due to which hece of Allahys seems, by the gra ing a dream in the olden dawas selected as vice-regent of Allah Almigthy. Man is superior to other living beings, because he has the capability and potentiality to understand as well as reason the consequences. Knowledge is obtained through the continuous process of education. This process is usually a life long process. This is also a fact that education is such an activity which is bilateral and participatory. It cannot be accomplished with out the two partners-teacher and student. This activity requires a transmitter and areceiver. If any one of them is missing the exercise would remain incomplete. To compare however, the two the- teacher appears superior to his pupils as he is the organiser and director of the teaching learning process. That is why since times immemoriable, search for significant teachers has ever been in progress and the same is still going on. No dobt the countable good teachers are there, but they are not countless. There is a need of producing a countless number od genuine educators/prospective educators to contribute in this regard. This objective in view the people at the helm of the affairs are trying their best to bring desirable changes in the education system, teacher education curriculum and teacher training programmes. The best teacher, being a dream in the old days, is about to become a reality, if the curse outlines and syllabi are properly dispensed, it is hoped that the required lot of teachers would be made available. The future educators/teachers are needed to well equiped in all skills not confining only to academic learning ignoring ITC, current affairs and contemporary issues. These objectives in view, improvements in the system are being carried out to achiev the goals. The new curricula, on which is based, this book of mine is the result of long deliberations and brain stormings
3
undertaken by the senior educators. This is now upto the implementers and the students to benefit from the same in the best possible capacity. The book is now in your hands and this is not claimed to be the final word. There is always place for improvement. The author would be highly obliged for any comments/recommendations, if conveyed to make it further better and improved. Dr. Arbab Khan Afridi The Author
4
ACKNOWLEDGEMENT All praises and glory be to Allah Almighty Who bestowed upon me His blessing to be able to produce my this book, named “Teacher Education in Pakistan”. My humble gratitude and thanks are due for Him with submission and heartiest admiration who guided me to the right path. This all became possible only due to Allah’s significance and benevolence. The rays of the light of Omni-Present Allah always took me out of the deep darkness of ignorance to the lightened path of knowledge, spreading its reflection to the needy. My thanks and gratitudes are due for my old student and now my colleague Dr. ______ , who provided certain reference books and substantive substances that were very much beneficial for the compilation of my this book. In addition to that, I am extremely thankful to my Composer Mr. Muhammad Nawaz Khan Abbasi, Peshawar who provided step by step expertise views regarding printing and book production process. My thanks are also due for Dr. Muhammad Rauf, the ______of the ______of IER University, Peshawar, who always took pains in searching certain reference books for me. He always showed great enthusiasm and pleasure in complying to any of my request regarding the Bibliographies to make them available at the earliest. Last but not the least, I am thankful to my family members who cooperated with me and made all sort of requirements available to me during the process of preparing the primary substance of this book. They maitained a very calm and conducive environment to me during all this period of compilation, otherwise this work would not have been possible to have come to light.
5
Autho r
6
7
UNIT-1: INTRODUCTION 1.1
EVALUATION, ASSESSMENT, MEASUREMENT AND TEST:
1.1.1
Evaluation:
Literally, the term evaluation means “appraisal”, “Judgment” or “assessment”, “calculation”, “estimation” or “rating” of a thing. According to the International Dictionary of Education (by G Terry & JB Thomas) evaluation means “value judgment” on an observation, performance test or any data whether directly measured or inferred. Evaluation is the qualitative assessment of a thing. It answers the question “How good”? A. D. Jones defines evaluation as “the process of finding the value of something”. He further says “the process of evaluation is the attempt to find the worth of any enterprise. The Oxford Advanced Learner’s Dictionary defines the term “Evaluate” as to find out or form an idea of the amount or values of something. When we evaluate something, we mean to determine the value or worth of that thing. Evaluation is, actually, the process through which we collect information of something and then make a decision in the light of that information. So we can say that evaluation is concerned with making judgments about things. When we act as evaluators, we attribute “value” or “worth” to behavior, objects and processes. In the wider community, for example, one may make evaluative comments about a play, clothes, restaurant, a book or someone’s behavior. We may enjoy a play, admire someone’s clothes, to speak about some restaurant and so on and so forth. Invariably, these are rather simple, straight forward comments of value or worth because this judgment is not based on appropriate and relevant data.
According to William Wiersma and Stephen G. Jurs, “The more effective evaluation requires the judgment which is based on appropriate and relevant data”. For example, to say that a film is ‘good’ or ‘bad’ is not the judgment based on appropriate and relevant data. It is, therefore, not the exact evaluation of the film. It well-written script, tight direction mood-enhancing music, suitable characters and so forth; because this judgment is based on some appropriate and relevant data. These are the characteristics upon which we can make a judgment about something. Educational Evaluation: The Concept of Evaluation in Education: Educational Evaluation is a specific term which is used for the judgment of the educational objectives. Educational Evaluation seeks to determine how the student has achieved the stated objectives of the learning situation. Different educationists have defined Educational Evaluation in different words some of which are discussed below: 1.
“Educational Evaluation is a systematic process of collecting, analyzing and interpreting information to determine the extent to which pupils are achieving instructional objectives”. –– Norman E. Gronlund
2.
“Educational Evaluation is the systematic process of collecting and analyzing data in order to determine whether, and to what degree, objectives have or being achieved”. –– L.R. Gay
3.
“Educational Evaluation is the estimation of the growth and progress of pupils towards objectives or values in the curriculum”. –– Writestone
4.
“Educational Evaluation is the defined as the process of determining the extent to which educational objectives are achieved by the student”. –– Remmers
Approaches to Evaluation Evaluation in our schools is essentially concerned with two major approaches to making judgments: 1.
Product Evaluation:
It is the evaluation of students’ performance in a specific learning context. Such kind of evaluation seeks to determine how well the student has achieved the stated objectives of the learning situation. In this sense the student’s performance is seen as a product of the educational experience. A school report is an example of Product Evaluation. 2.
Process Evaluation:
It is that kind of evaluation that seeks to examine the experiences and activities involved in we learning situation. It makes judgment about the process by which students acquired learning. In more simple words, it examines the process of learning experience before it has concluded. For example, the evaluation of the nature of studentsteacher interaction, instructional methods, school curricula, a specific programmes etc. are the best examples of Process Evaluation. 1.
Curriculum Evaluation
Curriculum Evaluation, as is clear from the name, is the evaluation of a certain curriculum i.e. an instructional programme. It is used to determine the outcome of a programme and to decide whether to accept or reject a programme. This evaluation helps in the further development of the curriculum materials for continuous improvement. For a better learning, it is necessary to assess a new programme in order to find out whether the desired outcomes are being achieved or not. The use of evaluation techniques should enable the curriculum workers to
make steady progress in improving the curriculum. Curriculum evaluation should not only be a means for judging educational effectiveness, but also should lead to useful decisions that can serve as a powerful force to improve the educational process. Careful evaluation should demonstrate the strengths and weaknesses in the curriculum so that necessary changes can be made in the instructional programme. 2.
Programme Evaluation
Programme Evaluation is used for judging the effectiveness of a programme or a special project. This evaluation is used to make a decision about programme installation and modification. It helps to obtain evidence to support or oppose a programme. Outside education, ‘programme evaluation' is used as a means of determining the effectiveness, efficiency and acceptability of any form of programme. But within education, we can use the term in a similar way as in the case of evaluating the effectiveness of a new writing, or reading programme in primary schools. A curriculum evaluation may qualify as a programme evaluation if the curriculum is focused on change or improvement. Programme evaluation, however, does not involve appraisal of curricula (e.g. evaluation of a computerized student record keeping system.) 3.
Personnel Evaluation
The evaluation of personnel is the assessment of the performance of a working personnel in an organization. That is why it is also called performance appraisal or staff evaluation. In education, 'personnel evaluation' is very much necessary for adopting appropriate appraisal, plans and procedures to achieve the goal of education. According to McNeil, J.D. "Evaluation of the performance of working personnel can be an effective instrument for helping people in growing and developing in their roles. It could be used as a mechanism of continuing education and learning from one another. Through a wellorganized appraisal system every employee (r.3) can create learning spaces for himself in the system in which he works. A good personnel evaluation helps the employee to recognize his/ her own strengths and weaknesses in order to enable him to improve his performance in a given
role. It also helps in identifying people for the purpose of motivating, training and developing them for new roles or existing roles. 4.
Institutional-Evaluation:
Institutional Evaluation is the evaluation of the total programme of a school, college, university or other educational institution. The evaluation of an institution is used to collect information and data on all aspects of the function of that institution. The basic aim of this evaluation is to determine the degree to which instructional objectives are being met and to identify areas of strength and weakness in the total programme. An institutional evaluation involves more than the administration of tests to students; it may require any combination of questionnaire, interviews, and observations with data being collected from all persons in the institution community, including administrators, teachers, and counsellors. The major component of institutional evaluation is the institution testing. The more comprehensive the testing program, the more valuable are the resulting data. That is why, for achieving the most valuable resulting data, institutional testing programme should include measurement of achievement, aptitude, personality and interest. Tests selected for an institutional evaluation must match the objectives of the institution and be appropriate for the students to be tested. Need or Importance of Evaluation Evaluation plays pivotal role in teaching-learning process. It helps in providing information about the success or failure of an educational objective. It shows whether the student has achieved the required objective or not, and to what degree has the goal been reached? So evaluation provides relevant information to the decision-makers need about input, output, operation of a programs, and placement of student in programs. Levels of understanding can be assessed. and future educational objectives set, based on student needs. Similarly, appropriate activities can be planned by the teacher based on the knowledge of the attributes of the student. Evaluation also makes it easy for the teacher to form objective, select content, and plan for learning experiences. It also provides a guideline about all aspects of the teaching- learning process.
Without evaluation we cannot be aware of the effectiveness or ineffectiveness of an educational program or objective. Evaluation is as necessary for student as for teacher or decisionmakers. Its importance for the student is great because the whole process of education is for the benefit of the student. The student is the centre of interest in the teaching learning process. For the student, evaluation provides feedback regarding better strengths and weaknesses. It encourages the student for better study and increases his motivation. Improvement of the teacher's teaching and the student's learning through judgment, using available information is the ultimate need of the evaluation process. In a nutshell, evaluation plays central role in the teaching learning process. It serves as a guiding principle for the selection of supervisory techniques and also as a means for improving schoolcommunity relation. 1.1.2 Assessment: Concept of Assessment Literally assessment means the act of judging or assessing a person or situation or event. It is the classification of someone or something with respect to its worth. Assessment is a general term that includes the full range of procedures used to gain information about student learning (observations, ratings of performances or projects, paper-and-pencil tests) and the formation of value judgments concerning learning progress. A test is a particular type of assessment that typically consists of a set of questions administered during a fixed period of time under reasonably comparable conditions for all students. (Linn and Groundlund, 2000)/ Assessment may include both quantitative descriptions (measurement) and qualitative descriptions (nonmeasurement) of students. In addition, assessment always includes value judgments concerning desirability of the results. Assessment may or may not he base on measurements; when it is, it goes beyond simple quantitative description.
The process of collecting, synthesizing, and interpreting information to aid in decision making is called assessment. For many people, the, words,, classrooms, assessment evoke images of pupils taking paper-and-pencil, test, teachers scoring them, and grades being assigned to the pupils based upon their performance. Assessment, as the term is used here, includes the full range of information that teacher gather in their classrooms; information that helps them understand their pupils, monitor their instruction, and establish a viable classroom culture. It also includes the variety of ways teachers gather, synthesize, and interpret that information.' Assessment is a general term that includes all the ways through which teachers gather information in their classrooms. Need for Assessment in Education As long as there is need for the educator to make some instructional decisions, curricular decision, and selection decision. Placement or classification decisions based on the present or anticipated educational status of the child so long will there be need for assessment in educational enterprise. To the modern educator, the ultimate goal of assessment is to facilitate learning. This could be done in a number of ways, in each way a separate type of decision is required. The assessment decision also determines which of tests is to be used for assessment. Thus there is a close relationship between the purpose of evaluation, evaluate decisions and types of tests to be used for them. The purposes of assessment are as follows: Selection Decision Whenever there will be choice, selection decision is to be made. In our daily life we see that institutions and organization need persons for their work, they get responses from several people but they cannot take all of them. They have to make selection out of them. Assessment of these persons is to be made on the bases of tests given to them. Tests will provide information, which will help in selection decision. Some persons will be acceptable while others will not be acceptable. Similarly the universities have to make section decisions for admitting the students to various courses. Courses in which hundreds of candidates are applicants,
Selection decision is to make on stronger footing. Naturally some tests are given to the candidates to help in selection decision such as Aptitude tests, Intelligence tests. Achievement tests or Prognostic tests are generally given for the purpose of selection decision. There has been ruling from the judiciary that the scores on these tests should have a good relationship with the success in the job or the course for which the tests has been given. If any selection tests does not fulfill this requirement it needs to be improved or replaced by a better one I Although perfection of such tests cannot be guaranteed but any institution or organization which is interested in the best students or workers will continue to make efforts in improving the tests being used for the purpose of selection. Placement Decision Since school education should be provide to all in a welfare state the schools must make provision for all, they cannot reject the candidates for admission as the universities or colleges can do. How these candidates placed in different programmes of school education is to be determined on the basis of their assessment. Such school determinations are called placement .decision. These decisions are required not only in the case of those who are with some disadvantage but also with those who are gifted and talented. The schools have to find one or the other programme for all school age children depending upon their weakness or strength. Placement tests have to be different and more useful from selection, tests because they improve the decision to differentially assign students to teaching programmes. Achievement test and interview are generally used for placement decision. Classification Decisions Assessment is also required to help in making decisions in regard to assigning a person to one of several different categories, jobs or programmes. These decisions are called classification decisions because in one particular job or programme, there may be several levels or categories. To which level or category a particular person of child be assigned, depends upon the results of the test. Aptitude tests, achievement tests, interest inventories value questionnaires attitude scale
and personality measures are used for classification decision. There is a minor difference in classification placement and selection. Classification refers to the cases, where categories are essentially unordered, placement refers to the case where the categories represent level of teaching or treatment and selection refers to the case where the persons can be selected or rejected. Diagnosis and Remedial Decisions Assessment is required to locate the students who need special remedial help. For example what instructional strategies the teacher should use to help a particular students or a group of student so that the opportunities are maximized to achieve the objective. Aptitude tests, intelligence tests, diagnostic achievement tests, diagnostic personality measures etc. may be used to achieve the purpose. Feed Back It is not sufficient to assessment student through a test and doing nothing after that. A good teacher will use tests for the purpose of providing feedback to students. Feedback may be effective or ineffective depending upon the circumstances. Feedback will facilitate learning if it confirms the learner's correct responses or identifies errors and corrects them. Test results made available to parents may be used for making feedback evaluation device. It is also to be remembered that feedback are both for the student and teacher because it provide information to both and help in knowing how will students have learnt and how well the teacher has taught. Motivation and guidance of learning: Assessment is also used to motivate the students for more study and providing for learning. However motivation device can be used positively as well as negatively. Unfortunately most of the schoolteacher use examination or refusing to grant annual promotion to next class can motivate the student but if they are motivated with using such evaluation techniques which provide more confidence to the students in the subject, they will be more effective and lasting. Aptitude tests, achievement tests, attitude scales, personality
measures, interest inventories, surprise quizzes encourage student for more study and understanding. Assigning Makers to Students: The instructional programme remains incomplete if it is not followed by assessment. Although no teacher chooses teaching profession because he is interested in evaluating the students but no teacher confines his job to teaching only. He regularly evaluates his students and assigns them makers. Actually most of the teachers are giving most of their time to this purpose. If teachers do not evaluate their students, do not assign those marks or grades, how can they check their effectiveness of teaching and learning outcome of the students? Role of Assessment in Education Process The assessment of learning takes place in an instructional context and. Consequently, that learning environment shapes the reasons why we evaluate, influences the purpose for evaluating as well how we evaluate the determines how we should use the outcomes of our assessment. Assessment is an integral part of instruction; it is not a separate entity that somehow is loosely attached to the teaching process. The instruction process and the role of evaluation in it both must be understood as background to the study of educational measurement. To that end, the role of assessment in instruction will be described using a model that explains how the teaching process works. (A) There are many models that describe the variety of approaches to teaching found in schools, but the Basic Teaching Model (BTM), introduced by Glaser (1962) accounts for the fundamental components of most other specific teaching models, such as the Socratic approach, the individualized instruction approach, or the computer dominated instructional approach (Joyce and well, 1980). Few teachers probably follow the BTM steps explicitly to guide their instructional activities. And though we do not specifically endorse the use of the BTM or any other particular model, we do advocate instructional approaches, by
whatever name, the account for the fundamental functions represented in the BTM as described next. The main purposes of the BTM are to identify the major activities of the teacher and to describe the relationship between activities figure III is a diagram of the mode. Our primary interest is the Performer Assessment component, but we cannot understand completely the role of evaluation without understanding how Performance Assessment affects, and is affected by, other teaching activities. Instructional Objectives, the first component of the BTM, represents the teacher's starting point in providing instruction. What should students learn? What skills and knowledge should be the focus of instruction? What is curriculum and how is it denned? The second component, Entering Behaviour, indicates that the teacher must try to assess the students' level of achievement and readiness to learn prior to beginning. Instruction. What do the students know already and what are their cognitive skills like? How receptive to learning are they? Which ones seems self-motivated? This component indicates a need for evaluation information before instruction actually begins. Once the teacher has decided what will be taught and to whom the teaching is to be directed, the "How?" must be determined. The Instructional Procedures component deals with the material and methods of instruction the teacher selects or develops to facilitate student learning. Does the text need to be supplemented with illustration? Should small group projects be developed? Is there computer software available to serve as a refresher for prerequisites? At this point instruction could begin, and often it does, but unless the teacher makes plans to evaluate student's performance, the students and teacher will never be sure when learning is complete. The performance Assessment component helps to answer the question, "Did we accomplish what we set out to do? Tests, quizzes, teacher observations, projects, and demonstration are evaluation tools that help to answer this question. Thus evaluation should be a significant aspect of the teaching process; teaching does not occur, according to the model, unless evaluation of learner performance occurs.
A
C
B
D
Instructional Objectives Instructional Objectives Instructional Objectives Instructional Objectives
E
Feedback Loop The model shows a fifth component, the Feedback Loop that can be used by the teacher as both a management and a diagnostic procedure. If the results of evaluation indicate that sufficient learning has occurred, the loop takes the teacher back to the Instructional Objectives component, and each successive component, so that plans for beginning the next instructional unit can be developed. (New objectives are needed, entering behavior is different, and methods will need to be reconsidered,) But when evaluation results are not so positive, the Feedback Loop is a mechanism for identifying possible explanations. (Note the arrows that return to each component.) Were the objectives too vaguely specified? Did students lack essential prerequisite skills or knowledge? Was the film or text relatively ineffective? Was there insufficient practice opportunity? Such questions need to be asked and frequently are. However, questions need to be asked about the effectiveness of the performance assessment procedures also, perhaps more frequently than they are. Were the test questions appropriate? Were enough observations made? Were directions clear to students? The Feedback Loop returns to the Performance Assessment component to indicate that we must review and assess the quality of out evaluation procedures, after the fact to determine the appropriateness of the procedures and the accuracy of the information. Unless the tools of evaluation are developed with care, inadequate learning may go undetected or complete learning may be misinterpreted as deficient. In sum, good teaching requires planning for and using good evaluation tools, Furthermore, evaluation does not take place in vacuum. The BTM shows that other components of the teaching process provide cues about what to evaluate, when to evaluate, and how to evaluate. Our
purpose is to identify such cues and to take advantage of them in building tests and other assessment devices that measures achievement as precisely as possible. (B) Assessment decision maker who is concerned about all aspects of the educational endeavour. The key point to consider and keep in mind is that evaluation involves appraisal of particular goals or purposes. Useful information may be obtained for evaluation procedures by both formal and informal mean and should include information collected during instruction as well as in the end of the course date. According to Ahmanrt and Giock (1985) School Administrators, guidance personnel, classroom teacher, and individual students require information that will allow them to make informed and appropriate decision regarding their respective educational activities. Ideally, they should be aware of all the alternatives open to them, the possible outcomes of each alternative, and the advantages and disadvantages of the respective outcomes, Educational and psychological measurement can help individuals with these matters. (C)
Tyler, 1966: Airasian and Madaus. 1972: Gronlund 1976:
Thorndike and Hagen, 1977: rightly observe that the data secured through testing procedures may have uses as give below: First, measurement data may be employed in the placement of students on one or another instructional programme. Usually pupils take a pretest to measure whether they have mastered the skills that are prerequisite to admittance to a particular course or instructional, sequence. For instance, foreign language and mathematics programmes are usually arranged in some hierarchical order so that achievement at each level of learning depends on mastery of the preceding level. The student is lead from the entering position in the hierarchy to the terminating phase via intermediate steps, based upon the information provided by a pretest a student can be placed: (1)
At the most appropriate point in the instructional sequence.
(2)
In a programme with a particular instructional strategy on
(3)
With an appropriate teacher.
Second, measurement data can be used in formative evaluation. Tests are administered to students to monitor their success and to provide them with relevant feedback. The information is employed les to grace a student than to make instructions responsive to the student's strengths anorweaknesses as identified by the measurement device, Mastery learning procedures emphasize the use of formative tests to provide detailed information about each student's grasp of a unit's objectives. Third, measurement data has a place in diagnostic evaluation. Diagnostic testing takes over where formative testing /eaves off When a student fails to respond to the feedback corrective activities associated with formative testing a more detailed search for the source of the learning difficulty is indicated. Remediation is only possible when teacher understands the basis of a student's problem and then designs instruction to address the need. Forth, measurement data may be used for summation purposes. Such testing is employed to certify or grade students at the completion of a course or unit of instruction. Often the result is `final' and follows the student throughout his or her academic career (as in the case or college and university transcripts). It is this aspect of evaluation that some educators final particularly objectionable. Fifth, measurement data are used by employers educational institutions in making the selection by decisions. Many jobs and slots in education& programme are limited in number, and there are more applicants than positions. In order to identify the most promising candidates standardized tests may be administered to the applicants. The information provided by the tests presumably increases the accuracy and objectivity of administrator's decisions. College Board examinations are used by many universities in admitting students to graduate and professional schools likewise employ data from standardized testing programme make their entrance decisions.
Sixth, school officials in making curricular decisions in order to evaluate existing programme use measurement data and to decide among instructional alternative. School administrators need to assess their students' current levels of performance the strengths and weaknesses of the evidence. Seventh, measurement data finds a place in personal decisionmaking. Individuals confront a variety of choices at any number of points in their lives. Should they attend college or pursue some other type of post-high school training? What kind of Job seems most suited to their needs? What sort of training programme should they enter? Measures of interest, temperament, and ability can give individuals insights that can prove helpful in the decision-making process. Types of Assessment Tests/ and other assessment procedures can be classified in terms of their functional role in classroom instruction. One such classification system follows the sequencer which assessment procedures are likely to be used in the classroom. These categories classify the assessment of student performance in the following – manner: 1.
Placement assessment
To determine student performance at the beginning of instruction. 2.
Formative assessment To monitor learning progress during instruction-
3.
Diagnostic assessment To diagnose learning difficulties during instruction.
4.
Summative assessment To assess achievement at the end of instruction.
Although a single instrument may sometimes be useful for more than one purpose (e.g., both form formative and summative assessment purposes), each of these types of classroom assessment typically requires instruments specifically designed for the intended use.
All these types of assessment are discussed below in detail. Placement Assessment This is also called Need Analysis Assessment. Placement assessment is concerned with the student's entry performance and typically focuses on questions such-as the following: (1) Does the student possess the knowledge and skills needecF to begin the planned instruction? For example, is a student's reading comprehension at a level that allows him or her to do the expected independent reading for a unit in history, or does the beginning algebra student have a sufficient command of essential arithmetic concepts? (2) To what extent has the students already developed the understanding and skills that are the goals of the planned instruction? Sufficient levels of comprehension and proficiencies might indicate-the desirability of skipping certain units or of being placed in a more advanced course. (3) To what extent do the student's interests, work habits, and personality characteristics indicate that one mode of instruction might be better than another (e.g., group instruction versus independent study)? Answers to questions like these require the use of a variety of techniques: records of past achievement, pretests on course objectives, self-report inventories, observational techniques, and so on. The goal of placement assessment is to determine for each student the position in the instructional sequence and the mode of instruction that is most beneficial. Formative Assessment According to Gron Lund (1990): Formative assessment of work is used while it is in process of being carried out so that the assessment affects the development of the works. Formative Assessment is a part of the instructional process. When incorporated into classroom practice, it provides the information -needed to adjust teaching and learning while they are happening. In this sense, formative assessment informs both teachers and students about student understanding at a point when timely adjustments can be made.
These adjustments help to ensure students achieve, targeted standardsbased learning goals within a set time frame. Although formative assessment strategies appear in a variety of formats, there are some distinct ways to distinguish them from summative assessments. Formative assessment is used to monitor learning progress during instruction; its purpose is to provide continuous feedback to both student and teaching concerning learning successes and failures. Feedback to students provides reinforcement of successful learning and identifies the specific learning errors and misconceptions that need correction. Feedback to the teacher provides information for modifying instruction and for prescribing group and individual work. Formative assessment depends heavily on specially prepared tests and assessments for each segment of instruction (e.g., unit, chapter. Tests and other types of assessment tasks used for formative assessment are most frequently teacher made, but customized tests for publishers of textbooks and other instructional materials also can serve this function. Observational techniques are, of course, also useful in monitoring student progress and identifying learning errors. Because formative .assessment is directed toward improving learning and instruction, the results typically are not used for assigning course grades. Diagnostic Assessment According to Gron Lund (1990): Diagnostic assessment is concerned with those educational' problems which remains unsolved even after the corrective prescription of formative assessment. Diagnostic assessment is a highly specialized procedure. It is concerned with the persistent or recurring learning difficulties that are left unresolved by the standard corrective prescriptions of formative assessment. If a student continues to experience failure in reading, mathematics, or other subjects, despite the use of prescribed alternative methods of instruction, then a more detailed diagnosis is indicated. To use a medical analogy, formative assessment provides first-aid treatment
for simple learning problems and diagnostic assessment searches for the underlying causes of problems that do not respond to first-aid treatment. Thus, diagnostic assessment is much more comprehensive and detailed. It involves the use of specially prepared diagnostic tests as well as various, observational techniques. Serious learning disabilities also are likely to require the services of educational, psychological, and medical specialists, and given the appropriate diagnosis, the development of an individualized education plan (IEP) for the student. The aim of diagnostic assessment is to determine the causes of persistent learning problems and to formulate a plan for remedial action. Summative Assessment The assessment that is carried out at the end of a piece of work is called summative assessment. Summative assessment typically comes at the end of a course (or unit) of instruction. It is designed to determine the extent to which the instructional goals have been achieved and is used primarily for assigning course grades or free certifying student mastery of the intended learning outcomes. The techniques used in summative assessment are determined by the instructional goals, but they typically include teacher made achievement tests, ratings on various types of performance (e.g., laboratory, oral report), and assessments of products (e.g., themes, drawing, research reports). These various sources of information about student achievement may be systematically collects into a portfolio of work that may be used to summarize or showcase the student's accomplishments and progress. Although the main purpose of summative assessment is grading, or the certification of student achievement, it also provides information for judging the appropriateness of the course objectives and the effectiveness of the instruction. 1.1.3
Measurement
Meaning &Definition of Measurement Literally the verb measure means to find or determine the 'size', `quantity' or 'quality' of anything. According to Chambers Dictionary the
term 'measure' means `to find out the size or amount of something'. "Measurement" in the International Dictionary of Education (by G Terry Page & J.B. Thomas) means "the act of finding the dimension of any object and the quantity found by such an act. The 'Oxford Advance Learner's Dictionary defines `measurement' as the 'standard or system used in stating the size, quantity or degree of something.' It is the way of assessing something quantitatively. It answers the question "How much?" In other words we can say that measurement is the quantitative aspect of evaluation. With the help of measurement we can easily describe students' achievement by telling their scores. These definitions show that 'measurement' is the quantitative assessment of something. Now let's see how the term is defined specifically in education. L. R. Gay, (1985) defines measurement as "a process of quantifying the degree to which someone or something possesses a given trait, i.e. quality, characteristics or features." Educational Measurement (The concept of measurement in education) In Education, the term 'measurement' is used in its specific meanings. It is the quantitative assessment of the performance of a student, teacher, curriculum or an educational program. We can say that the quantitative score used for educational evaluation is called measurement. The term is used for the data collected about student or teacher performance by using a measuring instrument in a given learning situation. It shows the exact quantity or degree of the performance, traits or character of the person or thing to be measured. For example instead of saying that Hamid is underweight for his age and height, we can say that Hamid is 18 years old, 5' 8" tall, and weight only 85 pounds. Similarly, instead of saying that Hamid is a more intelligent than Zahid, we can say that Hamid has a measured. IQ of 125 and Zahid has a measured IQ of 88. In each of the above cases, the numerical statement is more precise, more objective and less open to interpretation than the corresponding verbal statement.
Steps of measurement There are two steps used for in the process of measurement. The first step is to devise a set of operations to isolate the attribute and make it apparent to us. Just a standard is used for judging the durability of a thing, in the same way educators and psychologists use various methods for testing the behaviour or performance of a student. For this purpose they often use Stanford-Binet Tests or other tests that include operations for eliciting behaviour that we lake to be indicative of intelligence. The second step in measurement is to express the results of the operations established in the first step in numerical or quantitative terms. This involves an answer to the questions, how many or how much? Just millimetre is used as a unit for indicating the thickness of a thing, in the same way educators and psychologists use some numerical units for gauging intelligence, emotional maturity and other attributes. Thus each step in measurement rests on human- fashioned definitions. In the first step, we define the attribute that interests us. In the second step, we define the set of operations that will allow us to identify the attribute, and express the result of our operations. Difference between Evaluation and Measurement Some people use 'evaluation' and 'measurement' in the same meaning. Both the terms are used for the process of assessing the performance of the student and collecting information about an educational objective. Both tell how effective the school programme has been and refer to the collection of information, appraisal of students, and assessment of programme. Some recognize that measurement is one of the essential components of evaluation. But there is difference between the two terms. Roughly speaking, `measurement' is the quantitative assessment whereas 'evaluation' is the quantitative as well as qualitative assessment of the performance of a student or an educational objective. Measurement is a limited process used for the assessment of limited and specific educational objectives. On the other hand, evaluation is much more comprehensive term used for all kinds of educational objectives. Moreover, for measurement Evaluation is the continuous inspection of all
available information concerning the student, teacher, educational programme and the teaching- learning process to ascertain the degree of change in students and form valid judgements about the students and the effectiveness of the programme. On the other hand 'measurement' is the collection of data about the performance of a student, teacher or curriculum etc. However, both 'evaluation' and 'measurement' are closed closely related. We cannot separate one from the other. Both are used for assessing the effectiveness of a programme of the appraisal of student. Measurement collects data directly from the objects of concern of the students. Other information is collected from students by non-testing procedures. Information provided by testing and non-testing is the best thought of material to be used in the evaluation process. The Importance of Measurement in Education Measurement plays very important role in the teaching-learning process. Without measurement we cannot assess the effectiveness of an educational programme, the school or its personnel. For effective teaching, it is necessary for the teacher to be aware of the strengths and weaknesses of his teaching method. Similarly, for an effective learning, it is necessary for the student to be aware of the possible outcomes of all the alternatives. He should also be informed about the advantages and disadvantages of the respective outcomes. All this is impossible without measurement. Without measurement, how can a teacher be aware his method of teaching or how a student can be informed about the outcomes of the alternatives. Without measurement, evaluation is impossible and without evaluation we cannot get knowledge of the effectiveness of an educational programme. Measurement tells us about the characteristics of students, their progress in studies and their achievements in various subjects. It also tells how much or to what extent the instructional objectives of the school and the individual classroom teacher being achieved? Measurement serves as a guideline for students to develop their educational and vocational plans for the future. With the help of measurement, information is gathered about school programmes,
policies, and objectives. This information is conveyed to parents and other members of the community. Similarly, measurement data are used by employers and educational institutions in making the selection by decision. With the help of standardized tests, the administrators collect information about every applicant. The information provided by the tests increases the accuracy and objectivity of administrators & decision makers. In this way measurement data are employed by school officials in making curricular decisions. In short, measurement occupies the central place in the process of teaching and learning. It is the only mean through which the educational condition can be improved. The Function of Measurement and Evaluation Measurement' and 'evaluation' are interdependent ("N. We cannot separate one from the other just as we cannot separate the two sides of a coin. Evaluation is the qualitative aspect of anything, which is based on the quantitative value (measurement) of that thing. Without measurement we cannot make an exact evaluation of a thing. In this respect evaluation and measurement perform the same functions in the education. Cron Back, in his book "Essentials of psychological testing" has discussed the following functions of measurement and evaluation. (1)
Effectiveness of Educational Programme
In education, the concerned people and personnel must be aware of the effectiveness of an educational programme. This is possible only by making an evaluation of that programme. By evaluation a teacher is able to know as to what extent the method of teaching is effective. He is also able to know as to what extent the equipment of laboratory is effective. This will enable him to improve his method of teaching make learning process effective. (2)
Prediction
After evaluation it is possible to predict the performance of students in future. By evaluation we know the aptitude and interest etc. with the help of which we guide them to take admission in institution which is according to his aptitude and interest. So, on the basis of evaluation we can plan for the future. (3)
Selection
Measurement and evaluation is used during the selection of suitable persons for different jobs in Govt. as well as semi Govt. departments. (4)
Classification
Evaluation is helpful in the classification in all educational institutions. At the end of every year, some tests are given to students to check their ability and make classification on the basis of results obtained from these tests. Another educational psychologist, Camp, adds that evaluation plays important function in making maladjusted students, students as useful members of the society by finding their interests and attitudes. Students suffering from inferiority complex can also be treated after their proper evaluation. In short, evaluation and measurement have important functions in education. They serve as guidelines for students, teachers, ' counsellors and administrators. 1.1.4
Test
Measurement and evaluation are the two processes that are used to collect information about the strengths and weaknesses of an educational programme or the performance of a student, teacher or other personnel. But these processes need some instruments for their operations. Such instruments are called tests. So, the instruments that are used to measure the sample of students' behaviour under specific conditions are called tests. In other words we can say that:
"A test is a systematic procedure for measuring a sample of students' behaviour under specific conditions." Some other definitions of test are given below: 1.
A procedure for critical evaluation; a means of determining the presence, quality, or truth of something.
2.
A series of questions, problems, or physical responses designed to determine knowledge, intelligence, or ability.
3.
The means by which the presence, quality, or genuineness of anything is determined: (e.g. a test of a new product.)
4.
The trial of the quality of something: (e.g. to put to the test.)
5.
A particular process or method for trying or assessing.
6. A set of problems, questions, etc., for evaluating abilities or performance. A test consists of a number of questions to be answered, a series of problems to be solved, or a set of tasks to be performed by the examinees. The questions might ask the examinees to define a word, to do arithmetic computations, or to give some information. The questions, problems and tasks are called test items. Difference between Test, Measurement and Evaluation: William Wiersma and Stephen G. Jurs (1990) in their book "Educational Measurement and Testing" remarks that the terms of Testing, measurement, assessment and evaluation are used with similar meanings but they are not synonymous though they are related with each other. They define these terms as follows:Test: "(It) has a narrower meaning than either measurement or assessment. Test commonly refers to a set of items or questions under specific conditions. When a test is given, measurement takes place; however, all measurement is not necessarily testing". Measurement: "For all practical purposes assessment and measurement can be considered synonymous. When assessment is taking place,
information or data are being collected and measurement is being conducted". Evaluation: "Evaluation is a process that includes measurement and possibly testing but it also contains the notion of a value judgment. If a teacher administer a test to a class and computes the percentages of correct responses, measurement and testing have taken place. The scores must be interpreted which may mean converting them to values like As Bs Cs and so on or judging them to be excellent, good, fair or poor. This process is evaluation because the value judgments are being made". Another distinction is given by Normane E. Gronlund (1985) who defines these terms as follows in the book "Measurement and Evaluation in Teaching". Test: "An instrument or systematic procedure for measuring a sample of behaviour. (Answers the question "How well does the individual perform-either in comparison with others or in comparison with a domain of performance tasks"? Measurement: "The process of obtaining numerical description of the degree to which an individual possesses a particular characteristic. (Answers the question "How much?"). Evaluation: "The systematic process of collecting, (Classroom) analyzing and interpreting information to determine the extent to which pupils are achieving instructional objectives. (Answers the question "How good"). Similarly Anthony J. Nitko (1983) in his book "Educational Tests and Measurement" makes the distinction between Test, Measurement and Evaluation in the following words: Tests: "Tests are systematic procedures for observing persons and describing them with either a numerical scale or a category system. Thus tests may give either qualitative or quantitative information". Measurement: "Measurement is a procedure for assigning numbers to specified attributes or characteristics of persons in a manner that
maintains the real world relationships among persons with regard to what is being measured". Evaluation: "Evaluation involves judging the value or worth of a pupil of an instructional method or of an educational program. Such judgements may or may not be based on information obtained from tests". Robert L. Ebel and David A. Frisible (1986) in their book "Essentials of Educational Measurement" rightly observe. "All tests are a subset of the quantitative tools or techniques that are classified as measurements. And all measurement techniques are a subset of the quantitative and qualitative techniques used in evaluation." Table showing relationship Between Testing, Measurement and Evaluation: Test
Measurement
An instrument or systematic procedure for measuring a sample of behavior
The process of obtaining a numerical description of the degree to which an individual possesses a particular characteristic”
A systemati collecting and order to make
Answers the question ‘How well does the individual perform-as compared to others.
Answers much?’
‘How
It answers the
It is means information
collecting
It gives Numerical Value to some trait.
Involves quantitative decision-maki
Its objective is to find out the facts pertaining to some aspect.
Its objective is to present the information objectively.
Its objective decisions abo of educational
Test is only a instrument to obtain data
Measurement quantifies data and is essential part of evaluation
Depends up measurement
Types of Tests
of
the
question
Eva
TESTS Ability Tests
Achievement Test
Aptitude Test
Essay Tests
Objective Tests
Attitude Tests
Character Tests
Personality Tests
Intelligent Test
Interest Tests Adjustment Tests
As it is shown in the diagram above, tests can be classified into two broad categories according to the behaviour tested: ability tests and personality tests. These two types are discussed in detail and are further classified into sub-types in the following lines. (A) Ability Tests These tests are used to test the ability of a student. These tests measure the maximum performance of a student that a student can do. Ability tests are further classified into three types; (1) achievement tests, (2) aptitude tests, (3) intelligence tests. These are discussed in the lines below. (1) Achievement tests: These tests are used to appraise the outcomes of classroom instruction. They measure the attained ability of a student i.e. what a student has learnt to do. Achievement tests are further classified into two types of tests i.e. 'Essay type tests' and 'objective type tests'. (These two types of tests will be discussed in detail in the next question). (2) Aptitude tests: Aptitude Tests are those tests that are used to measure the potential ability of a student i.e. what a student can learn to do. They measure the capacity of a student to learn a given content.
According to Hull, C. L. "An aptitude test is a psychological test designed to predict an individual's potentialities for success or failure in a particular occupation, subject for study, etc. this shows that an aptitude test is a test designed to discover what potentiality a given person has for learning some particular vocation or acquiring some particular skill. Achievement tests and aptitude tests seem to be the same. But the distinction between the two is that they are different in use. If a test is used to measure the present attainment, it is called achievement test. And if a test is used to predict the future level of performance, it is called an aptitude test. (3) Intelligence Tests: Intelligence Tests are those tests that are used to measure the native capacity or the overall mental ability of a student. These are also called scholastic aptitude tests or tests of mental ability. There are many kinds of intelligent tests but the most popular one is the concept of (IQ) introduce by Termen. IQ is computed by dividing the mental age (MA) of a student by his physical age or chronological age (PA or CA) i.e. the actual age of the student. Then the result is multiplied by 100. I.Q =
MA CA
× 100
Where: I.Q. = Intelligence Quotient M.A. = Mental Age C.A. = Chronological Age (Physical Age) (B)
Personality Tests
Tests used for the assessment of personality of a student are called personality tests. They measure the typical performance of a student i.e. what a student will do. They are universally administered almost all over the world in various fields, vocations, institutions, and for the selection of recruits. In Pakistan, too, personality tests are used for
job selection and for the selection of army recruits like ISSB examinations. Personality tests include attitude tests, interest tests, adjustment and temperament tests, character tests, and tests of other motivational and interpersonal characteristics. Uses of Tests Tests play important role in teaching- learning process. Without tests we cannot make evaluation or assessment of a student's or neither teacher's performance nor we can collect information about the effectiveness of an educational programme. That is why tests are very important in education. They motivate students for learning. They serve a number of purposes in a variety of educational activities. The following are the different uses of tests; 1.
Uses of tests in teaching process
With the help of the result obtained from tests, teachers can easily collect information about aptitude, intelligence, interests, attitude and the overall performance of the students. He comes to know the strengths and weaknesses of his teaching method. It becomes easy for the teacher to grade students in a subject. Teats' results enable him to know how the future success of a student in a subject can be predicted. 2.
Uses of tests in learning process
The student is the centre of intere ;t in teaching –learning process. All kinds of educational activiti .;s are performed for th sack of student. That is why the use and importance of tests in th process of learning is greater than in any other activity. Tests hel ,students in knowing their strengths and weaknesses in a subject. The resul,S obtained from these tests serve as guideline for students. They motivate students to study. 3.
Uses of Tests in Guidance
Tests show the overall performance of the students. Therefore; they enable the examiner to know how to guide students educational and vocational choice. Tests also make parents aware o the aptitude of their
children and can make a plan for their proper guidance. The result of the tests in itself serves as a guideline for the students. 4.
Uses of Tests in Administration
The results obtained from the tests provide the administrators of the deportment with useful information In the light of these tests, they can easily decide how to promote students, how to admit them an&how to modify (trie.7) school objectives, instructional methods and curricula. They can then easily decide how to make the teaching–learning processes effective. 5.
Uses of Tests in Research
The data collected from tests are uses as powerful tools in research and experimentation in classroom. The research workers use these data in their genetic or ease study research. In short, tests are used in almost all educational activities. They are the real tools with the help of which information about teachers, students, curricula and etc. are gathered. And in the light of this information, teaching and learning process is improved. 1.2
THE PURPOSE OF TESTING
Introduction: The purpose of test is usually included the test is announced or at the beginning of the semester when the evaluation procedures are described as a part of the general orientation to the course. Should there be any doubt whether the purpose of the test is clear to all pupils, however it culd be explained again at at the time of testing. This is usually done orally. The only time, a statement of the purpose of the test needs to be included in the written direction is, when the test is to be administered to several sections taught by different teachers, then a written statement of purpose ensures, greater uniformity. There are various types of test being applied in the educational institutions, because no a child’s ability interests and personality. One test measures only a specific ability that is why school administers use many different types of
tests even in one single area such as intelligence, move than one test are needed over a period of years to obtain a reliable estimate of ability each test serves its own purpose, however, testing and evaluation serve following purposes. Types of Testing: There are four types of testing. Placement Testing:
Most placement tests constructed by classroom teachers are pretests designed to measure. 1. 2.
Whether pupils possess the prerequisite skills needed to succeed in a unit or course or. To what extent pupils have already achieved the objectives of the planned instruction. In the first instance we are concerned with the pupils readiness to begin the instruction. In the second we are concerned with the appropriateness of our planned instruction for the group and with proper placement of each pupil in the instructional sequence.
Formative Testing:
Formative tests are given periodically during instruction on monitor pupils learning progress and to provide ongoing feedback to pupils and teacher, formative testing reinforces successful learning and reveals learning weaknesses in need of correction. A formative test typically covers some predefined segment passes a rather limited simple of learning tasks. The test items may be easy or difficult, depending on the learning tasks in the segment of instruction being tested, formative tests are typically criterion referenced mastery test, but norm-referenced survey tests can also survey this function, ideally, the test will be constructed in such a way that corrective prescription can be given for missed test items or sets of lest items. Because the main purpose of the test is to improve learning the result are seldom used for assigning grades.
Diagnostic Testing:
Diagnosis of persistent learning difficulties involves much more then diagnostic testing, but such tests are useful in the total process. The diagnostic test takes up where the formative test leaves off if pupils do not respond to the feedback corrective prescriptions of formative testing. A more detailed search for the source of testing we will need to include a number of a test items in each specific area, with some slight variation from item to items in diagnosing pupils, difficulties in adding whole numbers, for example we would want to include addition problems containing. Various numbers combination with some not requiring carrying and some requiring carrying, to pinpoint the specific types of error each pupil is making. Because our focus is on the pupils learning difficulties, diagnostic test must be constructed in accordance with the most common sources of error that pupils encounter. Such tests are typically confined to a limited area of instruction, and the test items tend to have a relatively low level of difficulty.
Summative Testing:
The summative test ig given at the end of a course or unit of instruction, and the results are used primarily for assigning grades or certifying pupil mastery of the instructional objectives. The result can also be used for evaluating the effectiveness of the instruction. The end of the course test (final examination) is typically a norm-referenced survey test that is broad in coverage and includes test items with a wide range of difficulty. The more restricted end of unit, summative test might be norm referenced or criterion referenced depending on whether mastery or developmental outcomes are the focus of instruction. Purpose of Testing: 1.
To Certify Pupils’ Achievements / Grading:
Tests are given to the students to ascertain their achievements tests provide the teacher with student’s actual achievements instead of an intuitive generalization based on simple observation. These tests given the teacher an objective and comprehensive picture of each pupil’s
progress. This is important because all concerned persons (students themselves, student’s parents, teachers, counselors, administrators, employers, admission officers, and even community) need to know students performed in school and in particular courses.
To report Student’s Progress to Parents:
Testing gives the teacher in objective and comprehensive picture of each pupil’s progress, so that it could be presented to the present. These reports from the foundation for most effective cooperation between parents and teachers, which results improved learning.
To Report to Administrators:
The results of tests indicate the extent to which the school’s objectives are being achieved from the results of evaluation the administrators become able to identify the weak points and strengths in the teaching programs of their schools and take necessary action for their improvement.
To Assess Learner’s Needs:
To test the pupils’ knowledge and skills at the beginning of instruction enables the teacher to answer the questions like: Do the pupils possess the abilities and skills needed to proceed with the instruction? What, and to what level have the pupils already mastered the intended outcomes? This information helps the teacher in planning his instructional activities.
To Provide Relevant Instruction:
Testing provide a type of continuous feedback, about the usefulness of the instructional process it helps the teacher in changing and adapting the instructional activities continuously according to the student’s needs.
To Furnish Instruction:
Testing factions as an instructional device it not only increases the self-knowledge of the students, but also the attainment of specific
objectives. This practice of giving ‘tests’ is common in our institutions through these the students become aware of their speed of progress, errors, and present status on the basis of which they plan their further efforts.
To Provide Guidance and Counseling:
The results of tests are especially useful for guidance and counseling of the students. These are useful in assisting the students with educational and vocational decisions, guiding them in the selection of curricular and co-curricular activities, and helping them solve personal and social adjustment problems.
To Know the level of Achievement of Objectives:
The first step in the instructional process is to determine the extent to which the pupils achieved the instructional objectives. Testing and evaluation help in this regard tests are useful in determining the learning outcomes of classroom instruction. The teacher can evaluate the success or failure of classroom learning in relation to the test results. The teacher then accordingly adjusts the level and direction of classroom instruction.
To Analyze the Instruction Objectives:
The information from carefully developed tests and evaluation is used to assess the appropriateness and attainability of the instructional objective. The instructional objectives are modified in the light of the evaluation information.
To Discover Maladjusted Children:
In every school there are some students who present severe problems of educational or social adjustment. These include the withdrawn, the unhappy, the mentally retarded, and others who are not adjusting to the pattern of the school. The standardized tests help the teachers and counselors to understand and help such students.
To Appraise Educational instrumentalities:
Testing and evaluation is useful, in appraisal for educational instrumentalities such as teachers, teaching methods, teaching materials and text books.
To Conduct Research:
Test and evaluation data is important in research programs. The information obtained from evaluation is used to compare the effectiveness of different curricula, different teaching methods and different organizational plans techniques of evaluation, and to find out the ways to improve to teaching learning process.
To Change the Curricula:
One purpose of the tests and evaluation is to find out the weak points in the curriculum so that it could be changed in accordance will the need, of the society.
To measure Behavior in Controlled Situation:
Another purpose of tests is to measure the behavior of the subject or student under controlled conditions. 1.3
GENERAL PRINCIPLES OF ASSESSMENT:
Assessment is an integrated process for determining the nature and extent of student learning and development. In order to make this process effective, the following principles are taken into consideration. 1)
2)
Clearly specify what is to be assessed the priority in the assessment process. The effectiveness of assessment depends as much on a careful description of what to assess as it does on the technical qualities of the assessment procedure used. When assessing student learning, this means clearly specifying the intended learning goals before selecting the assessment procedures to use. An assessment procedure should be selected because of its relevance to the characteristics or performance to be measured. Assessment procedures are frequently selected on the basis of their objectivity, accuracy or convenience.
3)
4)
5)
Comprehensive assessment requires a variety of procedures. No single type of instrument or procedure can assess the vast array of learning and development outcomes emphasized in a school program. Multiple choice and short answer tests of achievement are useful for measuring knowledge, understanding, and application outcomes, but essay tests and other written projects are needed to assess the ability to organize and express ideas. A complete picture of student achievement and development requires the use of many different assessment procedures. Proper use of assessment procedure requires an awareness of their limitations. Assessment procedures range from very highly developed measuring instruments to rather crude assessment devices. Even the best educational and psychological measuring instrument yield results that are subject to various types of measurement error. Not best or assessment asks all the questions or poses all the problems that might appropriately be presented in a comprehensive coverage of the knowledge, skills and understanding relevant to the content standards or objectives of a course or instructional sequence. Instead only a sample of the relevant problems of questions is presented. Even in a relatively narrow part of a content domain, such as understanding photosynthesis or the addition and subtraction of fractions, there are a host of problems that might be presented, but any given test or assessment samples but a small fraction of those problems. Limitations of assessment procedures do not negate the value of tests and other types of assessments. A keen awareness of the limitations of assessment instruments makes it possible to use them more effectively. Cruder the instrument, the greater its limitations and consequently, the more caution required in its use. Assessment is a means to an end, not an end in itself. The use of assessment procedures implies that some useful purpose is being served and that the user is clearly aware of this purpose. To blindly gather data about students and then file the information
away is a waste of both time and effort. Assessment is best viewed as a process of obtaining information on which to base educational decisions. 1.4
TYPE OF EVALUATION PROCEDURE
The evaluation process can basically be carried out at two main levels; programme and student. Student evaluation can be further be subdivided into formative and summative evaluation. Evaluation Procedure Student Evaluation Programme Evaluation
Formative Evaluation
Summative Evaluation Diagnostic Evaluation
Proagramme Evaluation: program evaluation is systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency. When our concern is judging the compatibility between the aims and the learning out comes of a programme, the emphasis is on the efficacy of that programme. On the other hand a ‘good’ programme may be badly implemented. The task of quality and control is to maintain and maximize the efficiency of a programme. The quality content of a programme is to determined, among other factors, by i. ii. iii. iv. v.
Its conceptual quality. Logical relevance to the need of the student. Simplicity and comprehensibility in terms of readability and literacy level of the content. Relative stability and survival value in the literature. Applicability to familiar and novel situation.
To matter how good a programme may be; the maintenance system must be well facilitated. The school administrators head of subjects unit, supervisors. The teacher and the pupils must be activity involved if successful implementation of the programme is to be realized, the teacher being the main executor of the programme must be will trained not just to be able to teach facts but so select facts that relate to other facts and principles. The teacher education programmes in the advanced teacher colleges and the universities must prepare teachers to be able to teach their subjects effectively. In order to be implemented a programme should be designed in such a way that under favourable conditions certain intended learning outcomes will emerge. The school teacher, the headmaster and supervisor must gather information from time to time in order determine the success or seakness of the programme. If desirable outcomes are observed, the focus of all concerned with instruction should be to improve the programme through an effective maintenance system. If the product (students) produced are of poor quality, corrective measures are selected and applied in order to achieve the desired results. If after all these efforts, the products are still found to be poor the programme is usually abandoned. Several process are involved in the input out put process. The teacher is the most important component of the maintenance process of the programme.t he interacts with the students with the staff, experts and administrators and forms a bridge between hem and learning materials. Often he acts as the input analyzer and an identifier as well as the teaching agent of the programme.t he external sensor examines the learning environment to identify changes perhaps economic, political and psychological or social within the environment that can destabilize the system. The input analyzer processes all the information supplied by the external sensor and transmits it to the school administrator for appropriate action. He analyze and organize information obtained form the input variable into a comprehensible structure to be used in planning
activities. The identifier (usually the teacher or his head of department) examines the out put and the internal working conditions of the maintance system. It is he who provide the decision rules (head master) with a realibel picture of the internal condition of the system. The input output information provided by the analyzer and the identifier becomes the input of decision rule and it is utilized by the headmaster to produce a decision policy or instruction to the teacher. Any given programme introduced into school setting is not left in its naked form but assumes a different from for that setting. It contents are emphasized as teacher, administrators and students. Programme education can be carried out through the use of surveys, interview, experimental students and so on. Student Evaluation: As pointed out earlier, testing forms an integral part of student evaluation. The purpose of this type of evaluation is to determine how well a students is performing in a programme. Through a series of oral questions, paper-pencil tests, manipulative still tests. Tutorials discussions, tutorials, individualized instruction, assignments, projects and so on the student is gradually guided towards a desired goal. Basically there are two types of student evaluation. i. i.
Forative and
ii. Summative
Formative Evaluation:
Formative evaluation aims at ensuring a healthy acquisition and development of knowledge and skills by students. Formative evaluation is also used to identify students in order to guide them towards desiable goals. As student needs and difficulties are identified, appropriate remedial measures are taken to solve such problems. The purpose is to find out whether after learning experience students are able to do what they were previously unable to do. A short term objectives of formative evaluation may be to help student perform well at the end of the programme. It is a process of channeling input variables through a process that will yield expected outputs. The classroom is variables
through a process that will yield expected outputs. The classroom teacher is the best formative evaluator. Formative evaluation attempts. 1. 2.
3.
To identify the content (knowledge or skills). To appreciate the level of cognitive abilities such as memorization, classification, comparison, analysis, explanation, quantification, application and so on. To specify the relationship between content and levels of cognitive abilities.
In other words, formative e evaluation provides the evaluator with useful information about the strength or weakness of the student within an instruction context. 1. 2. 3. 4.
Formative evaluation is alone during an instructional programme. The instructional programme should aim at the attainment of certain objectives during the implementation of the programme. Formative evaluation is done to monitor learning and modifying the programme if needed before its common completion. Formative evaluation is for current students.
Characteristics of Formative Evaluation 1. 2. 3. 4. 5. 6.
ii.
It relatively focuses on molecular analysis. It is because seeking. It is interested in the broader experience of the programme users. It is designing exploratory and flexible. It seeks to identify influential variables. It requires analysis of instructional material for mapping the hierarchical structure of the learning tasks and actual teaching of the course for a certain period. Summative Evaluation
Summative evaluation is primary concerned with purposes progress and outcomes of the teaching learning process attempts as far as possible to determine to what extent the broad objective of a programme have been achieved. It is based on the following assumptions.
1. 2. 3. 4.
That the programmer’s objectives are achieved. That the teaching learning process has been conducted effiently. That the teacher student material interaction have been conductive to learning. That there is uniformity in classroom conditions for all learners.
Unlike formative evaluation, which is guidance oriented summative evaluation is judgmental in nature. Promotion examination, the first school leaving certificate examination, the public examination belongs to this form of evaluation. Summative evaluation carries threat with it in that the student may have no knowledge of the evaluator. According to A.F Nikto (1983) summativn already completed programme, procedure or product. Summative evaluation is done at the conclusion instruction and measures the extent to which student have attained the desired out comes. Chief Characteristics of Summative Evaluation: 1. 2. 3. 4. 5. 6. 7.
It lends to the use of well-defined evaluation design. It Focus on analysis. It provides descriptive analysis. It trends to stress local effects. It is unobtrustive and non-reactive as far as possible. It is concerned with broad range of issues. Its instruments are reliable and valid.
Difference between the Summative and Formative Evaluation In the beginning these terms applied for the evaluation of curricular work only. M. Seriven explains the difference between these terms as follows in his book Evaluation the asurus (1980). “Formative evaluation is conducted during the development or improvement of a programme or product (or person) it is an evaluation conducted for in-house but is may be done by an internal or external evaluator (preferably) a combination. Summative evaluation, on the other hand, is conducted after completion of a programme. (or a course of study) and for the benefit of some external audience or decision malker.
(e.g funding agency or future possible users) though it may be done by an internal or an external evaluator or by a combination”. Gloria, Hitchok and other (1986) state the difference between the summative and formative avaluation in these words. “It is fairly straight forward to produce an “ideal” type of either a summative or a formative profiles. It is far more difficult to combine the two into one unified system. The undervaluing philosophies of the two appear difficult to reconcile”. Following are the main differences between these types of evaluation: 1. 2.
3. 4.
They differ in purpose, nature and timing. Summative evaluation is the terminal assessment of performance at the end of instruction but formative evaluation is the assessment made during the instructional phase to inform the teacher about progress learning and what more is to be done. The summative evolution limits the use of profile and record of achievement but they are regulary use in formative evaluation. In summative evaluation, the assessment is done to test learning outcomes against a set of objectives criteria with out revealing the details of the route to the teacher, which the student followed in reaching that point. Formative evaluation takes the form of a dialogue between the student and teacher in which both determine the task.
Broad Differences Formative and Summative Characteristic
Formative
Su
Purpose
To monitor progress of student getting feedback
To check final
Content focus
Detailed narrow scop
Gernal Board
Methods
Daily assignments
Projects
Observations
Projects
Daily
Weekly, quarte
Frequency
1.5
NORM- REFERENCED AND CRITERION REFERENCED TEST:
Test designed to provide, a measure of performance that is interpretable in terms of an individual’s relative standing in some known group is called norm-referenced test. A norm group may be made up of a students at the local level, district level, provincial level or national level. Types of Norms: There are two type of norms which are following. a)
b)
National Norms: Most standardized achievement and aptitude test require national norms because the tests are intended for used across the country. The norm group should represent the population of student in the country. Local Norms: There are many communities where local norms are more useful than the national for example there may be some cities where the citizen who are above national averages in educational and socioeconomic level.
Characteristics of Norm-Referenced Test 1.
Its basic purpose in to measure student’s achievement to curriculum based skill. Therefore it covers majority of the course.
2.
It is prepared for a particular grade level. As the test is curriculum based therefore. It can only be applied to a particular class for which it is prepared.
3.
It classifies achievement as above average, average or below average for a given grade.
4.
It is generally reported in the form or percentile rank, linear standard score, normalized standard score and grade equivalent score.
5.
Norm-referenced test is likely to have times that are very difficult for the grade level so student can be ranked.
Drawbacks of Norm–Referenced Test 1.
Test item that are answered correctly by most of the pupils are not included in these test because of their inadequate contribution to response variance. They will be the items that deals with the important concepts of course content.
2.
Norm-Referenced test compare an individual performance to the performance of a group called norm group an entirely different conclusion will be reached if the norm group is a collection of university seniors majoring in physics.
a)
Criterion – Referenced Test:
1.
According to Gronlund (1985) a test designed to provide a measure of performance that is interpretable in terms of a clearly defined and delimited domain of learning talks is called criterion-referenced test.
2.
According to wiersna and Stephen (1990) criterion referenced test describe. The performance of the student in the herm of actual skills or task that are included in the test.
b) 1. 2. 3. 4. 5. c)
Haracteristics of Criterion-Referenced Test: It measures student’s achievement of curriculum based skills. It is prepared for a particular grade or course level. It has balanced representation of goals and objectives. It can be administered before and after instruction. It is used to evaluate the curriculum plan, instructional progress and group student’s interaction. Limitation of CRTS:
CRTs tell only whether a learner has reached proficient in a task area but does not show how good or poor in the learner’s level of ability.
Task included in CRTs may be highly influenced by a given teacher interests or biases, leading to general validity problem.
Only some area readily land themselves for listing specific test can be built and this may be a constructing element for teacher.
1.6
EDUCATIONAL:
“Educational assessment can be defined as the process of documenting knowledge skills, attitude and beliefs”. Or “The process of collecting synthesizing and interpreting information to assessment.” General Principles of Assessment: Following are the main principles of assessment. 1. 2. 3. 4. 5.
Clearly specify what is to be assessed has priority in the assessment process. An assessment procedure should be selected because of its relevance to the characteristics or performance to be measured. Comprehensive assessment requires a variety of procedures. Proper use of assessment procedures requires an awareness of their limitations. Assessment is a means to an and not an end in itself.
Clearly specify what is to be assessed: General statements from, content standard or from course objectives can be a helpful starting point but in most cases teachers needs to be more specific for assessment process to be effective. Thus specification of the characteristic to be measured should precede the selection or development of assessment procedures. Specify the intended learning goals before selecting the assessment procedure to use. Example: Content standard in the field of physics night specify that students. Should understand idea documents in field of physics. 1. 2. 3. 4.
Assessment may be in the form of multiple choice. Short answer Essay questions Numerical questions
To establish assessment priorities for such a standard teacher needs to answer the questions such as the following.
Q1.
What idea?
Q2.
What document?
Q3.
What concepts of physics?
The general statement in standard does not answer such questions, but they must be either explicitly or implicitly, to develop assessments. Assessment must be relevant to the performance to be measured: Assessments procedures are frequently selected on the basics of their objectivity, accuracy or convenience although there criteria are important they are secondary to main criterion. Examples: If teachers goal is that students should learn written skills or creative writing, composition, sentence structure so the if multiple choice will be option for assessment then it will be poor one, teacher must include story writing, easy, summaries or such type of things for improving writing stills of a child. “Close match between the intended learning goals and type of assessment is must”. Comprehensive assessment requires a variety of procedure: A lot of procedures are required to assess the knowledge of a person about anything. Things which are to be assessed also play a vital role in connectivity with the procedure some of the procedures are given below.
Multiple choice Short answer Essay test Written projects Observational technique
Multiple-Choice and short answer test of achievement are useful for measuring knowledge, understanding, and application outcomes, but
essay tests and other written project are needed to assess the ability to organize and express ideas. Projects that require students to formulate problems, accumulate to formulate problems, accumulate information through library research or collect data (e.g through experimental observations or interviews) are needed to measure certain skills in formulating and sawing problems observational techniques are needed to assess performance skills and various aspects of students behavior and self-report techniques are use full for assessing interests and attitudes. A complete picture of students achievement and development requires the use of many different assessment procedure. Proper use of Assessment Procedure Requires an Awareness of their Limitations Not a single test can assess whatever the teacher want every procedures has its plus points and negative pointes or we can say it is not suitable for the things to be assessed. So one must how about it and takes care of it so that we can get correct assessment results. Some of the major problems are 1. 2. 3.
Sampling error Chance factor Incorrect interpretations
Sampling Error: An achievement test may not adequately sample a particular domain of instructional content. An observational instrument design to assess a student’s social adjustment may not sample enough behavior for a dependable index of this trait. Sampling can be controlled though careful application of established measurement procedures. Chance Factor: A second source of error is caused by chance factors influencing assessment results, such as guessing on objective Tess, subjective scoring on essay test, errors in judgment on observation devices and in consistent responding on self report instrument.
Through the careful use of assessment procedure we are able to keep these error of assessment to a minimum. Incorrect Interpretation: The incorrect interpretation of measurement results constitutes another major source of error. We usually, more precise the result than the requirement that’s why this problem waists Result must be precise accurately. Assessment is a mean to an end, not an end in itself: The use of assessment procedure implies that some useful purpose implies that some useful purpose is being served and that the user is clearly aware of his purpose. The blindly gather data about students and then file the information away is a waste of time and effort. Assessment is best viewed as a process of obtaining information on which to base educational decisions. Conclusion: All the principles are very important because they are directly linked with the inter predation of information if the requirement is not fulfilled than assessment will be wrong.
UNIT-2: JUDGING THE QUALITY OF THE TEST Definition: Test percentile scores are just one type of test scores you will find on your child's testing reports. Many test reports include several types of scores. Percentile scores are almostalways reported on major achievement that are taken by your child's entire class. Percentile scores will also be found on individual diagnostic test reports. Understanding test percentile scores is important for you to make decisions about your child's special education program. Test percentile scores commonly reported on most standardized assessments a child takes in school. Percentile literally means perhundred. Percentile scores on teacher-made tests and homework assignments are developed by dividing the student's raw score on her work by the total number of points possible. Converting decimal scores to percentiles is easy. The number is converted by moving the decimal point two places to the right and adding a percent sign. A score of .98 would equal 98%. Test percentiles on a commercially produced, norm-referenced or standardized test, are calculated in much the same way, although thecalculations are typically included in test manuals or calculated with scoring software. If a student scores at the 75th percentile on a norm-referenced test, it canbe said that she has scored at least as well, or better than, 75 percent of students her age from the normative sample of the test. Several othertypes of standard scores may also appear on test reports. Percentile rank From Wikipedia, the free encyclopedia The percentile rank of a score is the percentage of scores in its frequency distribution that are the same or lower than it. For example, a test score
that is greater than or equal to 75% of the scores of people taking the test is said to be at the 75th percentile rank. Percentile ranks are commonly used to clarify the interpretation of scores on standardized tests. For the test theory, the percentile rank of a raw score is interpreted as the percentages of examinees in the norm group who scored at or below the score of interest.mm Percentile ranks (PRs) are often normally distributed (bell-shaped) while normal curve equivalents (NLEs) are uniform and rectangular in shape. Percentile ranks are not on an equal-interval scale; that is, the difference between any two scores is not the same between any other two scores whose difference in percentile ranks is the same. For example, 50 _ 25 = 25 is not the same distance as 60 _ 35 = 25 because of the bell-curve shape of the distribution. Some percentile ranks are closer to some than others. Percentile rank 30 is closer on the bell curve to 40 than it is to 20. The mathematical formula is
ce+0.5 fi X 100% N where c is the count of all scores less than the score of interest, f is the frequency of the score of interest, and Nis the number of examinees in the sample. If the distribution is normally distributed, the percentile rank can be inferred from the standard score. 2.1
VALIDITY, METHODS OF DETERMINING VALIDITY:
Introduction: Tests play a central role in the evaluation of pupil learning. They provide relevant measures of many important learning outcomes. Tests and other evaluation instruments serve a variety of uses in the school, for example test of achievement might be used for selection, placement, diagnosis or certification of mastery.
When constructing or selecting tests and other evaluation instruments, the most important question is to what extent will be the interpretation of the scores be appropriate, meaningful and useful for the intended application of the results?So validity is always concerned with the specific use of results. Factors Influencing Validity: A careful examination of test items will indicate wether the test appears to measure the subject matter content and the mental function that the teacher is interested in testing. Following are the factors that prevent the test items from functioning as intended and there by lower the validity of the interpretation. 1.
2.
3. 4.
5.
6.
7.
8.
Unclear Direction: Directions that do not clearly indicate to the pupil how to respond to the items will reduce validity. Reading Vocabulary and Sentence Structure too Difficult: Vocabulary and sentence structure that is too complicated for the pupils will distort the meaning of the test results. Inappropriate level of difficulty: Items that are too easy or too difficult also lower validity. Poorly constructed items: Test items that provide clues to the answers will measure the pupil’s alertness in detecting clues as well as those aspects of pupil performance that the test is intended to measure. Ambiguity: Ambiguous statements confuse the pupil and so causes to discriminate in a negative direction. Inadequate time limits: Time limits that do not provide pupil with enough time to consider the items reduce the validity. Test too short: If the test is too short to provide a representative sample of the performance we are interested in, its validity will suffer accordingly. Improper arrangement:
9.
Test items are arranged in order of difficulty, with the easiest items first. Placing difficult items early may cause pupils to spend too much time. Identifiable pattern of answers: Placing correct answers in some systematic patern will enable pupils to guess the answers more easily.
Methods of Determining Validity: There are several methods of determining the validity of measuring instruments which we may call. 1.
2.
3.
4.
Content Validity: Content validity is evaluated by showing how well the content of the test samples the class of situations. It is especially important in the case of achievement and proficiency measures. It is also known as “face validity”. Concurrent Validity: It is evaluated by showing how well test scores correspond to already accept measures of performance or status made at the same time. For example, we may give a social studies class a test on knowledge of basic concepts in social studies and at the same time obtain from its teacher report on these abilities as far as pupils in the class are concerned. If the relationship between the test scores and the teacher’s report of abilities is high. The test will have high concurrent validity. Predictive Validity: It is evaluated by showing how well prediction made from the tests are confirmed by evidence gathered at some subsequent time. When the tester wants to estimate how well a student may be able to do in college courses on the basis of how well he has done on test he took in secondary schools. Construct Validity: It is evaluated by investigating what psychological qualities a test measures. It is ordinarily used when the tester has no definitive criterion measure of what he is concerned with and hence must use indirect measures. This type of validity is
usually involved in such tests as those of study habits, appreciations, understanding and interpretation of data. Conclusion: In short we can say that validity is specific to the purpose and situation for which a test is used. A test can be reliable without being valid but the converse is not true in other words. It is conceivable that a test can measure some quality with a high degree of consistency without measuring at all the quality it was actually intended to measure. 2.2
FACTORS AFFECTING VALIDITY
Test experts generally agree that the most important quality of test is its validity. The word “Validity” means “effectiveness” or “Soundness”. It refers to the accuracy with which a thing is measured. Types of Validity: Validity is classified into three categories. 1) Content Validity. 2) Criterion related validity. 3) Construct Validity. A good measuring instrument is that which is valid with respect to all these three categories. These are discussed below. i.
Content Validity: Content validity is the degree to which a test measures an intended content area. In other words the content validity of a test refers to the extent to which the test content represents a specified universe of content. For Example: for example if a teacher taught a course of biology and would like to give a test at the end of the course.
ii.
Construct Validity: Construct validity is the degree to which a test measures an intended hypothetical construct. In other words construct validity refers to the extent to which the test throughout the major. There will be a correlation between the new measuring
iii.
iv.
v.
approach / tool with standardized measure of ability in this very discipline (like GRE subject test). Concurrent Validity: Concurrent validity is the degree to which the scores on a test are related to the scores on another, already established, test administered at the same time or to some other valid criterion available at the same time. For Example:We may give a social studies class a test based on knowledge of basic concepts in social studies and at the same time obtain from its teacher a report on these abilities. As far as pupils in the class are concerned measures to construct that it claims to measure. For Example: Examples of the construct are intelligence, creativity ability to apply principles and ability to reason. For example if a teacher wants to measure the ability to reason, and give two reasoning tests to his class. Criterion related Validity: This type of validity is used to predict about the upcoming or futures or current performance and it correlated the test results with another criterion of interest. (Coz by, Zool). For Example: If for an educational program, measures are developed to assess the cumulative student learning. Predictive Validity: Predictive validity is the degree to which a test can predict how well an individual will do in future situation. In other words, predictive validity means the validity of a test or examination which is based upon its correlation with some future variable. For Example: For example one speaks of the predictive validity of school examination for future success in higher education. Similarly, if a small test gives the same standing to an individual in a test which was achieved by him in a much longer test, it will be culled concurrently validity.
Methods of Determining Validity: The methods of deterning validity is also termed as, forms of Expressing validity. There forms, generally used for expressing validity index of the test. 1.
2.
3.
a. b. 2.3
Correlation Coefficient: Test scores are correlated with that of criterion scores. The obtained coefficient of correlation is the extent of validity index of the test. Expectancy Table: Test scores are evaluated or correlated with the rating of the supervisors. It provides empirical probabilities of the validity index. Cross Validation: It means to have another look for correlation coefficient with another criterion or expect any tables with other criterion. It is of two types. Empirical validation Logical or rationale validation RELIABILITY, AND METHODS OF DETERMINING RELIABILITY:
Meaning and Definition: Reliability means consistency of measurement in the words of Ebel and frisbie, “The ability of a test to measure the same quantity when it is administred to an individual on two different occasions by two different teser is called reliability. The reliability indicates the degree to which measurement can be relied upon, to measure the same thing each time is used. In simple words we can say that a good measuring instrument (test) should be reliable in reporting the results if it is done by the same group of student under the same conditions. Reliability is also called dependability or trustworthiness reliability is the degree to which a test consistently measures whatever it
measures. The more reliable a test is, the more confidence we can have that the scores obtained from the administration of the test are essentially the same scores that would be obtained if the test where re-administered. An unreliable test is essentially useless. For example, if an intelligence test was unreliable then a student soring an IQ of 120 today might score an IQ of 140 tomorrow and a 95 the day after tomorrow. On the other hand if the test is reliable then the IQ of a student will remain nearly the same each time the test is administered. The reliability of a test depends upon the number of questions consisted by it. A test will be more reliable if it possesses more questions. In this respect, objective type tests are more reliable because its sampling is more extensive. We can take another expert opinion to understand the meaning of reliability. “If a clinical thermo meter on three successive determinations, for example yielded reading of 97 o, 103o and 99.6ofor the same patient, it would not be considered very reliable. Reliability, of course is a necessary but not a sufficient condition for using a test. A highly reliable test may be totally invalid or may not measure anything that is psychologically of educationally significant. The reliability of a single of a single test score is expressed quantitatively in terms of the instruments standard error of measurement. If the standard error of measurement, for example, is 2.5, we can say that there are approximately two chances in three (more precisely 68 in 100) that the true score falls between 72.5 and 77.5 when the obtained score is 75. By definition, an unreliable test cannot possible be valid. The necessary degree of reliability however depends on the use that is made of test scores.” Methods of Determining Reliability: For determining reliability, it is necessary that the test should be valid and it should measure what it is designed to measure. It should be administered to an appropriate person or group of parsons for whom the test has been developed. Reliability is a statistical measure and therefore
it can be computed by using different statistical methods. Which have been stated in detail on next page. 1.
Test-retest method: When the reliability of the results are two measured, then at that very situation the test retest method is used in this method the tests are subjected to the group of students at different perrods of time. The scores obtained from first and second time can be correlated in order to check the stability and persistency of test. In test re-test reliability the time factor counts a lot in very close retesting the results are approximately the same, yielding high correlation. But when the retest is administered after an year or two, as the result of changes in the characteristics of students, there are expected to be large variation in the outcome and therefore stability will be low.\ Limitation:
i.
The co-efficient of reliability established through test-retest method is erroneous.
ii.
The reliability determined through test-retest method has memory of carry over effect.
iii.
The test retest method is not an objective method ascertaining reliability of the test.
2.
Equivalent forms method: The second method of ascertaining reliability is alternate form method or method of equivalence. Through this method one has to use two alternate or equivalent tests in order to establish the reliability. This method is used to see the reliability of test for measuring certain content area. It is applied to standardized tests only as they have two or more forms of the same test available. Equivalent forms are used in the same group and in close succession. The result of both the tests are correlated. The correlation shows the degree to which both tests are measurining the same content area. Sometimes the equivalent forms are used
with time interval. Results obtained by this method provide both stability and reliability of test. This method is generally considered to be the best method. Limitation: i.
Finding the reliability through this method is cumbersome because it is difficult to judge the quality of a test which is equivalent in each and every respect.
ii.
This process is more time consuming and also it is not free from carry over effect.
iii.
More over establishment of reliability through this method is not feasible for each and every type of test.
3.
Split half method: As the name indicates in Split-Half- method the approach is to split the test into two reasonable equivalent halves. Such independent sub-test are then used as a source of the two independent scores needed for reliability estimation. In this method a test is administered to a group of students. Before scoring, the test is split into two equal halves. Generally odds and evens are separated. By marking each part separately each student gets two different scores which are correlated. The correlation gives a measure of internal consistency, of the test. Reliability of the test is estimated by applying spearman Brown formula:
1 2 x Reliabi lity on test 2 1 1+ Reliability on test 2
Like equivalent forms method the split-half method helps in determining the reliability of test items are representative sample of the content. Limitation:
i.
The general criticism of split-half method is concerned with splitting on the test. As there is no rule, one may go for applying this own conscience in splitting the test into two halves. The way of splitting varies from person to person which affects the reliability coefficient.
ii.
The second criticism is concerned with the items difficulty. Generally the items of a test are arranged in ordered of difficulty but this fact is not true for each and every type of test. Say for example, without knowing the difficulty level of items if one goes for splitting all the difficulty items in one half and the simple item in another half, it will affect the reliability coefficient adversely.
4.
Kuder Richardson method: Richardson developed several formulas for measuring internal consistency of a test. Kudder Richard son formula zo and z1 and generally applied. But due to simplicity of the operation formula z1 is always preffered. Reliability (K R Z1) =
[
M ( K −M ) K 1− K −1 K S2
]
K= the number of items in the test. M= mean of the test scores; S= standard deviation of the test scores. Summary: The following methods are used for determining reliability of a test. A
Test – Retest method
i. Immediate (without interval)
B
Equivalent form method
ii. With time interval
C
Split half method
iii. Immediate
D
Kuder – Richardson formula
iv. With interval
5.
Parallel form Reliability: When the different sets or different parts of a test (suppose questionnaire a and questionnaire B) are developed but they must have a linkage (in a sense of knowledge, skills and
6.
2.4
behaviors) and then these assessments instruments are subjected on the same group. The result obtained from these groups are then correlated which can show the reliability of the test in regards of the alternate sets of instruments. Inter - rater method of Reliability: The Measures of the reliability about the different judges agree upon the decisions about the assessment is called inter rater method of rebility. The answers cannot effectively interpret by human observes and for that very purpose the inter-rater reliability is of utmost importance. FACTORS AFFECTING RELIABILITY:
Reliability: The degree or the extent of the similarities among the results obtained on several occasion or in other words it can be defined as the degree to which an assessment instruments elicit stable and consistent plethora results. Reliability means consistency of measurement. The words of Ebel & Frisbie “The ability of a test to measure the same quantity when it is administered, to an individual on two different occasions by two different testers is called reliability”. Reliability also called dependability or trust worthiness. Reliability is the degree to which a test consistently measures whatever it measure. Factors which affects the reliability: The factors which badly affects the reliability are as under:
The examinee: Fatigue burden, lack of motivation, carelessness. Trait of Test:
Ambiguous items, poorly worded direction tricky questions in familiar format. Conditions of test- taking and marking: Poor examination condition, excessive heat or cold carelessness in marling, disregards or lack of clear standards for scoring, computational errors. There are also some factors which affects on reliability, which are as under: 1.
A very important factor influencing test reliability is the number of test items. That is the greater number of items in a test, the more reliable the test.
2.
Other things being equal the narrower the rang of difficulty of the items of a test the granter the reliability.
3.
Evenness in scaling is factor influencing the reliability of a test other things being equal a test every scaled is more reliable than a test that has gaps in the scale of difficulty of its items.
4.
Other things being equal, inter-dependent items tend to decrease the reliability of a test.
5.
The more objective the scoring of a test the more reliable is the test.
6.
Chance in getting the correct answer to an items is a factor which lowers the test reliability.
7.
Other things being equal, the more homogenous the material of a test the greater its reliability.
8.
Other thing being equal, the more common the experiences called for in a test are the members of the group taking the test more reliable the test.
9.
Other things being equal the same test given late in the school year (i.e. after covering the unit in the class) is more reliable that when given Carly in the year (i.e. without teaching the unit).
10.
Other things being equal, each, question in a test lower the reliability of test. A test answered by the systematic relall or recognition of orderly facts or experience is more reliable than a test answered by sudden insight because of novelty.
11.
Lengthy items lower the reliability because certain factors in the item will be over or under estimated.
12.
Inadequate or faulty directions, failure to provide suitable illustrations of the task lower the reliability.
13.
Strange or unusual words of items lower the reliability.
14.
The accuracy with which a test is timed is an important factor in test reliability.
15.
Difference in incentive and effort tend to make tests unreliable. The appeal of a test is stronger with some individuals than with others, and is stronger with an individual at one time than at another.
16.
Accidents occurring during the examination such as breaking a pencil, running out of link, or defective test booklets influence the reliability of the test. Outside disturbances also lower the reliability.
17.
The interval between the test and retest is important for reliability estimate.
18.
Cheating in the examination is another factor which lowers the reliability because the score of the individual may increase or decrease unduly.
19.
Illness, worry, excitement though less important still they influence the reliability of the test.
References Murad Ali Katozai 1st Edition, June, 2013 Measurement and Evaluation.
Dr. Mohammad Nooman & Obaid Ullah 1st Edition June 27th 2013 A Manual of Educational & Social Science and Research Methodologies. 2.5
PRACTICALITY:
Meaning: The word “Practicality” means “feasibility” or “us ability”. A test will be practicable if it is easy to administrated, easy to interpret and economical in operation. A good test is that which have sufficiently simple instructions so that it can be administered even by a person of low level intelligence. Tests having difficult instructions and requiring high level training for administering them and expensive for wide use in schools are social to have low usability or practicability. Practicality refers to the economy of time effort and money in testing. In other words a test should be.
Easy to design Easy to administer Easy to interpret
Test of Practicality of a Measuring Instrument: The practicality attribute of a meaning instrument can be estimated regarding its economy convenience and interpretability. Economy consideration suggests that some mutual benefits is required between the ideal research project and that which the budget can afford. The length of measuring in strument is an important area where economic pressures are swiftly left. Convince test suggest that the measuring instrument should be easily manageable. For this purpose one should pay proper attention to the layout of the measuring instrument. For example 9, questionnaire with clear instructions and instrument, is examples of this questionnaire that lack these features.
Characteristics of Practicality: There are many characteristics of practicality they are. 1.
2. 3. 4. 5. 6.
The test should be free from drawbacks and limitations of both essay type and objective type tests. They should have they merits and good point to both these type of test. For this purpose a test should have both essay and objective type of test and questions so that it may cover at the time, the whole course as well as improve the writing skill f the students. It should not require long answer for essay type questions. It should have large number of short essay type questions so that it may cover the slow course in 9, short time. It should not be prepared for evaluation the knowledge and information of the students. It should be arranged in the social and economical conditions of the country. There should be no choice in the given questions. Students should have to answer all the questions. This will discourage selective study.
UNIT-3: APPRAISING CLASSROOM TESTS (ITEMS ANALYSIS) 3.1
THE VALUE OF ITEM
3.1.1
Item Analysis
Item is a statistical technique which is used for selecting and rejecting the items of a test on the basis of their difficulty value and discriminative power. Item analysis is a general term that refers to the specific methods used in education to evaluate test items, typically for the purpose of test construction and revision. Regarded as one of the most important aspects of test construction and increasingly receiving duration, it k an approach incorporated into item response theory (ERT), high serves as an alternative to classical measurement theory (GMT) or classical test theory (CIT). Classical measurement theory considers a score to Ile the direct result of a person's true score plus error. It is this error that is of interest as previous measurement theories have been unable to specify its source. However, item response theory uses item analysis to differentiate between types of error in order to gain a clearer(1) The main objective of item analysis is to select the appropriate understanding of any existing deficiencies. Particular attention is given to individual test items, item characteristics, probability of answering items correctly, overall ability of the test taker, and degrees or levels of knowledge being assessed. Item analysis is concerned basically with the two characteristics of an item--difficulty value and discriminative power. Need of Item Analysis Item analysis is a technique by which the test items are selected and rejected. The selection of items may serve the purpose of the designer or test constructor, because the items have the such characteristics. The following are the main purpose of the test:
(a)
Classification of students or candidates.
(b)
Selection of the candidates for the job.
(c)
Gradation is an academic purpose to assign grades or divisions to the students.
(d)
Prognosis and promotion of the candidates or students.
(e)
Establishing individual differences, and
(f)
Research for the verification of hypotheses.
The different purposes require different types of test having the items of different characteristics. The selection or entrance test includes the items of high difficulty value as well as high power of discrimination. The promotion or prognostic test has the items of moderate difficulty value. There are various techniques of item analysis which are used these days. The Objectives of Item Analysis (1)
The following are the main objectives of item analysis technique: items for the final drift and reject the poor items which do not contribute in the functioning of the test. Some items are to be modified.
(2)
Item analysis obtains the difficulty values of all the items of preliminary draft of the test. The items are classifieddifficulties, moderate and easy items.
(3)
It provides the discriminative power (item reliability; validity) to differentiate between capable and less capable examines of all the items preliminary draft of the test. The items are classified on the basis of the indexes-positive, negative and no discrimination. The negative and no discrimination power items are rejected out rightly.
(4)
It also indicates the functioning of the distructors in the multiple-choice items. The powerful and poor distructors are
changed. It provides the basis for the modification to be made in some of the items of preliminary draft. (5)
T3he reliability and validity of test depends on these characteristics of a test. The functioning of a test is increased by this technique. Both these indexes and considered simultaneously in selecting and rejecting the items of a test.
(6)
It provides the basis for preparing the final draft a test. In the final draft items are arranged in difficulty order. The most easy items are given in the beginning and most difficult items are provided at the end.
(7)
Item analysis is a cyclic technique. The modified items are tried out and their item analysis is done again to obtain these indexes (difficulty and discrimination). The empirical evidences are obtained for selecting the modified items for the final draft.
Functions of Item Analysis The main function of item analysis is to obtain the indexes of the items which indicate its basic characteristics. There are three characteristics (1)
Item difficulty value (D. V.) is the proportion of subjects answering each item correctly.
(2)
Discriminative power (D.P.) of item, this characteristic is of two type — (a) Item reliability— It is taken as the point-biserial correlation between an item and the total test score, multiplied by the item standard deviation. (b) Item validity— It is taken as the point biserial correlation between an item and a criterion score multiplied by the item standard deviation.
The test as a whole should fulfil its purpose successfully; each of its items must be able to discriminate between high and poor students on the test. In other words, a test fulfils its purpose with maximum
success when each .items serves as good predictor. Therefore it is essential that each item of the test should be analysed in terms of its difficulty value and discriminative power for the justification. Item analysis serves the following purpose (1)
To improve and modify a test for immediate use on a parallel group of subjects.
(2)
To select the best items for a test with regard to its purpose after a proper try out on the group of subjects selected from the target population.
(3)
To provide the statistical check-up for the characteristics of the test items for the judgment of test designer.
(4)
To set up parallel forms of a test. Parallel form of test should not require only to have Similar items content or type of items but they should also have the sky& difficulty value and discriminative power. Item analysis' technique that exactly parallel test can be developed, provides 'the empirical basis.
(5)
To modify and reject OF poor items of the test. The poor items may not serve the purpose of the test. The powerful distractor of items are changed an'tkpoor distracters are also changed.
(6)
Item analysis is usually done of a power test rather than speed test. It speed test all the items are of the same difficulty value. The purpose of speed test is to measure the speed and accuracy while speed is acquired through practice. There is no power test, because the time limit is imposed, therefore these are the speeded test. The speediness of the test depends on the difficulty values of the items of the test. Most of the students should reach to last items, in the allotted time for the test. Item analysis is the study of the statistical properties of test items. The qualities usually of interest are the difficulty of the item and its ability or power to differentiate between more capable and less capable examinees. Difficulty is usually expressed as the percent or proportion getting the item right, and discrimination as some
index comparing success by the more capable and the less capable students. Meaning of definition of Difficulty Value (D.V.) The term difficulty value of an item can be explained with the help of simple example of extreme ends. If an item of test is answered correctly by every examinee, it means the item is very easy the difficulty value is 100 percent or proportion is one. This item will not serve any purpose and there is no use to include such items in a test. Such items are generally rejected. If an item is not answered correctly by any of the examinees. None could answer correctly, it means the item is most difficult, the difficulty value is zero percent or proportion is also zero. This item will not serve any purpose and there is no use to include such items in a test. Such items are usually rejected. "The difficulty value of an item is defined as the proportion or percentage of the examinees who have answered the item correctly." —1.1). Guilford "The difficulty value of an item may be defined as the proportion of certain sample of subjects who actually know the answer of item." —Frank S. Freeman In the definition of difficulty value, it has been stated that it is the percentage and proportion of examinee's who answer the item correctly, but in the second definition, the difficulty value is defined as the proportion of certain sample of subjects who actually know the answer of an item. This statement seems to be most functional and dependable, because an item can be answered correctly by guessing but the examinee does not know the answer of the item. The difficulty value depends on actually knowing the correct answer of an item rather than answering an item correctly.
In the procedure of item-analysis "correction for guessing” formula is used for the scores rather than right answers. The difficulty value is also obtained in terms of standard scores or z-scores. Methods or Techniques of item Analysis A recent review of the literature on item analysis indicates that there are at least twenty three different techniques of item analysis. As it has been discussed that item analysis technique obtain the indexes for the characteristics of an item. The following two methods of item analysis are most popular and are widely used. 1)
Davis method of item analysis—It is the basic method of item analysis. It is used for the prognostic test for selecting and rejecting the items on the basis of difficulty value and discriminative power. The right responses are considered in obtaining the indexes for the characteristics of an item. The proportion of right responses on the items are considered for this purpose.
2)
Stanley method of item analysis. It is used for the diagnostic test items. The wrong responses are considered in obtaining the difficulty value and discriminative power. The wrong responses provide the cause of weakness of the students. The proportion of wrong responses on an item is considered for this purpose.
There are separate techniques for obtaining difficulty value and discriminative power of the items. (a)
Techniques of Difficulty Value.
There are two main approaches for obtaining difficult value. a1 – Proportion of right responses on an item technique. Davis and Haper have also used this technique. a2 – Standard scores or z-scores or normal probability curve. Technique of Discriminative Power.
b1 – Proportion of right responses on an item technique. Davis and Haper have used this technique. 3.2
THE PROCEDURE/ PURPOSE OF ITEM ANALYSIS:
The review of literature on item analysis indicates that there are two dozen techniques of item analysis have been devices to obtain the difficulty value and discriminative index of an item of a test. It is not possible to describe all the techniques of item analysis in this chapter. Therefore, most popular and widely used techniques have been discussed. Fredrick B. Davis method of Item Analysis of Prognostic test, and Stanley method of Item Analysis of Diagnostic test. "The item difficulty value may be defined as the proportion or percentage of certain sample subjects that actually know the answer of an item. --Frank S. Freeman The difficulty value depends on actually knowing the answer rather than answering correctly i.e. right responses. In objective type test, the items are answered correctly by guessing rather than actually knowing the answer. It means that an item may be answered without knowing its answer. Thus, correction for guessing is to be used for obtaining the scores which may be actual correct responses. It is important to note that in the procedure of item analysis item wise scoring is done, while subject wise scoring is done in general. There are several formulas have been developed by psychomatricians for 'guessing correction'. Some of the important formula-correction for guessing has been discussed. Formula-Correction for Guessing The following two formula-corrections for guessing have been explained. (a)
Guilford's formula-correction for guessing and
(b)
Horst's formula-correction for guessing.
(a) Guilford's formuia-correction for Guessing. J. P. Guilford has developed the following formula-correction for guessing which used for estimating the actual scores or actually know the answer. S=R, where
(1) (n – 1)
R = right responses on the item W = wrong responses on the item n = number of alternatives in the item S = Actual correct responses on the item.
Example. An item is administered on a group of 50 subjects. The following responses are obtained on different alternatives of the item. (a)
The functions of item analysis
(b)
Selection of good items-8
(c)
Rejection of poor items--7
3.2
MAKING THE MOST OF EXAMS: PROCEDURES FOR ITEM ANALYSIS:
One of the most important (if least appealing) tasks confronting faculty members is the evaluation of student performance. This task requires considerable skill, in part because it presents so many choices. Decisions must be made concerning the method, format, timing, and duration of the evaluative procedures. Once designed, the evaluative procedure must be administered and then scored, interpreted, and graded. Afterwards, feedback must be presented to students. Accomplishing these tasks demands a broad range of cognitive, technical, and interpersonal resources on the part of faculty. But an even more critical task remains, one that perhaps too few faculty undertake with sufficient skill and tenacity: investigating the quality of the evaluative procedure. Even after an exam, how do we know whether that exam was a good one? It is obvious that any exam can only be as good as the items it
comprises, but then what constitutes a good exam item? Our students seem to know, or at least believe they know. But are they correct when they claim that an item was too difficult, too tricky, or too unfair? Lewis Aiken (1997), the author of a leading textbook on the subject of psychological and educational assessment, contends that a "postmortem" evaluation is just as necessary in classroom testing as it is in medicine. Indeed, just such a postmortem procedure for exams exists-item analysis, a group of procedures for assessing the quality of exam items. The purpose of an item analysis is to improve the quality of an exam by identifying items that are candidates for retention, revision, or removal. More specifically, not only can the item analysis identify both good and deficient items, it can also clarify what concepts the examinees have and have not mastered. So, what procedures are involved in an item analysis? The specific procedures involved vary, but generally, they fall into one of two broad categories: qualitative and quantitative. Qualitative Item Analysis Qualitative item analysis procedures include careful proofreading of the exam prior to its administration for typographical errors, for grammatical cues that might inadvertently tip off examinees to the correct answer, and for the appropriateness of the reading level of the material. Such procedures can also include small group discussions of the quality of the exam and its items with examinees who have already taken the test, or with depaitinental student assistants, or even experts in the field. Some faculty use a "think-aloud test administration" (cf. Cohen, Swerdlik, & Smith, 1992) in which examinees are asked to express verbally what they are thinking as they respond to each of the items on an exam. This procedure can assist the instructor in determining whether certain students (such as those who performed well or those who performed poorly on a previous exam) misinterpreted particular items, and it can help in determining why students may have misinterpreted a particular item.
Quantitative Item Analysis In addition to these and other qualitative procedures, a thorough item analysis also includes a number of quantitative procedures. Specifically, three numerical indicators are often derived during an item analysis: Item difficulty, item discrimination, and distractor power statistics. Item Difficulty Index (p) The item difficulty statistic is an appropriate choice for achievement or aptitude tests when the items are scored dichotomously (i.e., correct vs. incorrect). Thus, it can be derived for true-false, multiple-choice, and matching items, and even for essay items, where the instructor can convert the range of possible point values into the categories "passing" and "failing." The item difficulty index, symbolized p, can be computed simply by dividing the number of test takers who answered the item correctly by the total number of students who answered the item. As a proportion, p can range between 0.00, obtained when no examinees answered the item correctly, and 1.00, obtained when all examinees answered the item correctly. Notice that no test item need have only one p value. Not only may the p value vary with each class group that takes the test, an instructor may gain insight by computing the item difficulty level for a number of different subgroups within a class, such as those who did well on the exam overall and those who performed more poorly. Although the computation of the item difficulty indexp is quite straightforward, the interpretation of this statistic is not. To illustrate, consider an item with a difficulty level of 0.20. We do know that 20% of the examinees answered the item correctly, but we cannot be certain why they did so. Does this item difficulty level mean that the item was challenging for all but the best prepared of the examinees? Does it mean that the instructor failed in his or her attempt to teach the concept assessed by the item? Does it mean that the students failed to learn the material? Does it mean that the item was poorly written? To answer these
questions, we must rely on other item analysis procedures, both qualitative and quantitative ones. Item Discrimination Index (D) Item discrimination analysis deals with the fact that often different test takers will answer a test item in different ways. As such, it addresses questions of considerable interest to most faculty, such as, "does the test item differentiate those who did well on the exam overall from those who did not?" or "does the test item differentiate those who know the material from those who do not?" In a more technical sense then, item discrimination analysis addresses the validity of the items on a test, that is, the extent to which the items tap the attributes they were intended to assess. As with item difficulty, item discrimination analysis involves a family of techniques. Which one to use depends on the type of testing situation and the nature of the items. I'm going to look at only one of those, the item discrimination index, symbolized D. The index parallels the difficulty index in that it can be used whenever items can be scored dichotomously, as correct or incorrect, and hence it is most appropriate for true-false, multiple-choice, and matching items, and for those essay items which the instructor can score as "pass" or "fall." We test because we want to find out if students know the material, but all we learn for certain is how they did on the exam we gave them. The item discrimination index tests the test in the hope of keeping the correlation between knowledge and exam performance as close as it can be in an admittedly imperfect system. The item discrimination index is calculated in the following way: 1.
Divide the group of test takers into two groups, high scoring and low scoring. Ordinarily, this is done by dividing the examinees into those scoring above and those scoring below the median. (Alternatively, one could create groups made up of the top and bottom quintiles or quartiles or even deciles.)
2. 3.
Compute the item difficulty levels separately for the upper (Pupper) and lower (Plower) scoring groups. Subtract the two difficulty levels such that D = P upper - Plower
How is the item discrimination index interpreted? Unlike the item difficulty levelp, the item discrimination index can take on negative values and can range between -1.00 and 1.00. Consider the following situation: suppose that overall, half of the examinees answered a particular item correctly, and that all of the examinees who scored above the median on the exam answered the item correctly and all of the examinees who scored below the median answered incorrectly. In such a situation P, upper, = 1.00 and P lower = 0.00. As such, thevalue of the item discrimination index D is 1.00 and the item is said to be a perfect positive discriminator. Many would regard this outcome as ideal. It suggests that those who knew the material and were well-prepared passed the item while all others failed it. Though it's not as unlikely as winning a million-dollar lottery, finding a perfect positive discriminator on an exam is relatively rare. Most psychometricians would say that items yielding positive discrimination index values of 0.30 and above are quite good discriminators and worthy of retention for future exams. Finally, notice that the difficulty and discrimination are not independent. If all the students in both the upper and lower levels either pass or fail an item, there's nothing in the data to indicate whether the item itself was good or not. Indeed, the value of the item discrimination index will be maximized when only half of the test takers overall answer an item correctly; that is, whenp = 0.50. Once again, the ideal situation is one in which the half who passed the item were students who all did well on the exam overall. Does this mean that it is never appropriate to retain items on an exam that are passed by all examinees, or by none of the examinees? Not at all. There are many reasons to include at least some such items. Very easy items can reflect the fact that some relatively straightforward concepts were taught well and mastered by all students. Similarly, an
instructor may choose to include some very difficult items on an exam to challenge even the best-prepared students. The instructor should simply be aware that neither of these types of items functions well to make discriminations among those taking the test. [material omitted...] Conclusion To those concerned about the prospect of extra work involved in item analysis, take heart: item difficulty and discrimination analysis programs are often included in the software used in processing exams answered on Scantron or other optically scannable forms. As such, these analyses can often be performed for you by personnel in your computer services office. You might consider enlisting the aid of your departmental student assistants to help with item distractor analysis, thus providing them with an excellent learning experience. In any case, an item analysis can certainly help determine whether or not the items on your exams were good, ones and to determine which items to retain, revise, or replace. Understanding Item Analysis Reports Item analysis is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole. Item analysis is especially valuable in improving items which will be used again in later tests, but it can also be used to eliminate ambiguous or misleading items in a single test administration. In addition, item analysis is valuable for increasing instructors' skills in test construction, and identifying specific areas of course content which need greater emphasis or clarity. Separate item analyses can berequested for each raw score' created during a given ScorePak® run. Sample Sample item analysis (30K PDF*) A basic assumption made by ScorePak® is that the test under analysis is composed of items measuring a single subject area or underlying ability. The quality of the test as a whole is assessed by
estimating its "internal consistency." The quality of individual items is assessed by comparing students' item responses to their total test scores. Following is a description of the various statistics provided on a ScorePak® item analysis report. This report has two parts. The first part assesses the items which made up the exam. The second part shows statistics summarizing the performance of the test as a whole. Item Statistics Item statistics are used to assess the performance of individual test items on the assumption that the overall quality of a test derives from the quality of its items. The ScorePak® item analysis report provides the following item information:
Item Number
This is the question number taken from the student answer sheet, and the ScorePak® Key Sheet. Up to 150 items can be scored on the Standard Answer Sheet.
Mean and Standard Deviation
The mean is the "average" student response to an item. It is computed by adding up the number of points earned by all students on the item, and dividing that total by the number of students. The standard deviation, or S.D., is a measure of the dispersion of student scores on that item. That is, it indicates how "spread out" the responses were. The item standard deviation is most meaningful when comparing items which have more than one correct alternative and when scale scoring is used. For this reason it is not typically used to evaluate classroom tests.
Item Difficulty
For items with one correct alternative worth a single point, the item difficulty is simply the percentage of students who answer an item correctly. In this case, it is also equal to the item mean. The item difficulty index ranges from 0 to 100; the higher the value, the easier the question. When an alternative is worth other than a single point, or when
there is more than one correct alternative per question, the item difficulty is the average score on that item divided by the highest number of points for any one alternative. Item difficulty is relevant for determining whether students have learned the concept being tested. It also plays an important role in the ability of an item to discriminate between students who know the tested material and those who do not. The item will have low discrimination if it is so difficult that almost everyone gets it wrong or guesses, or so easy that almost everyone gets it right. To maximize item discrimination, desirable difficulty levels are slightly higher than midway between chance and perfect scores for the item. (The chance score for five-option questions, for example, is 20 because one-fifth of the students responding to the question could be expected to choose the correct option by guessing.) Ideal difficulty levels for multiple-choice items in terms of discrimination potential are: Format
Ideal Difficulty
Five-response multiple-choice
70
Four-response multiple-choice
74
Three-response multiple-choice
77
True-false (two-response multiple-choice)
85
(from Lord, F.M. "The Relationship of the Reliability of Multiple-Choice Test to the Distribution of Item Difficulties," Psychometrika, 1952, 18, 181-194.) ScorePak® arbitrarily classifies item difficulty as "easy" if the index is 85% or above; "moderate" if it is between 51 and 84%; and "hard" if it is 50% or below.
Item Discrimination
Item discrimination refers to the ability of an item to differentiate among students on the basis of how well they know the material being tested. Various hand calculation procedures have traditionally been used to compare item responses to total test scores using high and low scoring groups of students. Computerized analyses
provide more accurate assessment of the discrimination power of items because they take into account responses of all students rather than just high and low scoring groups. The item discrimination index provided by ScorePak® is a Pearson Product Moment correlation2 between student responses to a particular item and total scores on all other items on the test. This index is the equivalent of a point-biserial coefficient in this application. It provides an estimate of the degree to which an individual item is measuring the same thing as the rest of the items. Because the discrimination index reflects the degree to which an item and the test as a whole are measuring a unitary ability or attribute, values of the coefficient will tend to be lower for tests measuring a wide range of content areas than for more homogeneous tests. Item discrimination indices must always be interpreted in the context of the type of test which is being analyzed. Items with low discrimination indices are often ambiguously worded and should be examined. Items with negative indices should be examined to determine why a negative value was obtained. For example, a negative value may indicate that the item was mis-keyed, so that students who knew the material tended to choose an unkeyed, but correct, response option. Tests with high internal consistency consist of items with mostly positive relationships with total test score. In practice, values of the discrimination index will seldom exceed .50 because of the differing shapes of item and total score distributions. ScorePak® classifies item discrimination as "good" if the index is above .30; "fair" if it is between . 10 and.30; and "poor" if it is below .10.
Alternate Weight
This column shows the number of points given for each response alternative. For most tests, there will be one correct answer which will be given one point, but ScorePak® allows multiple correct alternatives, each of which may be assigned a different weight.
Means
The mean total test score (minus that item) is shown for students who selected each of the possibleresponse alternatives. This information should be looked at in conjunction with the discrimination index; higher total test scores should be obtained by students choosing the correct, or most highly weighted alternative. Incorrect alternatives with relatively high means should be examined to determine why "better" students chose that particular alternative.
Frequencies and Distribution
The number and percentage of students who choose each alternative are reported. The bar graph on the right shows the percentage choosing each response; each "#" represents approximately 2.5%. Frequently chosen wrong alternatives may indicate common misconception among the students. Difficulty and discrimination Distributions At the end of the Item Analysis report, test items are listed according their degrees of difficulty (easy, medium, hard) and discrimination (good, fair, poor). These distributions provide a quick overview of the test, and can be used to identify items which are not performing well and which can perhaps be improved or discarded. Test Statistics Two statistics are provided to evaluate the performance of the test as a whole. Reliability Coefficient The reliability of a test refers to the extent to which the test is likely to produce consistent scores. The particular reliability coefficient computed by ScorePak® reflects three characteristics of the test:
The intercorrelations among the items -- the greater the relative number of positive relationships, and the stronger those relationships are, the greater the reliability. Item discrimination indices and the test's reliability coefficient are related in this regard.
The length of the test -- a test with more items will have a
higher reliability, all other things being equal. The content of the test -- generally, the more diverse the subject matter tested and the testing techniques used, the lower the reliability.
Reliability coefficients theoretically range in value from zero (no reliability) to 1.00 (perfect reliability). In practice, their approximate range is from .50 to .90 for about 95% of the classroom tests scored by ScorePak®. High reliability means that the questions of a test tended to "pull together." Students who answered a given question correctly were more likely to answer other questions correctly. If a parallel test were developed by using similar items, the relative scores of students would show little change. Low reliability means that the questions tended to be unrelated to each other in terms of who answered them correctly. The resulting test scores reflect peculiarities of the items or the testing situation more than students' knowledge of the subject matter. As with many statistics, it is dangerous to interpret the magnitude of a reliability coefficient out of context. High reliability should be demanded in situations in which a single test score is used to make major decisions, such as professional licensure examinations. Because classroom examinations are typically combined with other scores to determine grades, the standards for a single test need not be as stringent. The following general guidelines can be used to interpret reliability coefficients for classroom exams: Reliability
Interpretation
.90 and above
Excellent reliability; at the level of the best standardized tests
.80- .90
Very good for a classroom test
.70 - .80
Good for a classroom test; in the range of most. There are probably a few items which could be improved.
.60 - .70
Somewhat low. This test needs to be supplemented by other measures (e.g., more tests) to determine grades. There are probably some items which could be improved.
.50 - .60
Suggests need for revision of test, unless it is quite short (ten or fewer items). The test definitely needs to be supplemented by other measures (e.g., more tests) for grading.
.50 or below
Questionable reliability. This test should not contribute heavily to the course grade, and it needs revision.
The measure of reliability used by ScorePak® is Cronbach's Alpha. This is the general form of the more commonly reported KR-20 and can be applied to tests composed of items with different numbers of points given for different response alternatives. When coefficient alpha is applied to tests in which each item has only one correct answer and all correct answers are worth the same number of points, the resulting coefficient is identical to KR-20. (Further discussion of test reliability can be found in J. C. Nunnally, Psychometric Theory. New York: McGraw-Hill, 1967, pp. 172-235, see especially formulas 6-26, p. 196.) Standard Error of Measurement The standard error of measurement is directly related to the reliability of the test. It is an index of the amount of variability in an individual student's performance due to random measurement error. If it were possible to administer an infinite number of parallel tests, a student's score would be expected to change from one administration to the next due to a number of factors. For each student, the scores would form a "normal" (bell-shaped) distribution. The mean of the distribution is assumed to be the student's "true score," and reflects what he or she
"really" knows about the subject. The standard deviation of the distribution is called the standard error of measurement and reflects the amount of change in the student's score which could be expected from one test administration to another. Whereas the reliability of a test always varies between 0.00 and 1.00, the standard error of measurement is expressed in the same scale as the test scores. For example, multiplying all test scores by a constant will multiply the standard error of measurement by that same constant, but will leave the reliability coefficient unchanged. A general rule of thumb to predict the amount of change which can be expected in individual test scores is to multiply the standard error of measurement by 1.5. Only rarely would one expect a student's score to increase or decrease by more than that amount between two such similar tests. The smaller the standard error of measurement, the more accurate the measurement provided by the test. (Further discussion of the standard error of measurement can be found in J. C. Nunnally, Psychometric Theory. New York: McGraw-Hill, 1967, pp.172-235, see especially formulas 6-34, p. 201.) A Caution in Interpreting Item Analysis Results Each of the various item statistics provided by ScorePak® provides information which can be used to improve individual test items and to increase the quality of the test as a whole. Such statistics must always be interpreted in the context of the type of test given and the individuals being tested. W. A. Mehrens and I. J. Lehmann provide the following set of cautions in using item analysis results (Measurement and Evaluation in Education and Psychology. New York: Holt, Rinehart and Winston, 1973, 333-334):
Item analysis data are not synonymous with item validity. An external criterion is required to accurately judge the validity of test items. By using the internal criterion of total test score, item analyses reflect internal consistency of items rather than validity.
The discrimination index is not always a measure of item
quality. There is a variety of reasons an item may have low discriminating power: a) extremely difficult or easy items will have low ability to discriminate but such items are often needed to adequately sample course content and objectives; b) an item may show low discrimination if the test measures many different content areas and cognitive skills. For example, if the majority of the test measures "knowledge of facts," then an item assessing "ability to apply principles" may have a low correlation with total test score, yet both types of items are needed to measure attainment of course objectives. Item analysis data are tentative. Such data are influenced by the type and number of students being tested, instructional procedures employed, and chance errors. If repeated use of items is possible, statistics should be recorded for each administration of each item.
Raw scores are those scores which are computed by scoring answer sheets against a ScorePak® Key Sheet. Raw score names are EXAM1 through EXAM9, QUIZ1 through QUIZ9, MIDTRMI through MIDTRM3, and FINAL. ScorePak® cannot analyze scores taken from the bonus section of student answer sheets or computed from other scores, because such scores are not derived from individual items which can be accessed by ScorePak®. Furthermore, separate analyses must be requested for different versions of the same exam. Return to the text. (anchor near note 1 in text) A correlation is a statistic which indexes the degree of linear relationship between two variables. If the value of one variable is related to the value of another, they are said to be "correlated." In positive relationships, the value of one variable tends to be high when the value of the other is high, and low when the other is low. In negative relationships, the value of one variable tends to be high when the other is low, and vice versa. The possible values of correlation coefficients range from -1.00 to 1.00. The strength of the relationship is shown by the
absolute value of the coefficient (that is how large the number is whether it is positive or negative). The sign indicates the direction of the relationship (whether positive or negative). Return to the text. *Software capable of displaying a PDF is required for viewing or printing this document. Adobe Reader is available free of charge from the Adobe Web site at http://www.adobe.com/products/acrobatireadstea.html QUESTION: A few years ago in your Shiken column, you showed how to do item analysis for weighted items using a calculator (Brown, 2000, pp. 1921) and a couple of columns back (Brown, 2002, pp. 20-23) you showed how to do distractor efficiency analysis in a spreadsheet program. But, I don't think you have ever shown how to do regular item analysis statistics in a spreadsheet. Could you please do that? I think some of your readers would find it very useful. ANSWER: Yes, I see what you mean. In answering questions from readers, I explained more advanced concepts of item analysis without laying the groundwork that other readers might need. To remedy that, in this column, I will directly address your question, but only with regard to norm-referenced item analysis. In my next Statistics Corner column, I will address another reader's question, and in the process show how criterion-referenced item analysis can be done in a spreadsheet. The Overall Purpose of Item Analysis Let's begin by answering the most basic question in item analysis: Why do we do item analysis? We do it as the penultimate step in the test development process. Such projects are usually accomplished in the following steps: 1. 2.
Assemble or write a relatively large number of items of the type you want on the test. Analyze the items carefully using item format analysis to make sure the items are well written and clear (for guidelines, see Brown, 1996, 1999; Brown & Hudson, 2002).
3.
4.
5.
Pilot the items using a group of students similar to the group that will ultimately be taking the test. Under less than ideal conditions, this pilot testing may be the first operational administration of the test. Analyze the results of the pilot testing using item analysis techniques. These are described below for norm-referenced tests (NRTs) and in the next column for criterion-referenced tests (CRTs). Select the most effective items (and get rid of the ineffective items) to make a shorter, more effective revised version of the test.
Basically, those five steps are followed in any test development or revision project. Item Analysis Statistics for Norm-Referenced Tests As indicated above, the fourth step, item analysis, is different for NRTs and CRTs, and in this column, I will only explain item analysis statistics as they apply to NRTs. The basic purpose of any NRT is to spread students out along a general continuum of language abilities, usually for purposes of making aptitude, proficiency, or placement decisions (for much more on this topic, see Brown, 1996, 1999; Brown & Hudson, 2002). Two item statistics are typically used in the item analysis of such norm-referenced tests: item facility and item discrimination. Item facility (IF) is defined here as the proportion of students who answered a particular item correctly. Thus, if45 out of 50 students answered a particular item correctly, the proportion would be 45/50 = . 90. An IFof .90 means that 90% of the students answered the item correctly, and by extension,that the item is very easy. In Screen 1, you will see one way to calculate IFusing the Excel® spreadsheet for item 1 (I1) in a small example data set coded 1 for correct and 0 for incorrect answers. Notice the cursor has outlined cell C21 and that the function/formula typed in that cell (shown both in the row above the column labels and in cell B21) is = AVERAGE (C2:C19), which means average the ones and zeros in the range between cells C2 and C19. The
result in this case is .94, a very easy item because 94% of the students are answering correctly.
All the other NRT and CRT item analysis techniques that I will discuss here and in the next column are based on this notion of item facility. For instance, item discrimination can be calculated by first figuring out who the upper and lower students are on the test (using their total scores to sort them form the highest score to the lowest). The upper and lower groups should probably be made up of equal numbers of students who represent approximately one third of the total group each. In Screen 1, I have sorted the students from high to low based on their total test scores from 77 for Hide down to 61 for Hachiko. Then I separated the three groups such that there are five in the top group, five in the bottom group, and six in the middle group. Notice that Issaku and Naoyo both had scores of 68 but ended up in different groups (as did Eriko and Kimi with their scores of 70). The decision as to which group they were assigned to was made with a coin flip.
To calculate item discrimination (ID), I started by calculating IFfor the upper group using the following: = AVERAGE(C2:C6), as shown in row 22. Then, I calculated IFfor the lower group using the following: = AVERAGE(C15:C19), as shown in row 23. With IFupper and IFlower in hand, calculatingIDsimply required subtracting IFupper– IFlower. I did this by subtracting C22 minus C23, or = C22 -C23, as shown in row 24, which resulted in an IDof .20 for I1. Once I had calculated the four item analysis statistics shown in Screen 1 for Il, I then simply copied them and pasted them into the spaces below the other items, which resulted in all the other item statistics you see in Screen 1. [Note that the statistics didn't always fit in the available spaces, so I got results that looked like ### in some cells; to fix that, I blocked out all the statistics and typed alt oca and thusadjusted the column widths to fit the statistics. You may also want to adjust the number of decimal places, which is beyond the scope of this article. You can learn about this by looking in the Help menu or in the Excel manual. Ideal items in an NRT should have an average IFof .50. Such items would thus be well centered, i.e., 50 percent of the students would have answered correctly, and by extension, 50 percent would have answered incorrectly. In reality however, items rarely have an IFof exactly .50, so those that Ell in a range between .30 and .70 are usually considered acceptable for NRT purposes. Once those items that fall within the .30 to .70 range of IFs are identified, the items among them that have the highest IDs should be further selected for inclusion in the revised test. This process would help the test designer to keep only those items that are well centered and discriminate well between the high and the low scoring students. Such items are indicated in Screen 1 by an asterisk in row 25 (cleverly labeled "Keepers"). For more information on using item analysis to develop NRTs, see Brown (1995, 1996, 1999). For information on calculating NRT statistics for weighted items (i.e., items that cannot be coded 1 or 0 for
correct and incorrect), see Brown (2000). For information on calculating item discrimination using the point-biserial correlation coefficient instead of ID, see Brown (2001). For an example NRT development and revision project, see Brown (1988). Conclusion I hope you have found my explanation of how to do normreferenced item analysis statistics (item facility and item discrimination) in a spreadsheet clear and helpful. I must emphasize that these statistics are only appropriate for developing and analyzing norm-referenced tests, which are usually used at the institutional level, like, for example, overall English language proficiency tests (to help with, say,admissions decisions) or placement tests (to help place students into different levels of English study within a program). However, these statistics are not appropriate for developing and analyzing classroom oriented criterionreferenced tests like the diagnostic, progress, and achievement tests of interest to teachers. For an explanation of item analysis as it is applied to CRTs, read the Statistics Corner column in the next issue of this newsletter, where I will explain the distinction between the difference index and the B-index. 3.3
ITEM DIFFICULTY:
Definition: “item difficulty is a measure of the proportion of individuals who responded correctly to each test item.” Item difficulty in a test determined by two proportion of individuals who correctly respond to the item in particular. “item difficulty of a test for a particular group is evaluated by the percentage of participates who respond correctly.” Explanation: Item difficulty is simply the percentage of students taking two tests who answered the item correctly. The larger the percentage getting an item rights the easier two items. The higher the difficulty index, the
easier the item is understood to be (wood, 1960). To compute the item difficulty, divide the number of people answering the item correctly by the total number of people answering item. The proportion for the item is usually denoted by P and it called item difficulty. The range is from 0% to 100%. Examples: To determine the difficulty level of test items, a measure called difficulty Indene is used. This measure asks teachers to calculate the proportion of students who answered the test correctly. By looking each alternative (for multiple choice), we can also find out if there are answers choices that should be replaced. For example, let’s we give a multiple choice quiz and there were four answer choices (A,B,C and D). Two following talde illustrates how many students selected each answer choice for Question # 1 and # 2. Questions
A
B
C
#1
0
3
24*
#2
12*
13
3
*Devertes correct answers. For question # 1, we can see that A was not a very good distracter no one selected that answer. We can also compute the difficulty of item by dividing the number of students who choose two correct answers (24) by the number of total students (30) by using formula, the difficulty of Question # 1, P is equal to P=
24 30
P= .80 A rough “role – of thumb” is that if the item difficulty is more then 75, it is an easy item; if the difficulty is below 25, it is a difficult item. Given these parameters, this item could be regarded moderately
easy 10ts (80%) of students got it correct. In contrast, Question # 2 is
much more difficult.
( 1230 =.40) P=
.
12 30
P= .40 In fail, on question # 2, more students selected an incorrect answer (B) than selected the correct answer (A). This item should be carefully analyzed to ensure that B is an appropriate distracter. Therefore “Item difficulty” should have been named “item easiness;” it expresses the proportion or percentage of students who answered two items correctly. 3.4
THE INDEX OF DISCRIMINATION
Introduction 1.
2.
3.
The index of discrimination is a useful measure of item quality whenever the purpose of a test is to produce a spread of scores, reflecting differences in student achievement, so that distinctions may be made among the performances of examinees. This is likely to be the purpose of norm-referenced tests. It is a degree to which students with high overall exam scores also got a particular item correct. It is often referred to as Item Effect, since it is an index of an item's effectiveness at discriminating those who know the content from those who do not. The item discrimination index is a point biserial correlation coefficient. Its possible range is -1.00 to 1.00. A strong and positive correlation suggests that students who get any one question correct also have a relatively high score on the overall
exam. Theoretically, this makes sense. Students who know content and who perform well on the test overall should be ones who know the content. There's a problem if students getting correct answers on a test and they don't know content.
the the are the
Measurement of Index of Discrimination Examples I If we are using the Item Analysis provided by Scanning Operations, discrimination indices are listed under the column head ‘Disc.’ RESPNSE TABLE - FORMA ITEM NO 1 2 3
OMIT % 0 0 0
A % 0 79 4
B % 18 0 7
C % 82 0 89
D % 0 21 0
E % 0 0 0
KEY-
The Index of Discrimination We examine item discrimination; there are a number of things we should consider. 1.
2.
3.
4.
Item difficulty! Very easy or very difficult items are not good discriminators. If an item is so easy (e.g., difficulty = 98) that nearly everyone gets it correct or so difficult (e.g., difficulty = 30) That nearly everyone gets it wrong, then it becomes very difficult to discriminate those who actually know the content from those who do not. That does not mean that very easy and very difficult items should be eliminated. In fact, they are fine as long they are used with the instructor's recognition that they will not discriminate well and if putting them on the test matches the intention of the instructor to either really challenge students or to make certain that everyone knows a certain bit of content. A poorly written item will have little ability to discriminate.
C A C
Example 2 Another measure, the Discrimination Index, refers to how well an assessment differentiates between high and low scorers. In other words, you should be able to expect that the high-performing students would select the correct answer for each question more often than the low-performing students. If this is true, then the assessment is said to have a positive discrimination index (between 0 and 1) -- indicating that students who received a high total score chose the correct answer for a specific item more often than the students who had a lower overall score. If, however, you find that more of the low-performing students got a specific item correct, then the item has a negative discrimination index (between -1 and 0). Let's look at an example. Table 1 displays the results of ten questions on a quiz. Note that the students are arranged with the top overall scorers at the top of the table 1 Table-1:The Index of Discrimination Student
Total score (%)
Qu 1
Asif
90
1
Sam
90
1
Jill
80
0
Charlie
80
1
Sonya
70
1
Ruben
60
1
Clay
60
1
Kelley
50
1
Justin
50
1
Tonya
40
0
“1” indicates the answer was correct; “0” indicates it was incorrect. Steps to determine the Difficulty Index and the Discrimination Index.
1.
After the students are arranged with the highest overall scores at the top, count the number of students in the upper and lower group who got each item correct. For Question #1, there were 4 students in the top half who got it correct and 4 students in the bottom half. Determine the Difficulty Index by dividing the number who got it correct by the total number of students. For Question #1, this would be 8/10 or p=.80. Determine the Discrimination Index by subtracting the number of students in the lower group who got the item correct from the number of students in the upper group who got the item correct. Then, divide by the number of students in each group (in this case, there are five in each group). For Question #1, that means you would subtract 4 from 4, and divide by 5, which results in a Discrimination Index of 0. The answers for Questions 1-3 are provided in Table 1
2.
3.
4. Table-2 Item
# Correct (upper group)
# Correct (Lower group)
Difficulty (p)
Discrimination (D)
Question 1
4
4
.80
0
Question 2
0
3
.30
-0.6
Question 3
5
1
.60
0.8
In table 2 we can see that Question #2 had a difficulty index of . 30 (meaning it was quite difficult), and it also had a negative discrimination index of -0.6 (meaning that the low-performing students were more likely to get this item correct). This question should be carefully analyzed, and probably deleted or changed. Our "best" overall question is Question 3, which had a moderate difficulty level (.60), and discriminated extremely well (0.8).
Recommendations for Determining Index of Discrimination It is typically recommended that item discrimination be at least . 20. It's best to aim even higher. Items with a negative discrimination are
theoretically indicating that either the students who performed poorly on the test overall got the question correct or that students with high overall test performance did not get the item correct. Thus, the index could signal a number of problems:
There is a mistake on the scoring key. Poorly prepared students are guessing correctly. Well prepared students are somehow justifying the wrong answer.
In all cases, action must be taken! So, items with negative item difficulty must be addressed. Items with discrimination indices less than . 20 (or slightly over, but still relatively low) must be revised or eliminated. Be certain that there is only one possible answer, that the question is written clearly, and that your answer key is correct.
UNIT-4: INTERPRETING THE TEST SCORES 4.1
THE PERCENTAGE CORRECT SCORE:
What does score test mean? A test score is a piece of information, usually a number, that conveys the performance of an examinee on a test. One formal definition is that it is "a summary of the evidence contained in an examinee's responses to the items of a test that are related to the construct or constructs being measured." Test scores are interpreted with a norm-referenced or criterion-referenced interpretation, or occasionally both. A norm-referenced interpretation means that the score conveys meaning about the examinee with regards to their standing among other examinees. A criterion-referenced interpretation means that the score conveys information about the examinee with regards a specific subject matter, regardless of other examinees' scores. Types of Test Scores There are two types of test scores: raw scores and scaled scores. A raw score is a score without any sort of adjustment or transformation, such as the simple number of questions answered correctly. A scaled score is the results of some transformation applied to the raw score. The purpose of scaled scores is to report scores for all examinees on a consistent scale. Suppose that a test has two forms, and one is more difficult than the other. It has been determined by equating that a score of 65% on form 1 is equivalent to a score of 68% on form 2. Scores on both forms can be converted to a scale so that these two equivalent scores have the same reported scores. For example, they could both be a score of 350 on a scale of 100 to 500.
Two well-known tests in the United States that have scaled scores are the ACT and the SAT. The ACT's scale ranges from 0 to 36 and the SAT's from 200 to 800 (per section). Ostensibly, these two scales were selected to represent a mean and standard deviation of 18 and .6 (ACT), and 500 and 100. The upper and lower bounds were selected because an interval of plus or minus three standard deviations contains more than 99% of a population. Scores outside that range are difficult to measure, and return little practical value. Note that scaling does not affect the psychometric properties of a test, it is something that occurs after the assessment process (and equating, if present) is completed. Therefore, it is not an issue of psychometrics, per se, but an issue of interpretability. Interpretation the Score by Criterion Referencing The raw score is number of points received on a test when the test has been scored according to the instructions. Raw score is not very meaningful without further information. Criterion-referenced test interpretation permits us to describe an individual's test performance without referring to the performance of other individuals. Thus we might describe a student's performance in terms of the speed, precision with which a certain task is performed. Criterion-referenced interpretation of test scores is most meaningful when the test is designed to measure a set of clearly stated learning tasks. Enough items are used for each interpretation to make dependable Judgments. Interpretation the Score by Percentages In mathematics, a relationship with 100 is called percentage (denoted by %). Often it is useful to express the scores in terms of percentages for comparison. Consider the following example. Grade
Class A No. of Students
%
Class B No. of Students
%
A
10
12.50
8
40
B
25
31.25
6
30
C
30
37.50
4
20
D
15
18.75
2
10
Total
80
100
20
100
Ten students from class A and eight students from class B got grade A. It looks apparently that class A is better in getting A grade but 12.5% of the students from class A and 40% students from class B got grade A. It is clear from the percentages that class. B is far better in getting grade A than class A. Interpretation the Score by Norm Referencing Interpretation of scores by norm referencing involves making of scores and expressing a given score in relation, to the other scores Normreferenced test interpretation tells us how an individual is compared with other persons who have taken the same test. The simplest type of comparison is to rank the scores from highest to lowest and to note where an individual's score falls. The rest of the scores serve as the norm group. The given score is compared with the other scores by norm referencing. If a student's score is second from the top in a group of 20 students, it is a high score meaning that the scores of 90% of the students are less than him. Ordering and Ranking A first step in organizing scores in the listing of scores in order of magnitude from largest to the smallest score. The data so arranged are called ordered array. By scanning an ordered array, we can determine quickly the largest score, the smallest score and other facts about the data. Ranked data consists of scores in a form that shows their relative position on some characteristic but does not yield a numerical value for this characteristic. The order of finish of cars in a race is an example of ranking. If we list the cars as first, second, third etc. up to the last car, we can say that they were ranked on the characteristic of overall
speed. We know each car's position relative to any other car's position but we have no precise knowledge of the speed of any car. A high school teacher ranked Hamid 30th in a class of 100 means that Hamid did better than 70 of his classmates but poorer than 29. But nothing has been aid about Hamid's general level of achievement. Measurement Scales Measurement scales are of great significance in analyzing and interpreting results. The important types of measurement scales are: The Nominal Scale The lowest measurement scale is the nominal scale. In this scale, each individual is put into one of the distinct categories or classes. Each class has a name. The names are just labels. There is no order in these classes. We cannot say that one class is larger than the other class. You cannot do arithmetic operations (addition, subtraction, multiplication, division) on this scale. Examples of the nominal scale are Categorization of blood groups of the students of a college into A, B, AB and 0 groups. We cannot say that group A is better than group B. Classification of books in a college library according to subjects. Distribution of the population of Pakistan according to sex, religion, occupations, marital status, literacy etc., is examples of the nominal scale. The Ordinal Scale When measurements are not only different from category to category but can also be ranked according to some criterion, they are to be measured on an ordinal scale. The members of anyone category are considered equal but members of one category are considered lower than those in another category. The ordinal scale is one-step higher than the nominal scale because we distribute the individuals not only in classes but we also order these classes.
Examples of the ordinal scale are Categorization of schools according to their educational level into primary, middle, secondary or higher secondary is an ordinal scale. There is an order in these classes. The primary level is lower than the middle level and the middle level is lower than the secondary level. You cannot do arithmetic operations on this scale. Individuals may be classified according to socioeconomic status as low, medium, high. Intelligence of students may be average, above average or below average. Classification of examination results into different grades A), A, B, C, D, E etc. In this measurement scale, we can say that one individual is larger than the other but we cannot say how large it is. The Interval Scale In this scale, it is not only possible to order measurement but also the distance between two measurements is known. We can say that the difference between two measurements 30 and 40 is equal to the difference between measurements 40 and 50. The level of the interval scale is higher than the nominal and the ordinal scales. This is truly a quantitative scale. A unit of measurement and a zero point are required for this scale. The selected zero point is not necessarily a true zero. It does not have to indicate a total absence of the quantity being measured. We measure height in meters or feet, weight in kilograms or pounds, temperature in centigrade or Fahrenheit, income in rupees and the time in seconds. Arithmetic operations can be done on this scale. You can add the income of a wife to that of his husband. The Ratio Scale The highest level of measurement is the ratio scale. Equality of ratios as well as equality of intervals is determined in this scale. Fundamental to the ratio scale is the true zero point. The measurement of height, weight and length makes use of the ratio scale.
Frequency Distribution Data that have been originally collected is called raw data or primary data. It has not yet undergone any statistical technique. To understand the raw data easily, we arrange into groups or classes. The data so arranged is called groups data or frequency distribution. General rules far the construction of a frequency distribution: 1.
Determine the Range. Range is the difference between highest and lowest scores.
2.
Decide the appropriate number of class intervals: 'There is no hard and fast formula for deciding the number of class intervals. The number of class intervals is usually taken between 5 and 20 depending on the length of the data.
3.
Determine the approximate length of the class interval by dividing the range with number of class intervals.
5.
Determine the limits of the class intervals taking the smallest scores at the bottom of the column to the largest scores at the top.
5.
Determine the number of scores falling in each class interval. This is done by using a tally or score sheet.
Example: The marks obtained by 120 students of first year class in the subject of Education are given below-Construct a frequency distribution. 57
86
69
62
75
73 80
78
87 83
77
35 70
6 8
84
73 81 78
61
72
59
98
95
63 76
73
88 60
52
83 86
4 5
70
53 85 74
62
78
89
84
60
79 91
64
84 85
81
79 90
7 8
83
50 71 65
76
58
71
79
51
61 61
89
81 74
76
74 82
9 1
71
76 80 52
71
66
77
65
44
79 95
74
79 63
83
87 77
7 5
83
48 70 85
61
70
72
67
61
83 75
79
97 75
66
54 81
6
78
75 83 61
8 33
76
62
55
72 76
78
75 99
80
83 86
The following steps are followed to-make a frequency distribution. 1.
Step-1: Range = maximum score-minimum score = 99 — 33 = 66.
2.
Step-2: Number of approximate class intervals to be taken is 7.
3.
Step-3: Length of the class intervals, usually denoted by i, is.
I=
Range No . of class intervals
The length is usually rounded upward to whole number. Therefore 9.4 is taken as 10. 4.
Step-4: Determine the limits of the class intervals 90 — 99 80 — 89 70 — 79 60 — 69 50 — 59 40 — 49 30 — 39
The lowest class interval is taken in which the minimum scores can be included. The minimum score is 33. The lowest class interval can be started from 30, but it is convenient to start the lowest class interval from the score to which addition of the length of the class intervals is easy. So we start from 30. This is called lower limit of the class intervals. Add 9 (1 — 1 = 10 — 1 = 9) to the lower limit to get the upper limit of the first class interval. Now add consequently i = 10 to the lower limits and upper limits to get the remaining class intervals.
5.
Step-5: Distribute the scores in the class intervals by putting a tally mark in the relevant class interval and count the number of scores in each class interval.
Grade
Tallies
90 - 99
||||| |||
80 – 89
||||| ||||| ||||| ||||| ||||| |||||
70 – 79
||||| ||||| ||||| ||||| ||||| ||||| ||||| ||||| |||||
60 – 69
||||| ||||| ||||| ||||| ||
50 – 59
||||| |||||
40 – 49
|||
30 – 39
||
Frequency The number of scores lying in a class interval is called the frequency of that class interval. For Example two scores lie in the class interval 30-39. Therefore 2 is the frequency of the class interval 30-39. Mid-Point of Class Mark The middle of a class interval is called mid point or class mark and is usually denoted by X. It is calculated as Midpoint = X =
Lower limit +Upper limit 2
For Example, the mid point of the class interval 30-39 is X=
Lower limit +Upper limit 30+39 = 2 2
N
=
69 2
= 34.5
Measures of Central Tendency: A single score calculated to represent all the scores is called an average. Average tends to lie in the centre of an array. That is why averages are called measures of central tendency. Since averages locate the centre of a data set, these are also called measures of location. Several types of average can be defined. Most commonly used averages are arithmetic mean, median and mode.
The Arithmetic Mean or Mean The arithmetic mean is the most commonly used average. It is usually called mean or average. The arithmetic mean is defined as the number obtained by dividing the sum of the scores by their number. It is denoted by putting bar on the variable symbol e.g., X (reads as X bar). The formula for calculating the arithmetic mean ungrouped data is:
∑x X´ = N Where Read as sigma, is the Greek symbol means sum of. X Means sum of the values of variable X. N is the number of scores of measurements. In order to calculate the arithmetic mean for grouped data formula is: X fx / f / Where fx means the sum of product of values for ‘f’ and `x', f means frequency of the scores and x means score. f means the sum of all the frequencies of the distribution.
The Median: The median of a set of scores is the middle score of the arithmetic mean of two middle scores in an array. 50% of the scores are less than median and 50% of the scores are greater than median. Formula for calculating median for ungrouped data: Median =
( N 2+ 1 )
th score
Formula for calculating median for grouped data: Median = L +
i N + −C f 2
(
)
Where L = lower class boundary of the median class interval. I = length of the median class interval. F - the frequency of the median class interval. N= f C = the cumulative frequency of the class interval below the median class interval. The Mode The mode is the score that occurs greatest number of times in a data set. Mode does not always exist. If each score occur the same number of times, there is no mode. There may be more than one mode. If two or more scores occur greatest number of times, then there are more than one mode. The mode can be calculated for grouped data with the help of following formula.
f m−f l L+ ×i Mode = 2 f m−f 1−f 2 Where L = lower class boundary of the modal class interval. Fm = the maximum frequency. Fi = the frequency preceding to the modal class. f2 = the frequency succeeding to the modal class, I = the length of the modal class interval. Note: The mode lies in the class interval having maximum frequency. This class interval is called the modal class. Empirical Relationship between Mean, Median and Mode: For moderately skewed distributions, we have the following empirical relation: Mode = 3 Median — 2 Mean Mode = 3 (74.61) — 2 (73.42) Mode = 76.99 Comparison of Measures of Central Tendency: The numerical value of every score in a data set contributes to the mean. This is not true of the mode or median because only the mean is based on the sum of all the scores. In a single peaked symmetrical distribution mean = median = mode. In practice, no distribution is exactly symmetrical, so the mode, median and mean usually have different values. If a population is not symmetrical, the mean, median and mode will not be equal. The mean is affected by the presence of a few extreme scores which the median and mode are not. The mean is preferred if extreme values are not present in the data. Median is preferred if interest is centered on the typical rather than the total score and if the distribution is skewed. If some scores are missing so that the mean cannot be
computed directly, the median is appropriate. Mode is preferred only if the distribution is multimodal and a multi-valued index is satisfactory. The Quartiles The values that divide a set of scores into four equal parts are called quartiles and are denoted by Ql, Q2, and Q3. Q1 is called the lower quartile and Q3 is called the upper quartile. 25% of the scores are less than Ql- and 75% of the scores are less than Q3. Q2 is the median. The formulas for the quartiles are given as: Q1 =
( N 4+ 1 )
Q2 =
2 ( N +1 ) N + 1 = th score 4 2
Q3 =
3 ( N +1) th score 4
th score
and
4.2
THE PERCENTILE RANKS:
The Percentiles: The values that divide a set of scores into hundred equal parts are called percentiles and are denoted by P1, P2, P3, ……….. and P99.? P25 is the first quartile, P75 is the third quartile and P50 is the median. The Percentile Ranks (PR): The procedure for calculating percentile ranks is the reverse of the procedure for calculating percentiles. Here we have an individual's score and find the percentage of scores that lies below it. In the Example, we calculate P78== 83.37. It means that 83.37 is the score below which 78% of the scores fall. If a student has a score of 83.37, we can say that his percentile rank (PR) is 78 on a scale of 100.
Relationships with a Distribution Computing the Coefficient of Correlation A coefficient of correlation measures the degree of linear relationship between two sets of scores. The range of the coefficient is from -- 1 to + I with intermediate value 0 meaning no linear relationship. There are two extremes: r = + 1 indicates perfect positive correlation and r = —I indicates perfect negative correlation. The larger the value of r, the higher is the degree of linear relationship.
Positive Correlation
Negative Correlation
No correlation The most common methods of computing the Coefficient of correlation are: 1.
Rank-difference method:
This method is useful when the number of scores to be correlated'-is small or exact magnitude of the scores cannot be ascertained. The scores are ranked according to size or some other criterion using numbers 1, 2, 3 n The rank-difference coefficient of correlation can be computed by the following formula. 2
Rs = 1 –
6∑ D 2 N (N −1)
Where D == the difference between two rankings. N = Number of pairs of scores. 2.
The Product-moment method
The product-moment coefficient is usually used when the number of scores is large. Thus this method is used in most research studies. The product-moment coefficient is usually denoted by r.
∑ XY − ∑ X ∑ Y
( )( ) (∑ ) √ ∑ (∑ )
N
rxy =
√
∑ X2 N
N
X
N
2
N
Y2 N
Y
2
N
Measures of Variability: Measures of central tendency measure the centre of a set of scores. However, two data sets can have the same mean, median and mode and yet be quite different in other respects. For example, consider the heights (in inches) of the players of two basketball teams. Team-1: 72 73 76 76 78 Team-2: 67 72 78 76 84 The two teams have the same mean height. 75 inches, but it is clear that the heights of the players of team 2 vary much more than those of team 2. If we have information about the centre of scores and the manner in which they are spread out we know much more bout set of scores. The degree to which scores tend to spread about. an average value is called dispersion. The Range It is the simplest measure of dispersion. The range of a set of scores is the difference between maximum scores and minimum scores. In symbols Range = Xm – Xo Where Xm is the maximum score and Xo is the minimum score. Quartile Deviation: The quartile deviation is defined as half of the difference between the third and the first quartiles. In symbols Q. D. = (Q3 – Ql) / 2
Where Q1 is the first quartile and Q3 is the third The Mean Deviation or Average Deviation: The average deviation is defined as the arithmetic mean of the deviations of the scores from the mean or median; the deviations are taken as positive. In symbols
∑ ¿ X− X´ ∨¿ N ¿
M.D. =
For grouped data
M.D. =
∑ f ∨X− X´ ∨¿ ∑f ¿
The Standard Deviation: The standard deviation is the positive square root of the arithmetic mean of the squares of deviations of all the scores from their mean.
X −X ¿2 ¿ ¿ S= ∑¿ ¿ √¿ Short formula for calculating standard deviation
√
S=
∑ X2− ∑ X N
2
( ) N
The Coefficient of Variation: Karl Pearson introduced a relative measure of dispersion known as coefficient of variation (denoted by c.v). It expresses the standard deviation as a percentage of the arithmetic mean of a data set. It is number without units and is used to compare variation in two or more distributions. The smaller value of the c.v. indicates lesser variation. It is also used as a criterion for consistent performance of the students, players etc. C.V.
S X
× 100
Standard Scores: A frequently used quantity is statistical analysis is the standard score or Z-score. The standard score for a data value in the number of standard deviations that the data value is away from the mean of the data set. Z=
´ X− X S
The Normal Curve: Before explaining the normal distribution, some basic concepts of probability is given below an event is a specified result That mayor may not occur when an experiment is performed. For example, in tossing of a coin once, appearance of head is an event, which may or may not occur. The probability of an event is a measure of the likelihood of its occurrence. A probability near indicates that the event is very unlikely to occur. Whereas a probability near 1 indicates that the event is quite likely to occur.
Relative frequency interpretation of probability: Consider the, experiment of tossing a balanced coin once. There are 50-50 chances the head will appear. Consequently, we assign a probability of 0.5 to that event. The relative-frequency interpretation is that in a large number of tosses, the head will appear about half of the time. Some Basic Properties of the Normal Curve 1.
The total area under the normal curve is equal to 1.
2.
The normal curve extends indefinitely in both directions.
3.
The normal distribution is symmetric about the mean that is the part of the curve to the left of is the mirror image of the part of the curve to the right of it.
4.
The mean, the median and the mode are equal.
5.
Mean deviation is 0.7979 .
6.
Quartile deviation is 0.6745 .
7.
In a formal distribution, – 0.674+5 to + 0.6745 covers 50% of the area. – to + covers 68.27% of the area. - 2 to + 2 covers 95.45 of the area. - 3 to IA + 3 covers 99.73% of the area.
4.3
STANDARD SCORES:
Most educational and psychological tests provide standard scores that are based on a scale that has a statistical mean (or average score) of 100. If a student earns a standard score that is less than 100, then that student is said to have performed below the mean, and if a student earns a standard score that is greater than 100, then that student is said to have performed above the mean. However, there is a wide range
of average scores, from low average to high average, with most students earning standard scores on educational and psychological tests that fall in the range of 85-115. This is the range in which 68% of the general population performs and, therefore, is considered the normal limits of functioning. Classifying Standard Scores However, the normal limits of functioning encompass three classification categories: low average (standard scores of 80-89), average (standard scores of 90-109), and high average (110-119). These classifications are used typically by school psychologists and other assessment specialists to describe a student's ability compared to sameage peers from the general population. Subtest Scores Many psychological tests are composed of multiple subtests that have a mean of 10, 50, or 100. Subtests are relatively short tests that measure specific abilities, such as. vocabulary, general knowledge, or short-term auditory memory. Two or more subtest scores that reflect different aspects of the same broad ability (such as broad Verbal Ability) are usually combined into a composite or index score that has a mean of 100. For example, a Vocabulary subtest score, a Comprehension subtest score, and a General Information subtest score (the three subtest scores that reflect different aspects of Verbal Ability) may be combined to form a broad Verbal Comprehension Index score. Composite scores, such as IQ scores, Index scores, and Cluster scores, are more reliable and valid than individual subtest scores. Therefore, when a student's performance demonstrates relatively uniform ability across subtests that measure different aspects of the same broad ability (the Vocabulary, Comprehension, and General Information subtest scores are both average), then the most reliable and valid score is the composite score (Verbal Comprehension Index in this example). However, when a student's performance demonstrates uneven ability across subtests that measure different aspects of the same broad ability (the Vocabulary score is below average, the Comprehension score is below average, and the
General Information score is high average), then the Verbal Comprehension Index may not provide an accurate estimate of verbal ability. In this situation, the student's verbal ability may be best understood by Helping Children at Home and School II: Handouts for Families and Educators S2-8llooking at what each subtest measures. In sum, it is important to remember that unless performance is relatively uniform on the subtests that make up a particular broad ability domain (such as Verbal Ability), then the overall score (in this case the Verbal Comprehension Index) may be a misleading estimate. 4.4
PROFILE:
One advantage of converting raw scores to derived scores is that a pupil’s performance on different tests can be compared directly. This is usually done by means of a test profile, like the one presented in Figure 14.3. such a graphic representation of test data makes it easy to identify a pupil’s relative strengths and weaknesses. Most standardized tests have provisions for plotting test profiles. The profile shown in figure 14.3 indicates a desirable tend in profile construction, instead of plotting targets scores as specific points on the scales, test performance is recorded in the form of bands that extend one standard error of measurement above and below the pupil’s obtained scores. Recall from our discussion of reliability that there are approximately two chances out of three that a pupil’s true source will fall within one standard error of the obtained score. Thus, these confidence bands indicate the ranges of scores within which we can be reasonably certain of finding the pupil’s true standings. Plotting them on the profile enables us to take into account the inaccuracy of the test scores when comparing performance on different tests. Interpreting differences between tests is simple with these score bands. If the bands for two tests overlap, we can assume that performance on the two tests does not different significantly, and if the ands do not overlap, we can assume that there is probably a real difference in performance.
The score bands used with the differential aptitude test can be plotted by hand or by computer. The computer produced profile shown in figure 14.3 is based on the same sex percentiles. There are recorded down the left side of the profile and were obtained from the percentile norms in table 14.3 the opposite sex percentiles are listed down the right side of the report also to show how the scores compare with the female norms. The differences in percentiles for some tests plot the results directly on the profile. The use of such bands minimizes the tendency of test profiles to present a misleading picture. Without the bands we are apt to attribute significance to differences in test performance that can be accounted for the chance alone. When profiles are used to compare test performance, it is essential that the norms for all tests be comparable. Many test publishers provide for this by standardizing a battery of achievement tests and a scholastic aptitude test on the same population. Profile Narrative Reports Some test publishers are now making available to profile of each pupil’s cores, accompanied by a narrative report that describes how well the pupil is achieving. The graphic profile provides a quick view of the pupil’s strengths and weaknesses, and the narrative report aids in interpreting the scores and in identifying areas in which instructional emphasis is needed. A typical report of this type, for a widely used test battery, is shown in figure 14.4. Narrative reports should be especially useful in communicating test result to parents. They are, of course, also helpful to those teachers who have had little or no training in the interpretation and use fo scores from published tests.
UNIT-5: EVALUATING PRODUCT, PROCEDURES & PERFORMANCE 5.1
EVOLUTION THEMES AND TERMS PAPERS:
The evaluation is structured around a logical sequence of seventeen questions which fall under six evaluation themes. The following are major themes of evaluation. 1.
Learning Outcome:-
The quality of learning outcome is the first theme identified in teaching and learning from work under this theme one of important sub theme is identified. Attainment of Curriculum Objectives:
Considered the knowledge, skill and understanding of our pupil. How does the knowledge level of pupil reflect the curriculum
objectives for chosen area? What opportunities are pupil's afforded use and display their
ability applies their knowledge and skill? Can pupils use their skills for curriculum reasoning in problem
solving? What is the attitude of pupils to learning curriculum? Do our pupils enjoy learning? Are they motivate to learning. In numeracy what do you understand by each of the following
skills?
Applying and problem solving Communicating and expressing Implementing Integrating Reasoning Understanding of recalling
In the course of a week how many of the following future in yours numeracy lessons? What opportunities are provided to development of the following context
Oral language Reading Writing Digital literacy
If we have not completed a school improvement plan to date, what do we need to focus on the support learns out comes and attainment of curriculum objectives in each curriculum area. 2.
Learning Experience.
The quality learning experience is the second theme identified in teaching of three important sub theme are Identified.
Learning environment Engaged in learning Learning to learning
Engaged in Learning
Are students interested and enthused by the content and teaching
approaches used? Do we encouraged pupil questioning considered teacher input
V'S pupil participation in your class room. How pupils are active when teacher work? Collaborative and independent learning. Progressive skill learning and skill development. Challenge and support.
To support learning by referring to outcomes and related success criteria to allow for further enhancement of understanding.
Pupils enjoy learning in class room and are eager to find out
more, All students in class room afforded the opportunity to participate in lesson and engage with learning.
Learning Environment To involve the students in development rules which recognize the rights of responsibilities of the community. Prepare supervision of pupils both within the class and at break times within the school setting. All the recourses well organized, labeled and clear to all learners. Celebrates pupils learning and achievements through a range of display. Concrete and visual materials, centers of interest and display of pupil work. Learning to Learn Learning to learning is the third sub theme of learning experiences. To engage the pupils to monitor their own progress in learning for learning technique to utilize them properly in class room to develop the skills of learner by proper planning of lessons. To allow the learner to communicate work with other in the clam. How do we enable the student learner to develop their personal organization to plan out their own work study and revision skills do we teach. To teach the pupils how to organized prose nil the work. To make the pupil creative and give the opportunity for collaborative work. 3.
Teac’s Practice
The quality of teacher's practice is the third theme of teacher and learning from work. Under this theme four sub themes are identified
Preparation for teaching Teaching approach Management of pupils
Assessment
1.
Preparation for Teaching
Learning outcome: Do we provide class, relevant and differentiated learning out
comes to pupil? How pupils are made aware of what they are going to learn? Are pupil familiar with the expected success criteria in learning activities.
Written Plans
Are the long and short terms plans prepared in accordance with
the rules for primary teacher. Does the planning clearly indicate expected learning out comes, teaching approaches resources and activities.
Monthly Progress Report
Do our cantos miosuila provide a clear picture of the progression and continuity of pupil learning across the curriculum.
Literacy and Numeracy
Are the literacy and numeracy opportunities identified across
the curriculum? How we identified these opportunities an are whole school plans individual planning?
Resources How satisfied are we with the resources, materials and equipment we have with in our class room and available within the school? Are the necessary and relevant material & readily available?
Assessment
Reflect on the use of assessment as an aid to teaching and
learning how do we plan for assessment. Does our planning reflect whole school assessment policy? How do we incorporate best practice as ????? in assessment guide line 2007 into our teaching and learning?
Teaching Approach Learning Outcome
How are lessons guided by expected learning outcome and
linked to curriculum. What provision is made to ensure expected learning outcomes are achieved during lesson?
Focus of Learning
Is attention given within each curriculum area. To the systematic development and application of knowledge
and skill including ICI? Pupils leaving timely and does it happen at a regular interval
Analysis use of Assessment Information Information teacher's setting of learning targets and activities for individual pupils group, the whole class pupils group the whole class, Inform the school improvement plan and as revise and update whole school improvement target. What is term piper? Term Paper: Definition “A term paper has two purposes the student should demonstrate an understanding of the material as well as the ability to communicate that understanding effectively. Writing term papers gives students practical experience in writing at length communicating thoughts and idea through the written word is a necessary skill in any profession.
A term paper is a research paper written by students over an academic term accounting of a large part of a grade. Terms papers are generally intended to describe on event, a concept or argue paint. A term is a written original work discussing a topic in detail, usually several typed paper in length and is often due at the end of semester there is much overlap between term papers and research paper "The term Paper" was originally used to describe a paper (usually a research based) that was due to at the end of "term" either a semester or quarter, depending on which unit of measure a school used common usage has "term paper" and "research paper" as interchangeable but this is not completely accurate. Not all term papers involve academic research and not all research papers are one term papers. Term papers date back to the beginning of the 19th century when print could be reproduced cheaply and written text of all types (reports memoranda specifications, and scholarly articles) could be easily produced and disseminated during the year from 1870 to 1900, mouton and Holmes (2003) write that American education was transformed as writing become a method of discoursed and research the hallmark or learning. Importance: Right away that you are cognizant of the fundamentals of composing A+ research projects here are some extra mysteries to guarantee. Never forget to dependably edit your term paper articles. The term paper is a necessary evil for every college student. Many students wonder why the need to regurgitate lectures and research on paper, but the term paper actually serves an important purpose for a college education and farther careers.
Effects:
In addition to the immediate effects on a student's course grade and grade point average, a term paper will be valuable when searching for or advancing in careers. Term Paper Evolution Term paper has been graded according to the following criteria. The Cortical Section Range depth and quality of literature research on your topic.
The author has integrated a variety of key pieces of literature on
the topic but so representing the consent state of research as well as covering various view point. The author has integrated a variety of key pieces of literature but
focuses too much on one particular author or view point. No independent literature research has been carried out the author exclusively refers to pieces of literature that have been assigned as course reading.
Correctness of theoretical part
The content of individual pieces of literature and giving than
appropriate prominence. The contents of individual pieces of literature are largely
represented correctly although the student may give too much prominence to individual. The literature review reveals that the student has not fully under stood large parts of the literature. The content of theoretical part is, as a result incorrect to the considerable degree.
Presentation of Literature Review Development of Argument:
The student has represented the views of prominent scholars on the topic and has developed critical argument and support of or against the literature represented has / her literature review is focused on the research question and relevant to it.
The student presents the current state of research concerning the topic at hand too much on one particular view point.
The literature review shows lack of focus. The author presents bit and piece that are loosely related to the topic at hand.
Presentation of Results
The tables and figures are legible and easily to grasp at the first glance It is evident that the author has spent find best visual means of presentation.
The formatting of tables and figures is satisfactory yet not always easy to grasp to grasp at the first glace.
The student has not attempted to use tables and figures to support his / her argument.
The student's plain how he/she arrived at the result represented and indicates their significance to the topic or field of linguistics in which paper is written.
The author largely points out the most striking result at the same time; however he/she concentrates too much discussing aspects that are not entirely relevant to research question at hand.
The author mainly lists example from her data no comparison of her results with those of previous studies is offered.
Language (Vocabulary, Grammar, Style).
The student uses the academic writing register/ style with appropriate linguistic terminatories.
The language used is largely suitable for an academic piece of writing but the paper exhibit some mainly recurring.
The student uses writing style which is an inappropriate for an academic paper. There are great number of grammatical mistake and Paragraph Lake coherence.
Further instructions End rubric for term paper grading
Each student must submit an independently-written report of their term paper project. Team members are welcome to share literature related to their project theme and may work jointly to develop hypotheses, predictions and experimental design. Nonetheless, the organization and text of each report must be developed independently by each team member. Normal rules concerning plagiarism apply. If you have any questions about this, best ask us first. The written term paper must have the following structure and include all of the following elements: THE PAPER SHOULD BE A MAXIMUM IF 5 PAGES (doublespaced; Times New Roman 12 font), excluding the title page and literature cited. First (title) page must include: 1. 2. 3.
Descriptive title Author Abstract (NOTE: MAXIMUM 200 WORDS)
Subsequent pages must include: 4. 5. 6. 7. 8. 9.
Introduction Hypotheses and predictions (these can be incorporated into the introduction or presented below a separate sub-heading) Study Area and Organisms Methods and experimental design Significance of work Literature Cited (use a journal format of your choice (e.g. Journal of Ecology, Oecologia)
The following criteria will be used for grading the report: 1. Abstract: Does the abstract reflect the title of the project and the aim and scope of the work? Does it contain essential information on rationale, hypothesis, study system and significance? Is it written clearly? (10 points) 2. Introduction: Does the introduction start by introducing a significant question in community ecology? Are statements supported by appropriate
citations from published literature? Is the initial question refined through the introduction to statement of the objective of the study? (10 points) 3. Hypotheses and predictions: Are the hypotheses presented clearly related to the objective of the study and are they logically connected to the ideas presented in the introduction? Are hypothesis stated correctly (i.e., do they provide an explanation for an observation)? Are predictions logically connected to the hypotheses? Are alternatives to the primary hypothesis acknowledged (where appropriate)? (10 points) 4. Study area and organisms: Is necessary background information on the ecology/natural history of study organisms and the study site presented? Is there unnecessary/irrelevant information that could have been omitted? Is the choice of organism/study sit appropriate given the objective of the study? (10 points) 5. Methods and Experimental Design: Is the explanation of the methods clear (use figures if necessary)? Are the methods appropriate to test the hypothesis proposed? Are essential methodological details included? This might include, for example, replication of treatments, size of treatments, duration of experiment, description of independent and dependent variables. Has the author considered potential confounding effects that might interfere with the ability to test the hypothesis? Are these recognized/addressed (where possible)? (10 points) 6. Significance, originality and creativity of work: This is the justification statement for the project – this does NOT mean just the conservation/management (i.e., applied) importance of your proposed work. I am looking here for a statement to indicate how this project can move the field of community ecology forward. Does this study potentially provide new insights into how communities are organized? Are the results from this study system broadly applicable to other groups of organisms or other kinds of interactions? What other studies might build on the results from this one – or would the results of this study allow you to infer something new about this system that you could then go on to test? (10 points)
7. Literature Cited: Are the papers in the body of the proposal cited in here? Are the references cited here in the body of the text? Is consistent formatting used? (5 points) 8. Presentation and clarity: Are the different sections of the proposal well linked? Are the ideas presented clearly – and can they be followed from one section of the proposal to the next? Is the writing style clear (topic sentences introduce themes presented in each paragraph; concise language used; spelling and grammar acceptable)? Use of tense and active/passive voice is consistent? Note: Use the past tense to describe results found in previous studies; use future tense "we will..." to describe work that you propose to do (5 points). 5.2
EVALUATING GROUP WORK & PERFORMANCE
Evaluating group work can provide valuable information about the degree to which:
The use of group work enhanced (or otherwise) student
achievement of learning outcomes and engagement. The use of group work enhanced (or otherwise) evaluator delivery or assessment of the unit of study,
Specific Questions: Evaluator can ask more specific questions about:
The response of individual students to group work as compared
to individual work. Group work process versus the group work product. The effectiveness of group work in class and/or out of class to
enhance learning, The appropriateness of group work. Organizational, planning, management and monitoring issues. Strengths and weakness of group work and ideas for
improvement. Diversity issues (did some students find it easier or harder, benefit more
than others and why, what about issues of power), The ways in which explained, facilitated, managed and
monitored the groups. The overall nature of the unit o study.
Timing of Evaluation: Evaluation an occur at any time during the unit of study program, but it usually occurs at the end of the semester or at the end of the task that is being undertaken and evaluated. Ideally students should be given time to reflect upon their experiences prior to completing any form of evaluation especially if evaluator desire some specific information about their experiences of group work or have a specific reflection component within the work being evaluated.
It is also important to clearly explain why undertaking evaluating?
It's a good idea to explain all of this at the start of the unit of study and to provide opportunities for students to reflect along the way. Evaluation can also be built into the requirements of the group work tasks by asking students to complete an evaluation of their own or the whole groups experience of group. This could also be a requirement of their assessment. It is up to evaluator whether or not to allocate marks. Method for Collecting Data for Evaluation: There is no single method for designing or conducting an evaluation method can be quantitative or qualitative, formal or informal, formative or summative, self administrated or externally administered, or any combination of these. There are advantages and disadvantages to each method and evaluator will largely depend upon the purpose of the evaluation and the content, material practices, tasks or activities being evaluated,
1.
Questionnaire:
Questionnaire is a common method of approach that involves having students complete a survey in the class. When evaluator designing questionnaire ensure that there is an introduction which explains the purpose of the evaluation, that there are clear instructions for completion and that the questions are unambiguous. The questions posed can be open ended or closed, or a combination. Open Ended Questions:These questions have the advantage of allowing students to identify what were the most important elements of their experience. A disadvantage is that they may not write much or may be nothing at all. Closed Questions: These are statements that allow students to rate their agreement or disagreement with a comment or statement by using a Liker Scale. Strongly agree
Agree
Neural
Disagree
Students are usually willing to answer these questions, especially if the questionnaires are anonymous. A disadvantage is that they do not give detailed response or answer "why" or "how" questions. 2.
Checklist: Checklist is another method that can provide basic data.
An example may be a list of provided unit outcomes (knowledge, skills, attributes, abilities etc) and students circle or tick the ones that apply. Alternatively evaluator could ask to generate their own list of outcomes, For example group work provided me with......
Autonomy. Opportunity to get to know my classmates.
S
Opportunity to work on a real life problems students are usually willingly to complete these lists but again the disadvantages is that they do not give detailed responses or answer "why" or "how" questions. Evaluation Hand-Out: Some academies design their own evaluation hand-out that can combine a number of evaluation methods and are anonymous, quick and easy to complete. They can take any form, use images, diagrams, comment boxes or questions and lists as above. Interview: Interview can be done individually or in small groups and provide the opportunity for evaluator to probe for deeper analysis of the process and experience. The disadvantage of this method is that it can be time consuming for both evaluator and the students, and in a larger group may be some students may be more vocal than others. Focus Group: Focus group uses a facilitative rather than direct questioning approach and is a useful way of having students discusses the process of group work. This method allows students to work off and build upon each other's answers. The disadvantage is that it is time consuming for both evaluator and students and there is the added difficult of arranging a time that will suit everyone. Practicality of the Evaluation Process: Before making a choice about evaluation method also consider the following questions:
What resource is needed to undertake the evaluation? What has to be done in order to undertake the evaluation (printing of forms, preparation of one-line questionnaire, ordering questionnaires, arranging interview rooms)?
What levels of participation evaluator require from the students, tutors, organizations or any other party who were involved in the group work activities?
Uses of Evaluation: It is important to consider who will use the evaluations and how it will be used. This is a key part of the planning process which relates to the purpose of the evaluation. It is also important to reflect upon and consider the methods that have been used to gather information about the effectiveness of group work. 5.3
EVALUATING DEMONSTRATION:
1.
The evaluation portion of the demonstration-performance method is where students get an opportunity to prove that they can do the manoeuvre without assistance. For the simulated forced approach you should tell students that you will be simulating an engine failure and that they are to carry out the entire procedure including all checks and look-out. While the student is performing this manoeuvre you must refrain from making any comments. Offer no assistance whatsoever, not even grunts or head nods. You must, however, observe the entire manoeuvre very carefully, so that you can analyze any errors that the student may make and debrief accordingly.
2.
3.
NOTE: You would interrupt the student's performance, of course, if safety became a factor. 4.
Success or failure during the evaluation stage of the lesson will determine whether you carry on with the next exercise or repeat the lesson.
Demonstration 1.
The explanation and demonstration may be done at the same time, or the demonstration given first followed by an explanation, or vice versa. The skill you are required to teach might determine the best approach.
2.
Consider the following: You are teaching a student how to do a forced landing. Here are your options: a.
Demonstrate a forced approach and simultaneously give an explanation of what you are doing and why you are doing it; or,
b.
Complete the demonstration with no explanation and then give a detailed explanation of what you have done; or
c.
Give an explanation of what you intend to do and then do it.
You will find that different instructors will approach the teaching of this skill differently. The following represents a suggested approach that appears to work best for most instructors. On the flight prior to the exercise on forced landings, give a perfect demonstration of a forced landing. It may be better not to talk during this demonstration, since you want it to be as perfect as possible to set the standard for the future performance. There is another advantage of giving a perfect demonstration prior to the forced landing exercise. Your students will be able to form a clearer mental picture when studying the flight manual because they have seen the actual manoeuvre. a.
The next step would be for you to give a full detailed explanation of a forced landing. During this explanation you would use all the instructional techniques described previously. You must give reasons for what is expected, draw comparisons with things already known and give examples to clarify points. This explanation should be given on the ground using visual aids to assist student learning.
b.
When in the air, give a demonstration, but also include important parts of the explanation. Usually asking students
questions about what you are doing or should do, will give them an opportunity to prove they know the procedure, although they have not yet flown it. c.
After completing the forced landing approach, while climbing for altitude, clear up any misunderstandings the students may have and ask questions.
d.
The demonstration and explanation portion of the demonstration-performance method is now complete and you should proceed to the next part, which is the student performance and instructor supervi
Evaluation Matrix for the Demonstration When assessing the demonstration of teaching skills, attention is given to the applicant's use of didactic solutions. The following matrix transparently describes the criteria used to evaluate the demonstration. The matrix is indicative instead of normative, and is used for support when evaluating the demonstration of teaching skills. In other words, not all of the aspects listed in the matrix need to be assessed systematically. The evaluators use the criteria listed below to form an overall appraisal of the demonstration's standard by assessing the quality of the components that are of a good or better level. If the demonstration includes a preliminary assignment, all the individual components are assessed in relation to it. If well grounded, the demonstration may also be virtual, held in real time and interactively. Component of the demonstration of teaching skills Objectives
Passabl e
Satisfactor y
Good
Very good
The applicant The applicant specifies specifies the the objectives objectives. clearly The applicant specifies the objectives taking into account the context,
content and target group. Content Correspondence between the topic and content of the demonstration Academic nature of the content Consistency and clarity of presentation of the content Critical approach Many-sided argumentation Connection between theory and practice Aptness, diversity and topicality of the research data used Use of own research results Consideration given to the target group in the choice of content
Methods
The topic and content of the demonstrat ion correspond to each other. The content is academic. The applicant presents the content clearly and consistentl y. Where appropriate , the applicant uses his/her own research results during the demonstrat ion. The applicant takes the target group into considerati on when.
The topic and content of the demonstratio n correspond to each other. The content is academic and topical. The applicant presents the content clearly and consistently. The applicant examines the conte critically. The applicant discusses the topics from many angles. The applicant explains the connection between theory and practice. The research data discussed are relevant, many-sided and topical. Where appropriate, the applicant uses his/her own research results during the demonstratio n.
The
teaching The
teaching
Organization of teaching Motivation of target group Suitable use of teaching methods Suitable use of teaching aids and materials
Wrap-up Evaluation of the teaching situation in terms of the objectives set Consideration given to the target group in solutions related to evaluation
situation is organized appropriate ly.
situation is organized appropriate ly, taking into considerati on its objectives, contents, target group and context. The applicant inspires the target group to engage, stimulates the listeners’ interest and motivates them to participate. The applicant uses different teaching methods appropriate ly in terms of the situation, objectives.
The applicant The applicant evaluates evaluates the the teaching teaching situation in situation in terms of terms of the the objectives objectives set. set. The solutions related to evaluation are relevant and take the target group into
considerati on.
Interaction skills Use of voice Clarity and intelligibility of speech Coherence of oral and written communication Quality of interaction Other matters improving communication.
Alignment of the preliminary assignment and the demonstration of teaching skills
The applicant’s The applicant’s delivery is delivery is clear and clear and understand understand able. able. Oral and Oral, written written and visual communic communica ation is tion is coherent. coherent. The applicant interacts with the listeners in a natural and appropriate manner in teaching situation. The
The preliminar y assignment and the demonstrat ion of teaching skills are well aligned.
preliminary assignment lays the foundation for and supports the demonstrat ion, and the two form a consistent whole.
5.4
EVALUATION OF PHYSICAL MOVEMENTS AND MOTOR SKILLS:
Motor Skills A motor skill is a function, which involves the precise movement of muscles with the intent to perform a specific act. Motor skills are skills that are associated with the activity of the body muscles like the skills performed in sport. Fine motor skills arc the type that is associated with small movements of the wrists, hands, feet, fingers and toes. Motor skills are the ability to make particular bodily movements to achieve certain tasks. They are a way of controlling muscles to make fluid and accurate movements. These skills must be learned, practiced and mastered, and overtime can be performed without thought, for example, walking or swimming. Children are clumsy in comparison to adults, because they have yet to learn many motor skills that allow them to effectively accomplish tasks. Motor skills are also learned and refined in adulthood. If a woman takes up belly dancing, her first movements will not closely resemble that of the teacher. Overtime however, she will learn how to control her muscles to make the signature movements that a belly dancer makes. Genetic factors also affect the development of motor skills, for example, the children of a professional dancer are far more likely to be good at dancing, with good coordination and muscular control, than the children of a biochemist. Gross motor skills are usually learned during childhood and require a large group of muscles to perform actions, such as balancing or crawling. Fine motor skills involve smaller groups of muscles and are used for fine tasks, such as threading a needle or playing a computer game. These skills can be forgotten if disused over time.
Types of Motor Skills There are two major categories of motor skills 1. 2.
Gross Motor Skills Fine Motor Skills
Gross Motor Skills
Gross motor skills maneuver large muscle groups coordinating functions for sitting, standing, walking, running, keeping balance and changing positions. These skills involve skills are those that are typically acquired during infancy and young childhood to control the large muscles of the body. These skills include sitting, crawling, walking. According to Anna Maria Wilms Floet, MD, on Medicine. Throwing a ball, riding a bike, playing sports, lifting and sitting upright are brief descriptions of large motor movements. Gross motor skills depend upon muscle tone, the contraction of muscles and their strength for positioning movements. Fine Motor Skills Fine motor skills coordinate precise, small movements involving the hands, wrists, feet, toes, lips and tongue. Features of fine motor control include handwriting, drawing, grasping objects, cutting and controlling a computer mouse. Experts agree that one of the most significant fine motor achievements is picking up a small object with the index finger and thumb referred to as the pincher grip, which usually occurs between 8 and 12 months of age Fundamental Motor Skills Fundamental motor skills are common motor activities with specific observable patterns. Most skills used in sports and movement activities are advanced versions of fundamental motor skills. For example, throwing in softball and cricket, the baseball pitch, javelin throw, tennis serve and netball shoulder pass are all advanced forms of the overhand throw. The presence of all or part of the overhand throw can be detected in the patterns used in these sport specific motor skills.
Similar relationships can be detected among other fundamental motor skills and specific sport skills and movements. Assessment of Motor Skills A motor skills assessment is an evaluation of a patient to determine the extent and nature of motor skill dysfunction. Care providers like physical therapists and neurologists can perform the assessment, which may be ordered for a number of reasons. It is not invasive, but does require the completion of a number of tasks. The length of time required can vary, depending on the test or tests used. It may be necessary to set aside a full day for testing. One reason for a motor skills assessment may be to establish a child's baseline level of motor competency. This can provide a reference point for the future. Physical education teachers, for example, may perform brief assessments with new students to determine which kinds of activities would be safe and appropriate for them. Pediatricians also use such testing to assess their patients. If a child appears to have developmental delays, this may result in a referral for a more extensive examination Different Ways to Assess Motor Skills Motor Skills can be evaluated in different ways .some of them are as follow. 1. Test gross motor skills using range of motion. Assess gross motor skills by asking the individual to perform a series of movements known as range of motion. Evaluate range of motion by asking the individual to hold an arm out and move it in a circular direction. The arm should be able to move in a complete circle when fully extended. Then ask the individual to stand and place one leg out. Have the individual move the leg up and down, back and forth and left to right. Note any difficulty in movement, abnormalities or pain experienced by the individual. 2. Assess gross motor skills using games. Gross motor skills can be evaluated using games and sports. Ask the individual to kick a ball to test gross motor skills of the leg. Jumping rope is a great way to evaluate
motor skills, because it uses both the arms and legs working together to accomplish the task. Hopscotch, basketball and walking on a balance beam are also good ways to evaluate gross motor skills. Look for the fluidity of movement, problems with balance and hand-eye coordination. 3. Evaluate fine motor skills of arms and legs. Ask the individual to put a clothespin on the edge of a box. Stringing beads on a shoelace is another way to assess fine motor skills. Using a stapler and placing a paperclip on a sheet of paper are also ways to assess fine motor skills. Place an item on the floor and ask the individual to pick it up using his toes only. Watch the individual perform each task, looking at how smooth the movements are and how easily the task is completed, and note any difficulties. 4. Test fine motor skills using common household items. Give the individual a jar and ask her to unscrew the lid and screw the lid back on. Ask the individual to place items, such as coins or blocks, into containers such as a bowl, bucket or cup. Draw a straight line on a piece of paper and have the individual use a pair of scissors to cut the line on the paper. Using pencils or pens of different sizes, ask the person to pick up and grasp each pencil/pen. Then ask the individual to trace items drawn on the paper. Watch for the completion of each task, looking for any problems during each movement. 5. Assess fine motor skills while getting dressed. Ask the individual to put on and button up a shirt. Next, have the individual put on a pair of pants that have a snap closure and a zipper. Give the individual a pair of shoes, which have shoelaces and not Velcro closures, and ask him to tie the shoes. Watch the individual perform each task, looking for difficulties, abnormal movements and the ability to perform each task completely without help. Some Motor Skills and Their Evaluation for Preschoolers Dancing, either freestyle or through songs with movements, such as "I'm a Little Teapot Dance and movement classes, like pre-ballet, can be fun but aren't necessary for motor-skills developmen
Walking, around the house, neighborhood, or park. For variety, add in marching, jogging, skipping, hopping, or even musical instruments to form a parade. As they walk, tell stories, count, or play games. Observe the child how he walks on a piece of string or tape, a low beam or plank at the playground, or a homemade balance beam. Playing pretend: Kids boost motor skills when they use their bodies to become waddling ducks, stiff-legged robots, galloping horses, soaring planes—whatever their imagination comes up with! Riding tricycles, scooters, and other ride-on toys; pulling or pushing wagons, large trucks, doll strollers, or shopping carts. Playing tag or other classic backyard games, such as Follow the Leader, Red Light/Green Light, Tails, or Simon Says (avoid or modify games that force kids to sit still or to be eliminated from play, such as Duck Duck Goose or musical chairs). Throwing, catching, and rolling large, lightweight, soft balls Swinging, sliding, and climbing at a playground or indoor play space. Ball Control Skills The following ball skills are generic in that they are not specific to a particular sport, and they are grouped by whether they require one or two balls. Skills are listed in their approximate order of difficulty. Younger movers may use a plastic ball, volleyball, or child's basketball, and older movers may use a youth-sized or adult-sized basketball. Select the highlighted name of a given skill to view a short video clip. Assessing Motor Skills in Early Childhood - Using the PDMS (Peabody Developmental Motor Scale) Does your toddler have special needs? Early diagnosis of problems in developmental motor skills is crucial for helping children with special needs. One of the most popular assessment tools is the Peabody Developmental Motor Scale. Is it reliable and sufficiently responsive?
After more than ten years of extensive research, a second edition known as the PDMS-2 finally replaced the first edition of the Peabody Developmental Motor Scale. The authors, M. Rhonda Folio and Rebecca R. Fewell, claim that the new and updated version provides better and more in-depth assessment of the gross and fine motor skills of preschoolage children. The PDMS-2, of course, is just one of the most commonlyused assessments for measuring the motor skills of toddlers. However, for children with special needs, the Peabody Development Motor Scale is one of the most reliable testing instruments used by many professionals, such as therapists, psychologists, and diagnosticians. Purpose of the Test The main purpose of the Peabody Developmental Motor Scale is to test the motor skills of children. Gross motor skills involve using large muscles such as in bending, balancing, crawling, walking, and jumping. Fine motor skills, on the other hand, involve using smaller muscles, particularly the muscles in the hand. A child, at a specific age, is expected to display proficiency at certain motor skills. With the PDMS-2, most dysfunctions of motor skills will be identified. And using the results of the PDMS-2, the special education teacher, parents, and other professionals of the IEP team can develop a more responsive learning and remediation program for the child with special needs. Would you want your child to take this assessment test? The next part describes how the test will be administered. Administration of the Test This assessment test is composed of six sub-tests that include special instructions on how each is administered to the preschool-age child. To keep the results of the test reliable and precise, the actual instructions on how the test will be carried out are only given to the test administrators and psychologists. This will prevent the parents from "preparing" their child to pass the test. But the sub-tests are given below: Reflexes – A reflexive action is a quick and automatic reaction to a particular environmental stimulus. This reaction is measured in this sub-
test that is composed of eight items. This sub-test, however, is administered only to children who are 11 months and younger because reflexes have been observed to be extensively integrated within 12 months. Stationary – This sub-test aims to measure the child's ability to maintain balance or equilibrium. It involves mainly the ability of the preschoolage child to control his or her body. It is composed of 30 items. Locomotion – This sub-test evaluates the child's ability to move. The movement involves crawling, walking, running, and other similar actions. The sub-test has 89 items. Object Manipulation – In this sub-test, the object that is manipulated is the ball. Since it is developmentally impossible for babies to even hold a ball, this sub-test is administered only to children who are older than 11 months. This 24-item sub-test involves activities such as throwing, catching, and kicking balls. Grasping – This sub-test primarily measures the preschool-age child's ability to use the muscles of the hand. Made of 26 items, the sub-test progressively determines their ability to grasp objects and to control fingers. Visual-Motor Integration – This sub-test evaluates the child's eye and hand co-ordination. Aside from controlling muscles, the test determines the level of the child's visual perception. Some examples of the activities of this 72-item sub-test include building blocks and copying designs. 5.5
EVALUATING ORAL PERFORMANCE:
Communication skills are taught in a wide range of general education courses and students are in need of speaking and listening skills that will help them succeed in future courses and in the workplace. Thus, the assessment of communication skills is an important issue in general education .Oral assessment is often carried out to look for students' ability to produce words and phrases by evaluating students' fulfillment of a variety of tasks such as asking and answering questions
about themselves, doing role-plays, making up mini-dialogues, defining or talking about some pictures given them. The operations in an oral ability test are either informational skills or interactional skills. The testing of speaking is widely regarded as the most challenging of all language tests to prepare, administer and score. Kind of Oral Communication Oral communication can also be delivered individually or as part of a team. Therefore, knowing the kind of oral communication act that is expected is a necessary step in being able to give useful feedback and ultimately an accurate evaluation Pronunciations Pronunciation is a basic quality of language learning. Though most second language learners will never have the pronunciation of a native speaker, poor pronunciation can obscure communication and prevent a student from making his meaning known. When evaluating the pronunciation of students, listen for clearly articulated words, appropriate pronunciations of unusual spellings, and assimilation and contractions in suitable places. Also listen for intonation. Are students using the correct inflection for the types of sentences they are saying? Do they know that the inflection of a question is different from that of a statement? Listen for these pronunciation skills and determine into which level student falls. Vocabularies Vocabulary comprehension and vocabulary production are always two separate banks of words in the mind of a speaker, native as well as second language. Teacher should encourage students to have a large production vocabulary and an even larger recognition vocabulary. For this reason it is helpful to evaluate students on the level of vocabulary they are able to produce. Are they using the specific vocabulary instructed them in the class? Are they using vocabulary appropriate to the contexts in which they are speaking? Listen for the
level of vocabulary students are able to produce without prompting and then decide how well they are performing in this area. Accuracy Grammar has always been and forever will be an important issue in foreign language study. Writing sentences correctly on a test, though, is not the same as accurate spoken grammar. As students speak, listen for the grammatical structures and tools teachers have taught them. Are they able to use multiple tenses? Do they have agreement? Is word order correct in the sentence? All these and more are important grammatical issues, and an effective speaker will successfully include them in his or her language. Communications A student may struggle with grammar and pronunciation, but how creative is she when communicating with the language she knows? Assessing communication in the students means looking at their creative use of the language they do know to make their points understood. A student with a low level of vocabulary and grammar may have excellent communication skills if she is able to make other understand her/him„ whereas an advanced student who is tied to manufactured dialogues may not be able to be expressive with language and would therefore have low communication skills. Don't let a lack of language skill keep the students from expressing themselves. The more creative they can be with language and the more unique ways they can express themselves, the better their overall communication skills will be. Interactions Ask the students questions. Observe how they speak to one another. Are they able to understand and answer questions? Can they answer when teacher ask them questions? Do they give appropriate responses in a conversation? All these are elements of interaction and are necessary for clear and effective communication in English. A student with effective interaction skills will be able to answer questions and follow along with a conversation happening around him. Great oratory
skills will not get anyone very far if he or she cannot listen to other people and respond appropriately. Encourage your students to listen as they speak and have appropriate responses to others in the conversation. Fluency Fluency may be the easiest quality to judge in your students' speaking. How comfortable are they when they speak? How easily do the words come out? Are there great pauses and gaps in the student's speaking? If there are then your student is struggling with fluency. Fluency does not improve at the same rate as other language skills. You can have excellent grammar and still fail to be fluent. You want your students to be at ease when they speak to you or other English speakers. Fluency is a judgment of this ease of communication and is an important criterion when evaluating speaking. Suggestions for Improvement Offer suggestions (rather than criticisms) for improved delivery style. Many students are aware of their difficulties in delivering oral communication and want feedback and support, and they do want suggestions. Not so useful: "Don't wave your hands when you talk."Better: "Let's figure out what you're going to do with your hands so that you don't distract the audience from what you are saying. What feels more natural to you?" Present oral communication skills as a set of professional skills that all professionals learn and practice steadily throughout their lives.
UNIT-6: PORTFOLIOS 6.1
PURPOSE OF PORTFOLIOS:
Literally Definition:
A) a large, flat, thin case for carrying loose papers or drawings or maps; usually leather
B) a set of pieces of creative work collected to be shownto
C)
potential customers or employers; ' the artist had put together a portfolio of his work"; "every actor has a portfolio of photographs" A collection of various company shares, fixed interest securities or money-market instruments.
Terminological Ally Definition: A portfolio is a purposeful collection of student work that exhibits the student's efforts, progress, and achievements in one or more areas of the curriculum. The collection must include the following:
Student participation in selecting contents. Criteria for selection. Criteria for judging merits. Evidence of a student's self-reflection.
It should represent a collection of students' best work or best efforts, student-selected samples of work experiences related to outcomes being assessed, and documents according growth and development toward mastering identified outcomes. Purpose of Portfolios: In this new era of performance assessment related to the monitoring of students' mastery of a core curriculum, portfolios can enhance the assessment process by revealing a range of skills and understandings one students' parts; support instructional goals, reflect change and growth over a period of time; encourage student, teacher, and
parent reflection; and provide for continuity in education from one year to the next. Instructors can use them for a variety of specific purposes, including:
Encouraging self-directed learning. Enlarging the view of what is learned. Fostering learning about learning. To promote student control of learning To track student progress To demonstrate individual growth To respond to individual needs To evaluate and report on student progress To facilitate student-led conferences To show process and product To show final products To show student achievement with respect to specific curricular
goals To document achievement for alternative credit To accumulate "best work" for admission to other educational
institutions or program Demonstrating progress toward identified outcomes. Creating an intersection for instruction and assessment. , Providing a way for students to value themselves as learners. Offering opportunities for peer-supported growth.
Benefits of Portfolio:
One of the most important benefits of using portfolios is the
enhancement of critical thinking%, skills which result from the need for students tot Develop evaluation criteria Students are pleased to observe their personal growth, They have better attitudes toward their work, and They are more likely to think of themselves as writers.
Factors that go into the development of a student portfolio assessment:
1.
First, you must decide the purpose of your portfolio. For example, the portfolios might be used to show student growth, to identify
2.
3.
weak spots in student work, and/or to evaluate your own teaching methods. After deciding the purpose of the portfolio, you will need to determine how you are going to grade it. In (titer words, what would a student need in their portfolio for it to be considered success and for them to earn a passing grade. The answer to the previous two questions helps form the answer to the third: What should be included in the portfolio? Are you going to have students put of all their work or only certain assignments? Who gets to choose?
How to Build a Student Portfolio The following suggestions will help you effectively design a student portfolio. 1.
Set a Purpose for the Portfolio. First, we need to decide what your purpose of the portfolio is. Is it going to be used to show student growth or identify specific skills? Are we looking for a concrete way to quickly show parents student achievement, or are we looking for a way to evaluate your own teaching methods? Once we have figured out your goal of the portfolio, then we think about how to use it.
2.
Decide How ' You Will You Grade it. Next, we will need to establish how we are going to grade the portfolio. There are several ways you can grade students work, we can use a rubric, letter grade, or the most efficient way would be to use a rating scale. Is the work completed correctly and completely? Can we comprehend it? we can use the grading scale of 4-1. 4 = Meets all Expectations, 3 = Meets Most Expectations, 2 = Meets Some Expectation, 1 = Meets No Expectations. Determine what skills you will be evaluating then use the rating scale to establish a grade.
3.
What will b Included in it. How will we determine what will go into the portfolio? Assessment portfolios usually include specific pieces that students are required to know. For example,
work that correlates with the Common Core Learning Standards. Working portfolios include whatever the student is currently working on, and display portfolios showcase only the best work students produce. Keep in, mind that we can create a portfolio for one unit and not the next. We get to choose what is included and how it is included. If you want to use it as a long-term project and include various pieces throughout the year, we can. But, we can also use it for short- term projects as well. 4.
How Much Will You Involve the Students. How much we involve the students in the portfolio depends upon the students age. It is important that all students should understand the, purpose of the portfolio and what is expected of them. Older students should be give n a checklist of what is expected, and how' it will be graded. Younger students may 1 of understand the grading scale so we can give them the option of what w 11 be include d in their portfolio. Ask them questions such as, why did you choose this particular piece and does it represent your best work? Involving students in the portfolio process will encourage them to reflect on their work.
5.
Will You Use a Digital Portfolio. With the fast-paced world of technology, paper portfolios may'become a thing of the past. Electric portfolios (e-portfolios/digital portfolios) are Teat because they are easily accessible, easy to transport and easy to use. Today’s students are tuned into the latest must-have technology, and electronic portfolios arc part of that. With students using an abundance of multimedia outlets, digital portfolios seem like a great fit. The uses of these portfolios are the same, students still reflect upon the r work but only in a digital way.
The key to designing a student portfolio is to take the time to think about what kind it will be, and how we well manage it. Once we do that and follow the steps;above, we will find it will be a success.
Types C F Portfolios Duo 1)
Best Work Portfolio
This type of portfolio highlights and shows evidence of the best work of learners. Frequently, this type of portfolio is called a display or showcase portfolio. For Students, best work is often associated with pride am a sense of accomplishment and can result in a desire to share their work with o hers. Best work can include both product and process. It is often correlated with the amount of effort that few learners have invested in their work. A major advantage of this type of portfolio is that learners (an select items that reflect their highest level of learning and canexplain why these it (ms represent their best effort and achievement. Best work portfolios are used for the following purposes: Student Achievement. Students may select a given number of entries (e.g., 10) that reflect their best effort or achievement (or both) in a course of study. The portfolio can be presented in a student-led parent conference or at a community open house. As students publicly share their excellent work, work they have chosen and reflected upon, the experience may enhance their self-esteem. Post-Secondary Admissions. The preparation of g.post-secondary portfolio targets work samples from high school that can be submitted forconsideration in the process of admission to college or university. This portfolio should show evidence of a range of knowledge, skills, and attitudes, and may highlight particular qualities relevant to specific programs. Many colleges and universities are adding portfolios to the initial admissions process while others are using them to determine particular placements once students are admitted. Employability. The audience for this portfolio is an employer, .This collection of work needs to be focused on specific knowledge, skills, and attitudes necessary for a particular job or career. The school-to-work movements in North America are influencing an increase in the use of employ-ability portfolios. The Conference Board of Canada (1092), for example, outlines the academic, personal management, and teamwork
skills that are the foundation of a high-quality Canadian workforce. An employability portfolio is an excellent vehicle for showcasing these skills. 2)
Growth Portfolio
A growth portfolio demonstrates an individual's development and growth over time. Development can be focused on academic or thinking skills, content knowledge, self-knowledge, or any area that is important in your setting. A focus on growth connects directly to identified educational goals and purposes. When growth is emphasized, a portfolio will contain evidence of struggle, failure, success, and change. The growth will likely be' an uneven journey of highs and lows, peaks and valleys, rather than a smooth continuum. What is significant is that learners recognize growth whenever it occurs and can discern the reasons behind that growth. The goal of a growth portfolio isfor learners to see their own changes over time and, in turn, share their journey with others. A growth portfolio ca -I be culled to extract a best work sample. It also helps learners see how achievement is often a result of their capacity to self-evaluate, set goals, and work over time. Growth portfolios car be used for the following purposes: Knowledge. This portfolio shows students' growth in knowledge in a particular content area or across several content areas over time. This kind of portfolio can contain samples of both satisfactory and unsatisfactory work, along with reflections to guide further learning. Skills and Attitudes. This portfolio shows students' growth in skills and attitudes in areas such as academic discipline s, social skills, thinking skills, and work habits. In this type of portfolio,challenges, difficult experiences, and other growth events can be included to demonstrate students' developing skills. In a thinking skills portfolio; for example, students might include evidence showing growth in their ability to recall, comprehend, apply, analyze, synthesize, and evaluate information Teamwork. This portfolio demonstrates growth in social skills in a variety of cooperative experiences. Peer responses and evaluations are
vital elements in this portfolio model, along with self-evaluations. Evidence of changing attitudes resulting from team experiences can also be included, especially s expressed in self-reflections and peer evaluations. Career. This portfolio helps students identify personal strengths related to potential career choices: The collection can be developed over several years, perhaps beginning in middle school and continuing throt4;hout high school. The process of selecting pieces over time empowers young people to make appropriate educational choices leading toward meaningful careers. Career portfolios mat items from outside the school setting that substantiate students' choices and create a holistic view of the students as learners and people. This type of portfolio may be modified for employment purposes. 3)
Showcase Portfolios
Showcase portfolios highlight the best products over a particular time period or course. For example, a showcase portfolio in a composition class may include the best examples of different writing genres, such an essay, a poem, a short story, a biographical piece, or a literary analysis. In a business class, the showcase portfolio may include a resume, sample business letters, a marketing project, and a collaborative assignment that demonstrates the individual's ability to work in a team. Students are often allowed to choose What they believe to be their best work, highlighting the it achievements and skills. Showcase reflections typically focus on the strengths of selected pieces and discuss how each met or exceeded required standards
4)
Process Portfolios
Process portfolios, by contrast, concentrate more on the journey, of learning rather than the final destination or end pro lusts of the learning process. In the composition class, for example, different stages of the process—an outline, first draft, peer and teacher responses, early revisions, and a final edited draft—may be required. A process reflection
may discuss why a particular strategy was used, what was useful or ineffective for the individual in the writing process, and how the student went about making progress in the face of difficulty in meeting requirements. A process reflection typically focuses on many aspects of the learning process, including the following: what approaches fiches work best, which are ineffective, information about oneself as a learner, and strategies or approaches to remember in future assignments. 5)
Evaluation Portfolios.
Evaluation portfolios may vary substantially in their content. Their basic purpose, however, remains to exhibit a series of evaluations over a course and the learning or accomplishments of the student in regard to previously determined criteria or goals. Essentially, this type of portfolio documents tests, observations, records, or other assessment artifacts required for successful completion of the course. A math evaluation portfolio may include tests, quizzes, and written explanations of how me went about solving a problem or determining which formula to use, whereas a science evaluation portfolio might also include laboratory experiments, science project outcomes with photo ; or other artifacts, and research reports, as well as tests and quizzes. Unlike the showcase portfolio, evaluation portfolios do not simply include the best work, but rather a selection of predetermined evaluations that may also demonstrate students' difficulties and unsuccessful struggles as well as their better world. Students who reflect on why some work was successful and other work was less so continue their learning as they develop their met cognitive skills.
6)
Online or e-portfolios
Online or e-portfolios may be one of the above portfolio types or a combination of different types, a general requirement being that all information and artifacts are somehow accessible online. A number of colleges require students to maintain a virtual portfolio that may include digital, video, or Wet -based products. The portfolio assessment process
may be linked to a specific course or an entire program. As with all portfolios, students are able to visually track and show their accomplishments to a wide audience, Conclusion: The portfolio process will continue to be refined and efforts made to improve students' perceptions if the process as it is intended to develop the self-assessments skills they will need to improve their knowledge and professional skills throughout their education careers. 6.3
GUIDELINE AND STUDENTS ROLE IN SELECTION OF PORTFOLIO ENTRIES AND SELF-EVALUATION:
Portfolio: An organized presentation of an individuals education, work samples, and skills. Terminologically a portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress, and achievements in one or more areas of the curriculum. Guidelines:
Identify purpose
Select objectives.
Think about the kinds of entries that will best match instructional outcomes.
Decide who select the entries
Decide how much to include, how to organize the portfolio, where to keep it and when to access.
Set the criteria for judging the work (rating scales, rubrics, checklists) and make such student understand the criteria.
Review the student’s progress.
Hold portfolio conferences with students to discuss their progress.
These guidelines are discussed below in detail. Identify Purpose: Without purpose, a portfolio is only a collection of student work samples. Different purposes result in different portfolios. For example, if the student is to be evaluated on the basic of the work in the portfolio for admission to college, then his final version of his best work would probably be included in the portfolio. Select Objectives: The objectives ot be met be students should be clearly stated a list of communicative functions can be included for students to check when the feel comfortable with them and stapled to the inside lover. Students would list the little or the number of the samples which address this function. Portfolios also can be organized according the selected objectives addressing one skill such as writing. The selected objectives will be directly related to the stated purpose for the portfolio. At any rate, teachers must ensure that classroom instruction support the identified seals. Decide how much to include & how to Organize: Teachers may want to spend some time going over the purpose of the portfolio at regular intervals with students to ensure that the selected pieces do address the purpose and the objectives. At regular times, ask students to go through their entries, to choose what should remain in the portfolio, and what could be replaced by another work which night be move illustrative of the objectives. Other material no longer current and/or not useful to document student progress toward attain bent of the objective should be discarded. What is the student’s role? The student’s role of participation in the portfolio will be largely responsible for the success of the portfolio. For this reason, students must
be actively involved in the choices of entries and in the rationale for selecting those entries. i.
Selecting:
The student’s first role is in selecting some of the items to be4 pair of the portfolio. Some teachers give students a checklist for making choices. Others leave students almost freedom in selecting their entries. At an rate student should include their best and favorite pieces of work along with those showing growth and process. ii.
Reflecting and self-assessing:
An essential component of self-assessment involves the student in reflecting about their own work. At the beginning students might not know what to saw so teaching will need to model the kinds of reflection expected from students. Set the Criteria for Judging the Work: There are two kinds of criteria needed at this point.
Criteria for individual entries (refers to the section on rubrics for details).
Criteria for the portfolio as a whole.
Assessing the individual entries in a portfolio is different from assessing the portfolio as a whole. If the purpose of the portfolio is to now student progress then if is highly probable that some of the beginning entries may not reflect high quality; however, over several months, the student now have demonstrated growth toward the stated objectives. Criteria can be established by teachers alone and/or by teachers and students together. At and rate, criteria for evaluating the portfolios must be announced a head of time. Possibilities of criteria include teacher evaluation and/or observation, student self-evaluation, peer assessment, and a combination of several teacher’s comments.
Following is a list of suggested criteria for a portfolio as a whole. Variety: Selected pieces display the range of tasks students can accomplish and skills they have learned. Growth: Student work represents the student’s growth in content knowledge and language proficiency. Completeness: Students organized the contents systematically. Organization: Students organized the contents systematically. Fluency: Selected pieces are meaningful to the students and communicate information to the teacher. Accuracy: Student work demonstrates skills in the mechanics of the language. Goal Oriented: The contents reflect progress and accomplishment of curricular objectives. Following Directions: Students followed the teacher’s directions for pieces of the portfolio. Neatness: Student work is neatly written, typed or illustrated. Justification or Significance: Student include reasonable justifications for the work selected or explain why selected items are significant. Reference Katozai, Murad Ali. Measurement & Evaluation. Peshawar. University Publisher, 2013 6.4
USING PORTFOLIOS IN INSTRUCTION AND COMMUNICATION:
Portfolio: Literally the word “Portfolio” is used in the following meanings: 1.
A portable large things and flat briefcase especially of leather used for carrying papers, pictures, drawings or maps.
2. 3. 4.
A list of the financial assests held by an individual or a bank or other financial institution. The role of the head of a government department e.g. “He holds the portfolio for foreign affairs”. An organized presentation of an individual’s education, work samples and skills.
Using portfolios of studne4t work in Instruction and communication: The term portfolio has become popular buzz word. Unfortunately, it is not always clear exactly what is meant or implied by the term especially when used in the context of portfolio assessment. This training module is intended to clarify the notion of portfolio assessment and help users design such assessments in a thoughtful manner. We begin with a discussion of the rationale for assessment alternatives and the discuss portfolio definitions characteristics and design considerations. Educators and critics are currently reciting a litany of problems concerning the use of multiple-choice and other structured format tests for assessing many important students outcomes. This has been accompanied by an explosion of activity searching for assessment alternatives. 1.
Capture a richer array of what students know and can do than is possible with multiple-choice tests. Current goals for students go beyond knowledge of facts and include such things as problems solving critical thinking, lifelong learning of new information and thinking independently. Goals also include dispositions such as persistence, flexibility, motivation and selfconfidence.
2.
Portray the process by which students produce work. It is important for example that students utilize efficient strategies for solving problems as well as getting the right answer. It is also important for students to be able to do such things as
monitoring their own learning so that they do when they perceive they are not understanding. 3.
Make our assessment align with what we consider important outcomes for students in order to communicate the right message to students and other about what we valve. For example if we emphasize higher order thinking in instruction but only test knowledge because testing thinking is difficult, students figure out pretty fast figure out pretty fast what is really valued.
4.
Have realistic contexts for the production of work, so that we can examine what students know and can do in real-life situations.
5.
Provide continuous and ongoing information on how students are doing in order to chronicle development, give effective feedback to students and encourage students to observe their own growth.
6.
Integrate assessment with instruction in a way consistent with both current theories of instruction and goals for students. Specifically we want to encourage active student engagement in learning, and student responsibility for the control of learning. We also want to develop assessment techniques that in their use, improve achievement and not just monitor it.
7.
Using portfolios of student work for assessment, already an instructional tool in many places, it seen as one potential way to accomplish these things. But using portfolios will only have these desired effects if we plan them carefully.
Important Points in Portfolio Developing Process: Some important points in portfolio development process are as follows: 1.
It should be consulted to teachers, students, parents and school administrations in deciding which items would be placed in it.
2.
It should be created a shared, clear purpose for using portfolios.
3.
It should reflect the actual day-to-day learning activities of students.
4.
It should be on-going so that they show students efforts, progress and achievements over a period of time.
5.
Items in portfolio should be collected as a systematic, purposeful and meaningful.
6.
It should give opportunities for students in selecting pieces they consider most reprehensive of themselves as learners to be placed into their portfolios, and to establish criteria for their selections.
7.
It should be viewed as a part of learning process rather than merely as recordkeeping tools, as a way to enhance students learning.
8.
Students can access their portfolios.
9.
Share the criteria that will be used to assess the work in the portfolio as well as in which the result are to be used. Teachers should give feedback to students, parents about the use the portfolio.
In conclusion, in portfolio making process some necessary steps are; assessment of studies should be clearly explained the process should over a certain time period, portfolio should encourage students to learn, and items in the portfolio should be multi-dimensional and should address different learning areas. Besides, it is vitally important that the studies in a portfolio should be designed in order to present students, performance and development in any time period in detail. Reference Katozai, Murad Ali. Measurement & Evaluation. Peshawar. University Publisher, 2013
6.5
POTENTIAL STRENGTH AND WEAKNESSES OF PORTFOLIOS:
Potential Strength of Portfolios (Or Advantages of Portfolios as Method of Assessment) Portfolio can present a wide perspective of learning process for students and enables a continuous feedback for them. Besides this, it enables students to have a self-assessment for their studies and learning, and to review their progress. Since it provides visual and dynamic proofs about students' interests, their skills, strong sides, successes and development in a certain time period, portfolio which is the systematic collection of the student's studies helps assessing students as a whole. Portfolio is strong devices that help students to gain the impbrtant abilities such as self-assessment, critical thinking and monitoring one's own learning. Furthermore, portfolio provide pre-service teacher assessing their own learning and growth, and help them become selfdirected and reflective practitioners, and contribute them the individual and professional developments. Mullin (1998) stresses that portfolio provides teachers to have new perspective in education. For instance, portfolio can answer these questions: what kind of troubles do students have? Which activities are more effective or ineffective? What subjects are understood and not understood? How efficient is the teaching process? Some advantages or strengths of Portfolios are given below: 1.
Portfolio provides multiple ways of assessing students' learning over time
2.
It provides for a more realistic evaluation of academic content than pencil-and paper tests.
3.
It allows students, parent, teacher and staff to evaluate the students' strength and weakness.
4.
It provides multiple opportunities for observation and assessment
5.
It provides an opportunity for students to demonstrate his/her
strengths as well as weakness. 6.
It encourages students to develop some abilities needed to become independent, self-directed learners
7.
It also helps parents see themselves as partners in the learning process.
8.
It allows students to express themselves in a comfortable way and to assess their own learning and growth as learners.
9.
It encourages students to think of creative ways to share what they are learning
10.
It increases support to students from their parents and enhances communication among teachers, students and parents.
11.
It encourage teachers to change their instructional practice and it is a powerful way to link curriculum and instruction with assessment
12.
It assesses and promotes critical thinking.
13.
It encourages students to become accountable and responsible for their own learning (i.e., self-directed, active, peer-supported, adult learning).
14.
It can be the focus of initiating a discussion between student and tutor.
15.
It facilitates reflection and self-assessment.
16.
It can accommodate diverse learning styles, though they are not suitable for all learning styles.
17.
Portfolios can monitor and assess students' progress overtime.
18.
Portfolios can assess performance, with practical application of theory, in real-time naturalistic settings (i.e., authentic assessment).
19.
Portfolios use multiple methods of assessment.
20.
Portfolios take into account the judgment of multiple assessors.
21.
Portfolios have high face validity, content validity, and construct validity.
22.
Portfolios integrate learning and assessment.
23.
Portfolios promote creativity and problem solving.
24.
Portfolios promote learning about learning (i.e., metacognition).
25.
Portfolios can be standardized and used in summative assessment.
26.
Portfolios combine subjective and objective, as well as qualitative and quantitative, assessment procedures.
27.
Portfolios can be used to assess attitudes and professional and personal development.
28.
Portfolios enable identification of the unsatisfactory or struggling performer.
29.
Portfolios offer teachers vital information for diagnosing students' strengths and weaknesses to help them improve their performance (i.e., formative assessment).
30.
Portfolios reflect students' progression outcomes (i.e., student profiling).
31.
Portfolios allow the evaluators to see, the student, group, or community as individual, each unique with its own characteristics, needs, and strengths.
32.
Portfolios serve as a cross-section lens, providing a basis for future analysis and planning. By viewing the total pattern of the community or of individual participants, one can identify areas of strengths and weaknesses, and barriers to success.
33.
Portfolios serve as a concrete vehicle for communication, providing ongoing communication or exchanges of information among those involved.
34.
Portfolios Promote a shift in ownership; communities and
toward
learning
participants can take an active role in examining where they have been and where they want to go. 35.
Portfolio assessment offers the possibility of addressing shortcomings of traditional assessment. It offers the possibility of assessing the more complex and important aspects of, an area or topic.
36.
Portfolios cover a broad scope of knowledge and information, from many different people who know the program or person in different contexts (e.g., participants, parents, teachers or staff, peers, or community leaders).
Potential Weaknesses of Portfolios (Or Disadvantages of Portfolios as Method of Assessment) 1.
When portfolios are used for summative assessment, students may be reluctant to reveal weaknesses.
2.
Portfolios are personal documents, and ethical issues of privacy and confidentiality may arise when they are used for assessment.
3.
Difficulties may arise in verifying whether the material submitted is the candidate's own work.
4.
Portfolios take a long time to complete and assess.
5.
The portfolio process involves a large amount of paperwork.
6.
Portfolio assessment may produce unacceptably low inter-rater reliability, especially if the assessment rubrics .are not properly prepared or are used by untrained assessors.
7.
May be seen as less reliable or fair than more quantitative evaluations such as test scores.
8.
Can be very time consuming for teachers or program staff to organize and evaluate the contents, especially if portfolios have to be done in addition to traditional testing and grading.
9.
Having to develop your own individualized criteria can be
difficult or unfamiliar at first. 10.
If goals and criteria are not clear, the portfolio can be just a miscellaneous collection of artifacts that don't show patterns of growth or achievement.
11.
Like any other form of qualitative data, data from portfolio assessments can be difficult to analyze or aggregate to show change.
Portfolio Assessment is Most useful for: 1.
Evaluating programs that have flexible or individualized goals or outcomes. For example, within a program with the general purpose of enhancing children's social skills, some individual children may need to become less aggressive while other shy children may need to become more assertive.
2.
Each child's portfolio assessment would be geared to his or her individual needs and goals.
3.
Allowing individuals and programs in the community (those being evaluated) to be involved in their own change and decisions to change.
4.
Providing information that gives meaningful insight into behaviour and related change. Because portfolio assessment emphasizes the process of change or growth, at multiple points in time, it may be easier to see patterns.
5.
Providing a tool that can ensure communication and accountability to a range of audiences. Participants, their families, funders, and members of the community at large who may not have much sophistication in interpreting statistical data can often appreciate more visual or experiential "evidence" of success.
6.
Allowing for the possibility of assessing some of the more complex and important aspects of many constructs (rather than just the ones that are easiest to measure).
Portfolio Assessment is not as useful for: 1.
Evaluating programs that have very concrete, uniform goals or purposes. For example, it would be unnecessary to compile a portfolio of individualized “evidence” in a program whose sole purpose is full immunization of all children in a community by the age of five years. The required immunizations are the same, and the evidence is generally clear and straightforward.
2.
Allowing you to rank participants or programs in a quantitative or standardized way (although evaluators or program staff may be able to make subjective judgments or relative merit).
3.
Comparing participants or programs to standardized norms. While portfolios can (and often do) include some standardized test scores along with other kinds of “evidence”, this is not the main purpose of the portfolio.
4.
May be seen as less reliable or fair than more quantitative evaluations such as test scores.
5.
Can be very time consuming for teachers or program staff to organize and evaluate the contents, especially if portfolios have to be done in addition to traditional testing and grading.
6.
Having to develop you own individualized criteria can be difficult or unfamiliar at first.
7.
If goals and criteria are not clear, the portfolio can be just a miscellaneous collection of artifacts that don’t show patterns of growth or achievement.
8.
Like any other form of qualitative data, data from portfolio assessments can be difficult to analyze or aggregate to show change.
6.6
EVALUATION OF PORTFOLIO:
According to Paulson, Paulson and Meyer, (1991, p. 63): “Portfolios offer a way of assessing student learning that is different than
traditional methods. Portfolio assessment provides the teacher and students an opportunity to observe students in a broader context: taking risks, developing creative solutions, and learning to make judgments about their own performances”. In order for thoughtful evaluation to take place, teachers .must have multiple scoring strategies to evaluate students' progress. Criteria for a finished portfolio might include several of the following:
Thoughtfulness (including evidence of students' monitoring of their own comprehension, metacognitive reflection, and productive habits of mind).
Growth and development in relationship to key curriculum expectancies and indicators.
Understanding and application of key processes.
Completeness, correctness, and appropriateness of products and processes presented in the portfolio.
Diversity of entries (e.g., use of multiple formats to demonstrate achievement of designated performance standards).
It is especially important for teachers and students to work together to prioritize those criteria that will be used as a basis for assessing and evaluating student progress, both formatively (i.e., throughout an instructional time period) and summatively (i.e., as part of a culminating project, a.ctivity, or related assessment to determine the extent to which identified curricular expectancies, indicators, and standards have been achieved). As the school year progresses, students and teacher can work together to identify especially significant or important artifacts and processes to be captured in the portfolio. Additionally, they can work • collaboratively to determine grades or scores to be assigned. Rubrics, rules, and scoring keys can be designed for a variety of portfolio components. In addition, letter grades might also be assigned, where
appropriate. Finally, some form of oral discussion or investigation should be included as part of the summative evaluation process. This component should involve the student, teacher, and if possible, a panel of reviewers in a thoughtful exploration of the portfolio components, students' decision-making and evaluation processes related to artifact selection, and other relevant issues.
UNIT-7: BASIC CONCEPTS OF INFERENTIAL STATISTS 7.1
CONCEPT & PURPOSE OF INFERENTIAL STATISTICS:
Introduction: The role and importance of statistics in education cannot be denied. In education we come across with measurement, evaluation and research. Similarly, we have to make educational policies and budgets. In all these fields we need to make proper measurement and present the data quantitatively. Thus without statistics we cannot make proper measurement. As quoted in different statistics books "Planning is the order of the day, and planning without statistics is inconceivable". Good statistics and sound statistical analysis assist in providing the basis for the design of educational policies, monitor policy implications and evaluate policy impact. To generate reliable and relevant information the data should be collected using appropriate statistical methods. The materials one uses for data collection should be well designed. The data analysis should also be done using appropriate statistical method. All these show that statistics plays vital role in Education Management and educational planning. Concept of Inferential Statistics Definition: The branch of statistics concerned with using sample data to make an inference about a larger group of data is called inferential statistics. Example: For instance the college teacher decides to use the average grade achieved by one statistics class to estimate the average grade of all the
sections of the same statistics course. This is a problem of estimation, which falls in the inferential statistics. In educational research, it is never possible to sample the entire population that we want to draw a conclusion about. For example, we might want to determine how well a new way of teaching mathematics can affect mathematical achievement for all children in Primary 1. However, it would be impossible to test all children in Primary 1 because of time, resources, and other logistical factors. Instead, we choose a sample of the population to conduct a study. Then we want to make conclusions - or inferences, about the entire population based on the results of the study from the sample. Quantitative research in education and social science aims to test theories about the nature of the world in general (or some part of it) based on samples („;?) of "subjects" taken from the world (or some part of it). When we perform research on the effect of TV violence on children's aggression, our intention is to create theories that apply to all children who watch TV, or perhaps to all children in cultures similar to our own who watch TV. We of course cannot study all children, but we can perform research on samples of children that, hopefully, will generalize back to the populations from which the samples were taken. Recall that external validity is the ability of a sample to generalize to the population. Purpose of Inferential Statistics The main purpose of inferential statistics is .to determine whether the findings from the sample can generalize to the entire population. There will always be differences between groups in a research study. Inferential statistics can determine whether the difference between the two groups in the sample is large enough to be able to say that the findings are significant. If the findings are indeed significant, then the conclusions can be applied - generalized - to the entire population. On the other hand, if the difference between the groups is very small, then the findings are not significant and therefore were simply the result of chance.
To illustrate this practically, imagine an entire room full of socks. You want to determine whether there are more white socks than green socks in the room. However, there are too many socks in the room to count them all, so you want to take a sample of socks. Based on this sample of socks, you will draw a conclusion about whether there are more white socks than green socks. After you collect your sample, then you will need to calculate inferential statistics is to determine whether the colours chosen in your sample likely reflect the colours of socks in the entire room or if your results were due to chance. What factors will determine whether the colours in the sample of socks adequately represents the colours of the entire room? Sample size. If you only pick two socks, they would probably not represent the entire room. The larger the sample is, the more representative the sample will be of the entire room and the more likely the inferential statistics will find a significant result. This is why when conducting experiments, the larger the sample is, the better: with large samples, the results will more likely reflect the entire population. Inferential statistics is the mathematics and logic of how this generalization from sample to population can be made. The fundamental question is: can we infer the population's characteristics from the sample's characteristics? Descriptive statistics remains local to the sample, describing its central tendency and variability, while inferential statistics focuses on making statements about the population. Unlike descriptive statistics, inferential statistics provide ways of testing the reliability of the findings of a study and "inferring" characteristics from a small group of participants or people (your sample) onto much larger groups of people (the population). Descriptive statistics just describe the data, but inferential let you say what the data mean. 7.2
SAMPLING ERROR:
In statistics, sampling error is incurred when the statistical characteristics of a population are estimated from a subset, or sample, of
that population. Since the sample does not include all members of the population, statistics on the sample, such as mean and quantities, generally differ from parameters on the entire population. For example: If one measures the height of a thousand individuals from a country of one million, the average height of the thousand is typically not the same as the average height of all one million people in the country. Since sampling is typically done to determine the characteristics of a whole population, the difference between the sample and population values is considered a sampling error. Population and Samples: A population is the entire group to which we want to generalize our results. A sample if a subset of the population might be all adult humans but our sample might be a group of 30 friends and relatives. Types of sampling errors: 1. 2. 3. 1.
2.
Random sampling Bias problems Non-sampling error Random Sampling: In statistics, sampling error is the error caused by observing a sampling instead of the whole population. The sampling error can be found by subtracting the value of a parameter from the value of a statistic. In nursing research, a sample error is the difference between sample statistics used to estimate a population parameter and the actual but unknown value of the parameter. (Bunns and Grove, 2009) Parameters and statistics: A numerical summary of a population is called a parameter, while the same numerical summary of a sample is called a statistic. Bias Problems: Sampling bias is a possible source of sampling errors. It leads to the sampling error which either have a prevalence to be positive
3.
4. 1. 2. 3. 4. 5. 7.3
or negative. Such errors can be considered to be systematic errors. Non-sampling Error: Sampling error can be constrasted with non-sampling error. Non-sampling error is a catch all term for the deviations from the true value that are not a function of the sample chosen, including various systematic errors and any random errors that are not due o sampling. Non-sampling errors are much harder to quantify than sampling error. Example of non-sampling error: Answers given by respondents may be influenced by the desire to impress an interviewer. Characteristics: Sampling Error: Generally decreased as the sample size increases (but not proportionally) Depends on the size of the population under study. Depends on the variability of the characteristic of interest in the population. Can be accounted for and reduced by an appropriate sampling plan. Can be measured an controlled in probability sample surveys. NULL HYPOTHESIS:
Before defining the term null-hypothesis, it is necessary that we must know about Hypothesis and statistical hypothesis. Hypothesis: A hypothesis is any statement or assumption about any phenomena of nature. Statistical Hypothesis: A statistical hypothesis is a statement or assumption about the value of a population parameter. For example; 1 = 80
(The population mean is equal to 80)
> 22
(The population mean is greater than 22)
2 # 25
(The population variance is not equal to 25)
1 = 2
(Population mean 1 is equal to population
mean 2) 1 - 2 = 0
(there is no difference between 1 and 2)
Null Hypothesis: The hypothesis to be tested in a test of hypothesis is called null hypothesis. It is a hypothesis which is tested for possible rejection or mollification under the assumption that it is true. It is denoted by H 0 and usually contains and equal sign. For example if we want to test that the population mean is 80, then we write. H0 : = 80 Another definition of ‘Null-Hypothesis’: Null hypothesis is a type of hypothesis used in statistics that proposes that no statistical significance exists in a set of given observations. The null hypothesis attempts to show that no variation exists between variables, or that single variable is no different than ero. It is presumed to be true until statistical evidence nullifies it for an alternative hypothesis. Examples: Hypothesis: The loss of my socks is due to alien burglary. (Alien burglary means unfamiliar theft). Null Hypothesis: The loss of my socks is nothing to do with alien burglary. Alternative Hypothesis:
The loss of my socks is due to alien burglary. In statistics, the only way of supporting your hypothesis is to refute the null hypothesis. Rather than trying to brave your idea (the alternative hypothesis) right you must show that the null hypothesis is likely to be wrong. You have to ‘refute’ or ‘nullify’ the null hypothesis. 7.4
TESTS OF SIGNIFICANCE:
Once sample data has been gathered through an observational study or experiment, statistical inference allows analysts to assess evidence in favor or some claim about the population from which the sample has been drawn. The methods of inference used to support or reject claims based on sample data are known as tests of significance. Every test of significance begins with a null hypothesis HO. HO represents a theory that has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved. For example, in a clinical trial of a new drug, the null hypothesis might be that the new drug is no better, on average, than the current drug. We would write HO: there is no difference between the two drugs on average. The alternative hypothesis, Ha, is a statement of what a statistical hypothesis test is set up to establish. For example, in a clinical trial of a new drug, the alternative hypothesis might be that the new drug has a different effect, on average, compared to that of the current drug. We would write Ha: the two drugs have different effects, on average. The alternative hypothesis might also be that the new drug is better, on average, than the current drug. In this case we would write Ha: the new drug is better than the current drug, on average. The final conclusion once the test has been carried out is always given in terms of the null hypothesis. We either "reject HO in favor of Ha" or "do not reject HO"; we never conclude "reject Ha", or even "accept Ha".
If we conclude "do not reject HO", this does not necessarily mean that the null hypothesis is true, it only suggests that there is not sufficient evidence against HO in favor of Ha; rejecting the null hypothesis then, suggests that the alternative hypothesis may be true. Example Suppose a test has been given to all high school students in a certain state. The mean test score for the entire state is 70, with standard deviation equal to 10. Members of the school board suspect that female students have a higher mean score on the test than male students, because the mean score from a random sample of 64 female students is equal to 73. Does this provide strong evidence that the overall mean for female students is higher? The null hypothesis HO claims that there is no difference between the mean score for female students and the mean for the entire population, so that = 70. The alternative hypothesis claims that the mean for female students is higher than the entire student population mean, so that > 70.s Steps in Testing for Statistical Significance 1. 2. 3. 4. 5. 1)
State the Research Hypothesis State the Null Hypothesis Select a probability of error level (alpha level) Select and compute the test for statistical significance Interpret the results State the Research Hypothesis
A research hypothesis states the expected relationship between two variables. It may be stated in general terms, or it may include dimensions of direction and magnitude. For example, General: The length of the job training program is related to the rate of job placement of trainees. Direction: The longer the training program, the higher the rate of job placement of trainees.
Magnitude: Longer training programs will place twice as many trainees into jobs as shorter programs. General: Graduate Assistant pay is influenced by gender. Direction: Male graduate assistants are paid more than female graduate assistants. Magnitude: Female graduate assistants are paid less than 75% of what male graduate assistants are paid. 2)
State the Null Hypothesis
A null hypothesis usually states that there is no relationship between the two variables. For example, There is no relationship between the length of the job training program and the rate of job placement of trainees. Graduate assistant pay is not influenced by gender. A null hypothesis may also state that the relationship proposed in the research hypothesis is not true. For example, Longer training programs will place the same number or fewer trainees into jobs as shorter programs. Female graduate assistants are paid at least 75% or more of what male graduate assistants are paid. Researchers use a null hypothesis in research because it is easier to disprove a null hypothesis than it is to prove a research hypothesis. The null hypothesis is the researcher's "straw man." That is, it is easier to show that something is false once than to show that something is always true. It is easier to find disconfirming evidence against the null hypothesis than to find confirming evidence for the research hypothesis. (Definitions taken from Valerie J. Easton and John H. McColl's Statistics Glossary v1.1)
One Tailed and Two Tailed Significant Tests One important concept in significant testing is whether you use a one tailed or two tailed test of significance. The answer is that it depends on your hypothesis. When your research hypothesis states the direction of the difference or relationship, then you use a one tailed probability. For example, a one tailed test would be used to test these null hypothesis: Females will not score significantly higher than males on an IQ test. Superman is not significantly stronger than the average person. The one tailed probability is exactly half the value of two tailed probability. 7.5
LEVELS OF SIGNIFICANCE:
In hypothesis testing, the significance level is the criterion used for rejecting the null hypothesis. The significance level is used in hypothesis testing as follows. First, the difference between the results of the experiment and the null hypothesis is determined. Then alluring the null hypothesis is true, the probability of a difference that large or larger is computed. Finally, the probability is compared to the significance level. If the probability is less than ON equal to the significance level, then the null hypothesis is rejected & the outcome is said to be statistically significant. Traditionally experiments have used to be statistically significant. Traditionally, experiments have used either the 0.05 level (sometime called 5% level) on the 0.01 level (1% level), although the choice of levels is largely subjective. The lower the significance level, the more the data must diverge from the null the 0.01 level is more conservative than the 0.05 level. Symbols: The Greek word alpha () is sometime used to indicate the significance level. The above explanation shows that the significance level is a value associated to some statistical value, tests, which indicates
the probability of obtaining those on more extreme results. This value can be interpreted as the probability of obtain those results. If the null hypothesis were (true) when (sampling is random) on as the probability of obtaining those results by chance alone. (When sampling is less than random). The value of this probability (also known as “p”, “p” – value, alpha & Type I error) runs between 0& 1. The closer to “0” the lower the probability of the results being found if the null hypothesis were true, on the lower the probability of the result being a chance result. As stated in beginning, significance levels are used to reject the null hypothesis that, for example, there is no correlation between variables” there is no difference between groups on there is no change between treatments”. A significant level of 0.051 is conventionally used in the social sciences, although probity as high as “0.10” also be used. Probability greater than 0.10 are rarely used. A significance level of 0.05 for example indicates that there is a 5% probability that results are due to chance. A significance level of 0.10 indicates a 10% probability that the results are due to chance. Thus, using significance levels above 0.10 is rather risky: while using lower significance level is “safer”. History: The present day concept of statistical significance originated by Ronald Fisher when he developed statistical hypothesis testing which he described as test of significance in his 1925 publication. Fisher suggested a probability of one-in-twenty (0.05) as a convenient cut off level of rejection null hypothesis. Role in Statistics: Statistical significance play a pivotal role in statistical hypothesis testing where it is used to determine it a null hypothesis can be rejecting on retained. A null hypothesis is the greater on general default statement that nothing happened on changed. For a null hypothesis to be rejected on false, the result has to be identified as being statistical significant. i.e. unlikely to have occurred by chance alone.
To determine a result is statistically significant a researcher would have to calculate a p-value which is the probability of observing an effect given that the null hypothesis is true. References www.en.wikipedia.org/wiki/statistical_significance. M.A. Kotazoi, Measurement & Evaluation: 2013. 7.6
TYPE-I AND TYPE-II ERRORS: REMAINING:
Statistical Errors Even in the best research project, there is always a possibility that the researcher will make a mistake regarding the relationship between the two variables. This mistake is called statistical error. In statistical test theory the notion of statistical error is an integral part of hypothesis testing. The test requires an unambiguous statement of a null hypothesis, which usually corresponds to a default "state of nature", for example "this person is healthy", "this accused is not guilty" or "this product is not broken". An alternative hypothesis is the negation of null hypothesis, for example, "this person is not healthy", "this accused is guilty" or "this product is broken". The result of the test may be negative, relative to null hypothesis (not healthy, guilty, broken) or positive (healthy, not guilty, not broken). If the result of the test corresponds with reality, then a correct decision has been made. However, if the result of the test does not correspond with reality, then an error has occurred. Due to the statistical nature of a test, the result is never, except in very rare cases, free of error. Two types of error are distinguished: type I error and type II error. In statistics, a type I error (or error of the first kind) is the incorrect rejection of a true null hypothesis. A type 11 error (or error of the second kind) is the failure to reject a -false. null hypothesis. A type I error is a false positive. Usually a type I error leads one to conclude that a thing or relationship exists when really it doesn't, for example, that a
patient has a disease being tested for when really the patient does not have the disease, or that a medical treatment cures a disease when really it doesn't. A type II error is a false negative. Examples of type II errors would be a blood test failing to detect the disease it was designed to detect, in a patient who really has the disease; or a clinical trial of a medical treatment failing to show that the treatment works when really it does. When comparing two means, concluding the means were different when in reality they were not different would be a Type I error; concluding the means were not different when in reality they were different would be a 'Type II error. All statistical hypothesis tests have a probability of making type I and type II errors. For example, all blood tests for a disease will falsely detect the disease in some proportion of people who don't have it, and will fail to detect the disease in some proportion of people who do have it. A test's probability of making a type I error is denoted by a. A test's probability of making a type II error is denoted by β. The detail is given below: Type-I Error: The first is called a Type I error. This occurs when the researcher assumes that a relationship exists when in fact the evidence is that it does not. In a Type 1 error, the researcher should accept the null hypothesis and reject the research hypothesis, but the opposite occurs. The probability of committing a Type I error is called alpha (a). A type I error, also known as an error of the first kind, occurs when the null hypothesis (H0) is true, but is rejected. It is asserting something that is absent, a false hit. A type I error may be compared with a so-called false positive (a result that indicates that a given condition is present when it actually is not present) in tests where a single condition is tested for. Type I errors are philosophically a focus of skepticism and Occam's razor. A Type I error occurs when we believe a falsehood. In terms of folk tales, an investigator may be "crying wolf' without a wolf in sight (raising a false alarm) (Ho: no wolf).
The rate of the type I error is called the size of the test and denoted by the Greek letter a (alpha). -It usually equals the significance level of a test. In the case of a simple null hypothesis a is the probability of a type I error. If the null hypothesis is composite, a is the maximum (supremum) of the possible probabilities of a type I error. Explanation: A Type I Error is also known as a False Positive or Alpha Error. This happens when you reject the Null Hypothesis even if it is true. The Null Hypothesis is simply a statement that is the opposite of your hypothesis. For example, you think that boys are better in arithmetic than girls. Your null hypothesis would be: "Boys are not better than girls in arithmetic." You will make a Type I Error if you conclude that boys are better than girls in arithmetic when in reality, there is no difference in how boys and girls perform. In this case, you should accept the null hypothesis since there is no real difference between the two groups when it comes to arithmetic ability. If you reject the null hypothesis and say that one group is better, then you are making a Type I Error. Type-II Error The second is called a Type II error. This occurs when the researcher assumes that a relationship does not exist when in fact the evidence is that it does. In a Type II error, the researcher should reject the null hypothesis and accept the research hypothesis, but the opposite occurs. The probability of committing a Type II error is called beta. Generally, reducing the possibility of committing a Type I error increases the possibility of committing a Type II error and vice versa, reducing the possibility of committing a Type II error increases the possibility of committing a Type I error. Researchers generally try to minimize Type I errors, because when a researcher assumes a relationship exists when one really does not, things may be worse off than before. In Type II errors, the researcher
misses an opportunity to confirm that a relationship exists, but is no worse off than before. Type II Error is a statistical term used within the context of hypothesis testing that describes the error that occurs when one accepts a null hypothesis that is actually false. The error rejects the alternative hypothesis, even though it does not occur due to chance. A type II error accepts the null hypothesis, although the alternative hypothesis is the true state of nature. It confirms an idea that should have been rejected, claiming that two observances are the same, even though they are different. Example: An example of a type II error would be a pregnancy test that gives a negative result, even though the woman is in fact pregnant. In this example, the null hypothesis would be that the woman is not pregnant, and the alternative hypothesis is that she is pregnant. In other words, a type DI error, also known as an error of the second kind, occurs when the null hypothesis is false, but erroneously fails to be rejected. It is failing to assert what is present, a miss. A type II error may be compared with a so-called false negative (where an actual 'hit' was disregarded by the test and seen as a 'miss') in a test checking for a single condition with a definitive result of true or false. A Type II error is committed when we fail to believe a truth. In terms of folk tales, an investigator may fail to see the wolf ("failing to raise an alarm"). Again, Ho: no wolf. The rate of the type II error is denoted by the Greek letter f3 (beta) and related to the power of a test (which equals 143). What we actually call type I or type H error depends directly on the null hypothesis. Negation of the null hypothesis causes type I and type II errors to switch roles. The goal of the test is to determine if the null hypothesis can be rejected. A statistical test can either reject (prove false) or fail to reject
(fail to prove false) a null hypothesis, but never prove it true (i.e., failing to reject a null hypothesis does not prove it true). Explanation: A Type II Error is also known as a False Negative or Beta Error. This happens when you accept the Null Hypothesis when you should in fact reject it. The Null Hypothesis is simply a statement that is the opposite of your hypothesis. For example, you think that dog owners are friendlier than cat owners. Your null hypothesis would be: "Dog owners are as friendly as cat owners." You will make a Type II Error if dog owners are actually friendlier than cat owners, and yet you conclude that both kinds of pet owners have the same level of friendliness. In this case, you should reject the null hypothesis since there is a real difference in friendliness between the two groups. If you accept the null hypothesis and say that both types of pet owners are equally friendly, then you are making a Type II Error. 7.7
DEGREES OF FREEDOM:
In statistics, the numl er of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary. The number of independent ways by which a dynamic system Can move without violating any constraint imposed of it, is called degree of freedom. In other words, the degree of freedom can be defined as the min mum number of independent coordinates that can specify the position of the system completely: Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the estimate of a parameter is called the degrees of 7eedom. In general, the degrees of freedom of an estimate of a parameter is equal to the number o 'independent scores that go into the estimate minus the number of parameters used as intermediate steps in the estimation of the parameter itself (which, in sample variance, is one, since the sample mean is the only intermediate step).
In many statistical problems we are required to determine the degrees of freedom. This refers to a positive whole number that indicates the lack of restrictions in, our calculations. The degree of freedom is the number of values in a calculation that we can vary. One step in most statistical inference problems is to determine the number of degrees of freedom. The number of degree of freedom in a problem is related to the,precise probability distribution that is to be used in the inference procedure. This step is an often overlooked but crucial detail in both the calculation of confidence intervals and the workings of hypothesis tests.
There is not a single general formula for the number of degrees Of freedom for every inferenceproblem. Instead there are specific formulas to be used for each type of procedure in inferentialstatistics. In other worlds, the setting that we are working in will determine how we calculate thenumber of degrees of freedom. Determining Degree of Freedom:
Number of components that are free to vary about a parameter Df = Sample size – Number of parameters estimated Df is n-1 for one sample test of mean
A Few Examples For a moment suppose that we know the mean of data is 25 and that the values are 20,10, 50, and one unknown value. To find the mean of a list of data, we add all of the data and divide by the total number of values. This gives us the formula (20 + 10 + 50 + x)/4 = 25, where x denotes the unknown. Despite c ling this unknown, we can use some algebra to determine that x = 20.
Let's alter this scenario slightly. Instead we suppose that we know the mean of a data set is 25, with values 20, 10; and two unknown values. These unknowns Could be different, so we use two different variables, A and y to denote this. The resulting formula is (20 + 10 + x +y)/4 = 25. With some algebra we obtain y = 70 - x. The formula is written in this form to show that once we choose a value for x, the value fory is determined. This shows 'that there is one degree of freedom. Now we'll look at a t ample size of one hundred. If we know that the mean of this sample data is 20, but do not know he values of any of the data, then there are 99 degrees of freedom. All values must add up t ) a total of 20 x 100 = 2000. Once we have the values of 99 elements in the data set, then the last one has been determined. Example To compute the variance I first sum the square deviations from the mean. The mean is a parameter: it is a characteristic of the variable under examination as a whole and is part of describing the overall distribution of values. If you know all the, parameters you can accurately describe the data. The more parameters you know, that is to saythe more you fix, the fewer samples fit this mode of the data. If you know only the mean, there will be many possible sets of data that are consistent with this model but if you know the mean and the standard deviation, fewer possible sets of data fit this model. So in computing the Variance I had first to calculate the mean. When I have calculated the mean, I could vary any of the scores in the data except for one. If I leave one score unexamined it can always be calculated accurately from the rest of the data and the mean itself. Maybe an example can make this clearer. I take the ages of a class of students and find the mean. If I fix the mean, how many of the other scores (there are N of them remember) could still vary? The answer is N-1. There are N-1 independent pieces of information that could vary while the mean is known. These are the degrees of freedom. One piece of information cannot vary because its
value is fully determined by the parameter (in t its case the mean) and the other scores. Each parameter that is fixed during our computations constitutes the loss of a degree of freedom. If we imagine starting with a small number of data points and then fixing a relatively large number of parameter: as we compute some statistic, we see that as more degrees of freedom are lost, fewer and fewer different situations are accounted for by our model since fewer and fewer pieces of information could in principle be different from what is actually observed. So, the interest, to put it very informally, in our data is determined by the degrees of freedom: if there is nothing that can vary once our parameter is fixed (because we have so very few data points maybe just or e) then there is nothing to investigate. Degrees of freedom can be seen as linking sample size to explanatory power. The Standard Deviation is a measure of how spread out numbers are; Its symbol is a (the greek letter sigma) The formula is easy: It is the square root of the Variance. To calculate the variance follow these steps: Work out the Mean (the simple average of the numbers) Then for each number: subtract the Mean and square the resIult (the squared difference). Then work out the average of those squared differences. Let suppose we have five values i.e 600,470,170,430 & 300
Mean = = 394
600+ 470+170+ 430+300 5
=
1970 5
2 Variance: σ =n
−94 ¿ ¿ ¿2 2062 + 762 +(−224 )2+36 2+ ¿ ¿
=
42,436+5,776+50,176+1,296+ 8,836 5
=
108,520 5
2 Variance σ =
= 21,704
−94 ¿ ¿ ¿2 2062 + 762 +(−224 )2+36 2+¿ ¿
=
42,436+5,776+50,176+1,296+ 8,836 5
=
108,520 5
= 21,704
UNIT-8: SELECTED TESTS OF SIGNIFICANCE 8.1
T-TEST:
Definition: i)
ii)
iii)
A t-test helps you compare weather two groups have different average values (For example, weather men and women have different average heights). A t-test asks weather a different between two groups averages unlikely to have occurred because of random chance in sample selection. A difference is more likely to be meaningful and “real” if (a) the difference between, the average is large, (b) the sample size is large, and (c) Responses are consistently close to the average values and not widely spread out (the standard deviation is low). A statistical examination of two population means. A twosample. T-test examines weather two samples are different and is commonly used when the variances of two normal distribution are unknown and when an experiment uses a small sample size. For example, a t-test could be used to compare the average floor routine score of the U.S women’s Olympic gymnastic team to the average floor routine score of China’s women’s team.
The t-test’s statistical significance and the t-test’s effect size are the two primary outputs of the t-test. Statistical significance indicates weather the difference between sample averages is likely to represent an actual difference between population and the effect size indicates wither that difference is large enough to be practically meaningful. The “One sample t-test” is similar to the “independent samples t-test” except it is used to compare one group’s average value to a single number .x. for practical purposes you can look at the confidence interval around the average value to gain this same information.
The “paired t-test” is used when each observation in one group is paired with a related observation in the other group. For example do Kansans spend money on movies in January to February. Where each respondent is asked about their January from their February spending? In fact a period t-test subtracts each respondent’s January spending from their February spending (yielding the increase is spending), then take the average of all those increases in spending and looks to see wither that average is statistically significantly greater than Zero (using a one sample t-test). The “ranked independent t-test” ask a similar question to the typical unranked test but it is more robust to outliners (a few bad outliners can make the results of an unranked t-test invalid). T-test (Independent Samples) Dollars spend on movies per month. Stat-wing represents t-test results as distribution curves. Assuming there is a large enough sample size, the difference between these samples probably represents a “real’s” difference between population from they were sampled. Example: Let’s say you are curious about wether New Yorkers and Kansans spend a different amount of money per month on movies. It is impractical to ask every New Yorker and Kansans about their movie spending, so instead you ask a sample of each – may be 300 New Yorkers and 300 Kansans – and the average are 14 Dollars and 18 Dollars. The ttest asks wether that difference is probably representative of a real difference between Kansans and New Yorkers generally or whether that is most likely a meaningless statistical fluke. Technically, it asks the following. If there were in fact no difference between Kansans and New Yorkers generally, what are chances that randomly selected groups from those populations would be as different as these randomly selected groups are?
For example if Kansans and New Yorks as a whole actually spent the same amount of money on average. It is very unlikely that 300 randomly selected Kansans each spend exactly 14 Dollars and 300 randomly selected New Yorkers each spend. 18 Dollars exactly. So if you are sampling yielded those results, you would conclude that the difference in the sample groups is most likely representative of a meaningful difference between the populations as a whole. Statistical Analysis of the T-test: The formula for the t-test is a ratio. The top part of the ratio is just the difference between the two means or averages. The bottom part is a measure of the variability or dispersing of the scores. This formula is essentially another example of the signal-to-noise metaphor in research the difference between the means is the signal that in this case, we think our program or treatment introduced into the data, the bottom part of the formula is a measure of variability that is essentially noise that may make it harder to see the group difference. Signal noise: The top part of the formula is easy to compute----- Just find the difference between the means. The bottom part is called the standard error of the difference. To compute it, we take the variance for each group and divide it by the number of people in that group. We add these two values and then their square root. The specific formula is given in Figure. SE
( X´ T − X´ c )=
√
var T var c + nc nc
Remember that the variances is simply the square of the standard deviation. The final formula for the T-test is shown in the given figure as under.
T=
´ T − X´ C X
√
nar t nar c + nT nC
Formula for T-test.
References O’Mahony, Michael (1986). Sensory Evaluation of Food: Statistical Methods and procedures. William H.; Saul A. Teukolsky. William T. Vetterling Br Ain P. Flannery (1992). Numerical Recipes in C: The Art of \Scientific Computing. Internet Google, pre Encyclopedia. 8.2
CHI-SQUARE (X2):
The X2-distribution (X is the Greek letter Chi, pronounced Ki) was first obtained in 1875 by H.R Helmert a German physicist. Later in 1900, Karl Pearson showed that as n-increasing to infinity a discrete multinomial distribution may be transformed and made to approach a chi-square distribution. This approximation has broad application such as a test of goodness of fit, as a test of independence and a test of homogeneity. The chi-square distribution contains only one parameter, called the number of degree of freedom. Chi-Square Distribution: Let Z1, Z2 ----- Zn be normally and independently distributed variables with Zero mean and unit vassance (0, 1). Then the random variable expressed by the quantity. X2 = In otherworld’s it can be defined as “It is the sum of squares of n-indep endant standardized random variables”.
Properties of Chi-Square Distribution: Chi-square distribution has the following properties. 1.
The chi-square distribution is continuous ranging from Zero to infinity.
2.
Total area under the curve is unity.
3.
The mean of X2 distribution is equal to the number of degree of freedom i.e. n.
4.
The variance of f2 distribution is equal to twice the degree of freedom i.e. 2n.
5.
The carve of chi-square distribution is positively skewed.
6.
The X2 distribution tends to normal distribution an the number of degrees of freedom approaches to infinity.
7.
Moment generating function of x2 distribution is (1-2+)-n/2
8.
X2 distribution is leptokurtic as 2> 3.
Uses of X2 Distribution: 1. 2. 3. 4. 5.
6.
X2 is used to test the goodness of fit. X2 is used to test the independence of attributes. X2 is used to test the validity of a hypothetical ratios. X2 is used to test the homogeneity of soosal X2 variances. X2 is used to test whether the hypothical value S2 of population variances hypothical value S2 of population variances is true on not. X2 is used to test the equality of several population correlation co-efficient.
Goodness of Fit Test: This test is based on the property that cell probabilities depend upon unknown parameters, provided that the unknown parameters are replaced with their estimates and provided that and one degree of freedom is deducted for each parapets estimated”. To see whether there is evidence of small or large differences, the test statistic to use is;
2 /ei
oi−ei ¿ ¿ u (¿−npi)2 =¿ ∑ ¿ npi i=1 K
x2∑ ¿ i=1
With k-1-number of parameters estimated degrees of freedom. The symbol Oi and ei are represented observed and expected frequencies respectively. When the observed values are equal to the expected values, the X2 = 0. The larger the difference between the observed and expected frequencies, the larger will be the X 2 value. A small value of X2 indicates that the fit is good and leads to accept H 0. A large value of X2 indicates that the fit is poor and leads to accept H1. Contingency Table: A table consists two & more rows and two or more columns, into which n-observations are classified according to two different criteria (or variables) is commonly called, a contingency table. The simplest form of a contingency table is 2×2 table which is obtained when both criteria are dichotomized. The totals of the frequencies in each of the rows and columns are called the marginal total a frequencies. Contingency tables provides a useful method of comparing two variables. A 2 × 2 contingency table are as under. Classes
B1
B2
To
A1
O11
O12
(A
A2
O21
O22
(A
Total
(B1)
(B2)
N
A contingency table may be extended to higher dimension. i.e. r × c contingence table, where r represents number of rows and c represents number of columns. Testing Hypothesis of Independence in Contingency Table: The data presented in a contingency table can be used to test the hypothesis that the two variables of classification are independent. It this hypothesis is rejected, the two variables of classification are not independent and we say that there is some also citation (or interaction) between the two variables of classification. To do so, we must calculate the expected frequencies based on this hypothesis, keeping the marginal totals fixed. Let eij denote the expected frequency belonging to Ai and Bj. Assuming the hypothesis of independence is true, the proportion of members belonging to any class Ai should be the same and equal to the proportions in the total. Thus r
eij = ( Ai)
∑ eij i=1 4
∑ (Ai )
=
(Bj ) So that n
i=1
Eij =
( Ai ) (Bj) n
That is, under Ho: The classification are independent, the expected frequency in any cell is equal to the product of the marginal total common to that cell divided by the total number of observation. If our hypothesis of independence is true the difference between observed and expected frequencies are small and are attributed to sampling error. Large differences arise of the seeing false. The Chisquare statistic provides a means for deciding whether the differences are large or small overall. Hence the statistic to use is,
r
c
i=1
j=1
X =∑ ❑ ∑ ( oij−eij ) 2
1 /eij
With (r-1) (c-1) degrees of freedom. Where r represents rows and c represents the number of columns. A large value of X 2 indicates that the null hypothesis is false. The procedure for testing the null independence in contingency table is given below: i)
hypothesis
of
Formulate the null and altonative hypothesis as:
H0: The two variables of classification are independent OR There is no relationship / Association between the two variables. H1:
The two variables of classification are not independent; means they are associated.
ii)
Choose a significance level x. The commonly used levels are at x = 0.01, 0.05 etc.
iii)
The test statistic use to r
c
i=1
j=1
X =∑ ❑ ∑ ( oij−eij ) 2
2 /eij
Which, if H0 is true, has an approximate chi-square distribution with (r-1) (c-1) degrees of freedom. iv)
eij=
Compute the expected frequencies under H0 for each cell by the formula
( Ai )( Bj ) n
¿
( ith row total ) ( jth column total ) Total number of observation
Also calculate the value of X2 and the degrees of freedom.
v)
Determine the critical region which depends on X and the number of degrees of freedom.
iv)
Decide as below: (i)
Reject H0, if the computed value of X2> X2× (r-1) (c-1)
(ii)
Accept H0 if X2> X2× (r-1) (c-1)
References 1.
Chudry and Kamal (2004), Introduction to statistical theory part-I. Markazi Kutab Khana, Urdu Bazar, Lahore, Pakistan.
2.
B.L. Agarwal (2003), Programmed Statistics, 2 nd Edition. New Age International (P) Limited Publishers 4835/24, Ansori Road, Daryaganj, New Delhi – 110002, ISBN: 81-224-1458-3.
8.3
REGRESSION:
In statistics, regression analysis is a statistical technique for estimating the relationships among variables. It includes many techniques for modelling and analysing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables. In other words regression is a statistical measure that attempts to determine the strength of the relationship between one dependent variable (usually denoted by Y) and a series of other changing variables (known as independent variables). Types of 'Regression' There are two basic types of regression: (i)
Linear regression
(ii)
Multiple regression.
Linear regression uses one independent variable to explain and/or predict the outcome of Y, while multiple regression uses two or more independent variables to predict the outcome. The general form of each type of regression is: Linear Regression: Y = a + bX + u Multiple Regression: Y = a + b1 X1+ b2 X2 + B3 X3
B3X3 + …… Bt Xt u
Where: Y
=
the variable that we are trying to predict
X
=
the variable that we are using to predict Y
a
=
the intercept
b
=
the slope
u
=
the regression residual.
In multiple regression, the separate variables are differentiated by using subscripted numbers. Regression takes a group of random variables, thought to be predicting Y, and tries to find a mathematical relationship between them. This relationship is typically in the form of a straight line (linear regression) that best approximates all the individual data points. Regression is often used to determine how much specific factors such as the price of a commodity, interest rates, particular industries or sectors influence the price movement of an asset.
