Critique & Evaluation

Introduction

Overview

Review Objectives / Elements

What

The critique refers to the instructor’s role as a critic, and the evaluation section discusses creating and conducting effective evaluations.

Why

The instructor must be able to appraise student performance and convey this information back to the student. Also, throughout a course, and at the end of a course, a student must be evaluated to measure and document whether the course objectives have been met.

Critique

Purpose

  • Should provide the student with something constructive they can work on
  • Provide direction and guidance to raise their level of performance
  • Can be used for re-teaching in the case that several students falter at the same step

Characteristics

  • Objective
    • Focused on student performance, not personal opinions, likes, dislikes, or biases
    • To be objective, a critique must be honest and based on the performance as it was
  • Flexible
    • The performance must be examined in the context it was accomplished
    • Fit the tone, technique, and content of the critique to the occasion as well as the student
    • Allow for variables and be flexible to satisfy the requirements of the moment
  • Acceptable
    • Before accepting the critique, students must accept the instructor
      • Must be confident in qualifications, teaching ability, sincerity, competence, and authority
    • Present it fairly, with authority, conviction, sincerity, and from a position of competence.
  • Comprehensive
    • Cover strengths and weaknesses
    • What will provide the greatest benefit?
      • A few major points or more minor points
      • Critique what most needs to be improved or only what can be reasonably expected to improve
  • Constructive
    • The critique is pointless unless the student profits from it
    • Don’t offer a negative critique without a solution.
  • Organized
    • It needs to follow some pattern of organization otherwise it may lose its impact
      • ANy pattern is acceptable if it is logical and makes sense to student and instructor
      • Options include:
        • The sequence fo the performance itself
      • Work backwards from where the demonstration failed (or was successful)
        • Break the whole into parts or build the parts into a whole
  • Thoughtful
    • Reflects thoughtfulness to self-esteem, recognition, and approval from others
      • Ridicule, anger, or fun at a student’s expense have no place in a critique
  • Specific (rather than general)
    • Tell the student why something was not good and how to improve it
    • Students should have no doubt what was good, and what was poor, and how they can improve

Methods

  • Instructor/Student Critique
    • The instructor leads a group discussion in which students offer criticism of a performance
      • This should be controlled carefully and directed with a firm purpose.
  • Student Led Critique
    • A student is asked to lead the critique
    • This can generate student interest and learning, and be effective
  • Small Group Critique
    • Small groups are assigned a specific area to analyze and present their findings on
    • Results in a comprehensive critique
  • Individual Student Critique by Another Student
    • Another student is requested to present the entire critique
      • The instructor must maintain firm control over the process
  • Self-Critique
    • A student is required to critique a personal performance
    • Do not leave controversial issues unresolved, or erroneous impressions uncorrected
    • Make sure the student realizes the mistakes
  • Written Critique
    • Three Advantages
      • The instructor can devote more time and thought to it
      • The student can keep written critiques and refer to them whenever they wish
      • The student has a record of suggestions, recommendations, and opinions of all other students
    • Disadvantage
      • That the other members of the class do not benefit

Ground Rules

  • Do not extend the critique beyond its scheduled time limit and into the time allotted for other activities
    • Point of diminishing returns is reached very quickly
    • No more than 10 – 15 mins (not more than 30 mins)
  • Avoid trying to cover too much
    • Get the main points (4-5 things at most)
  • Allow time for a summary of the critique to reemphasize the most important things to remember
  • Avoid absolute statements (most rules have exceptions)
  • Avoid controversies with the class and don’t take sides
  • Never allow yourself to be maneuvered into defending criticism
    • Don’t let the student argue and tell you that you are wrong
  • If part of the critique is written, ensure it is consistent with the oral portion

Evaluation

Purpose

  • To determine how a student is progressing
  • Evaluations: Oral Quiz, Written Test, Performance Test

Oral Questions

  • Effective Oral Questions:
    • Have only one correct answer
    • Must apply to the subject of instruction
    • Should be brief and concise, but also clear and definite
    • Must be adapted to the ability, experience, and stage of training of the students
    • Center on only one idea
    • Should be limited to who, what, where, when, why, or how and not a combination
    • Must present a challenge to the student
    • Demand and deserve the use of proper English

Types of Questions to Avoid

  • “Do you understand?” “Do you have any questions?”
  • Puzzle Quesitons
  • Oversize
  • Toss-up
  • Bewilderment
  • Trick Questions
  • Irrelevant Questions

Responding to Questions

  • The question must be clearly understood by the instructor before an answer is attempted
  • Display interest in the student’s question and frame an answer that’s direct and accurate
  • Determine whether the question has been completely answered and if the answer is satisfactory
  • If it is unwise to introduce more complicated info explain that the question was good/pertinent, but an answer would complicate the learning tasks and reintroduce the question later in training if it’s not covered
  • If an answer is unknown, freely admit not knowing, but promise to get the answer or help look it up.

Written Questions

  • Characteristics of a Good Written Test
    • Reliability – The degree to which test results are consistent with repeated measurements
      • Does the test give consistent measurement to an individual or group?
    • Validity – The extent to which a test measures what it is supposed to measure
      • The most important consideration in test evaluation
      • Items that do not pertain directly to the objectives of the course should be removed
    • Usability – Refers to the functionality of tests
      • Easily read, clear and concise, clearly used and drawn graphics, easily graded.
    • Objectivity – Describes singleness of scoring of a test
      • Biases of the person grading are not reflected (Easily graded in a fair manner)
    • Comprehensiveness – The degree to which a test measures the overall objectives
      • A test must sample an appropriate cross-section of the objectives of instruction
    • Discrimination – The degree to which a test distinguishes the difference between students
      • A test must be able to measure small difference in achievement in relation to the objectives
      • When constructed this way, it has 3 features:
        • A wide range of scores
        • All levels of difficulty are included
        • Each item distinguishes between high and low achievers of course objectives

Developing Written Test Questions

  • Should test a concept or idea that is important the student knows/understands
  • Must be stated so that everyone competent in the area would agree on the correct response
  • Should be stated in language the student will understand
  • SHould be worded in a simple, direct, and unambiguous way, and it should be edited for brevity
  • Should include sketches, diagrams, etc, when necessary to visualize the problem/add realism
  • Should present a problem that demands knowledge of the subject or course

Performance Tests

Characteristics

  • Criterion-referenced – the objective is for all applicants to meet the FAR standards
    • Each student’s performance is evaluated against a written, measurable standard
  • The ACS purpose is to delineate the standards by which FAA inspectors and DPE conduct tests
  • Parts include Areas of Operations and Tasks
    • AoO define phases of the practical test arranged in a logical sequence within each standard
    • Tasks are titles of knowledge areas/flight procedures/maneuvers appropriate to an AoO

Uses

  • The instructor trains the applicants to ACS standards in all areas
  • The evaluation fo the student is only in relation to the standards listed in the ACS
  • Students should be trained to the very highest level possible!

Conclusion & Review

Briefly review the main lesson points anything in question

Review

  1. Critique
    1. Purpose and characteristics of an effective critique
    2. Methods and ground rules for a critique
  2. Evaluation
    1. Characteristics of effective oral questions and what types to avoid
    2. Responses to student questions
    3. Characteristics and development of the effective written test.
    4. Characteristics and uses of performance tests, specifically, the FAA Airman Certification
    5. Standards
    6. Collaborative assessment (or learner-centered grading (LCG))

 

360 Virtual Tours