Advisory Circular (AC) No. 408-001

From: Transport Canada

Pilot Examiner Monitoring Procedures - Aeroplane and Helicopter

Issuing Office: Civil Aviation Document No.: AC 408-001
File Classification No.: Z 5000-34 Issue No: 01
RDIMS No.: 7444047-V9 Effective Date: 2013-01-23

TABLE OF CONTENTS

1.0 INTRODUCTION

  1. This Advisory Circular (AC) is provided for information and guidance purposes. It describes an example of an acceptable means, but not the only means, of demonstrating compliance with regulations and standards. This AC on its own does not change, create, amend or permit deviations from regulatory requirements, nor does it establish minimum standards.

1.1 Purpose

  1. The purpose of this document is to set out the requirements for the Pilot Examiner (PE) Monitor. It is intended for anyone who is preparing for a monitor, whether it is for the initial issuance or renewal of a PE authority. The information provided is the expected PE standard. It is used by Transport Canada Civil Aviation Inspectors (CAI’s) conducting PE Monitors and informs PE’s carrying out their delegation.

1.2 Applicability

  1. This document applies to Transport Canada Civil Aviation (TCCA) employees and to individuals and organizations when they are exercising privileges granted to them under an External Ministerial Delegation of Authority. This information is also available to the aviation industry for information purposes.

1.3 Description of Changes

  1. Not applicable.

2.0 REFERENCES AND REQUIREMENTS

2.1 Reference Documents

  1. It is intended that the following reference materials be used in conjunction with this document:

    1. Subpart 408 of the Canadian Aviation Regulations (CARs) – and

    2. Standard 428 of the CARs – ;

    3. Transport Canada Publication (TP), 14277E, April 2010 — Pilot Examiner Manual;

    4. TP 12475E, April 2005 — Flight Test Guide – Recreational Pilot Permit - Aeroplane;

    5. TP 13723 — Private Pilot Licence Flight Test Guide – Aeroplane;

    6. TP 3077 — Private and Commercial Pilot Licence Flight Test Guide – Helicopter;

    7. TP 13462 — Commercial Pilot Licence Flight Test Guide – Aeroplane;

    8. TP 219 — Multi-Engine Class Rating – Aeroplane Flight Test Guide;

    9. TP 9939 — Instrument Rating – Groups 1, 2 and 3 Flight Test Guide;

    10. TP 15099 — Instrument Rating – Group 4 Flight Test Guide;

    11. TP 5537 — Flight Instructor Rating – Aeroplane, Helicopter, Aerobatic; and

    12. Authorized Person’s Guidelines and Personnel Licensing and Procedures Manual.

2.2 Cancelled Documents

  1. Not applicable.

  2. By default, it is understood that the publication of a new issue of a document automatically renders any earlier issues of the same document null and void.

2.3 Definitions and Abbreviations

  1. The following definitions are used in this document:

    1. Approved Training Organization: an Approved Training Organization pursuant to Subpart 407 of the CARs;

    2. Flight Training Unit: a Flight Training Unit pursuant to Subpart 406 of the CARs;

    3. FSTD: means a certified simulator or flight-training device.

    4. Monitor: means an event /flight test used to assess or confirm the manner, knowledge of a Pilot Examiner and method of conducting flight tests.

    5. Pilot Examiner: means a Pilot Examiner holding an accreditation/delegation from the Minister pursuant to Part I Section 4.3(1) of the Aeronautics Act or a Pilot Examiner trainee being monitored to confirm competence for an initial delegation/accreditation.

  2. The following abbreviations are used in this document:

    1. AC: Advisory Circular;

    2. ATC: Air Traffic Control;

    3. ATO: Approved Training Organization;

    4. CAI: Civil Aviation Inspector;

    5. CARs: Canadian Aviation Regulations;

    6. DFTE: Designated Flight Test Examiner;

    7. FTU: Flight Training Unit;

    8. PE: Pilot Examiner;

    9. TEM: Threat and Error Managements; and

    10. TC: Transport Canada.

3.0 BACKGROUND

  1. The Designated Flight Test Examiner (DFTE) program was introduced in June 1974, to provide prompt flight test service to the civil aviation training industry. The details of the program were established following discussions with the Royal Canadian Flying Clubs Association and the Air Transport Association of Canada. DFTEs were selected, trained, and appointed to conduct flight tests for private and commercial pilot licences and multi-engine class ratings.

  2. Before the DFTE program was established, flying clubs could conduct the flight test for private pilot candidates at schools offering approved courses. Even earlier, the privilege of testing private pilot candidates was given to all Class 1 and 2 flight instructors. Mounting concerns about the accident rate among private pilots resulted in this privilege being removed in 1969.

  3. Following discussions with Aerobatics Canada, the authority to conduct flight tests for some aerobatic instructor ratings was delegated to industry in 1985, subject to the same procedures for appointment and monitoring required of other authorities. Delegation of instrument rating flight tests was initiated in 1992. Delegation of flight instructor flight tests was initiated in 2003.

  4. The term “Designated Flight Test Examiner” was replaced with the term “Pilot Examiner” (PE) in November 2004. In addition to more accurately describing the duties of the examiner, the term pilot examiner is generally more widely recognized in other countries. More than 9,000 flight tests each year are performed by pilot examiners, making this group of industry professionals an essential part of a safe aviation system.

  5. The PE program is a service responding to the need to provide prompt, reliable, and objective flight testing for the issuance of licences, permits and ratings.

  6. The overall administration of the PE Program, including policy and standardization, is the responsibility of Transport Canada Civil Aviation (TCCA) – Headquarters (HQ). Final selection, on-site training, accreditation and monitoring of PE are the responsibility of the TCCA Regional Offices.

  7. An essential element of the PE Program is communication. PE workshops and routine contact between PEs and Flight Instructors combined with direct communication with TCCA Regional and HQ Inspectors are essential for standardization and continuous improvement in flight training and testing in Canada.

  8. Pilot Examiner authority is granted to individuals in accordance with the terms, conditions and guidelines of the Pilot Examiner Manual (TP14277).

  9. TCCA has a responsibility to exercise due diligence on behalf of the Minister of Transport to ensure that persons holding an accreditation, pursuant to Part 1, Section 4.3(1) of the Aeronautics Act, retain the required competence, professionalism and standardization while exercising their delegated duties.

  10. Standardization is achieved through the following events:

    1. A Headquarters PE Workshop every 2 years;

    2. A monitoring event during the first year, then recurrent monitoring every 2 years thereafter; and, as required;

    3. Special monitoring.

4.0 ADMISSION TO THE MONITOR

  1. PEs are required to contact an authorized CAI within 90 days prior to expiration for completion of a monitoring event. The PE will arrange, with the CAI, a mutually convenient date and time. Should the PE not be able to make the necessary arrangements, the CAI will arrange a monitoring event.

  2. For admission, the PE is required to present:

    1. A valid Commercial or Airline Transport Pilot Licence in the same aircraft category;

    2. Documentation for the conduct of a specific flight test. This may include , scripts, plans-of-action and a bank of oral questions, etc.;

    3. May be required to present their copies of completed Flight Test Reports, for recurrent PE monitors; and

    4. For attempting a monitor after an assessment of “Unacceptable” was assigned, must provide evidence of meeting the conditions under which the suspension would be terminated, as stated in the letter of notification, in accordance with the requirements for notification specified in the Aeronautics Act.

5.0 PILOT EXAMINER MONITOR RESULTS

  1. The Privacy Act protects the privacy of individuals with respect to personal information about themselves held by a government institution. A PE Monitor, and the resulting PE Monitor Report (Appendix A of this AC), measures the competency of the PE conducting a flight test.

  2. Personal information may be disclosed in accordance with Section 8(2) (a) of the Privacy Act, which allows disclosure “for the purpose for which the information was obtained or compiled by the institution or for a use consistent with that purpose”. The purpose for which PE Monitor information is obtained is to ensure the safety of aviation in Canada. The specific purposes are to measure whether the PE meets the minimum knowledge and skill standard for the delegation issued by the Minister pursuant to Section 4.3(1) of the Aeronautics Act.

  3. In accordance with 8(2)(a) of the Privacy Act, a copy of the PE Monitor Report (Appendix A of this AC) shall be given to the PE showing the results of a monitor and a copy shall be retained by TCCA. Specific information about the results of a PE Monitor will not be given by TCCA to anyone but the individual named on the PE Monitor Report except in accordance with the Privacy Act.

6.0 PILOT EXAMINER ASSESSMENT

Note: The following table includes definitions of each assessment. These definitions are not all-inclusive, but are meant as a guide for TCCA CAI’s.

Acceptable

Competency is acceptable, but may include few minor errors from ideal that are inconsequential. The likelihood of the error being repeated is minimal.

  • Knowledge of the function and application of PE and Authorized Person delegation is acceptable;
  • The pre-flight briefing includes acceptable coverage of content and use of oral questions;
  • The conduct of the flight test includes acceptable execution of the flight test profile/script, application of realism and time management;
  • The post-flight debriefing includes acceptable coverage of content and execution of technique;
  • Administration includes acceptable knowledge and application of flight test prerequisites, assessment validity and flight test report completion; and
  • Rapport with candidate is acceptable.
Acceptable with Briefing

Competency meets a lower standard and may include an action or inaction that has the potential to lead to an undesired outcome of the flight test. Includes a major error, limited knowledge and/or application of the element(s). Any major error is effectively mitigated through effective countermeasures and is deemed to be acceptable.

  • Knowledge of the function and application of PE and Authorized Person delegation is limited;
  • The pre-flight briefing includes basic coverage of content and use of oral questions;
  • The conduct of the flight test includes basic execution of the flight test profile, application of realism and time management;
  • The post-flight debriefing includes basic coverage of content and execution of technique;
  • Administration includes limited knowledge and application of flight test prerequisites, assessment validity and flight test report completion; and
  • Rapport with candidate(s) is guarded, but courteous.
Unacceptable

Competency includes an action or inaction that may include frequent major errors or a critical error leading to a mismanaged or undesired outcome of the flight test. The PE exercised his/her powers in an unacceptable manner that required direct or immediate intervention by the monitoring CAI. These may include non compliance with Canadian Aviation Regulations (CARs) or non-adherence to PE Manual policy and procedures.

  • Knowledge of the function and application of PE and Authorized Person delegation is unacceptable;
  • The pre-flight briefing includes unacceptable coverage of content and use of oral questioning techniques;
  • The conduct of the flight test includes unacceptable execution of the flight test profile, application of realism and time management;
  • The post-flight debriefing includes unacceptable coverage of content and execution of technique;
  • Administration includes unacceptable knowledge and application of flight test prerequisites, assessment validity and flight test report completion; and
  • Rapport with candidate is unacceptable

7.0 CONDUCT OF THE PILOT EXAMINER MONITOR

  1. The PE will assess all flight tests using the 4-point marking found in the applicable Flight Test Guide. The standards specified in the guides are not exhaustive and they do not define all expected performance criteria. PEs must apply their knowledge and experience in conjunction with the rating definitions to arrive at an assessment. A candidate is expected to operate the aircraft in accordance with the manufacturer's specifications, recommended speeds and configurations in the Pilot's Operating Handbook/Aircraft Flight Manual (POH/AFM/HFM) or other approved data under normal circumstances. However, it is important for a PE to apply discretion for unusual circumstances outside the control of the candidate such as wind, traffic or weather conditions. A PE may also tolerate an excursion from specified limits in the performance criteria if the candidate recovers in a timely manner. However, an excursion from prescribed limits, with or without a timely recovery, which jeopardizes the safety of the aircraft is unacceptable.

  2. A PE's principle function during a flight test is that of an examiner assigned by the Minister to assess an individual's performance. PEs should make a concerted effort to be relaxed and non-threatening. PEs will refrain from making personal remarks and inappropriate motions such as shaking their heads, stiffening to rigidity or gasping at inappropriate times, laughing at performance or commenting subsequent to a poorly conducted manoeuvre.

  3. PEs will apply the standards in the applicable Flight Test Guide regardless of the training and experience of the candidate. PEs will assess the candidate against what would be “well executed considering existing conditions". For example, if it is windy and turbulent, the candidate will not be able to maintain altitude, heading and airspeed as well as on a calm day. Give credit where credit is due, and do not be influenced by poor performance on a previous flight test item when assessing a subsequent item.

  4. PEs will assess each exercise as soon as possible after it has been completed. A “2” or “1” evaluation requires a written remark to justify the assessment.

  5. The CAI will use this AC in conjunction with other guidance for the conduct of PE monitors including Staff Instructions and any Regional Supplementary Staff Instructions.

  6. The delegation sought will determine the nature of the monitor(s) required. Although the PE must be prepared to demonstrate the ability to effectively conduct any flight test for authorities sought, the monitor is a sampling of the PE’s performance.

8.0 ROLE PLAYING

  1. Suitable flight test candidates or aircraft with more than two seats are not always available. Industry has expressed concern in regard to the negative impact on the candidate of having a third person on board that is in a “position of authority”. The applicable privileges expressed in the regulations for the Student Pilot Permit, the Private Pilot Licence also apply to Commercial Pilot Licence flight tests and in each case state that “no passenger other than the person referred to in paragraph 401.15(1)(a) [the examiner] is carried on board”. Although it might be desirable for examiner training purposes to observe and monitor actual RPP, PPL, or CPL flight tests, this practice is in contravention of Sections 401.19, 401.26 and 401.27 of the CARs and will not be used for PE training or monitoring. Another consideration is that during the conduct of emergency manoeuvres, having a third person on board may compromise safety by altering the recovery characteristics of the aircraft.

  2. These flight tests will be done with a mock candidate or simulated with the CAI role-playing the part of a candidate. Mock candidates should be persons nearing completion of their training for, or have recently acquired, the licence or rating for which examiner authority is sought. Again, be aware that emergency manoeuvres with a third person on board may compromise safety by altering the recovery characteristics of the aircraft.

  3. For Multi-engine Class Rating flight tests, role-playing is strongly recommended to avoid having a CAI riding in a second-row seat due to safety issues that may occur during stalls and simulated engine failures.

  4. Pilot examiners seeking authority to conduct instrument rating flight tests may be trained or monitored by observing all or part of a formal licensing flight test. The options of using a mock candidate or a CAI role-playing as a candidate are also available.

  5. Flight instructor rating flight tests may be trained or monitored by observing all or part of a formal licensing flight test. The options of using a mock candidate or a CAI role-playing as a candidate are also available. If emergency manoeuvres are to be demonstrated during a flight instructor flight test, avoid having a third person on board;

  6. CAls may role-play the part of a candidate or use their experience to develop realistic scenarios for sampling of flight test items that may be flown by a mock candidate to sample a PE’s evaluation skills and judgement during the ground and flight portions of a flight test.

  7. Scenarios should include exercises in which performance is simulated to be:

    1. well executed during a test;

    2. executed with minor or major errors during a test; and

    3. executed with critical errors during a test.

  8. Although a majority of the monitors are conducted in aircraft, some may be conducted in:

    1. a certified Level 3, 5 or 6 flight training device (FTD) having a visual system appropriate to the task being evaluated, if the aircraft is an aeroplane; or

    2. a certified Level A or higher simulator.

9.0 PILOT EXAMINER MONITOR PHASES AND ELEMENTS

  1. The PE monitor consists of six (6) phases:

    1. Delegated Authority;

    2. Rapport with Candidate;

    3. Pre-Flight Briefing;

    4. Conduct of the Flight Test;

    5. Post-Flight Debriefing; and

    6. Administration.

  2. Each phase is subdivided into elements for a total of thirteen (13) elements that will be assessed. Details of each phase and element are given under the headings Aim, Description, and Performance Criteria. The performance criteria is not all-inclusive, but are meant as a guide for the CAI.

10.0 DELEGATED AUTHORITY

10.1 Aim

  1. To determine that the PE has an acceptable knowledge of the duties and responsibilities of the PE Delegation and Authorized Person Delegation and that he or she can demonstrate the application of those duties and responsibilities during a flight test.

10.2 Description

  1. The CAI will assess the PE’s knowledge through oral questions and by observation.

  2. The PE will be expected to have the required knowledge as indicated in the PE Manual and apply those duties and responsibilities in a professional manner during the conduct of a flight test.

10.3 Performance Criteria

  1. Assessment of “DELEGATION AUTHORITY” will be based on the PE's ability to:

    1. display a positive attitude towards the PE program;

    2. exercise all duties and responsibilities outlined in the PE Manual;

    3. demonstrate a broad understanding of the PE Manual policies;

    4. act within the limits of his/her delegation;

    5. demonstrate knowledge of TP11825 – Liability Through the Exercise of Delegated Authority; https://tc.canada.ca/en/aviation/publications

    6. understand and apply the "Privacy Act" with respect to flight test report forms;

    7. demonstrate an exceptional knowledge of the PE Manual, relevant Flight Test Guides and various FTU/ATO company manuals and operations specifications as they relate to the FTU/ATO Operator’s flight operations;

    8. demonstrate a satisfactory knowledge of the contents and interpretation of the CARs and appropriate Transport Canada publications; and

    9. diligently exercise his/her delegation of authority with the expected reasonable duty of care.

  2. Assessment of “AUTHORIZED PERSON DELEGATION” will be based on the PE’s ability to:

    1. understand and carry out the function of an Authorized Person on behalf of the Minister in accordance with the powers, duties, functions contained in Part IV of the CARs, TCCA policies, procedures and guidelines;

    2. exercise all reasonable duty of care to ensure that all requirements have been met prior to issuing privileges in a candidate’s Aviation Document for:

      1. Instrument Rating;

      2. Multi-engine Class Rating;

      3. Flight Instructor Rating; and

      4. Individual Type Rating.

    3. accurately complete the “Application for Endorsement of a Rating” – Form 26-0083 for the issuance of Multi-engine Class Rating - Aeroplane, Flight Instructor Rating, Instrument Rating or Type Rating or the “Application for a Licence”, as applicable;

    4. determine and apply the fee requirements for a licence or rating; and

    5. complete any other Authorized Person administrative requirements as outlined in the PE Manual and the Personnel Licensing Procedures Manual (TP 2943) or appropriate Staff Instructions.

11.0 RAPPORT WITH CANDIDATE

11.1 Aim

  1. To determine that the PE can establish, foster and maintain a professional, non-intimidating rapport with a flight test candidate.

11.2 Description

  1. The PE will be expected to demonstrate and maintain a professional non-intimidating rapport with the candidate and maintain that rapport throughout the conduct of the flight test.

11.3 Performance Criteria

  1. Assessment will be based on the PE’s ability to:

    1. be an effective communicator and exhibit appropriate speaking and listening skills including voice, volume, pace expression and vocabulary;

    2. seek candidate participation and involvement by fostering open communication and a positive environment by displaying energy, enthusiasm and professionalism;

    3. address concerns openly and directly with those involved and maintain a friendly and approachable demeanour at all times to keep the candidate at ease;

    4. be well versed in the assessment of non-verbal communication, including posture, eye contact and other body language;

    5. treat all candidates fairly, cordially and with respect. Sensitivity will be shown towards social, cultural and gender issues.

    6. be empathetic; and

    7. avoid rudeness, profanity and slang at all times.

12.0 PRE-FLIGHT BRIEFING

12.1 Aim

  1. To determine that the PE can prepare and conduct an effective pre-flight briefing.

12.2 Description

  1. The PE will be expected to conduct a pre-flight briefing by selecting a suitable location free from distraction. The pre-flight briefing will consist of a discussion between the PE and the candidate prior to the conduct of the flight test to ensure everyone involved understands exactly what will take place.

  2. The PE is expected to ask a series of practical questions pertaining to the candidate’s knowledge of flight planning, procedures and aircraft equipment.

12.3 Performance Criteria

  1. Assessment of “CONTENT” will be based on the PE’s ability to:

    1. conduct a comprehensive pre-flight briefing ensuring the candidate is aware of what is expected and includes the items to be covered and their relationship to the applicable Flight Test Guide and the PE Manual;

    2. ensure the briefing items are clear, concise and presented in a logical and organized sequence;

    3. for a single pilot flight test in a multi-crew aircraft, brief how multi-crew requirements will be demonstrated;

    4. for a flight test in a single-crew operation, brief how the single-crew requirements will be demonstrated;

    5. ensure safety is briefed as a priority, including methods for simulating emergencies in an aircraft;

    6. where a “qualified safety pilot” is used in the aircraft, brief what their role will be during simulated emergencies;

    7. where applicable, ensure a simulator safety briefing is provided including shutdown and escape procedures;

    8. review operational flight plan, weight and balance, meteorological and aerodrome conditions, NOTAMs and obtain candidate involvement. In addition, the PE is expected to ensure the weather is conducive to flight testing requirements; and

    9. ensure all of the candidate’s questions or concerns are answered.

  2. Assessment of “ORAL QUESTIONS” will be based on the PE’s ability to:

    1. apply effective oral questioning techniques. This includes ensuring questions are easily understood, clear, concise, composed of common words, thought provoking, practical, operationally relevant and appropriate flight planning, regulatory or flight procedures and aircraft systems as outlined in the applicable Flight Test Guide;

    2. allow adequate time for the candidate to answer;

    3. exhibit appropriate listening skills such as: avoiding interruption, listening attentively, probing for additional information or paraphrasing for understanding, as needed;

    4. refrain from asking leading questions or from teaching or briefing the candidate on the correct answer;

    5. ask questions of the candidate concerning reasons for their action/inaction, when in doubt;

    6. know the answers to the questions asked and use reference material to ensure the accuracy of answers;

    7. use training aids to understanding, where necessary (i.e. handouts, whiteboards, computer or cockpit mock-ups, etc.);

    8. avoid affecting the candidate’s self-confidence with body language and/or comments; and

    9. assess once the exercise is complete and take notes, where applicable.

13.0 CONDUCT OF THE FLIGHT TEST

13.1 Aim

  1. To determine that the PE can prepare and conduct an effective airborne or simulator assessment.

13.2 Description

  1. An airborne or FSTD flight test will be conducted immediately after the pre-flight briefing. The PE will be expected to have a “plan-of-action” or “script” to conduct an effective and realistic flight profile that adheres to the guide, without undue burden on the pilot or flight crew.

  2. If the PE utilizes a third party to operate the FSTD Instructor Operating Station (IOS), the PE is expected to ensure the following additional items are covered:

    1. The FSTD operator is provided with a copy of the plan-of-action or script and briefed on the specifics and logistics of the document;

    2. The PE ensures that Air Traffic Control (ATC) Clearances, weather, malfunctions and emergencies are provided and programmed as required by the plan-of-action or script;

    3. In the event of a plan-of-action/script deviation or FSTD fault, a means of communication is determined between the PE and the FSTD operator to mitigate the situation; and

    4. The FSTD operator does not participate in the conduct of the flight test with the exception of role playing (i.e. ATC, ground personnel etc.), as delegated by the PE.

13.3 Performance Criteria

  1. Assessment of “FLIGHT PROFILE/SCRIPT” will be based on the PE’s ability to:

    1. cover all flight test items as briefed and in accordance with the applicable Flight Test Guide;

    2. organize a well-designed scenario to afford the candidate the opportunity to demonstrate both technical and non-technical Cockpit Resource Management (CRM) concepts (i.e. scenarios will be flown to completion and unrealistic endings should be avoided);

    3. where a single candidate is being assessed in a multi-crew aircraft, assess his/her PF duties;

    4. where a candidate is required to operate in a multi and single crew operation, ensure that the single crew requirements are completed;

    5. demonstrate competence in the operation of the simulator when occupying the FSTD IOS;

    6. observe the candidate and not be programming the FSTD or be distracted at key moments;

    7. for airborne flight tests, follow the supplementary guidance material outlined in the PE Manual;

    8. accurately follow the plan-of-action and effectively adapt to changes from the briefed procedure and initiate a candidate briefing to clearly explain the changes and why they are necessary;

    9. use good judgement regarding how to conduct the required items for an airborne flight test and ensure clear communication is achieved between the PE and the training pilot, Pilot Not Flying (PNF) or Pilot Monitoring (PM), as applicable, for the purposes of maintaining safe flight and simulating abnormalities/emergencies;

    10. apply any of the four (4) conditions for a manoeuvre to be repeated (i.e. discontinuance, collision avoidance, misunderstood request and other factors); and

    11. rate each item as soon as possible after it has been completed and to take notes, where applicable.

  2. Assessment of ‘ENVIRONMENTAL REALISM” will be based on the PE’s ability to:

    1. conduct FSTD flight tests in real time as much as possible;

    2. ensure the flight test, whether airborne or simulated, is conducted in an efficient logical flow without rushing the candidate;

    3. provide consistent, fair and effective pilot/crew assessment scenarios;

    4. allow the candidate to use all available technology to the maximum;

    5. introduce malfunctions that are only representative of the list of malfunctions. The consensus is one major and one minor in addition to the mandatory engine failure(s) per pilot flying;

    6. introduce simulated system abnormalities that are realistic and do not lead to multiple unrelated failures;

    7. ensure simulated emergencies are conducted in a manner that enables the candidate to understand the aircraft status;

    8. take time to evaluate the candidate’s action when a major or critical error (imminent failure) has been committed before “freezing” the simulator;

    9. where applicable, effectively play the role of ATC, dispatch, maintenance, etc. When acting as ATC, the PE must provide clear and unambiguous clearances and instructions using standard terminology that closely duplicate ATC MANOPS requirements;

    10. provide assistance that would normally be provided by ATC and would not compromise the integrity of the flight test;

    11. not use initiatives intended to prevent the candidate from making an error. For example, not leading the candidate as ATC – this would include key airspeed and altitude clearances that would make an error less significant;

    12. refrain from teaching or providing “over-the-shoulder” instruction to the candidate on the correct completion of a flight test item or from taking any action that will prompt the candidate towards a specific action;

    13. ask questions of the candidate concerning reasons for their action/inaction, when in doubt;

    14. ensure the flight is conducted under the flight crew concept for multi-crew aircraft;

    15. for airborne flight tests, liaise with ATC to ensure an effective flight test scenario;

    16. for airborne flight tests, conduct the test on an actual filed flight plan with ATC, where practicable;

    17. make clear and “on-time” verbal calls during an airborne flight test (i.e. “field in sight” or “simulated” etc.); and

    18. for flight tests conducted in a FSTD, ensure that the airport(s) used, meet the basic model airport content requirements for testing.

  3. Assessment of “TIME MANAGEMENT” will be based on the PE’s ability to:

    1. ensure the flow and pace of the flight test is efficient without rushing the candidate/crew;

    2. ensure required sequences are completed within an acceptable time frame;

    3. afford the candidate/crew sufficient time to complete all emergency actions and follow-up items; and

    4. use real time to the extent possible and refrain from using the flight/position freeze unless justified or included in the script for a specific reason.

14.0 POST FLIGHT BRIEFING

14.1 Aim

  1. To determine that the PE can prepare and conduct an effective post flight briefing.

14.2 Description

  1. The PE will be expected to provide a post flight briefing consisting of a review and discussion of the flight test in a suitable location free from distraction.

  2. Depending on the outcome of the flight test, the PE is expected to utilize one of three (3) post-flight briefing methods. The methods are as follows:

    1. The traditional method for a routine flight test with only minor errors where a facilitated debriefing will add no value;

    2. The facilitated method to ensure the underlying Threat and Error Managements (TEM) and CRM issues are adequately addressed to promote crew participation, self-assessment and enhance future performance; and

    3. The method for failed flight tests that focuses on a review of strengths, weaknesses and opportunities for improvement.

  3. Using discussion and/or facilitation, within a reasonable time frame, the PE is expected to lead the candidate(s) to self-discovery for their performance, including threats, errors and error management and methods available to correct the errors and to enhance future performance. The PE is expected to assist the candidate(s) to focus on the actions taken during the flight test by limiting extraneous discussions.

14.3 Performance Criteria

  1. Assessment of “CONTENT” will be based on the PE’s ability to:

    1. inform the pilot/crew of the outcome (i.e. pass/fail) at the earliest opportunity;

    2. regardless of the briefing method chosen, the time taken should be reasonable commensurate with the candidate’s performance and include the following mandatory elements:

      1. strengths, weaknesses and opportunities for improvement;

      2. all items assessed either as a one (1) or two (2);

      3. reviewing anything written on the Flight Test Report, including proposed remarks that will be written on the form; and

      4. review anything the PE deems to be a safety issue.

    3. review threats, errors and error management;

    4. provide the candidate/crew with sufficient information by accurately referencing Standard Operating Procedures, regulations, procedures, limitations, tolerances, etc. for all assessments of one (1) and two to minimize misunderstandings;

    5. debrief items with a goal to improving performance;

    6. for a failed flight test:

      1. debrief the candidate(s):

        1. on the reason for the failure;

        2. the administrative course of action; and

        3. who may conduct the re-test.

      2. notify the Chief Flight Instructor/Director of Training:

        1. of the failed item(s); and

        2. make recommendations for corrective action.

      3. provide a copy of the Flight Test Report to the candidate(s) as soon as practicable;

    7. make recommendations to the candidate(s) or FTU/ATO, where applicable; and

    8. inform the candidate/crew when the debriefing is complete and ask if there are any questions concerning the conduct of the flight test or other related topics.

14.4 Performance Criteria

  1. Assessment of “TECHNIQUE” will be based on the PE’s ability to:

    1. select the appropriate debriefing method consistent with the outcome of the flight test;

    2. conduct a debriefing highlighting the strengths, weaknesses and opportunities for improvement of both major and minor errors;

    3. not allow personal perspective to result in debriefing elements that deviate from objective criteria but result in acceptable evaluations;

    4. for flight tests assessed as pass, make effective use of facilitated debriefing techniques including;

      1. the use of methods for debriefings as outlined in the PE Manual;

      2. integrate TEM, Technical and Non-Technical (NOTECH) issues in a manner that enhances pilot/crew performance and flight safety;

      3. adjust facilitation level needed to engage the candidate(s) to the maximum extent;

      4. use questions to promote in-depth candidate participation, including:

        1. using open ended questions (i.e. what, how and why);

        2. using silence/pauses to elicit thoughtful candidate responses;

        3. rewording and/or redirecting questions;

        4. asking follow-up questions that require in-depth analysis.

      5. ensure all crewmembers are involved by asking appropriate questions;

      6. use discussion to improve the knowledge and confidence of candidate(s);

      7. bring out CRM issues and root causes and discuss them with the candidate(s);

      8. not focus on the assessment, but rather the error(s) and the method(s) for correcting them in the future;

      9. finish discussions with ways to improve performance.

    5. for flight tests assessed as fail:

      1. showing empathy, conduct a debriefing in a non-confrontational manner while highlighting the strengths, weaknesses and opportunities for improvement; and

      2. not focus on the assessment of fail, but rather the undesired state(s), the circumstances leading to the undesired state and method(s) to prevent reoccurrence.

    6. ask questions of the candidate/crew concerning thought processes, actions, inactions and reasons, when in doubt.

15.0 ADMINISTRATION

15.1 Aim

  1. To determine the PE’s ability to complete the administrative duties in accordance with the procedures outlined in the PE Manual and the Authorized Person Delegation.

15.2 Description

  1. The PE is expected to demonstrate competence in the following areas:

    1. Candidate prerequisites for admittance to the flight test;

    2. Completion of the Flight Test Form;

    3. Assignment of additional licensing privileges, if applicable; and

    4. Provision of counselling and direction with respect to a failed flight test, when required.

15.3 Performance Criteria

  1. Assessment of “FLIGHT TEST PREREQUISITES” will be based on the PE’s ability to:

    1. determine a candidate’s eligibility for the flight test (i.e. training, valid Aviation Document and medical, letter of recommendation, etc) and privileges of licences;

    2. ensure the candidate meets the knowledge and experience requirements for the issuance of licence or rating sought in accordance with Part IV of the CARs;

    3. for simulator flight tests, determine that the simulator to be used is serviceable and can meet the requirements of the flight test; and

    4. for flight tests in an aircraft, determine that the aircraft is airworthy and is adequately equipped to meet the requirements of the flight test, utilizing the MEL where applicable.

  2. Assessment of “ASSESSMENT VALIDITY” will be based on the PE’s ability to:

    1. cover all items on the flight test report;

    2. correctly apply the one (1) to four (4) marking scale, including:

      1. considerations for momentary deviations due to weather, turbulence, ATC requests, emergencies, etc;

      2. correctly integrating Threat and Error Management concepts in making assessments;

      3. correctly apply the six (6) elements when making assessments;

      4. integrate non-technical skills;

      5. terminate the flight test when:

        1. more than two (2) items have been assessed as one (1);

        2. more than one (1) item has been assessed as one (1) for the Instrument Rating; or

        3. one (1) item has been assessed as fail (F) for a Flight Instructor Rating

    3. in the case of multi-crew, conduct the flight test under the flight crew concept and apply jeopardy requirements, when warranted and applicable.

  3. Assessment of “FLIGHT TEST REPORT COMPLETION” will be based on the PE’s ability to:

    1. complete the applicable flight test report form as directed in the PE Manual to meet the following minimum requirements:

      1. the form must be legible and contain no deletion making the document unreadable;

      2. all appropriate circles are properly filled;

      3. for abnormal/emergencies, the type of emergency is documented;

      4. correctly indicate the applicable valid-to-dates, where applicable;

    2. structure clear, appropriate and legible remarks that support assessment(s) of a two (2) by referencing the applicable element(s), the associated performance criteria outlined in the applicable Flight Test Guide, regulations, aircraft limitations, SOP’s, etc. and the countermeasures employed;

    3. ensure remarks written for an assessment of two (2) do not warrant an assessment of one (1);

    4. structure clear, appropriate and legible remarks that support the assessment of one (1) by referencing the applicable element(s), the specific performance criteria outlined in the applicable Flight Test Guide, regulations, aircraft limitations, SOP’s etc. that was breached resulting in an undesired aircraft state and an absence of countermeasures;

    5. ensure no additional remarks are written that were not debriefed with the candidate/crew;

    6. for a failed flight test, provide a copy of the flight test report form to the candidate as soon as practicable; and

    7. demonstrate or explain the distribution requirements of flight test report forms.

16.0 FINAL REVIEW

  1. The final review with the PE is an opportunity to provide both positive and negative feedback to the PE and to give some further information or techniques that will improve their performance. This must be done in private with no interruptions.

  2. Advise the PE if they were successful or not. Debrief the PE with a summary and a review of their performance, highlighting positive performance whenever possible. At this time, conduct a review of the PE’s completed required forms for content and accuracy.

  3. It is important that the assessment of the PE be straightforward and direct. There is a need for awareness where strengths and weaknesses lie, and where and how to improve.

  4. Complete the PE Monitoring Report Form as soon as feasible. If possible, hand a copy of the report to the PE during or at the conclusion of the final review. It is important that the PE Monitoring Report Form is accurate – so do not rush it.

  5. During the final review of the forms completed by the PE, the CAI should:

    1. Verify remarks for grades of the 1 or 2 recorded on the flight test report form concur with the criterion of the PE Manual and the relevant flight test guide. Remarks should clearly state “What happened!”.

    2. Verify wording of remarks that are rated “2” are not written with wording that constitutes a “1” assessment.

    3. Verify accuracy and completeness of flight test report form and all other required forms.

    4. For an initial monitor, countersign the flight test report and the candidate’s licence, if required.

  6. During the review of the overall performance of the PE during the flight test, the PE Monitoring Report Form should be used to go over each sub-item and discuss the topics of interest while highlighting positive and negative performance and discussing methods for improvement where necessary. The PE must be included in the discussion to explain his or her methods and reasons for doing things a certain way. This review may include the following:

    1. An informal discussion format for all past monitors;

    2. Advise the PE what you consider to be his/her strong and weak points as a PE;

    3. Examples from the flight test to highlight your points (positives and negatives);

    4. Examples from the PE Manual to show the required standard for a PE;

    5. Examples from the additional guidance section of the PE Manual for methods to conduct various exercises during the flight test;

    6. Provide briefing or counselling to improve areas assessed as “Acceptable with Briefing”;

    7. Point out guidelines in the PE Manual that were not followed;

    8. Ensure that the PE understands why they need improvement and where;

    9. Ensure a thorough understanding of the 1-4 grading scale and the 6 elements that form its basis;

    10. Ensure that the PE understands the grading methodology and discuss how it could have been used better;

    11. Discuss the PE’s briefing and debriefing techniques and ensure that they are following the PE Manual guidelines;

  7. A review of the administrative process for the monitor and any review of Authorized Person’s duties will be discussed. This is especially important for initial monitors in that the PE may not conduct any flight tests until they have received their Letter of Authority. A review of any final areas and a summary of what happens to the monitor paperwork will be provided. There will be a reminder of the requirement to have another monitor prior to the first day of the 13th month from today or the first day of the 25th month, as applicable. If the PE Monitor Report Form is complete, the PE will be given a copy. If not, he/she will be advised that a copy will be sent to them shortly. The CAI will ask for any questions and whether they have any feedback.

17.0 UNACCEPTABLE PILOT EXAMINER MONITOR

  1. When the CAI assesses at least one item as “Unacceptable”, the PE Monitor will receive an overall assessment of “Unacceptable”.

  2. In addition, a PE Monitor that has three (3) or more items assessed as “Acceptable with Briefing” will also receive an overall assessment of “Unacceptable”.

  3. The CAI will indicate to the PE that a “Notice of Suspension” will be issued for a recurrent monitor grade of “Unacceptable”, or a “Refusal to Issue” for an initial monitor grade of “Unacceptable”, as applicable; and

  4. Advise the PE of his/her right to appeal to the TATC for an unacceptable monitor.

18.0 INFORMATION MANAGEMENT

  1. Not applicable.

19.0 DOCUMENT HISTORY

  1. Not applicable.

20.0 CONTACT OFFICE

For more information, and to submit suggestions for amendment, please contact the:
Chief, Commercial Flight Standards (AARTF)

Phone: 613-990-1022
Fax: 613-990-6215
E-mail: arlo.speer@tc.gc.ca

Suggestions for amendment to this document are invited, and should be submitted via:
AARTinfodoc@tc.gc.ca

Original signed by Arlo Speer for

Aaron McCrorie
Director, Standards
Civil Aviation
Transport Canada

APPENDIX A – PILOT EXAMINER MONITOR REPORT FORM

 

 

AC 408-001 - Pilot Examiner Monitoring Procedures - Aeroplane and Helicopter
(PDF, 554 KB)

 

Report a problem with this page

Date modified: