Advisory Circular (AC) No. 408-002

Subject: Limiting the Number of “2” Assessments on Part IV Flight Tests

Issuing Office:	Civil Aviation, Standards
Document No.:	AC 408-002
File No.:	Z 5000-34
Issue No.:	01
RDIMS No.:	15097126-v6
Effective Date:	2019-09-25

Table of contents

• 1.0 Introduction
  • 1.1 Purpose
  • 1.2 Applicability
  • 1.3 Description of changes
• 2.0 References and requirements
  • 2.1 Reference documents
  • 2.2 Cancelled documents
  • 2.3 Definitions and abbreviations
• 3.0 Background
• 4.0 Analysis of Flight Test Data
• 5.0 New Evaluation Policy
• 6.0 Document history
• 7.0 Contact

1.0 Introduction

• (1) This Advisory Circular (AC) was originally scheduled for an effective date of September 1st, 2019, but was withheld to coordinate with the necessary Program changes in the Flight Training and Aviation Education (FTAE) software to enable the new business rules for the acceptance of flight test reports in accordance with the new policy of this advisory circular. The new effective date is September 25th, 2019 in conjunction with Release 3.3.1 of FTAE.

1.1 Purpose

• (1) The purpose of this document is to introduce a policy to limit the total number of combined “1” and “2” evaluation on Part IV flight tests in order to enhance the competency of applicants for pilot licences and ratings.

1.2 Applicability

• (1) This document applies to all Transport Canada Civil Aviation (TCCA) employees, to individuals and organizations when they are exercising privileges granted to them under an External Ministerial Delegation of Authority. This information is also available to the aviation industry for information purposes.

1.3 Description of changes

• (1) Not applicable.

2.0 References and requirements

2.1 Reference documents

• (1) It is intended that the following reference materials be used in conjunction with this document:
  • (a) Aeronautics Act (R.S.C., 1985, c. A-2);
  • (b) Part IV, Subpart 408 of the Canadian Aviation Regulations (CARs) — Conduct of Flight Tests;
  • (c) Standard 428 of the CARs — Conduct of Flight Tests;
  • (d) Advisory Circular 408-001 — Pilot Examiner Monitoring Procedures — Aeroplane and Helicopter;
  • (e) Staff Instruction SUR-020 — Pilot Examiner Monitoring Procedures;
  • (f) Transport Canada Publication (TP) 14277 — Pilot Examiner Manual.

2.2 Cancelled documents

• (1) Not applicable.

2.3 Definitions and abbreviations

• (1) The following abbreviations and definitions are used in this document:
  • (a) CAR: Canadian Aviation Regulation;
  • (b) CPL (A): Commercial Pilot Licence — Aeroplane;
  • (c) FTU: Flight Training Unit;
  • (d) PPL (A): Private Pilot Licence — Aeroplane.

3.0 Background

• (1) The Minister of Transport for Canada has a mandate under the Aeronautics Act to ensure civil aviation safety for Canadians.
• (2) The Minister of Transport for Canada has the responsibility to provide the programs and oversight to ensure that all pilots licensed in Canada are as competent as can be reasonably expected by the public, the domestic aviation industry and the foreign sponsors of trainees and their Civil Aviation Authorities.
• (3) The Canadian pilot skill standard is one of the highest amongst the major Civil Aviation Authorities in the world. Each flight test item, task or manoeuvre, is assessed individually based on the published “Aim”, “Description” and prescriptive “Performance Criteria”. The evaluation is objectively driven with a minimum of subjectivity.
• (4) Present Transport Canada (TC) management involved with pilot training and licensing and some of the foreign Civil Aviation Authorities are questioning the veracity and quality of flight training in Canada, revealed by the marginal competence of some Canadian flight training and licensing graduates when they perform flight duties upon their return to their native country. These foreign authorities are now contemplating alternative countries for the training of their airline cadets.
• (5) At the root of their concern is a safety issue that has been determined through flight test data analyses indicating that a significant number of flight test candidates obtained licences and ratings despite having been evaluated and awarded with a “2” on a large number of the flight test items, indicating major errors or deviations. This situation has existed for decades, whether with the present 4-Point Marking Scale or the previous 6-level (0 to 5) marking scale but present TC management has expressed a new willingness to address the issue. The present 4-Point Marking Scale, which is based on the evaluation method used by airline Advanced Qualification Programs (AQP) using adult education principles is designed to determine the level of performance relative to an “Ideal performance under existing conditions” versus the number of minor or major errors and unacceptable critical errors. This evaluation matrix accepts all levels of skill performance except unacceptable performances on condition that the minimum percentage is achieved.
• (6) In order to better ensure an improved quality of training, competency and proficiency of our licencees, TC must undertake a change in evaluation policy that more closely mirrors the principles now being applied by policy to pilots of Canadian Air Operators (Part VII), limiting Captains to a maximum of two “2”s and First Officers to four “2”s on the Pilot Proficiency Checks and to pilots working for Private Operators (Subpart 604) who are limited to a maximum of four “2”s on their Competency Check (Private Operator).
• (7) Evaluation of a score of “2” means:
  • (a) Performance is observed to include major errors:
  • (b) Aircraft handling is performed with major deviations and/or an occasional lack of stability, over/under control or abrupt control input;
  • (c) Technical skills reveal deficiencies either in depth of knowledge or comprehension of procedures, aircraft systems, limitations or performance characteristics that do not prevent the successful completion of the task;
  • (d) Situational awareness appears compromised as cues are missed or attended to late or the candidate takes more time than ideal to incorporate cues or changes into the operational plan;
  • (e) Flight management skills are not consistent. Instrument displays, aircraft warnings or automation serve to avert an undesired aircraft state by prompting or remedying threats and errors that are noticed late;
  • (f) Safety margins are not compromised, but poorly managed.

4.0 Analysis of Flight Test Data

• (1) Data indicating successful PPL (A) flight tests with occurrences of “2s”are illustrated in the graph below. No action is generally undertaken to address the items assessed with major error or deviation, as these are deemed to be “Passed, successful attempts”.

  Successful PPL Flight Tests – National

  

  Successful PPL Flight Tests – National – Text version

  From August 22, 2017 to August 22, 2018

  Number of tests	Number of 2s / test
  331	1
  344	2
  256	3
  243	4
  147	5
  96	6
  69	7
  31	8
  19	9
  14	10
  4	11
  2	12

   
• (2) Flight test data analyses also indicate that a high percentage of the flight tests that are assessed with one or two failed items include a significant number of items assessed with a “2” indicating major errors or deviations. During the partial retests, only the previously failed items are retested. Generally, no supplementary training is undertaken to address the weak areas previously identified with a “2”.
• (3) Occurrences of “1s” and “2s” on Failed Flight Tests (At least one “1”): (Reports without 2s were not counted.)

  PPL Flight Tests Eligible for Partial Retest – National

  

  PPL Flight Tests Eligible for Partial Retest – National – Text version

  From August 22, 2017 to August 22, 2018

  Number of tests	Number of 2s and 1s / test
  28	1
  62	2
  62	3
  49	4
  51	5
  50	6
  37	7
  26	8
  18	9
  10	10
  8	11
  7	12
  1	13
  3	14

   

5.0 New Evaluation Policy

• (1) Since section 408.18 of the CARs does not address the occurrence of “2” evaluations on Part IV flight tests, a policy to resolve the issue and introduce a new consequence for poor skill demonstrations with an excessive number of “2”s during formal flight tests is required.
• (2) In addition to the circumstances stipulated in section 408.18 of the CARs in regard to flight test failures, if during a flight test a candidate is evaluated a combined number of “1s” and/or “2s” that exceeds the maximum number in the table below for the flight test being undertaken, the examiner will terminate the flight test and a complete retest will be required after supplementary training to address the weak areas identified. The number of “2s” allowed is based on the number of items on a flight test to bring the equivalency to 70% or better.

  Passenger-carrying Rating Ultra-light Pilot Permit	Recreational Pilot Permit (A)	Private Pilot Licence (A)	Commercial Pilot Licence (A)	Private Pilot Licence (H)	Commercial Pilot Licence (H)	Multi-engine Class Rating	Instrument Rating and Instrument Proficiency Check
  4	6	5	4	6	5	4	3

• (3) If the proposed policy had been in effect between August 22, 2017 and August 22, 2018, the total number of flight tests requiring a complete retest, in addition to that required by the existing provisions of 408.18, would be 636 complete retests out of 3959 PPL (A) and CPL (A) flight tests or 16.1%.

6.0 Document history

• (1) Not applicable.

7.0 Contact

For more information, please contact:

Standards Branch, Civil Aviation – Safety and Security
Pilot Training and Licensing – AARTFE
Telephone: 613-991-5667
E-mail: wayne.chapin@tc.gc.ca

We invite suggestions for amendment to this document. Submit your comments to:
Civil Aviation Communications Centre
Telephone: 1-800-305-2059
Civil Aviation Communications Centre contact form

Document approved by

Robert Sincennes
Director, Standards
Civil Aviation, Transport Canada