Audit of Special Flight Operations Certificate Processes Related to Special Aviation Events - Air Shows

From: Transport Canada

November 2015

 

Audit of Special Flight Operations Certificate Processes Related to Special Aviation Events - Air Shows
(PDF, 270 KB)

 

EXECUTIVE SUMMARY

INTRODUCTION

Transport Canada establishes regulations and standards of safety for the protection of the general public during Special Aviation Events. Individuals and companies must apply for a Special Flight Operations Certificate in order to carry out aviation events such as air shows, balloon festivals, aerobatic competitions, low-level air races, and fly-ins. On average, Transport Canada issued 49 Special Flight Operations Certificates annually over the two-year period between April 2013 and March 2015 for the conduct of air shows. The Department is responsible for ensuring the consistent interpretation of regulations for the issuance of a certificate and for the monitoring and enforcement of the conditions contained in the certificate.

AUDIT OBJECTIVE AND SCOPE

The Audit of Special Flight Operations Certificate Processes Related to Special Aviation Events – Air Shows was included in Transport Canada's 2015/16 to 2017/18 Risk-Based Audit Plan. The audit objective was to review the adequacy and effectiveness of the standard operating procedures, to assess compliance with this functional guidance and to ensure that a review process is in place for Special Flight Operations Certificates related to special aviation events – air shows.

We examined the processes in place for the granting of Special Flight Operations Certificates under the

  • Canadian Aviation Regulations 603
    • Part VI – General Operating and Flight Rules
      • Subpart 3 – Special Flight Operations
        • Division I – Special Aviation Events
          • Certification Requirements for Special Aviation Events
  • Special Flight Operations Standard 623 – Special Flight Operations
    • Division I – Special Aviation Events
      • Chapter 1 – Air Shows

We also examined the processes in place for the monitoring and enforcement of the certificates' conditions.

CONCLUSION

We found that the activities related to the issuance of Special Flight Operations Certificates for air shows and the enforcement of their requirements are adequate and effective. Inspectors stated the Canadian Aviation Regulations and Special Flight Operations Standards are straightforward and, for the most part, provide sufficient guidance to assess applications for the issuance of Special Flight Operations Certificates for air shows. There are mechanisms in place to seek clarification of the regulations and the standards. Air shows are being monitored and, for the period we examined, no enforcement actions have resulted from any investigation of air shows.

We did find however two instances where procedures were not followed according to the staff instruction. The first instance refers to an inspector signing certificates without delegated authority. The second instance has to do with the majority of inspectors not following their own staff instruction when it comes to documenting their work in RDIMS .

{ ATIP removed}

We are recommending that signing authorities are respected and that the issuance and monitoring of Special Flight Operations Certificates for air shows should be subject to periodic review to help ensure consistent application. And finally, that a risk-based approach to carry out inspections of Special Flight Operations Certificates for air shows should be finalized and implemented.

STATEMENT OF CONFORMANCE

This Audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of an external assessment of Internal Audit's Quality Assurance and Improvement Program.

Signatures

Signed

Dave Leach (CIA, MPA) Director, Audit and Advisory Services

 

2015-11

Date

 

Signed

Martin Rubenstein (CPA, CIA, CFE) Chief Audit and Evaluation Executive

 

2015-11

Date

 

1. INTRODUCTION

1.1 PURPOSE

The Audit of Special Flight Operations Certificate Processes Related to Special Aviation Events – Air Shows was included in Transport Canada's 2015/16 to 2017/18 Risk-Based Audit Plan. The audit was to review the adequacy and effectiveness of the standard operating procedures, to assess compliance with this functional guidance and to ensure that a review process for Special Flight Operations Certificates related to air showsFootnote 1 is in place.

1.2 BACKGROUND

Transport Canada's Safety and Security group is responsible for advancing the safety and security of Canada's transportation system through various programs. The Civil Aviation program promotes the safety of the national air transportation system through its regulatory framework and oversight activities. As part of the regulatory framework, Civil Aviation develops policies, guidelines, regulations, standards, and educational materials to advance civil aviation in Canada. As part of the oversight activities, Civil Aviation verifies that the aviation industry complies with the regulatory framework through service activities to the aviation industry (e.g. licences and certifications) and surveillance activities (e.g. assessments, validations, inspections, and enforcement).

The Civil Aviation program is delivered through the Civil Aviation Directorate at Headquarters and through the five regions.

The Aeronautics Act governs civil aviation in Canada. One of the main goals of the Act is to bring into force the Canadian Aviation Regulations. Transport Canada is responsible for administering the regulations on behalf of the Government of Canada. The Canadian Aviation Regulations are a compilation of regulatory requirements designed to enhance safety and the competitiveness of the Canadian aviation industry and include such items as definitions, general administrative and compliance provisions, regulatory authorities and fees for services provided by the Department. In addition, the regulations contain general rules applicable to all non-military aircraft operations, including regulations respecting special types of operations such as air shows, parachuting, and balloon operations. Section 603Footnote 2 of the regulations stipulates the need and requirements for Special Flight Operations Certificates for several types of aviation activities such as an air showFootnote 3, a low level air race, an aerobatic competition, a fly-inFootnote 4, and a balloon festival. In support of the regulations, the Special Flight Operations Standards outline the requirements for complying with the regulations for special flight operations. These standards are separated into a number of Divisions and Chapters covering various aspects of special flight operations and their requirements

  • Division I – Special Aviation Events
    • Chapter 1 – Air shows
    • Chapter 2 – Balloon festivals
    • Chapter 3 – Aerobatic competitions
    • Chapter 4 – Low level air races
    • Chapter 5 – Fly-ins
  • Division II – Balloons with fare-paying passengers
  • Division III – Parachute descents
  • Division IV – Miscellaneous special flight operations such as take offs, approaches and landings within built-up areas and unmanned air vehicles

In order to issue a Special Flight Operations Certificate, an application must be submitted to the appropriate Regional Civil Aviation office sixty (60) days prior to the proposed date of the event to allow sufficient time to process the application. Additional information in support of the application must be received no later than ten (10) working days prior to the date of the event. Most regions forward applications to a central email where the work is then distributed to the appropriate Technical Team Lead who assigns it to an inspector.

Inspectors in each region review the application on a case-by-case basis to assess it against the requirements of the Canadian Aviation Regulations and the Special Flight Operations Standards. When the inspector determines that the requirements of the regulations and standards have been met, and that any identified risks will be managed to an acceptable level, a Special Flight Operations Certificate is prepared for the signature of the Regional Technical Team Lead for Flight Operations or the Associate Director of Operations, Civil Aviation. Once the Special Flight Operations Certificate is issued for an event, inspectors may monitor the event for compliance with the terms of the Special Flight Operations Certificate.

Where a contravention is identified and the contravention is considered minor, the inspector may simply provide oral counsel to the certificate holder, thus providing immediate guidance on the need for future compliance. The decision to conclude the incident with oral counselling or to refer the matter to the Enforcement Unit for further investigation is entirely at the discretion of the inspector.

If the contravention is of a more serious nature, the inspector must immediately refer the incident to the appropriate Regional Manager, Enforcement Unit. To refer an incident to the Regional Manager, Enforcement Unit, the inspector provides a Detection Notice together with any notes or evidence collected to date such as photographs, logs, and tapes. Upon completing an investigation, the Enforcement Unit can take a variety of actions depending on the severity of the contravention: For less severe contraventions, it can provide oral counseling and issue warnings and notices; for moderately severe contraventions it can issue fines and detentions, and for severe contraventions it can pursue judicial prosecution.

The Detection Notice is only one trigger to start an investigation. Others include complaints from the public, police reports, and a review of daily incident reports. Regardless of the trigger, once a potential issue is identified, the Enforcement Unit is responsible for investigating the issue and determining the appropriate course of action.

1.3 AUDIT OBJECTIVE, SCOPE, APPROACH AND CRITERIA

Audit Objective

The objective of the audit was to review the processes regarding the issuance, monitoring, and enforcement of Special Flight Operations Certificates (SFOC) related to special aviation events - air shows. Specifically, the audit assessed whether the Department has the following in place:

  • a process to ensure the consistent issuance of SFOCs to special aviation events - air shows.
  • a risk-based inspection plan to ensure special aviation events - air shows comply with the terms of their SFOC .
  • a process to deal with issues of non-compliance related to SFOCs for special aviation events - air shows.

Audit Scope

This audit specifically focused on Transport Canada activities as they relate to the

  • Canadian Aviation Regulations 603
    • Part VI – General Operating and Flight Rules
      • Subpart 3 – Special Flight Operations
        • Division I – Special Aviation Events
          • Certification Requirements for Special Aviation Events and
  • Special Flight Operations Standard 623 – Special Flight Operations
    • Division I – Special Aviation Events
      • Chapter 1 – Air Shows.

Audit Approach

The audit team reviewed documents related to the processes and procedures surrounding the issuance and enforcement of Special Flight Operations Certificates for air shows. Interviews were conducted both at Headquarters and in the regions. Finally, the audit team reviewed all air show applications and enforcement files between December 1, 2013, and November 30, 2014, Pursuant to the staff instructionFootnote 5 developed for the review and processing of an application, all documentation related to the issuance of a Special Flight Operations Certificate is required to be stored in the Department's electronic repository, the Records, Document, and Information Management System (RDIMS). As such, the review of application files focused on the electronic files that should be filed in RDIMS in accordance with the criteria found in the staff instruction.

Audit Criteria

To address the audit objective, we used the following audit criteria:

  1. There are adequate and effective standard operating procedures for issuing Special Flight Operations Certificates related to special aviation events – air shows.
    • The standard operating procedures for issuing Special Flight Operation Certificates for air shows are in place and are clear, concise, and communicated.
    • The standard operating procedures for issuing Special Flight Operation Certificates for air shows are followed and applied consistently throughout the regions. The quality of the inspectors' work is being reviewed.
    • There is a process to seek clarification on the interpretation of the Canadian Aviation Regulations and Special Flight Operations Standards related to special aviation events – air shows and this information is communicated throughout the regions.
  2. There is an adequate and effective monitoring program in place to ensure certificate holders are abiding by their Special Flight Operations Certificates for air shows.
    • There is a risk-based approach to monitoring Special Flight Operations Certificate compliance.
    • There are processes in place to deal with non-compliance.

1.4 REPORT STRUCTURE

For each of our criteria, we have included contextual information, what we expected to find and what we did find and, where appropriate, recommendations. The last section of the report contains management's action plan to address our audit recommendations.

2. FINDINGS AND RECOMMENDATIONS

2.1. ISSUANCE OF SPECIAL FLIGHT OPERATIONS CERTIFICATES

Context

The Canadian Aviation Regulations prohibit the conduct of special aviation events unless a Special Flight Operations Certificate is issued by Transport Canada and the provisions of a Special Flight Operations Certificate – Special Aviation Event are complied with. The regulations outline the restrictions, and spectator safety criteria to be considered when issuing a Special Flight Operations Certificate for the conduct of an air show. Under the regulations, an application must be submitted. Once the application is received and the applicant demonstrates the ability to conduct the event in accordance with the Special Flight Operations StandardsFootnote 6, the Minister issues the Special Flight Operations Certificate to conduct the event.

The requirement for a Special Flight Operations Certificate detailed in the Canadian Aviation Regulations only applies to civilian aircraft operations. Military aircraft operate under the authority of the Department of National Defence and therefore are not required to apply for a Special Flight Operations Certificate pursuant to the Canadian Aviation Regulations.

Between December 1, 2013, and November 30, 2014, Transport Canada issued 43 Special Flight Operations CertificatesFootnote 7.

Standard Operating Procedures

What We Expected

We expected that there would be standard operating procedures in place for the issuance of Special Flight Operations Certificates for air shows and that these are clear, concise, and communicated.

What We Found

There are standard operating procedures for the issuance of Special Flight Operations Certificates for air shows. The standard operating procedures are clear, concise and communicated.

"Standard operating procedures" is a generic term used throughout all of the modes in Transport Canada. In Civil Aviation, the more commonly used term is staff instructions. For the purpose of this audit report, we do not distinguish between standard operating procedures and staff instructions as both documents serve as a tool to assist employees in carrying out their work.

There is a staff instructionFootnote 8 in place that provides inspectors with the information, procedures, and guidelines necessary to process an application and prepare a Special Flight Operations Certificate - Special Aviation Event required by the Canadian Aviation Regulation 603.01 to conduct a Special Aviation Event - Air Show. The staff instruction makes reference to Canadian Aviation Regulations and well as the Special Flight Operations Standards. The staff instruction, which mirrors the regulations and the standards, clearly explains that inspectors must review the application to ensure the standards are met and can be adhered to by the applicant. The staff instruction clearly states that inspectors must bring deficiencies to the attention of the applicant. The staff instruction is communicated through the department's internet site.

In addition to the staff instruction, inspectors also base their work directly on the Canadian Aviation Regulations and the associated Special Flight Operations Standards, their knowledge of the industry, years of experience, and on the job training. Inspectors reported to the audit team that the requirements outlined in the Canadian Aviation Regulations and the Special Flight Operations Standards are clear.

Procedures for Issuing Special Flight Operations Certificates and Functional Review

What We Expected

We expected that the steps for issuing a Special Flight Operations Certificate are being followed and are applied consistently throughout the regions. We also expected that there would be a review of inspectors' quality of work.

What We Found

Some of the required steps for issuing a Special Flight Operations Certificate are not always being followed. { ATIP removed}

The audit team set out to review 100% of the air show event files but was not provided information for all 43 events reported by Civil Aviation to the International Council of Air Shows in December 2014. The audit team was only able to locate and review information recorded in RDIMS for 38 events.

Through our file review of 38 events based on the criteria outlined in the staff instruction we found that inspectors were following the steps for the issuance of Special Flight Operations Certificates, with the following exceptions:

  • One inspector signed Special Flight Operations Certificates on behalf of the Technical Team Lead, which contravenes the staff instructionFootnote 9 and the Transport Canada Civil Aviation Consolidated Record of Authorities.
  • Inspectors were not documenting complete air show information in accordance with the staff instruction.Footnote 10 Note that this does not mean that inspectors aren't doing their job or that the documents do not exist. It is possible that they do exist in hard copy somewhere, just not in RDIMS . All 38 events had at least one piece of documentation not found in RDIMS .
    1. For instance, some key documents such as pilot licenses, medical certificates, statement of aerobatic competency, aircraft registration, aircraft certificate of airworthiness, insurance, operating conditions, signed participants' statementFootnote 11 etc., were not found in RDIMS .
    2. Evidence of review by the inspector was not always evident. For example, some regions use the Distributed Air Personnel Licensing System (DAPLS) to verify Canadian pilots' credentials. However, there was no evidence on file to confirm that DAPLS had been used in those cases where a copy of a pilot's license was not on file.
  • Evidence of review was clearly demonstrated by one inspector who completed a series of checklists throughout the process and saved them in RDIMS . The use of these checklists was not found in any of the other regions.
  • In some regions, sensitive information was not being classified according to departmental record keeping practices. For example, the audit team found information in RDIMS classified as "Unprotected" that should have been classified as "Protected A" due to the personal information contained in the documents.
  • Although all regions made use of the Special Flight Operations Certificate form letter, some regions have either modified the standard letter or the Participant's Statement to stress the importance of having all participants sign the statement and return it to the department on the first working day following the event.
  • The monitoring of an air show was not documented.

The audit team learned that the functional authority does not assess inspectors' issuance or monitoring of Special Flight Operations Certificates for air shows. { ATIP removed}

Recommendation

  1. Headquarters should ensure that signing authorities are respected and that the issuance and monitoring of Special Flight Operations Certificates for air shows be periodically reviewed to help ensure the process is consistently applied and followed throughout the regions.

Clarification on the Interpretation of the Regulations

What We Expected

We expected mechanisms to be in place for inspectors to seek clarification regarding the interpretation of the regulations and that interpretations are shared throughout the regions.

What We Found

There are processes in place to seek clarification on the interpretation of the regulations.

While the staff instruction on air shows provides guidance and direction regarding processes, it does not provide definitive interpretations of all the various terms in the regulations or the standards. Each special event presents its own unique characteristics and it is unrealistic to have regulations and standards for every possible situation. { ATIP removed} regions and their inspectors must apply their own interpretation with the coordination of national guidance.

To assist them in clarifying the interpretation of the regulations, inspectors can refer to the Special Flight Operations Standards which contains Information Notes that provide additional clarification on how to meet the various requirements of the regulations.

When inspectors require additional help or guidance regarding interpretations, the normal process is for the inspector to consult their fellow inspectors, their supervisor, and finally the Standards Coordination Branch in the region which is considered the regional subject matter expert. If the Standards Coordination Branch cannot answer the request, they will seek additional clarification and guidance from the subject matter expert at Headquarters. The subject matter expert in Headquarters for Special Flight Operations Certificates disseminates any new guidance or clarifications to both regional inspectors currently involved in Special Flight Operations Certificates as well as the Standards Coordination Branch in each region. A body of knowledge and interpretations is being maintained both at Headquarters and in the regional Standards Coordination Branches to support responding to future queries.

2.2. MONITORING COMPLIANCE AGAINST SPECIAL FLIGHT OPERATIONS CERTIFICATE CONDITIONS

Context

A Special Flight Operations Certificate to conduct the Special Aviation Event – Air Show is issued under the authority of the Minister pursuant to the Aeronautics Act. It certifies that the certificate holder is adequately equipped and able to conduct a safe operation, subject to the observance and performance by the certificate holder of the conditions set out in the certificate.

If the requirements of the certificate are not followed, Civil Aviation's Enforcement Procedures Staff Instruction describes the procedures to be used by all Headquarters and regional enforcement personnel when authorized enforcement duties and responsibilities are to be performed.

Information regarding complaints and investigations for Special Flight Operations Certificates is captured electronically in the Department's Enforcement Management System.

Risk-Based Approach to Monitoring

What We Expected

We expected a national set of risk-based criteria would be used to help determine which air shows to monitor to ensure certificate holders are abiding by the conditions of their certificate.

What We Found

While special aviation events have been monitored,{ ATIP removed}. Headquarters is in the process of developing a risk-based monitoring tool.

Between December 1, 2013, and November 30, 2014, six air shows were monitored with each region monitoring at least one air show. Inspectors from each region choose to monitor special aviation event – air shows based on various criteria such as:
{ ATIP removed}

{ ATIP removed}Headquarters has recently developed draft national risk criteria for air shows and has solicited input from the regions. This tool would enable a region to review the anticipated air shows at the beginning of the season and then determine which may present the greatest risk and the results would be integrated into a comprehensive risk-based oversight plan. This tool is to be formalized in a new staff instruction dealing with monitoring. As this tool is currently a work in progress, we were not in a position to comment on the robustness of the new process.

Recommendation

  1. Headquarters should finalize and implement a formal risk-based approach to carrying out inspections of special aviation events – air shows.

Processes Dealing With Non-Compliance

What We Expected

We expected that there would be processes in place to deal with occurrences of non-compliance.

What We Found

There are processes in place to deal with non-compliance.

There are procedures in place to deal with situations of non-compliance and these are outlined in the Enforcement Procedures Staff Instruction. Generally, inspectors or the general public bring these situations to the attention of the Enforcement Unit via the Civil Aviation Issues Reporting System or through the Civil Aviation Daily Occurrence Reporting System. As previously described in the background section, Instances of non-compliance could result in oral counseling, monetary penalties, or suspensions. In addition, the Chief of Aviation Enforcement publishes monthly on Transport Canada's websiteFootnote 12 a list of corporate and non-corporate offenders that have been fined or suspended to serve as deterrent and to increase public awareness and education of aviation safety. The Regional Manager of Enforcement has the final decision on how to deal with a situation of non-compliance (e.g. laying a monetary penalty).

Of the 43 air shows reported that took place between December 1, 2013, and November 30, 2014, only one event was investigated and it was concluded that there was no violation.

The Chief of Aviation Enforcement in Headquarters reviews monthly the electronic investigation files located in the Enforcement Management System to ensure that monetary penalties for the same violations are dealt with consistently across the country. The Chief of Aviation Enforcement discusses cases with the Regional Managers of Enforcement on a regular basis and informs them of any other issues identified.

3. CONCLUSION

We found that the activities related to the issuance of Special Flight Operations Certificates for air shows and the enforcement of their requirements are adequate and effective. Inspectors stated the Canadian Aviation Regulations and Special Flight Operations Standards are straightforward and, for the most part, provide sufficient guidance to assess applications for the issuance of Special Flight Operations Certificates for air shows. There are mechanisms in place to seek clarification of the regulations and the standards. Air shows are being monitored and, for the period we examined, no enforcement actions have resulted from any investigation of air shows.

We did find however two instances where procedures were not followed according to the staff instruction. The first instance refers to an inspector signing certificates without delegated authority. The second instance has to do with the majority of inspectors not following their own staff instruction when it comes to documenting their work in RDIMS .

{ ATIP removed}

We are recommending that signing authorities are respected and that the issuance and monitoring of Special Flight Operations Certificates for air shows should be subject to periodic review to help ensure consistent application. And finally, that a risk-based approach to carry out inspections of Special Flight Operations Certificates for air shows should be finalized and implemented.

4. RECOMMENDATIONS AND MANAGEMENT ACTION PLAN

It is recommended that the Assistant Deputy Minister, Safety and Security should ensure that the following audit recommendations are addressed.

 

Recommendation

Management Action Plan

Completion Date
(for each action)

OPI direct report for each specific action

1.

Headquarters should ensure that signing authorities are respected and thatthe issuance and monitoring of Special Flight Operations Certificates for air shows be periodically reviewed to help ensure the process is consistently applied and followed throughout the regions.

Civil Aviation plans to implement internal procedures which will include components that address signing authorities, documentation, and reporting. This will be achieved as follows:

Civil Aviation is currently developing a Staff Instruction entitled SI SUR-023 Unscheduled Surveillance Activities. This Staff Instruction will indicate: how and when unscheduled surveillance activities take place (including air shows); provide details regarding documentation accountabilities (e.g. signing processes) and outline appropriate quality control and reporting processes. (December 2016 – approved and published)

Civil Aviation will then provide training to Regions (developed from the final staff instruction). (March 2017 – training delivered)

March 2017

Safety and Security

2.

Headquarters should finalize the development of a formal risk-based approach to carrying out inspections of special aviation events – air shows.

Civil Aviation will implement a quality control plan by April 2017.

Subsequently, QC verification will be integrated to Quarterly Surveillance reporting by August 2017. This will ensure the new formal process is being implemented consistently and its feedback is shared with the Regions in a structured process, as required.

August 2017

Safety and Security

Report a problem with this page

Date modified: