Evaluation of Transport Canada’s Program of Payments to Other Government or International Agencies for the Operation and Maintenance of Airports, Air Navigation, and Airways Facilities (Also known as the Den-Ice Agreements Program)

Executive Summary

This evaluation was required as per Section 42(1) of the Financial Administration Act. It addresses the relevance and performance of the Den-Ice Agreements Program, which provides a contribution to the International Civil Aviation Organization (ICAO) in support of the Denmark and Iceland Joint Financing Agreements (Den-Ice Agreements). The evaluation examined the period from 2009-10 to 2013-14. Given that the Program has been ongoing since 1956 and is of low materiality and low risk, a streamlined approach to the evaluation was adopted, updating key information from the previous evaluation using financial data and data in ICAO Working Papers.

The contribution to ICAO supports the delivery of meteorological and communication services by Denmark and Iceland. Canada is one of 23 Contracting States that are party to the Den-Ice Agreements. The share that each Contracting State pays to ICAO is based on the total crossings made by their civil aircraft. The amount that Transport Canada paid annually through the contribution to ICAO has increased from an annual average of $37,500 between 2005-06 and 2008-09 to an annual average of $41,885 between 2009-10 and 2013-14. The increase in annual funding is attributed to the growth in Canadian air traffic in the North Atlantic region covered by the Agreements.

The evaluation found the Program to be relevant, fulfilling a continuing need. Without the contribution to ICAO in support of the Den-Ice Agreements, Canada would be removed from international discussions which inform Contracting States of emerging aviation safety issues in the North Atlantic region. Decisions made during the discussions would affect Canadian aircraft flying across this region whether Canada chooses to participate in the Agreements or not. For Canada to unilaterally withdraw from the Agreements could also reflect negatively on Canada’s engagement efforts in international aviation.

The evaluation also found that the Program achieved its expected outcomes; there have been no fatal accidents in the North Atlantic region covered by the Den-Ice Agreements during the 2009-10 to 2013-14 time period. Aviation safety has therefore remained at a high level in this region, despite an increase in air traffic.

The evaluation found that the projected number of Canadian crossings in the North Atlantic will continue to grow, along with the share that Canada pays to ICAO in support of the Den-Ice Agreements. In order to forecast Canada’s share as a Contracting State for upcoming years and improve program management, it is recommended that Transport Canada track Canada’s annual crossings in the North Atlantic and share of costs. This information is available to the Program in the annual ICAO Working Papers.

{ATIP REMOVED}

Introduction

Transport Canada’s Evaluation and Advisory Services conducted an evaluation of the Den-Ice Agreements Program in the summer of 2015. As per Section 42(1) of the Financial Administration Act, transfer payment programs must be evaluated every five years. The Program was last evaluated in 2009, and its Terms and Conditions are set to expire in March 2016. This evaluation examined the period from 2009-10 to 2013-14.

Background

The Den-Ice Agreements were established in 1956 to cover the operation and financing of facilities and services provided by Denmark and Iceland for civil aircraft flying across the North Atlantic (north of the 45^o N latitude). Facilities are located in Greenland, Iceland and the Faroe Islands. The Den-Ice Agreements have evolved over time and Canada has been a signatory since 1956. While the contribution is provided to ICAO as the administrator of the Den-Ice Agreements, the signatories are foreign governments.

Canada is one of 23 Contracting States that are party to the Den-Ice Agreements. Together, Contracting States finance 9.5% of the total costs of providing the services covered in the Agreements. Contracting States are responsible for financing meteorological services (i.e. surface and upper air synoptic observations) and communication services at Prins Christian Sund (i.e. a radio transmitter that provides aviation navigation services on a continuous basis at a specific location). Although Contracting States finance together 9.5% of the total costs to deliver the services, the amount that each Contracting State contributes is determined by the number of crossings made by their civil aircraft in the North Atlantic region.

Program Profile

During the 2009-10 to 2013-14 time period, Transport Canada paid an average annual amount of approximately $41,885 to ICAO to support the delivery of services covered by Contracting States in the Den-Ice Agreements. By comparison, Transport Canada contributed an average annual amount of $37,500 during the 2005-06 to 2008-09 time period. The increase in average annual funding can be attributed to the growth in Canadian air traffic across the North Atlantic region covered by the Den-Ice Agreements. Transport Canada’s contributions to ICAO in support of the Den-Ice Agreements, as indicated in the Public Accounts of Canada, are presented by Fiscal year in Table 1^{Footnote 1}.

Table 1: Transfer payments to ICAO for the operation and maintenance of airports, air navigation and airways facilities during the 2009-10 to 2013-14 time period^{Footnote 2}

Fiscal Year	2009-2010	2010-2011	2011-2012	2012-2013	2013-2014
Contribution ($)	53,296	30,193	32,402	48,697	44,835

The Program was renewed in 2011-12. During the renewal, no changes were made to the services covered by the Contracting States in the Den-Ice Agreements.

Evaluation Approach

Given the Program’s low materiality and low complexity of delivery, the evaluators adopted a streamlined evaluation approach^{Footnote 3}, relying primarily on a review of relevant documents.

The evaluation assessed the relevance/rationale and performance of the Program, as well as whether economy could be demonstrated in the utilization of financial resources. Performance was assessed based on the extent to which the Program achieved its expected outcomes, as outlined in its performance measurement strategy. The logic model for the Program is presented in Table 2 and was adapted by evaluators to include the immediate and intermediate outcomes stated in the performance measurement strategy.

Table 2: Logic Model for the Den-Ice Agreements Program

Activities	Outputs	Immediate Outcomes	Intermediate Outcomes	Ultimate Outcomes
Prepare Funding Agreement Attend ICAO Joint Support Committee Meetings Review Status Reports and ICAO Audits	Funding Agreement	International collaboration on funding service delivery Delivery of international meteorological and communication services	Demonstration of Canada’s engagement in international aviation Historical information is available for meteorological modeling and forecasting	A safe aviation system

The research questions below were used to support the evaluation.

Relevance/Rationale

1. 1. Is there a continued need for the Program?
2. 2. Does the Program have a clear rationale?
3. 3. Is the Program aligned with federal roles and responsibilities?
4. 4. Is the Program aligned with Transport Canada strategic outcomes?

Achievement of Expected Outcomes

1. 5. Has Canada fulfilled its financial obligations to the Den-Ice Agreements?
2. 6. Have funds been made available on a timely basis to support the Den-Ice Agreements?
3. 7. Have services been provided according to the requirements of the Den-Ice Agreements?
4. 8. Has Canada demonstrated its engagement in international aviation?
5. 9. Is historical information available for modeling and forecasting?
6. 10. Has aviation safety been maintained or improved in the North Atlantic region covered by the Den-Ice Agreements?

Economy

1. 11. Does the Program demonstrate economy in the use of financial resources?

Lines of Inquiry

The evaluation used the following lines of inquiry to answer the research questions and inform the overall evaluation:

Document Review

Documentary evidence served as the primary source of information for this evaluation and was used to assess the relevance and performance of the Program.

Key documents include Speeches from the Throne, federal budgets, and departmental reports on performance. Transport Canada’s financial records, invoices from ICAO, ICAO’s Den-Ice Agreements, and Transport Canada’s Terms and Conditions for Transfer Payments to ICAO were reviewed to assess outputs and economy in the use of resources, and to confirm the fulfillment of Transport Canada’s obligations to the Den-Ice Agreements.

In addition, ICAO Working Papers were reviewed and analyzed to understand trends in Canadian air traffic in the North Atlantic (i.e. number of crossings made by Canadian aircraft).

Counterfactual Analysis

Counterfactual analysis was used to assess the rationale and ongoing need for the Program and to assess the impact if Canada were to withdraw from the Den-Ice Agreements.

Evaluation Findings

Relevance/Rationale

Between 2009 and 2013, the growth in Canadian crossings in the North Atlantic region covered by the Den-Ice Agreements was 21.5% (from 20,991 to 25,500 crossings). Flying across the North Atlantic to Europe is considered to be a full crossing. A crossing between Greenland and Canada is counted as one-third of a crossing, while a crossing between Iceland and Canada is counted as two-thirds of a crossing. As Table 3 demonstrates, flying across the North Atlantic has generally been increasing, as indicated by total crossings by Contracting States, as has Canada’s percentage of total crossings.

Table 3: Canadian air traffic crossing the North Atlantic region covered by the Den-Ice Agreements during the 2009-10 to 2013-14 time period^{Footnote 4}

Fiscal Year	2009-2010	2010-2011	2011-2012	2012-2013	2013-2014
Total Canadian Crossings	20,991	23,052	25,826	26,793	25,500
Total crossings by Contracting States	331,212	331,229	348,480	336,875	336,959
Proportion of Canadian Crossings as a Percentage of Total Crossings (%)	6.34	6.96	7.41	7.95	7.57

With increasing Canadian air traffic in the North Atlantic, it is in Canada’s best interest to continue to support the delivery of the meteorological and communication services outlined in the Den-Ice Agreements.

Canada has been the host country for ICAO since the organization was founded in 1947 and ICAO plays a key role in the governance of civil aviation in Canada and world-wide. Canada’s Permanent Mission to ICAO provides oversight of the Den-Ice Agreements. Transport Canada attends the annual Den-Ice Joint Support Committee meetings and as such demonstrates its engagement in international aviation.

Participation in these meetings enables Transport Canada to remain informed of aviation safety issues in the North Atlantic and helps ensure the safety of Canadians traveling in this region. If Canada were to withdraw from the Agreements, Canada would no longer be part of the discussions that would affect its aircraft in the North Atlantic, and more importantly, such actions could impact Canada’s reputation as an important player in international aviation decision-making.

Aviation safety is a federal responsibility in Canada, as is the responsibility for entering into multilateral international agreements. ICAO plays a key role in the governance of civil aviation in Canada.

According to Transport Canada’s Program Alignment Architecture, the Program is aligned with Strategic Outcome 3: ‘A Safe and Secure Transportation System’. Specifically, it is part of Program Sub-Activity 3.1.1, ‘Aviation Safety Regulatory Framework’, which promotes a safe and harmonized aviation safety framework for Canadians and Canada’s aviation industry.

Performance

In terms of performance, as measured according to the Program’s performance measurement strategy, the evaluation found the following:

Invoices from ICAO indicate that Canada has fulfilled its obligations as a party to the Den-Ice Agreements for the 2009-10 to 2013-14 time period through the provision of annual funds to support the delivery of meteorological and communication services in the North Atlantic. Annual ICAO Working Papers submitted to Transport Canada indicate that Denmark and Iceland have provided the meteorological and communication services as stated in the Agreements for the time period covered by the evaluation.

On January 1^st of each calendar year, each Contracting State receives a statement indicating its assessed amount owing for the current calendar year. Invoices indicate that Transport Canada’s payments were received annually by ICAO between January 1^st and March 31^st, for the 2009-10 to 2013-14 time period. Payments to ICAO are made twice annually, the first in support of the Denmark Agreement and the second in support of the Icelandic Agreement.

As per the Program’s performance measurement strategy, ICAO Working Papers prepared for the Den-Ice Joint Support Committee are to provide Transport Canada with historical information to enable the department to undertake meteorological modeling and forecasting. However, the performance measurement strategy does not require Transport Canada to track the information found in the Working Papers, such as Canada’s air traffic data and share of costs. This information could be used by Transport Canada to model and forecast future costs to deliver these services.

Tracking this information would also provide a comparison between the amounts reported by ICAO and the amounts paid to ICAO by Transport Canada, and would allow for the identification and correction of discrepancies as they arise, as well as improved program management.

Working Papers indicate that aviation safety has remained high in the areas covered by the Den-Ice Agreements during the 2009-10 to 2013-14 time period; despite increased air traffic levels, there have been no fatal accidents in the North Atlantic airspace.

Economy

In terms of economy^{Footnote 5}, evaluators found the following:

The Terms and Conditions do not state what the annual amount for the contribution to ICAO should be. The performance measurement strategy states that the annual contribution paid by Transport Canada to support the delivery of the services in the Agreements can vary between a total of $50,000 and $70,000. The Program’s expenditure data indicates the contribution amounts have varied between approximately $30,000 and $53,000 over the time period covered by the evaluation.

According to the performance measurement strategy for the Program, which was last revised in 2011, the annual budget for the Program covers salary and performance monitoring costs as well as the contribution funding, as indicated in Table 4.

Table 4: Expected departmental resources per fiscal year, as indicated in the performance measurement strategy for the Den-Ice Agreements Program

Fiscal Year	2010-2011	2011-2012	2012-2013	2013-2014
Salaries ($)	14,000	6,400	6,400	6,400
Transfer Payments (contribution to ICAO) ($)	70,000	70,000	70,000	70,000
Performance Monitoring Costs ($)	14,000	6,400	6,400	14,000
Total Projected Program Costs ($)	84,000	76,400	76,400	76,400

However, the evaluation found that costs for salaries and performance monitoring are not outlined in the Terms and Conditions for the contribution to ICAO. The actual cost of the Program, as reported in the Public Accounts of Canada, has been equal to the contribution to ICAO in support of the Agreements, reconfirming that the Transfer Payment is the Program’s only cost.

The Program indicated that it received $100,000 per year between 2011-12 and 2013-14, of which a maximum of $70,000 is reserved for the contribution to ICAO. Combined with zero expenditures on salaries and performance monitoring, there has been a surplus each year since 2009-10 (see Table 5). The surplus is returned to Transport Canada where it can be redistributed to other grant and contribution programs. The Program has consistently not been spending all of its funds, and as such, cannot demonstrate economy in the use of financial resources.

Table 5: Comparison of actual costs and annual budget for the Den-Ice Agreements Program, as indicated in the Public Accounts of Canada for each fiscal year

Fiscal Year	2009-2010	2010-2011	2011-2012	2012-2013	2013-2014
Annual budget for the Program ($)	55,000	50,000	100,000	100,000	100,000
Actual costs ($)	53,296	30,194	32,402	48,697	44,835
Surplus from the Program ($)	1,704	19,806	67,598	51,303	55,165

In support of the economical use of resources, the Program would have to estimate how much it realistically needs and adjust the amount it receives, accordingly. To estimate the contribution portion, a forecasting of costs could be undertaken by tracking the information contained in the ICAO Working Papers, as already discussed.

Conclusion

The contribution funding helps ensure that meteorological and communication services are delivered by Denmark and Iceland in the North Atlantic region. Due to the continuing growth in Canadian air traffic in the North Atlantic, Canada’s contribution to ICAO to support the delivery of these services in the Agreements has been slowly increasing. The projected growth in Canadian air traffic in this region suggests that there will be a continuing need for the Program.

The contribution to ICAO also enables Canada to engage in international aviation discussions, participate at the annual Den-Ice Joint Support Committee meetings and thereby be involved in decisions that affect Canadian aircraft flying over the North Atlantic. More importantly, withdrawing from the Agreements would affect Canada’s reputation as a key player in international aviation.

In terms of performance, aviation safety has remained high in the North Atlantic during the 2009-10 to 2013-14 time period. However, the evaluation found that there are inconsistencies between the Program’s Terms and Conditions, its performance measurement strategy and its expenditures with regard to the allocation of monies to the Program and how the money is to be spent. Their alignment would add clarity to program funding. More importantly, the evaluation found that a sizeable amount of the funds allocated to the Program are not spent each year.

The evaluation also found that the Program is not making use of the information it obtains in ICAO Working Papers (i.e. Canadian air traffic and financial data). This information would allow for the verification of the accuracy of ICAO claims and the forecast of future costs and air traffic growth. The tracking of this information could therefore improve program management and monitoring.

Recommendations

The evaluation team proposes the following two recommendations to improve program management and monitoring.

Management Action Plan

Recommendation 1
Transport Canada should consider tracking annual air traffic data and Canada’s share of costs so that Transport Canada can verify the accuracy of ICAO billing, and forecast air traffic growth and future costs.

Transport Canada Civil Aviation Response to Recommendation 1

Transport Canada Civil Aviation previously responded to this recommendation indicating that it will explore the availability of the specific aircraft movement data and assess the complexity of undertaking this exercise in relation to the amount of Canada’s small expenditure and reasonably stable relative share. This response was based on the reasoning that to address this recommendation; the program would need to obtain third party sources of traffic data.

Further to feedback received from Evaluation and Advisory Services which provided clarification with respect to this recommendation, Transport Canada Civil Aviation now agrees with the recommendation and will chart the information referenced in the recommendation on an ongoing basis using data published by ICAO.

1 Recommendation	2 Proposed Action	3 Forecast Completion Date	OPI
Transport Canada should consider tracking annual air traffic data and Canada’s share of costs so that Transport Canada can verify the accuracy of ICAO billing, and forecast air traffic growth and future costs.	Further to a reply received from Evaluation and Advisory Services which clarified this recommendation, the program has revised its original response as follows: Transport Canada Civil Aviation agrees with the recommendation. In order to verify that the amounts ICAO invoices Transport Canada are correct and in line with the Agreement, Transport Canada Civil Aviation will input into a spreadsheet on an annual basis, the number of Canadian aircraft crossings and the corresponding financial figures using data published by ICAO in Working Papers.	February 2017	Safety & Security

Recommendation 2:
{ATIP REMOVED}

4 Recommendation	5 Proposed Action	6 Forecast Completion Date	OPI
{ATIP REMOVED}	{ATIP REMOVED}	{ATIP REMOVED}	{ATIP REMOVED}