Evaluation of the Northern Transportation Adaptation Initiative

| Print Version |

Table of Contents

• Executive Summary
  • Evaluation Scope and Approach
  • Major Findings, Conclusions and Recommendations
• Program Profile
  • Background
  • Program Objectives
  • Program Activities and Expected Results
  • Program Management and Resources
• About the Evaluation
  • Evaluation Rationale and Scope
  • Evaluation Methodology
  • Evaluation Considerations
• Evaluation Findings: Relevance
  • Continuing Need
  • Alignment with Government Priorities and Departmental Strategic Outcomes
  • Alignment with Federal Roles and Responsibilities
• Evaluation Findings: Performance
  • Effectiveness – Achievement of Expected Outcomes
  • Efficiency and Economy
• Lessons Learned from Implementation
• Conclusions and Recommendations
• Management Action Plan

Executive Summary

The evaluation of Transport Canada’s Northern Transportation Adaptation Initiative (NTAI) was conducted to inform the renewal of adaptation programming beyond the current phase of the Government of Canada’s Clean Air Agenda, which expires in March 2016. The NTAI comprises Transport Canada’s contribution to the Clean Air Agenda’s Adaptation Theme, a set of ten initiatives at nine federal departments and agencies designed to help Canadians adapt to the effects of climate change.

The NTAI is a five-year (2011-12 to 2015-16) $11 million program that funds research and development (R&D) projects through grants, contributions and contracts, and manages two networks of expertise on adaptation of transportation infrastructure built on permafrost and northern marine transportation adaptation. The NTAI is a collaborative initiative that includes the participation of stakeholders in the territorial and provincial governments, academia, and the private sector in building capacity and developing effective adaptation practices in order to ensure a resilient northern transportation system.

The program is managed by the Strategic Policy and Innovation directorate of Transport Canada’s Policy group at National Headquarters, with the Quebec Regional Office managing an R&D project examining infrastructure adaptation at the Kuujjuaq airport in northern Quebec.

Evaluation Scope and Approach

The evaluation of the NTAI was conducted in the summer of 2014, three years after the NTAI was launched. Consequently, the evaluation focused on the effectiveness of program activities, achievement of short-term outcomes, and lessons learned from implementation. As per the Policy on Evaluation, the evaluation also examined program relevance and efficiency.

The evaluation was based on 20 interviews (with program management, staff, members of the networks, and funding recipients), a review of project files (n=37), a review of key documents, and observation of a network of expertise workshop.

Major Findings, Conclusions and Recommendations

The evaluation found that support for northern transportation adaptation continues to address an ongoing need, as all modes of the northern transportation system require adaptive measures in the face of climate change, and knowledge of effective adaptive practices remains limited.

The NTAI continues to be aligned with federal priorities and roles, as evidenced in the Northern Strategy, recent Speeches from the Throne and Budgets. The NTAI also aligns with federal adaptation roles as outlined in the Federal Adaptation Policy Framework (2011), and with the significant federal transportation investments in the North. Although currently situated within Transport Canada’s Program Alignment Architecture under two Strategic Outcomes, the objectives of the NTAI most directly support the departmental Strategic Outcome of “An Efficient Transportation System”.

In its first three years, the NTAI has made progress in achieving its short-term outcomes of building capacity, awareness and facilitating collaboration. The program has fully established two networks of expertise, which include stakeholders from the territories, provinces, universities and colleges, and, to a lesser extent, private sector. These networks have proven to be effective venues for building collaboration between territorial governments and researchers, including through developing research projects.

During the period covered by the evaluation, the NTAI funded 46 research and development projects. The projects, and related discussions and presentations through the networks, have increased awareness and understanding of the interplay of climate and infrastructure, scoped potential impacts of climate change on the northern transportation system, and identified specific vulnerabilities in the transportation system. New technologies and adaptive techniques such as methods of reducing permafrost degradation due to groundwater around highways, and reducing the impact of climate change on the surface friction of northern airport runways, are being tested.

A few early examples were identified of NTAI -funded research being taken into consideration in northern transportation infrastructure design and adaptation. These included research projects on permafrost degradation informing the Iqaluit Airport Improvement Project, and the construction of interceptor ditches along a portion of the Alaska Highway in the Yukon.

Spending of NTAI funding was slower than originally planned, due to the time required to build program delivery capacity/processes and to develop and approve project proposals. The amount of Other Operating Costs (OOC) budgeted for the initiative appears to have been too high, which resulted in a relatively large number of projects being funded through OOC , a few of which appear to be less directly valuable to achieving expected outcomes.

Lessons learned from the implementation of the initiative include:

• Formulating clear and specific R&D priorities is important to ensuring the initiative’s activities directly address the adaptation information needs of end-users. While the focus on adaptation of infrastructure built on permafrost was appropriately narrow, the initiative’s activities related to marine transportation adaptation were varied and broad in scope, which limited the ability of projects to build on each other, and may impede knowledge transfer.
• While the intent is for the networks of expertise to be self-sustaining in the future, the networks appear to require continuing support from Transport Canada to sustain their momentum.
• Maintaining the right balance between membership continuity and openness to new members is a key consideration in a successful network. The networks would benefit from new members to “refresh” themselves and maximize transparency and their impact in their respective fields.
• There were considerable barriers to developing an online information management and sharing tool (i.e., a web portal) for the networks of expertise. While initial planning was undertaken to develop a web portal for the networks, work on this ceased due to barriers related to privacy, data and information management/storage, ownership, and other issues.
• Considerable time was required to develop program administrative processes, including the project application guide, as Strategic Policy and Innovation staff needed to build capacity in setting up and administering a transfer payment program and the Centre of Expertise on Transfer Payments was developing new departmental templates.

Areas requiring further attention were identified in the evaluation. First, as the field of northern transportation adaptation has many players—including provinces and territories, universities and colleges, other federal organizations, and the private sector—Transport Canada should exercise its leadership in this area to work with other organizations to identify any areas of potential overlap, and opportunities for streamlining or partnering. To the extent possible, private sector involvement in future activities should be bolstered.

Second, while considerable information and data are being generated on monitoring, vulnerabilities, and potential climate impacts, there is a need for increased attention on testing of adaptation techniques in order to identify effective practices that can be implemented.

Third, given the considerable data and information that are becoming available through the initiative, there is a need to ensure that information is widely communicated to those who can make use of it, including those responsible for northern infrastructure planning and maintenance. Effective knowledge transfer will be key to ensuring the program can contribute to the achievement of longer-term outcomes related to the implementation of effective adaptation measures in existing and future infrastructure.

Finally, robust project- and program-level performance monitoring, including related to knowledge transfer and implementation of adaptation, will be important moving forward. At the time of the evaluation, project-level performance data collection was weak.

The evaluation includes the following recommendations:

Program Profile

Background

As part of Budget 2011, the Government of Canada announced an investment of $148.8 million over five years for federal adaptation programming. This programming comprised the second phase of the Adaptation Theme under the Government of Canada’s Clean Air Agenda, which was launched in 2007 to improve Canada’s environment and prepare for the challenges of climate change. The second phase of the Clean Air Agenda’s Adaptation Theme, which enhanced and broadened support for adaptation programming, consisted of 10 programs in nine departments and agencies including, for the first time, Transport Canada.

Transport Canada’s contribution to the Adaptation Theme was designed to build on and expand activities previously undertaken by the department relevant to the north. Due to requests from territorial governments, in 2008 the department had launched the Northern Transportation Research Initiative, with $1 million in funding from the department’s Gateways and Border Crossings Fund dedicated to research and development (R&D) related to northern transportation. Through this funding, the department had begun initial conceptualization and establishment of two networks of expertise in northern transportation and funded projects to support research and related projects. In addition, through the Quebec Regional Office, research had been undertaken at the Kuujjuaq airport to study the condition of the permafrost beneath the airstrip and its future stability in light of a warming climate.

Stakeholder consultations and initial network meetings identified the need for an increased level of support for R&D related to adapting the northern transportation system to the effects of climate change beyond the Northern Transportation Research Initiative. Furthermore, the Kuujjuaq airport was identified as a site well- suited to the testing of adaptation techniques given that previous studies had already been undertaken at this site. As a result, the Northern Transportation Adaptation Initiative (NTAI) was launched in 2011 to continue the work of the department in northern transportation adaptation.

Program Objectives

The objectives of the NTAI are to:

• improve the resilience, responsiveness and adaptability of Canada’s northern transportation system by incorporating climate change considerations into infrastructure design and maintenance, by mitigating future maintenance costs and losses in economic productivity, and by building northern science capacity; and
• work with stakeholders to develop strategic adaptation measures for northern transportation infrastructure to ensure a sustainable northern transportation system.

Program Activities and Expected Results

The NTAI includes two major activities:

• Managing two networks of expertise: the Network of Expertise on Transportation Infrastructure in Permafrost Regions (the Permafrost Network) and the Network of Expertise on Transportation in Arctic Waters (NEXTAW).
• Managing grants, contributions and contracts to fund research and development and related projects.

In the immediate term, the NTAI is expected to increase collaboration and capacity in climate change adaptation, build awareness of adaptation practices, and increase the availability of new and tested northern adaptation measures. This is expected to contribute to adaptation measures being implemented and climate change considerations being factored into northern transportation planning. Ultimately, in the long term, the NTAI is expected to contribute to a northern transportation system that is more resilient to climate change.

Program Management and Resources

The NTAI is managed by Innovation Policy in the Strategic Policy and Innovation directorate of Transport Canada’s Policy group, at National Headquarters. The Quebec Regional Office (Environmental Affairs group, Programs directorate) is responsible for managing the Kuujjuaq airport projects.

As it is the overall policy lead for adaptation within Transport Canada, the Environmental Policy directorate participates, with Strategic Policy and Innovation, on the (interdepartmental) Clean Air Agenda Adaptation Theme Renewal Director General Management Committee.

The NTAI was allocated $11.0 million over five years (2011-12 to 2015-16), including $4.1 million for grants and contributions, $5.5 million for Other Operating Costs (OOC), and $1.4 million for employee salaries, benefits, and accommodations.

About the Evaluation

Evaluation Rationale and Scope

The evaluation of the NTAI was conducted between June and September 2014 to inform the renewal of federal Adaptation programming beyond the Clean Air Agenda. The evaluation comprises Transport Canada’s contribution to the horizontal evaluation of the Clean Air Agenda’s Adaptation Theme being led by Environment Canada. The evaluation was also undertaken to identify lessons learned from the implementation of the NTAI to inform future Transport Canada programming.

To assess the initiative’s performance, the evaluation focused primarily on the effectiveness of program activities and achievement of short-term outcomes. The evaluation also sought to assess the extent to which longer-term outcomes are likely to be achieved given progress to date. In addition, as per the Policy on Evaluation, the evaluation examined program relevance and efficiency/economy.

Evaluation Methodology

The evaluation included the following lines of inquiry:

• Interviews were conducted with a sample of key program stakeholders, including program staff and managers, members of the networks and recipients of project funding, and members of the design team working on the Iqaluit Airport Improvement Project. A total of 20 interviews were conducted.
• A project file review was conducted to examine the expected and actual results achieved to date from funded NTAI projects, as well as any issues related to the progress of projects. Thirty-seven (37) project files were reviewed, which comprised all funded projects at the start of the evaluation. (The number of projects increased during the course of the evaluation to 46; financial information was available and incorporated into the evaluation for these nine additional projects.)
• A document review examined key foundational, background, contextual, and program operational documents. In addition to these documents, evaluators also reviewed the responses from a qualitative survey of Permafrost Network members, conducted by Innovation Policy in June 2014 with input from the evaluation team.
• The evaluation manager attended a Permafrost Network workshop on June 4^th to 6^th, 2014, in Dawson City, Yukon. Attending the workshop allowed the evaluation manager to learn about the network and its members, and view a series of presentations on NTAI -funded research projects. In addition, the workshop included a site-visit to locations along the Dempster Highway to examine permafrost conditions and research monitoring stations that received funding through the NTAI . Attendance at the workshop also allowed the evaluation manager to speak informally with network participants to obtain their perspectives on the network.

Evaluation Considerations

A number of considerations and limitations influenced the conduct of the evaluation.

• Most of the research and development funded by the NTAI was ongoing at the time of the evaluation, limiting the extent to which the evaluation could examine project results. Of the 46 contracts, grants and contributions funded by the NTAI by the end of the evaluation period, 28 were complete, representing 24% of the total value of committed project funds.
• While the program had identified performance measures for contribution projects, only one contribution was complete at the time of the evaluation and no performance data had been collected from this project. Interviews with project proponents helped to mitigate this limitation.
• Innovation projects can take time to realize longer-term impacts, including those related to applying new/tested techniques and technologies. As a result, the evaluation focused on short-term outcomes and the extent to which longer-term outcomes are likely to be achieved.

Evaluation Findings: Relevance

To assess the relevance of the NTAI , the evaluation examined the continuing need for the initiative, and alignment with federal government priorities, departmental strategic outcomes, and federal roles and responsibilities.

Continuing Need

Climate change in the North is having a significant impact on the transportation system and its infrastructure. Increases in average annual air temperature have begun to cause permafrost degradation, which is resulting in increased and more frequent occurrences of sinkholes, potholes, slumping and settlement issues in transportation infrastructure.^Footnote1 Changes to the conditions of permafrost have the potential to affect up to 7,000 km of all-weather roads in the territories, 2,000 km of winter roads in the Northwest Territories, over 60 airports and much of the Arctic marine system.^Footnote2

In addition to permafrost degradation, the marine transportation system is susceptible to rises in sea levels, increased storm events, stronger waves, changes to currents, and coastal erosion as a result of climate change. Milder Arctic summers have also sent potentially dangerous hard-packed ice into shipping lanes. At the same time, a warmer climate provides opportunities for increased marine transportation, as marine routes become more numerous and existing routes are open longer due to reduced sea ice.^Footnote3

Maintenance and replacement of the existing transportation infrastructure is a central priority for the North as much of the infrastructure is old and has reached its peak level of capacity.^Footnote4 Furthermore, significant expansion of the northern transportation system is ongoing and expected to continue.^Footnote5 Yukon and the Northwest Territories are extending the Dempster Highway from Inuvik to Tuktoyaktuk. The Government of Nunavut is leading a $300-million expansion and renovation of its primary airport in Iqaluit, and will expand its marine resupply system, among other projects.^Footnote6 Significant expansion of Quebec’s northern transportation system is also expected as part of the province’s plan for northern resource development.^Footnote7 Other projects across the North are in progress or under development to address a growing population and expected increases in resource development and travel.

While infrastructure replacement and expansion are priorities in the North, effective adaptation practices for northern transportation infrastructure are not well understood. Traditional transportation infrastructure planning, design, construction and maintenance techniques are not well adapted to permafrost conditions, and climate change is adding increasing complexity as permafrost and other conditions become less predictable.^Footnote8 Challenges for adaptation in the North include limited financial and human capacity, risk identification, and limited baseline data and climate change scenario projections.^Footnote9 Indeed, adaptation, as a field of practice in Canada, is still in an early stage: a recent Canadian review of adaptation practices across all sectors concluded that adaptation has largely focused to date on planning and policy development, building capacity and raising awareness.^Footnote10

Failure to address transportation infrastructure vulnerabilities will increase infrastructure costs. The significant costs of maintaining northern roads was noted by territorial stakeholders at the Permafrost Network workshop as an ongoing area of concern, particularly as road infrastructure continues to be expanded. In the Yukon, the annual maintenance costs for the section of the Alaska Highway built on permafrost has averaged $30,000 per kilometre per year, and permafrost-free stretches of the highway have averaged $4,000 per kilometre.^Footnote11 Without effective climate change adaptation practices, the already considerable costs of maintaining northern transportation systems will increase as infrastructure becomes more unstable.^Footnote12

Unreliable transportation infrastructure can hinder or discourage economic development, which is reliant on a dependable transportation system to facilitate exploration and resupply.^Footnote13 For example, in 2006, a shortened season for the Northwest Territories’ Tibbitt to Contwoyto Winter Ice Road due to unusually warm weather resulted in the need for a massive airlift operation by the Diavik Diamond mine to resupply fuel, at an additional cost to the company of $11.25 million.^Footnote14

While northern transportation adaptation is a relatively new field of practice, the NTAI is not the only initiative active in R&D relevant to this area:

• In 2012, the Arquluk program was launched as a five-year (2012-2017) $1.32M Cooperative Research and Development Program funded by Natural Sciences and Engineering Research Council (NSERC) and 12 partners. Led by Université Laval, Arquluk’s goal is similar to the permafrost-related activities of the NTAI : to improve current adaptive methods beneath transportation infrastructure by developing expertise for mitigating permafrost instability.^Footnote15 NSERC -funded basic research on permafrost is also being undertaken through the Arctic Development and Adaptation to Permafrost in Transition (ADAPT) initiative, which, although not examining transportation practices, is studying changes to permafrost.
• The recently announced National Research Council-led Arctic Program may also include northern transportation R&D projects of relevance to adaptation.

Other federal organizations (e.g., Natural Resources Canada (NRCan), Aboriginal Affairs and Northern Development), and provincial-territorial organizations (the Yukon Government’s Department of Transportation, Transports Québec) are also undertaking, or have recently completed, R&D initiatives or projects relevant to northern transportation adaptation.

The networks of expertise have provided forums for communicating and sharing results between initiatives. For example, the director of Arquluk, a member of the Permafrost Network, provided an update on Arquluk projects at the June 2014 Permafrost workshop, and NRCan has regularly participated in the network. In addition, the Clean Air Agenda Adaptation Theme Director General committees have provided venues for sharing information on Adaptation Theme programming among member organizations.

However, the risk of duplication remains given the increasing level of activity in adaptation and northern research and development generally. As planning is underway for federal adaptation programming following the current phase of the Clean Air Agenda, consultation and communication with other organizations in the field is especially critical at this juncture. Furthermore, ensuring that Transport Canada’s role in supporting northern transportation adaptation is clearly differentiated from those of other initiatives will reduce the risk of duplication of activities going forward.

Much of the NTAI -supported R&D has focused on filling information gaps through scoping studies, vulnerability assessments and monitoring projects. These studies are building foundational knowledge on the impacts of climate change on the northern transportation system, and the ways in which climate and related characteristics (like moisture or groundwater) affect the performance of transportation infrastructure.

Interviewed stakeholders identified the need to shift to an emphasis on research and development that tests or develops adaptive techniques and tools, and communicating effective practices to those who can implement them. With a growing body of data and information available from NTAI -funded projects, there is a need to ensure the information is translated into practical solutions for those responsible for planning and maintaining the northern transportation system.

Alignment with Government Priorities and Departmental Strategic Outcomes

The NTAI is clearly aligned with the Government of Canada’s priority of development of the North. The Government of Canada’s Northern Strategy (2009) outlines economic development as one of its four priorities, and identifies infrastructure as an area requiring “urgent attention”.^Footnote16 Recent Speeches from the Throne and Budgets have similarly prioritized northern economic development, including through investment in transportation infrastructure like the extension of the Dempster Highway from Inuvik to Tuktoyaktuk.^Footnote17

In addition, investment in R&D is a key policy lever to support the federal priority of economic development. Recent Speeches from the Throne and Budgets have emphasized the importance of innovation, science, technology, and research in positioning Canada for future prosperity.^Footnote18 Through supporting R&D projects, the NTAI aligns with the federal priority of innovation as a means to sustain Canada’s prosperity.

The NTAI is currently situated under both Strategic Outcome 1 – An Efficient Transportation System and Strategic Outcome 2 – A Clean Transportation System in the department’s Program Alignment Architecture. Specifically, the program is situated under:

• Program sub-activity 1.1.5 – Transportation Analysis & Innovation (under 1.1 Transportation Marketplace Frameworks), for all NTAI activities managed directly by Strategic Policy and Innovation; and
• Program 2.3 Environmental Stewardship, for the Kuujjuaq airport project managed by the Quebec Regional Office.

The objectives and expected results of the initiative align with Strategic Outcome 1 – An Efficient Transportation System. While the NTAI examines the impact of environmental factors resulting from climate change, the objectives of the initiative are focused on resiliency, responsiveness and adaptability of the transportation system; the initiative’s objectives are not explicitly focused on improved environmental stewardship. In addition, the NTAI ’s focus on infrastructure adaptation aligns with other programs under SO 1 related to infrastructure.

Alignment with Federal Roles and Responsibilities

In 2011, the Government of Canada released its Federal Adaptation Policy Framework, which guides federal action to address adaptation, including outlining expected roles of the federal government. Generating and sharing knowledge, building adaptive capacity, and facilitating collaboration amongst stakeholders and sectors are roles identified in the framework that align directly with Transport Canada’s roles in the NTAI .

Supporting initiatives like the NTAI that work to improve the resilience of northern transportation infrastructure is appropriate given the significant federal investments in infrastructure in the North through federal transfers and programs such as the Building Canada Plan and P3 Canada Fund. This investment includes, for example, $200 million to support the Inuvik to Tuktoyaktuk Highway, and $49.7 million for Nunavut’s first small craft harbor in Pangnirtung.^Footnote19

In addition, innovation in infrastructure adaptation can help Transport Canada to maintain and adapt the transportation infrastructure that it manages. For example, adaptation practices related to drainage, friction and pavement reflectivity are being piloted at the Kuujjuaq airport, which is owned by Transport Canada.

Evaluation Findings: Performance

Effectiveness – Achievement of Expected Outcomes

The effectiveness of the program was assessed by examining progress in achieving immediate outcomes, including building collaboration, capacity and awareness of adaptation, as well as testing/developing adaptation measures for northern transportation. Early results related to the implementation of adaptation measures, and climate change considerations being factored into northern transportation planning, were also examined.

Building collaboration and establishing linkages between stakeholders (including federal, territorial and provincial government officials, academic researchers, and the private sector), were, from the establishment of the NTAI , central components of the initiative. Collaboration was expected to strengthen the quality of research and development, and facilitate partnerships and sharing of effective practices.

The two networks of expertise, through their workshops and meetings, have been the major venues for building collaboration. The Permafrost Network has had five workshops, while the NEXTAW , which took longer to establish, has had two. Other meetings have taken place between workshops.

The Fifth Annual Workshop of the Permafrost Network in Dawson City, Yukon, for example, had 32 participants from two federal organizations, three territorial and provincial governments, five universities and colleges, and two private sector engineers. The workshop included presentations from project managers and graduate students, group discussions of research findings and activities, site visits at project monitoring sites along the Dempster Highway and at Dawson City, and discussion of ongoing and future priorities for the network. Feedback received on the workshop through a follow-up survey of participants was uniformly positive.

The networks have supported building project teams and collaboration in developing R&D projects funded through the NTAI . All of the NTAI ’s G&C -funded projects funded have involved multiple collaborators from different sectors. Interviewed members of networks noted the value of the collaboration developed as a result of their participation in the networks, particularly between academic researchers and territorial-provincial governments.

Collaboration has also included the development of partnerships to share the costs of research and development projects funded through the initiative. Among the 37 NTAI projects examined in the file review, 12 projects included planned financial or in-kind support from partners outside of Transport Canada. Overall, expected non-federal funding support comprised $1.05 million.

Recognizing that, as a field of practice, adaptation in Canada is in its relative infancy, the NTAI was designed to build capacity in northern transportation adaptation practices and techniques. Interviews, the follow-up survey of Permafrost network participants, and observation of the Permafrost workshop all indicated that capacity is being built among various stakeholder groups:

• Academic researchers are building a body of knowledge related to adaptation through funded research projects and through communicating with other researchers as part of the networks;
• Territorial and provincial policy managers are building knowledge through managing research being conducted by academic researchers, through regular contact with these experts in the networks, and through contact and dialogue with other territories; and
• Technical and engineering staff at territorial governments, some of whom have been in communication with researchers over adaptive design considerations for transportation infrastructure as part of R&D projects.

Network meetings and workshops have been key venues for capacity building, and have been well attended, partly as a result of NTAI support. There were 91 instances of the NTAI sponsoring individuals to participate in workshops, conferences and other forums in order to widen participation and information-sharing.

Capacity building has included building new research capacity among junior or student researchers. This was felt by territorial government officials to be important to ensure that Canada has future expertise in the areas of adaptation and northern transportation. Students working on northern transportation-related fields (e.g., geography, engineering) have been actively engaged in all aspects of the initiative. Among the funded projects examined in the project file review, 11 specifically indicated in project documentation that they included graduate students within the research teams. The Permafrost Network workshop in Dawson City included the participation of seven students, including two sponsored by the NTAI . Students at the workshop emphasized the value of their participation: they were able to learn from other researchers sharing project findings, build their research and professional networks, and discuss their own projects with leading researchers.

The NTAI has funded numerous projects to increase knowledge and build capacity. As of August, 2014, the NTAI had funded 46 contributions, grants and contracts, which included:

• Nine contributions and six grants, for a total value of $2.97 million; and
• Thirty-one (31) contracts, for a total value of $2.55 million.

Project funding was most often being provided to universities and colleges, which were recipients of 38% of the committed project funding, while territorial governments comprised 29% . The remainder was provided to other federal organizations (23%), and not-for profit and industry (9%).

Projects have predominantly focused on scoping studies, vulnerability assessments, and monitoring/baseline studies. Over three-quarters (76%, or 28 of 37) of projects reviewed for this evaluation included aspects related to monitoring, vulnerability assessment or scoping. For example, scoping studies and vulnerability assessments for marine transportation have included projects related to:

• strategies for managing Arctic yacht tourism;
• a climate change adaptation assessment for transportation in Arctic waters;
• a scenario-based approach to charting a course for marine transportation in the Canadian Arctic;
• a vulnerability assessment of infrastructure and operations of the Port of Churchill and associated shipping routes;
• Arctic transportation needs of the oil industry;
• the potential impacts of climate change on the future streamflow and water levels of the Mackenzie River and on its transportation system; and
• the potential impacts of a northern shipping route into Canada via the Arctic.

Scoping and vulnerability assessments related to permafrost have included a study of transportation risk in the Arctic due to climatic conditions. In addition, several permafrost monitoring/baseline studies were funded, including the establishment of baseline data collection sites and the assessment of permafrost response to climate warming on the Dempster Highway, monitoring of permafrost under the Iqaluit airport, and monitoring construction test sections along the Yellowknife Highway #3.

As a result of its participation in the networks, capacity has been built within Transport Canada.
Innovation Policy indicated it is now functioning as a de facto centre of expertise for the department in northern transportation as a result of the knowledge it has built on the issues of climate change, adaptation and the north, and due to its access to a network of stakeholders. Among its roles in northern transportation, Innovation Policy is the lead on Transport Canada’s Northern Transportation Strategy, its Steering Committee on Arctic and Northern Affairs, and supports the Transport Canada Climate Change Adaptation Plan. In addition, it is the lead in the Northern working group of the NRCan Adaptation Platform, Arctic Marine and Aviation Transportation Infrastructure Initiative, and Arctic Council initiatives related to climate change, among other initiatives.

Private sector participation in NTAI activities has been limited and is an area requiring further attention going forward. At the time of the evaluation, each network had two members from the private sector (although the lists of associate network members included others from the private sector). In addition, only six NTAI -funded projects were led by the private sector, and these were small projects ($25,000 or less in value). Permafrost Network participants noted that the limited private sector participation has been a weakness of the NTAI , given that private sector engineering firms are major participants in northern transportation infrastructure design, construction and maintenance. Challenges with increasing private sector participation in networks were also noted, as many firms may not see a clear financial incentive for participation.

Effective communication and knowledge transfer are important to ensuring that the investments made in R&D initiatives result in the application of innovative practices. As the body of information grows as more NTAI projects are completed, it is increasingly important that what is learned is compiled and shared with those who can make use of it.

The need for active communication and knowledge transfer was also expressed in interviews with network members. Many members of the networks indicated they were not fully aware of the status and results of projects and other activities of network members, and there is no central repository or portal of network communications, reports and products. Several members of the networks mentioned that a summary or compilation of what had been learned through the projects would be useful.

Knowledge transfer is especially important for an initiative like the NTAI , where many final project reports are technical and more suited to researcher audiences than to those responsible for the design or maintenance of northern transportation infrastructure.

Knowledge transfer should extend beyond network participants to achieve maximum impact. Network members can leverage their existing networks (like the Centre for Northern Studies, the Arquluk program) to broaden dissemination of information, while Transport Canada’s Strategic Policy and Innovation directorate can make use of its ongoing participation in various forums to share results and effective practices. Transport Canada should also ensure that information is shared between different areas of the department (including between headquarters and regional offices), and that results of the Kuujjuaq airport project are shared through the Permafrost Network and other venues.

Potential products to facilitate knowledge transfer could include, for example, a best practices manual, an update to the Transportation Association of Canada guidelines on building infrastructure on permafrost, and training materials.

Other federal research funding agencies (including NSERC , the Social Sciences and Humanities Research Council of Canada, and the Canadian Institutes of Health Research) have considerable experience with R&D knowledge transfer that may be useful to Transport Canada going forward.

Of the 37 projects reviewed as part of the file review, eight (22%) involved testing or developing new transportation adaptation techniques. As noted, as a result of the considerable information and knowledge gaps related to the effects of climate on transportation systems, the focus of research and development was more frequently on building foundational knowledge.

Two projects had completed testing of adaptation techniques:

• Culverts were instrumented and studied on the Alaska Highway to test the impact of different temperature and hydrological conditions affecting drainage systems built on permafrost. This is expected, as a next stage, to lead to further information on key factors in designing effective culverts to reduce permafrost degradation along northern highways.
• The ongoing R&D at the Kuujjuaq airport undertook runway surface friction and texture testing, and recommended an approach to maintaining effective runway friction at the airport in the face of climate change impacts. The project is also examining the albedo (reflectivity) of different surface coating colours, and water drainage, and their impact on permafrost stability in order to reduce costs and disturbances at northern airports.

In addition, six projects were testing new technologies to better monitor climate change-related vulnerabilities and/or impacts. These included a new technique for measuring hydraulic conductivity of active layer permafrost, research on testing remote sensing technologies, and testing a new temperature sampling method using fibre optics along transportation infrastructure. In addition, the Interdisciplinary Centre for the Development of Ocean Mapping (CIDCO) developed and is testing a new method for underwater surveying using floatplanes. Successful implementation of the technology could contribute to more cost-effective charting of northern harbour approaches compared to traditional ship-based surveying, which is expected to lead to safer and more efficient resupply.

An expected outcome of the NTAI is that adaptation measures are implemented in the northern transportation system, and climate change considerations are factored into northern transportation planning. Given that many of the NTAI projects were ongoing, and that implementation of new practices can take time to materialize following the testing or development of innovation practices, it was not expected that the initiative would have significantly contributed to this outcome at the time of the evaluation. However, a few early examples were identified in which NTAI -research had been taken into consideration in infrastructure design and adaptation.

The $300 million Iqaluit Airport Improvement Project has been informed by NTAI -funded permafrost research { ATIP Removed }.

In addition, the Government of Yukon indicated that NTAI - funded permafrost research related to groundwater and heat transfer contributed to the decision to pilot interceptor ditches to deflect groundwater along a limited section of the Alaska Highway. According to the territorial government, further testing of the interceptor ditches is required before more widespread adoption would be undertaken.

As part of the overall Adaptation Theme of the Clean Air Agenda, the NTAI undertook performance reporting using five indicators of expected results at the output and short-term outcome levels. While the program had identified performance measures for contribution projects, no performance data had been collected for the contribution-funded project that was complete at the time of the evaluation. The program indicated that it had monitored the performance of this project informally, and a more formal approach will be taken to performance monitoring for other contribution agreements in the future.

Strong and ongoing project-level performance monitoring, as well as program level reporting on knowledge transfer and adaptation testing, will be important going forward. This information will allow the program to systematically measure and report its results across all aspects of the program.

Efficiency and Economy

To assess the efficiency and economy of the NTAI , the evaluation examined the extent to which the initiative used resources as planned, and the suitability of the financial resources allocated to the program.

During its first three years, the NTAI spent $5.0 million of the $6.3 million allocated for the initiative, or 21% less than initially planned. Most of the underspending was of funds budgeted for grants and contributions (G&Cs), for which the initiative underspent by $820,000, or 36% of the original G&C budget. Of this amount, $200,000 has been reprofiled to 2014-15.

Lower-than-expected spending for grants and contributions was due to a number of factors. The program indicated it received fewer proposals from the territories-provinces than expected during the first years, and the process of developing project proposals through the networks of expertise was sometimes long and iterative, in order to ensure projects best reflected priorities. The duration of the approval process for contributions delayed the start of projects in some cases, and the timing of approvals sometimes did not align with the short seasonal window for project data collection, which slowed the spending of resources. In addition the program indicated that it took longer than expected to develop program administrative processes, including the project application guide.

At the time of the evaluation, there remained $2.0 million in G&C funding to be spent for the last two years of the initiative, of which $0.9 million was committed for existing projects. The program indicated that additional project proposals were being assessed and it expected to spend the remaining G&C funds before the end of the initiative.

The NTAI ’s planned and actual spending for the first three years of the initiative is shown in Table 1.

Other Operating Costs (OOC) comprised one-half of the total initiative budget, $5.54 million of the total $11.00 million. Spending of OOC included funding 31 contracts, including those managed by the Quebec Region, as well as costs related to the networks of expertise. Twenty contracts were for small projects of $25,000 or less undertaken through sole source contracts. Managing this number of contracts was administratively heavy for the program. In addition, some of the small contracts appear to have more limited value to achieving outcomes directly related to transportation adaptation. This included projects on northern transportation logistics indicators, and an initial review of marine transportation training needs of the North.

Lessons Learned from Implementation

The following section presents lessons learned from the implementation of the NTAI .

While the focus on adaptation of infrastructure built on permafrost was appropriately narrow, the initiative’s activities related to marine transportation adaptation were broad and varied. The topic of northern marine transportation infrastructure requires specific R&D priorities to be established in order to focus on the key information gaps/needs of the territories and other stakeholders. Although the marine transportation network identified five research focus areas following its establishment, these areas primarily represented types of projects (including innovation and technology development and testing, best practices, training and capacity building) rather than specific research questions or topics to examine in the area of adaptation.

The broader approach on marine transportation may limit the impact of these projects as a whole, as the projects do not build on each other to create a body of knowledge in targeted areas, and may impede knowledge transfer going forward.

Foundational documents developed for each of the two networks at their establishment noted that the networks were expected to become self-sustaining (i.e., continue without significant ongoing support from Transport Canada). However, Transport Canada continues to be heavily involved in its role as secretariat and as a major participant in the networks, and has sponsored other participants to travel to workshops and other events.

In addition, Transport Canada has been the significant funder of network-developed projects, which has enhanced participation in the networks, particularly among researchers. Transport Canada’s average share of funding among R&D projects funded through contribution agreements was 74%. Furthermore, interviews suggested that there may be limited capacity for other stakeholders to take on a significantly larger role in the networks at present.

It is likely that Transport Canada will need to continue to take a central role in the networks in the short term, in order to continue to maintain the momentum that has been built.

The evaluation noted the importance of building cohesion among network members while still maintaining transparency and openness to new members.

Overall, collaboration, networking, and relationship building were identified as key successes of the networks. This was especially true of the Permafrost Network, which was established first and has benefited from a tighter focus of activities to build collaboration. The continuous involvement of a core group of members has been central in creating momentum for the networks during their relative infancy.

There is, however, a risk that, with both the networks now well established, they may appear too closed if new membership is not regularly and widely encouraged. This is especially relevant given the central role that the networks have had in developing and recommending projects to Transport Canada for funding. Furthermore, continuing to develop new linkages with other stakeholders in transportation adaptation will help to facilitate successful knowledge transfer.

Strategic Policy and Innovation directorate initially pursued developing a web-based tool for the networks of expertise. This was expected to complement the regular in-person meetings of the networks, and allow members to undertake discussion and share information and reports online.

While consultations and planning were initiated by Transport Canada to develop an online information portal, this activity was eventually abandoned due to significant unanticipated challenges, including related to privacy, data and information management/storage, ownership, and other issues.

Many network members indicated they would see considerable benefit from an online portal, which would help to facilitate communication and knowledge-sharing between meetings. Transport Canada is continuing to examine the development of an online tool such as SharePoint as a tool to share information online.

The program faced a significant learning curve in setting up and administering the NTAI , according to interviews. Program staff had limited experience in administering a transfer payment program, and considerable time was spent in developing program capacity and administrative processes, including the program application guide and other documents. While the department’s Centre of Expertise on Transfer Payments provided advice and guidance to the program, it was itself newly established, and time was required for the Centre to develop departmental templates for contribution agreements and other documents, which the program required. These factors, among others, were said to have contributed to slower-than-expected progress during the first years of the initiative.

Management and administration of the NTAI ’s transfer payment program rested in a policy unit. This does not conform with the Policy-Program Continuum Standard Functional Model for Transfer Payment Programs at Transport Canada (2011), which outlines specific roles for policy and program units in developing and delivering transfer payment programs. According to NTAI program staff, at the time the NTAI was established, the decision was made to maintain administration of the initiative within Policy group (which had administered the forerunner, the Northern Transportation Research Initiative) given that this would continue to facilitate regular and ongoing contact with northern transportation stakeholders, thus informing other Strategic Policy work related to the North, and given the relatively small size of the transfer payment program ($4.07 million over five years).

Conclusions and Recommendations

The evaluation found that support for northern transportation adaptation continues to address an ongoing need, as all modes of the northern transportation system require adaptive measures in the face of climate change, and knowledge of effective adaptive practices in the field remains limited. Support for northern transportation adaptation aligns with federal priorities related to economic development, innovation, and the North. The NTAI aligns with federal adaptation roles as outlined in the Federal Adaptation Policy Framework (2011), and with the significant federal transportation investments in the North. Although currently situated within Transport Canada’s Program Alignment Architecture under two Strategic Outcomes, the objectives of the NTAI most directly support the departmental Strategic Outcome of “An Efficient Transportation System”.

In its first three years, the NTAI has made progress in achieving its short-term outcomes of building capacity, awareness and facilitating collaboration. The program has fully established two networks of expertise, which include stakeholders from the territories, provinces, universities and colleges, and, to a lesser extent, the private sector. These networks have proven to be effective venues for building collaboration between territorial governments and researchers, including through developing research projects. At the time of the evaluation, the NTAI had funded 46 research and development projects through contracts, grants and contributions.

The networks and projects have increased awareness and understanding of the interplay of climate and infrastructure, scoped potential impacts of climate change on the northern transportation system, and begun to identify vulnerabilities. Some new technologies and adaptive techniques are being tested, including, for example, methods of reducing permafrost degradation due to groundwater around highways, and reducing the impact of climate change on the surface friction of northern airport runways.

A few early examples were identified of NTAI -funded research being taken into consideration in northern transportation infrastructure design and adaptation. These included research projects on permafrost degradation informing the plan for the renovation of the Iqaluit Airport Improvement Project and the construction of interceptor ditches along a portion of the Alaska Highway in the Yukon.

Spending of NTAI funding was slower than originally planned, due to the time required to build program delivery capacity and processes and to develop and approve project proposals. The amount of OOC funding budgeted for the initiative appears to have been too high, which resulted in a relatively large number of projects being funded through OOC , including some smaller projects which are less directly linked to achieving expected outcomes.

Lessons learned from the implementation of the initiative include:

• Formulating clear and specific R&D priorities is important to ensuring the initiative’s activities directly address the adaptation information needs of end-users. While the focus on adaptation of infrastructure built on permafrost was appropriately narrow, the initiative’s activities related to marine transportation adaptation were broad and more varied, which limited the ability of projects to build on each other, and may impede knowledge transfer.
• While the intent is for the networks of expertise to be self-sustaining in the future, both networks appear to require continuing support from Transport Canada to sustain their momentum.
• Maintaining the right balance between membership continuity and openness to new members is a key consideration in a successful network. The networks would benefit from new members to “refresh” themselves and maximize transparency and their impact in their respective fields.
• There were considerable barriers to developing an online information management and sharing tool (i.e., a web portal) for the networks of expertise.
• Considerable time was required to develop program administrative processes, including the project application guide, as Strategic Policy and Innovation staff needed to build capacity in setting up and administering a transfer payment program and the Centre of Expertise on Transfer Payments was developing new departmental templates.

Areas requiring further attention were identified in the evaluation. First, as the field of northern transportation adaptation has many players—including provinces and territories, universities and colleges, other federal organizations, and the private sector—Transport Canada should exercise its leadership in this area to work with other organizations to identify any areas of potential overlap, and opportunities for streamlining or partnering. To the extent possible, private sector involvement in future initiatives should be bolstered.

Second, while considerable information and data are being generated on monitoring, vulnerabilities, and potential climate impacts, there is a need to increase attention on testing of adaptation techniques in order to identify effective practices.

Third, given the considerable data and information that are becoming available through the initiative, there is a need to ensure that information is widely communicated to those who can make use of it, including those responsible for northern infrastructure planning and maintenance. Effective knowledge transfer will be key to ensuring the program can contribute to the achievement of longer-term outcomes related to the implementation of effective adaptation measures in existing and future infrastructure.

Finally, robust project- and program-level performance monitoring will be important moving forward. At the time of the evaluation, project-level performance data collection was weak.

The evaluation includes the following recommendations:

Management Action Plan

#	Recommendations	Proposed Actions	Forecast Completion Date	OPI
1	Transport Canada should work with other organizations active in northern transportation adaptation to ensure coordination, avoid duplication, and identify opportunities for streamlining and/or partnering.	Transport Canada regularly invites officials from other government departments to participate in network of expertise meetings and shares information related to studies and results, (including Natural Resources Canada, Environment Canada, National Research Council, and the Department of Fisheries and Oceans). Transport Canada will continue to regularly engage other departmental officials. In addition, in order to ensure opportunities for streamlining are identified and partnerships are considered, Transport Canada will engage other organizations, where applicable, during the development of the renewed program details and in the identification of specific project proposals over the 5-year period of 2016/17-2020/21.	Consultations to begin Spring-Summer 2015	Policy
2	Transport Canada should work with the networks of expertise to develop a knowledge transfer plan.	Network members are keen on ensuring a broader knowledge transfer for program results; however, the proper means to achieve this has not yet been identified. This recommendation will be brought forward as an agenda item at future network meetings for discussion and action. Following the spring 2015 network meetings, Transport Canada will finalize the knowledge transfer plan.	Fall 2015	Policy
3	Transport Canada should ensure any future northern adaptation R&D initiatives prioritize adaptation pilot-testing and deployment/implementation of adaptation techniques.	This recommendation was taken into consideration during the preparations for Transport Canada’s proposed renewal initiative under Adaptation funding. In particular, one of the three core activities of the initiative will be the support of pilot projects in the North that address system vulnerabilities in all modes of transportation through the implementation of innovative adaptive solutions using existing knowledge, tools and best practices.	Completed	Policy

The following document is available for downloading or viewing:

• Evaluation of the Northern Transportation Adaptation Initiative ( Version, 235 kb )

To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet:

• Adobe Reader
• Foxit Reader
• Xpdf
• eXPert PDF Reader