Evaluation of the Trade and Transportation Corridors Initiative

Report outlining the results of the Evaluation of the Trade and Transportation Corridors Initiative.

On this page

• Introduction
• Findings
• Summary of findings and recommendations
• Appendix A. Management Action Plan
• Appendix B. List of RPAS research and development projects

Executive summary

About the initiative

The Trade and Transportation Corridors Initiative (TTCI) aims to enhance Canada's transportation infrastructure and drive economic growth through its three main components: the National Trade Corridors Fund (NTCF), the Trade and Transportation Information System (TTIS), and measures to support transportation innovation.

• The NTCF is an 11-year initiative (2017-18 to 2027-28) with a funding envelope of $4.3 billion. It aims to strengthen the efficiency, fluidity, and resiliency of trade corridors and supply chains.
• The TTIS was initially allocated $50 million over 11 years and was later expedited for completion within 8 years (2017-18 to 2024-25). The purpose is to make high quality, timely data on the transportation system available for decision-making.
• Measures to support transportation innovation include $50 million over five years (2017-18 to 2021-22) to support the safe integration of transportation technologies into the Canadian transportation system, specifically remotely piloted aircraft systems (RPAS) and connected and automated vehicles. This includes the Program to Advance Connectivity and Automation in the Transportation System (ACATS).

About the evaluation

The Evaluation of the TTCI examined all three components, considering questions of relevance, delivery, and effectiveness. Given the concurrent audit of transportation corridors and supply chains by the Office of the Auditor General of Canada (OAG), we took a calibrated approach when assessing the NTCF, meaning where feasible, we avoided unnecessary overlap with the OAG audit. We used several methods, including interviews, document review, literature review, project file review, and administrative data analysis.

What we found

For the NTCF, we found that:

• There is a clear need for the NTCF, and the program aligns strongly with TC’s mandate and federal government priorities.
• The NTCF has taken meaningful steps to improve its performance measurement practices by requiring all projects to report on common indicators. However, the utility of these indicators will depend on the quality of data provided by funding recipients.
• The expected contribution of NTCF project-level results to desired system-level changes is not clearly defined.
• The involvement of Transport Canada’s Transportation and Economic Analysis (TEA) group in NTCF decision-making processes is a key strength of the program.
• Uncertainty in timelines for NTCF project approval and funding has been identified as a concern for stakeholders.
• More time is needed for the impacts of NTCF-funded projects to materialize.

For the TTIS, we found that:

• TTIS is well-aligned with TC’s mandate and priorities.
• The implementation of the TTIS evolved as challenges were encountered.
• TTIS has succeeded in increasing the availability of transportation data and there is a clear understanding of information gaps and next steps.

For ACATS, we found that:

• While ACATS is relatively well-aligned with TC’s strategic objectives, part of the funding it provides is not strongly linked with any legislated TC mandate.
• Through its first call for proposals, ACATS contributed to awareness and knowledge-building for stakeholders and reached the targets for its immediate and intermediate expected outcomes.

For RPAS, we found that:

• There is a clear need for TC involvement in RPAS and the program is well-aligned with TC’s mandate.
• The RPAS Task Force has by and large advanced its objectives related to regulatory development, authorizations, and research and development.
• While the RPAS Task Force has strong engagement with industry stakeholders and other governments, more awareness among recreational users is needed.

Based on these findings, we recommend the following:

Recommendation 1

The NTCF should:

• Map out the relationship between project outcomes and the overall system-level objectives.
• Work with TEA to assess the feasibility of accounting for impacts of funded projects in trade corridors and supply chains metrics.

Recommendation 2

The NTCF should improve communications with stakeholders to enhance predictability about the application and approval process.

Recommendation 3

The RPAS Task Force should identify impediments to effective engagement and information dissemination and develop a strategy to meet its public engagement objectives.

1.0 Introduction

The Evaluation of the Trade and Transportation Corridors Initiative (TTCI) is an interim evaluation covering all Initiative components:

• The National Trade Corridors Fund (NTCF), which funds projects in support of trade corridors and supply chains;
• The Trade and Transportation Information System (TTIS), which aims to provide data and information for decision-makers; and
• Measures to support transportation innovation, which involve work on connected and automated vehicles, including the Program to Advance Connectivity and Automation in the Transportation System (ACATS), and efforts related to remotely piloted aircraft systems (RPAS).

The NTCF, as a transfer payment program with annual expenditures of over $5 million, must be evaluated every five years as required by the Policy on Results. As such, this evaluation was included in Transport Canada’s (TC) Five-Year Departmental Evaluation Plan for 2022-23.

1.1 About the Initiative

The TTCI supports stronger, more efficient transportation corridors and economic growth for Canadians. Through its three discrete components, the Initiative targets needs related to infrastructure and supply chains, data, and innovative technology.

Parts of the TTCI (NTCF and TTIS) fall under the Government of Canada's Investing in Canada Plan as part of the Trade and Transportation Infrastructure stream. The programs in this stream are intended to improve flow of the transportation system by building stronger, more resilient, and more efficient trade corridors to access global markets. ^{Footnote 1}

1.1.1 The National Trade Corridors Fund

The NTCF is an 11-year (2017-18 to 2027-28) merit-based transfer payment program intended to strengthen the efficiency, fluidity, and resiliency of Canada’s trade corridors and transportation infrastructure. Initial funding was granted via Budget 2017 and additional funding was allocated through Budgets 2019, 2021, and 2022, for a total funding envelope of $4.3B.

To date, the NTCF has had seven calls for proposals addressing various needs and vulnerabilities (Table 1). These calls for proposals aim to collect detailed information on potential projects that could support program objectives. Project proposals are assessed based on pre-defined criteria that enable the NTCF to fund meritorious projects.

Table 1. Themes and timeframes for NTCF calls for proposals

#	Call theme	Launch date	Closing date
1	National Call to Address Capacity Constraints	July 4, 2017	November 6, 2017
2	Canada’s North	November 19, 2018	March 29, 2019
3	Continuous Call for Trade Diversification	January 15, 2019	December 9, 2021
4	Arctic and Northern Call	October 23, 2020	March 15, 2021
5	Increasing the Fluidity of Canada’s Supply Chains	December 9, 2021	June 30, 2022
6	Relieving Supply Chain Congestion at Canadian Ports	January 31, 2022	February 25, 2022
7	Advancing Supply Chain Digitalization	February 13, 2023	April 11, 2023

Since 2006, the department has provided support aimed at improving Canada’s trade corridors through two previous merit-based infrastructure funding programs (Figure 1).

The Asia-Pacific Gateway and Corridor Initiative (APGCI) was launched in 2006 with a total of $1.17 billion in funding, of which $915 million was transfer payment funding. The APGCI was a set of infrastructure, policy, and research initiatives focused on enhancing the capacity and efficiency of the Asia-Pacific Gateway and Corridor; it looked to support the movement of both goods and people between North America and Asia specifically.

The Gateways and Border Crossings Fund (GBCF), guided by the National Policy Framework for Strategic Gateways and Trade Corridors, was launched in 2007-08 with $2.105 billion in funding. It aimed to fund projects which would improve the flow of goods and people between Canada and the rest of the world.

The APGCI and GBCF were designed to address congestion and capacity issues in trade corridors to ensure efficient movement of goods to market. Launched through the TTCI, the NTCF is intended to build on the results achieved by its predecessor programs.

Figure 1. TC Gateway programming and other key events leading up to the NTCF ^{Footnote 2}

Description

Figure 1 provides a chronological overview of key milestones leading up to the NTCF, spanning from October 2006 to March 2018.

It starts in October 2006 with the launch of the Asia-Pacific Gateway and Corridor Initiative (APGCI). July 2007 marked the signing of a Memorandum of Understanding (MOU) between the ministries of transport in Ontario and Quebec, aimed at developing the Ontario-Quebec Continental Gateway and Trade Corridor Initiative. This same month saw the release of the National Policy Framework for Strategic Gateways and Trade Corridors.

The year 2008 witnessed significant progress with the announcement of the Gateway and Border Crossing Fund (GBCF) and the initiation of the P3 process for the New Windsor-Detroit Crossing, known as the Gordie Howe Bridge. March 2011 marked the launch of the Atlantic Gateway and Trade Corridor Strategy. Fast forward to the fall of 2016, when the Minister of Transport announced the Transportation 2030 Strategic Plan. November of the same year saw the announcement of the Trade and Transportation Corridors Initiative (TTCI).

In spring of 2017, funding announcements were made, with Budget 2017 including the Trade and Transportation Corridors Initiative, allocating an initial funding of $1.9 billion for the NTCF. July of that year marked the launch of the call for proposals for the NTCF. In October 2017, evaluations of both the APGCI and GBCF were completed. Finally, March 2018 marked the conclusion of the APGCI.

1.1.2 The Trade and Transportation Information System

The TTIS was allocated $50 million to ensure that high quality, timely, and accessible data on the transportation system is available to stakeholders in support of both public and private planning and decision-making. Initially planned to span 11 years, the 2018 Fall Economic Statement announced the acceleration of the TTIS timeframe to 8 years (2017-18 to 2024-25) (Table 2).

Table 2. Original and accelerated funding for the TTIS

Year	Original	Accelerated
2017-18	$4,545,455	$4,545,455
2018-19	$4,545,455	$4,545,455
2019-20	$4,545,455	$9,090,910
2020-21	$4,545,455	$9,090,910
2021-22	$4,545,455	$9,090,910
2022-23	$4,545,455	$4,545,455
2023-24	$4,545,455	$4,545,455
2024-25	$4,545,455	$4,545,455
2025-26	$4,545,455	-
2026-27	$4,545,455	-
2027-28	$4,545,455	-
Total	$50,000,000	$50,000,000

The TTIS was implemented as a partnership between TC and Statistics Canada (StatCan) through the establishment of the Canadian Centre on Transportation Data (CCTD). The CCTD is intended to be the authoritative source of transportation data in Canada. Through the TTIS, TC aims to enhance the availability and quality of transportation data, as well as strengthen analytical capacity regarding system performance, to better support decision-making.

1.1.3 Measures to support transportation innovation

Allocated $50 million over 5 years (2017-18 to 2021-22) (Table 3), these measures support work related to two transportation technologies, namely, connected and automated vehicles and remotely piloted aircraft systems (RPAS). Their objective was to enable the safe integration of these technologies into the Canadian transportation system through the development of an appropriate regulatory regime and support for research and testing.

TC aimed to support the safe use of RPAS ($33.5 million) through:

• Developing and implementing new standards, regulations, and guidelines;
• Streamlining service delivery and authorizations; and
• Facilitating research and participating in select pilot projects.

For connected and automated vehicles, TC planned to undertake regulatory support and research ($6 million) and to build knowledge and capacity ($10.5 million). The latter aimed to help Canadian jurisdictions prepare for connected and automated vehicles through the Program to Advance Connectivity and Automation in the Transportation System (ACATS), a research and transfer payment program which supports trials, pilot projects, the development of codes and guidance, and knowledge-sharing activities. Activities related to regulatory support are not included in the scope of this evaluation.

Table 3. Funding allocations for TTCI's innovation measures

Year	RPAS	Connected and Automated Vehicles – Regulatory Support	Connected and Automated Vehicles– Advancing readiness for deployment (ACATS)
2017-18	$4,265,728	$435,798	$1,298,770
2018-19	$7,256,591	$1,476,406	$2,267,299
2019-20	$7,326,366	$1,362,048	$2,311,882
2020-21	$7,326,367	$1,361,542	$2,312,386
2021-22	$7,326,367	$1,329,062	$2,344,866
Total	$33,501,419	$5,964,856	$10,535,203 Footnote 3

1.2 About the evaluation

1.2.1 Evaluation approach and scope

Several factors shaped the approach and scope of this evaluation:

• During the conduct of this evaluation, the Office of the Auditor General of Canada (OAG) was carrying out an audit of transportation corridors and supply chains that focused on the NTCF specifically. We therefore calibrated our evaluation, where feasible, to avoid significant overlap with the OAG audit. Of note, both reviews examined performance measurement; however, our assessment is complementary, rather than duplicative, of work done by the OAG.

  In addition, TC’s Internal Audit completed an audit of the NTCF in 2023. This audit examined the design, implementation, monitoring, and performance reporting of the NTCF. The evaluation considered findings of the audit and, where possible, leveraged information and documents collected. Importantly, while the practice of reporting on results was considered in the audit, it did not assess NTCF performance measurement in depth, leaving room for the evaluation to explore this topic.

• Each TTCI component is distinct from the others. There is some complementarity between the NTCF and the TTIS, as both have direct implications for trade corridors and supply chains, and information from the TTIS is used in NTCF decision-making. Nevertheless, they are separate programs. The innovation measures (i.e., RPAS and connected and automated vehicles) are not directly linked to the infrastructure (NTCF) and data (TTIS) work under the TTCI. Further, the RPAS Task Force and connected and automated vehicles initiatives focus on two distinct technologies and do not intersect with each other. As such, each component has been assessed individually.

• Each component has a unique timespan. The NTCF and the TTIS are 11-year and 8-year programs, respectively. The innovation component had a shorter timeframe, ending in 2022. As a result, for the NTCF and TTIS, the evaluation focused on determining what progress has been made towards the achievement of their respective intended outcomes, and whether program delivery proceeded as anticipated. The next evaluation of the NTCF, which will be conducted five years after this evaluation, will focus on results achieved.

  For the innovation measures ACATS and RPAS, given their funding timeframe under the TTCI is finished, the evaluation focused on determining whether they achieved their intended results for the funding period.

• Regulatory work on connected and automated vehicles, delivered by TC’s Motor Vehicle Safety, was funded partially through the TTCI. However, it is covered through a separate evaluation and therefore not considered here. This evaluation focuses on ACATS.

1.2.2 Evaluation questions

The evaluation explored the relevance, delivery, and effectiveness of the TTCI through the following questions:

Relevance

Are the objectives of each of TTCI’s components linked to (i) federal government priorities and (ii) TC’s mandate? Is there a continuing need for each TTCI component?

Delivery and Effectiveness

Is work under the NTCF and TTIS on track to achieve their intended outcomes, and to what extent is this supported by current performance indicators? Has delivery of various TTCI components proceeded as desired? Does data collection and analysis through the TTIS effectively support internal decision-making related to the NTCF and supply chains? To what extent have TTCI-funded measures to support transportation innovation (i.e., RPAS and ACATS) achieved their intended results?

1.2.3 Evaluation methods

Interviews

We conducted semi-structured interviews with 25 internal stakeholders (i.e., Policy and Programs staff and senior management) and 12 external stakeholders (i.e., ACATS funding recipients, RPAS stakeholders, Statistics Canada staff).

Interviews explored topics such as:

• Progress and results for each TTCI component
• The need for and impact of TTCI's transfer payment programs, i.e., the NTCF and ACATS
• Supply chains issues in Canada
• Data and information needs for supply chains
• Considerations related to RPAS and connected and automated vehicles

This evaluation did not focus on individual NTCF-funded projects primarily because only a small portion had been completed at the time it was conducted (30 out of 183 approved projects, representing 5% of the total funding envelope) and several of these were studies or design projects. Consequently, we did not conduct interviews with NTCF funding recipients or other project stakeholders. As mentioned in the Evaluation Approach and Scope section, the next evaluation of the NTCF will take place in five years and will focus on the results achieved. That evaluation will likely include interviews with funding recipients and other external stakeholders, as most projects will be completed by then.

Document review

We reviewed documents from each TTCI component. This included foundational documents, internal reports and presentations, summary placemats, applicant guides and selection criteria for the NTCF and ACATS. Documents provided information such as history and rationale for the Initiative, progress under each component, funding priorities and related evidence, project selection processes, and performance measurement.

Literature review

We conducted a review of reports and papers by academics, industry organizations, and experts, as well as various media articles and industry websites. This provided contextual and supplementary information, particularly on supply chain issues and priorities in Canada, and progress, considerations, and future applications for RPAS and connected and automated vehicles.

Project file review

We reviewed final reports from projects funded through the NTCF and ACATS:

• For the NTCF, a total of 30 projects had been completed at the time of evaluation. We reviewed a sample of 13 final reports from completed projects. These represented projects from different areas of Canada. The aim was to assess intended outcomes, performance metrics, and actual results, as they relate to overall NTCF objectives. We did not assess granular details such as project-specific challenges.
• For ACATS projects, we reviewed 13 out of 15 final reports (reports from two projects were unavailable as they had not been provided by the recipients^{Footnote 4}) to assess the ways in which funded projects related to the program’s intended goals. From these final reports, we extracted the intended outcomes, performance indicators, and actual results.

Administrative data analysis

We analyzed administrative and financial information from the NTCF and ACATS to determine the number of applications received, the number of projects funded, and the distribution of funding by region, project type/focus, and in the case of the NTCF, transportation mode. For the NTCF, this information also provided insight into the length of time spent on each call for proposals.

1.2.4 GBA Plus and Official Languages assessment

We assessed potential Gender-Based Analysis Plus (GBA Plus) and Official Languages considerations during the planning phase of the evaluation using Evaluation and Advisory Services' (EAS) internal GBA Plus and Official Languages Checklist. There were no immediately obvious considerations related to GBA Plus or gaps in how the initiative accounted for GBA Plus or Official Languages. We determined that the TTCI is intended to benefit all Canadians. In addition, two NTCF calls were specifically dedicated to Northern communities, recognizing the unique context and needs there. Calls for proposals and related guidance and communications for the NTCF and ACATS were in both official languages. As such, we did not include a separate evaluation question on these topics, though both GBA Plus and Official Languages were kept in mind during the conduct phase in case relevant information emerged. Ultimately, there were no significant findings related to GBA Plus or Official Languages.

2.0 Findings

The evaluation findings below are organized by TTCI component.

The Findings section also includes a brief commentary on GBA Plus and Official Languages.

2.1 NTCF

For the NTCF, we report on program:

• Relevance
• Delivery, specifically:
  • NTCF performance measurement
  • Role of TC’s Transportation and Economic Analysis group
  • Timelines related to proposal review
• Effectiveness

2.1.1 Relevance

Finding 1: There is a clear need for the NTCF and the program aligns strongly with TC’s mandate and federal government priorities.

Budget 2023 ^{Footnote 5} communicated that supply chains remain a key priority for the Government, noting vulnerabilities in Canada’s supply chains exposed through the COVID-19 pandemic, floods in British Columbia, and other challenges. Investment in supply chain infrastructure and trade corridors is described as important for stability and economic growth, to place “Canadian workers and Canadian businesses at the heart of essential global supply chains”. ^{Footnote 6} The issue of supply chain bottlenecks was similarly highlighted in the 2021 Mandate Letter for the Minister of Transport. ^{Footnote 7} The NTCF directly supports the department’s goal of an efficient transportation system.

Nearly all of those we spoke to about the NTCF stated that there is a continued need for such a program, noting the oversubscription of the program, persistent infrastructure needs, and the uniqueness of the NTCF as a program dedicated specifically to trade and transportation infrastructure. Data demonstrates ongoing supply chain challenges for Canadian businesses, such as delivery delays, price increases, and shortages, though some improvement was observed in 2023. ^{Footnote 8} Key needs moving forward include stronger resilience in the face of disruptions, addressing labour needs, ageing infrastructure, the role of new technologies in freight optimization, and climate change impacts. ^{Footnote 9}

2.1.2 Delivery

Transportation and Infrastructure Programs is responsible for delivering the NTCF, which includes planning and leading the calls for proposals and subsequent reviews and recommendations, engaging with applicants, and managing contribution agreements.

Multiple other groups within the department support the delivery of the program:

• Strategic Policy and Innovation conducts and disseminates relevant research and analysis related to trade and transportation infrastructure, establishes priorities for calls for proposals, and supports review processes of project proposals.
• Transportation and Economic Analysis (TEA) provides supporting data and analysis on key supply chain and transportation issues and is formally involved in reviewing project proposals.
• Regional staff and modal experts sit on proposal evaluation teams, providing knowledge and insights based on their areas of expertise.

The following discussion is particularly relevant for the Programs group and Transportation and Economic Analysis.

2.1.2.1 Performance measurement

In the interviews, it was mentioned that a clear understanding of how funded projects impact system-level outcomes such as supply chain fluidity would enable the NTCF to better evaluate and report on the effectiveness of the program as well as to better target calls for proposals.

Several key pieces necessary to achieve this, such as consistent data on results across funded projects or system-level data on supply chain performance, already are or will shortly be available to the program. However, these have yet to be organized around a comprehensive performance measurement approach that clearly maps out the linkages between project impacts and system-level outcomes.

There are two key steps to determining how project results contribute to system-level outcomes: (1) ensuring meaningful data on the impacts of funded projects is collected, and (2) successfully mapping out project outcome linkages to system-level outcomes.

1) Collecting meaningful data on the impacts of funded projects

Finding 2: The NTCF has taken meaningful steps to improve its performance measurement practices by requiring all projects to report on common indicators. However, the utility of these indicators will depend on the quality of data provided by funding recipients.

At the project level, the NTCF has recently developed new indicators to be included in all contribution agreements. Previously, indicators for a project were generally defined by the proponent as appropriate for the specific project. The program has since developed common project indicators (Table 4), to collect data consistently across projects, allowing for the comparison and aggregation of project-level impacts. These metrics are intended to “allow for greater understanding of the impact of funded interventions … and support project evaluation in future calls for proposals.” ^{Footnote 10}

Table 4. Common project indicators for NTCF-funded projects

Economic Impacts	Jobs created (direct) Jobs created (indirect and induced) Investment leveraged ($) Impact on GDP ($) Value of additional goods moved ($)
Capacity	Increase in throughput capacity (volume measured in MTs, TEUs, rail cars, other applicable unit) Maintained throughput capacity due to repair or rehabilitation (MTs, TEUs, rail cars, other applicable unit) Increase in storage capacity (m2, TEUs, rail cars, other applicable unit) Maintained storage capacity due to repair or rehabilitation (m2, TEUs, rail cars, other applicable unit) Length of rail or road constructed, repaired, or rehabilitated (m, km)
Efficiency	Increased useful life of asset (years) Length of rail with increased weight/speed capability (km) Cost savings for transporters or shippers ($) Time savings, e.g., reductions in travel time, transloading time, train marshalling time, (minutes, hours, days)
Environmental	Annual reduction of CO2 (MTs) CO2e (MTs)
Technology/Digital	New participants accessing and using established digital platforms, as reported by recipients (#) Partnerships established (#)

Prior to this, performance indicators for a project were defined individually by each recipient and were not necessarily comparable across projects. The common project indicators initiative is therefore a significant step forward. However, the ultimate value of these indicators could potentially be limited by the following:

• The ability of recipients to collect data for relevant common project indicators. The onus will be on funding recipients to “ensure that appropriate data collection measures are in place to enable the capture and reporting of the performance indicators used to measure the achievement of project outcomes.”^{Footnote 11} The common project indicators are a new requirement, and it remains to be seen whether a critical mass of funding recipients will be able to successfully collect the necessary data to report on the performance indicators found in their respective contribution agreements. For example, recipients will have to take baseline measurements so that they can provide to TC “updated data on performance indicators listed in Schedule B:1 (Project description) compared to the start of the project”. We note that the final payment to the recipient will not be made until the final report is completed to TC’s satisfaction, providing TC with leverage to obtain the information it expects.
• Some common project indicators are difficult to measure. Some common project indicators (e.g., ‘increase in storage capacity’, ‘length of rail or road constructed, repaired or rehabilitated’) are relatively straightforward and measuring them should not present a challenge to funding recipients. However, measurements of other indicators such as ‘jobs created (indirect and induced)’, ‘annual reduction of CO² (MTs)’, or ‘value of additional goods moved’ are likely to be more challenging to accurately measure.

• Actual project results can take years to manifest, after final reports have already been submitted to TC. The timing of the measurement of many of these indicators could hinder their usefulness. While several indicators will only materialize sometime after the project is completed, the latest opportunity for TC to obtain this information is upon submission of the final report to TC, at the conclusion of the project. For example, data on ‘new participants accessing and using established digital platforms, as reported by recipients’ as of the conclusion of a project is not likely to reflect the real impact, as new participants will only access and use digital platforms after some time has passed. Similarly, the indicators ‘time savings, e.g., reductions in travel time, transloading time, train marshalling time’ or ‘cost savings for transporters or shippers’ might take time to materialize. As noted in reviews of previous TC infrastructure programming (e.g., 2012 audit of the GBCF) “usage” of new infrastructure does not happen immediately and requires time to fully take shape. The NTCF acknowledges this: where targets cannot be realized until years after a project is completed, recipients are permitted to submit forecasts in their final reports. While forecasts can be acceptable proxy measures, they do present a few issues. The 2017 evaluation of APGCI and GBCF noted that when recipients submitted “impact statements that were based on projections or other analysis (e.g., forecasting models using a multitude of behavioral assumptions about things like traffic flows estimated over [several] years) … [t]he likely quality of the projections [were] highly variable.”^{Footnote 12} That said, there are no other feasible options available to TC. Requiring post-project measurements from recipients is not a viable prospect as it has been tried in the past and the results were not positive. Several agreements under the GBCF and the APGCI required recipients to provide ‘retrospective analysis reports’ two to five years after project completion but it proved very difficult to compel recipients to report back after a project was completed.

  Given the NTCF will have to often rely on forecasts to understand the impacts of funded projects, taking steps to ensure that these are of the highest quality possible would mitigate the risk of inaccurate estimated impacts that can skew the overall program results. This may include, for example, requirements for recipients to provide methods, data sources, and assumptions involved in calculating forecasted results, or involving TEA in validating project reports, similar to their role in proposal reviews (discussed in detail in the next section).

In summary, assuming successful implementation with these hurdles in mind, common project indicators are a positive step towards demonstrating NTCF projects’ contribution to system-level outcomes.

2) Connecting project-level outcomes to system-level impacts

Finding 3: The expected contribution of NTCF project-level results to desired system-level changes is not clearly defined.

The next step in determining how project results support system-level outcomes is to link in a clear manner the common project indicators to broader system-level outcomes (e.g., the fluidity of Canadian trade; modal interconnectivity and operability). Many of these system-level outcomes relate to supply chain performance and are already tracked by the department through a variety of metrics, such as end-to-end transit time, container throughput, and trends in trade growth.

There are several methods to mapping out the relationship between common project indicators and system level outcomes. Whatever method the program chooses to use, it should articulate the precise linkages between a program’s activities, and short-, medium- and long-term outcomes. It should also account for underlying assumptions, risks and external factors that impact the achievement of desired outcomes.

Undertaking an exercise to map out how funded projects impact system-level outcomes would result in several key benefits:

1. Organization of common project indicators in accordance with the timeframe or level of outcomes they relate to. Currently, these indicators are only grouped by type of impact (e.g., capacity, efficiency). They are not arranged in a way that indicates whether they are connected to an immediate (short-term) or an intermediate (medium-term) outcome. For example, indicators such as ‘increased storage capacity’ or ‘length of rail or road constructed’ are clearly short-term, as they would be observed immediately upon project completion. On the other hand, indicators such as time or cost savings relate to medium-term outcomes, since they would only materialize once the immediate outcome of a project is achieved (for example, length of rail or road constructed). As noted earlier, medium-term outcomes typically relate to usage of infrastructure and therefore take time to fully take shape. Organizing common project indicators by level of outcomes is important when building a theory of change which focuses on identifying the logical order of all the outcomes that must occur for the desired ultimate outcomes to materialize.
2. Identification of gaps in intended outcomes as currently described. More outcomes may be identified, especially intermediate outcomes that typically link project-level outcomes to system-level outcomes. In addition, outcomes related to emerging risks may be identified. For example, the department has recently flagged infrastructure damage in the transportation system as high-risk and we understand that NTCF has been identified as an active measure to help mitigate that risk. If and when NTCF is positioned as the primary mitigation vehicle, appropriate outcomes and indicators would need to be developed and included in the program performance framework.
3. Identification of additional data needs to track and assess additional outcomes or external factors where feasible.
4. Better targeting calls for proposals through a stronger evidence base and a clearer understanding of supply chain health and what types of actions can help address problem areas.

Once the relationships between project impacts and trade corridors/supply chains are mapped out, typically the next step is to determine the extent to which project outcomes “move the needle” in terms of system-level supply chain metrics. Indeed, in interviews, the topic of quantifying the effect of project outcomes on the overall system was mentioned.

There is currently no mechanism to account for NTCF-funded projects in the analysis of system-level metrics. Should the program wish to pursue this, departmental metrics on fluidity and trade will need to be assessed to determine whether it is feasible and useful to incorporate data on short- and medium-term impacts of funded projects into calculations. This data would include information collected through common project indicators (whether actual measurements or forecasts reported by recipients), as well as additional data that may need to be collected to address gaps identified through the mapping out exercise.

Recommendation 1:

The NTCF should:

• Map out the relationship between project outcomes and the overall system-level objectives.
• Work with TEA to assess the feasibility of accounting for impacts of funded projects in trade corridors and supply chains metrics.

Considering and addressing the challenges discussed in this section will better position the NTCF in demonstrating the long-term impact of the program and targeting future investments.

2.1.2.2 Role of TC's Transportation and Economic Analysis group

Finding 4: The involvement of Transport Canada’s TEA group in NTCF decision-making processes is a key strength of the program.

Internal interviews found that the role played by TEA in supporting the NTCF is valuable and adds rigour to the program. TEA and the analyses it carries out support the NTCF during three stages: call design, proposal evaluation, and Ministerial briefings (Figure 2).

Figure 2. Areas of TEA involvement in the NTCF

Description

Figure 2 serves as a guide to the involvement of Transportation & Economic Analysis (TEA) in NTCF. Each stage of an NTCF call is listed as a timeline: Call Design, Call Launch, Application Intake, Proposal Evaluation, Recommendations & Approval, Notifications & Announcements, Project Implementation, and Project Close-out. Three out of eight stages are highlighted to note where TEA is involved, specifically: Call Design, Proposal Evaluation, and Recommendations & Approval. In the Call Design stage, TEA collects and analyzes data on the transportation sector, including labour issues, logistics, and efficiency, providing information to identify current and future pressures to help NTCF identify priorities and needs.

During the Proposal Evaluation phase, TEA, alongside other members of the evaluation team, assesses expressions of interest, assigns pass/fail scores, and contributes to the review of comprehensive project proposals submitted by applicants. In the Recommendations and Approval stage, when eligible and recommended projects are presented for approval, TEA offers additional context on trade and transportation infrastructure considerations based on departmental research, analysis, and stakeholder engagement.

Broadly speaking, TEA’s role in the department is to continually conduct research and analysis using a wide variety of data, from both internal (e.g., modal performance data collected under the Canada Transportation Act, safety and security data) and external (e.g., market intelligence, economic and socio-economic data, StatCan survey data, voluntary data from other departments or private stakeholders) sources. This information helps to identify bottlenecks and system pressures, performance, economic conditions, and various other trends and considerations regarding the Canadian transportation system. Region-specific challenges, needs, and opportunities are identified in Regional Transportation Assessments.

During call design, the information and analyses from TEA is used to identify supply chain priorities and inform the focus of the call.

Once expressions of interest (EOI) and, subsequently, Comprehensive Project Proposals (CPP) or Streamlined Project Proposals (SPP) are received, analysts from TEA are directly involved in the evaluation process. Initially, this support was more ad hoc but beginning in 2019 with the third call for proposals, TEA analysts formally joined multidisciplinary evaluation teams alongside NTCF Programs staff, regional analysts, and modal experts. As needed, subject matter experts from other teams are also involved. Proposals are evaluated using specific evaluation criteria, and evaluation teams review, meet, and discuss to come to a consensus on a final score for each proposal. Interview evidence highlights that TEA analysts are well placed to provide additional data and system insights, and often are able to validate or comment on data provided by applicants. In this way, the expertise of TEA positively contributes to the review process.

TEA has also supported the recommendation and approval process by providing supplementary material to be included in the briefing package for the Minister. This provides additional context and informs the Minister of key issues to consider in decision-making.

While the involvement of TEA in the program is clearly valuable, the evaluation notes that TEA does not receive results data or information about completed projects. Involvement of TEA at project close-out would provide them access to additional data source (i.e., project performance information) to draw from for departmental analyses. This would have the potential to strengthen NTCF performance measurement by facilitating an improved understanding of the relationship between project-level and system-level impacts.

2.1.2.3 Timelines

Finding 5: Uncertainty in timelines for NTCF project approval and funding has been identified as a concern for stakeholders.

Concerns from NTCF stakeholders were highlighted in recent reports (e.g., the final report of the National Supply Chain Task Force; Addressing Port Infrastructure Expansion in Canada, Report of the Standing Committee on Transport, Infrastructure, and Communities) and raised as a challenge by internal interview participants. Uncertainty in the length of time between project proposal and project approval can result in increased costs, difficulty in planning for materials, and changes to project timelines.

While, as noted by some interview participants, factors outside of program control (e.g., consultations, assessments) contribute to overall stakeholder uncertainty, this evaluation focuses on the timelines for NTCF approvals and funding.

Program representatives indicated that anticipated timelines for the review and approval of proposals are communicated to stakeholders during webinars. These details are generally not outlined in applicant's guides (Table 5). These documents explain the review process and criteria, but do not provide an estimate of the time required for review and approvals or an expectation of when results will be communicated to applicants. This could contribute to the sense of uncertainty stakeholders experience.

Table 5. Dates and service standards indicated in applicant's guides

Step	Call 1	Call 2	Call 3	Call 4	Call 5	Call 6	Call 7
EOI deadline	5 Sept 2017	15 Jan 2019	9 Dec 2021	21 Dec 2020	31 Mar 2022	N/A	N/A
Acknowledge receipt of EOI	Within 10 business days	Not specified	Automatic email	Not specified	Not specified	N/A	N/A
Communicate result of EOI review	Within 30 days of submission	Within 20 days of submission	Not specified	Within 20 days of submission	Not specified	N/A	N/A
CPP/SPP deadline	6 Nov 2017	29 Mar 2019	9 Dec 2021	15 Mar 2021	30 Jun 2022	25 Feb 2022	11 Apr 2023
Acknowledge receipt of CPP/SPP	Within 10 business days	Within 10 business days	Within 5 business days	Within 10 business days	Within 10 business days	Not specified	Within 10 business days
Communicate result of CPP/SPP	Not specified	Not specified	Not specified	Not specified	Not specified	April 2022	Not specified

We note there is currently no formal service standard for NTCF funding decisions. At TC, there are two mandatory service standards for grants and contributions, which the NTCF follows: ^{Footnote 13}

1. Acknowledging receipt of an application within 10 business days
2. Issuing payment within 20 business days

The department is currently updating service standards, which may include a standard for funding decisions, including one for the NTCF. This could support clearer expectations for applicants regarding timelines.

Though not presented in detail in applicant's guides, the NTCF sets internal deadlines and estimations for its review and approval processes. Our analysis shows that for Calls 3, 5, and 6, the actual time required for proposal review has been longer than expected timelines. This is particularly true for Call 5, where 92 days were estimated but in practice took 151 days between the closing date of the call and sending the recommendations to the Minister for approval (Figure 3). Reasons for this include a high volume of proposals and, as explained by program representatives, the decision to redirect some applicants from Call 6 to be considered under Call 5 (this decision, which is reasonable, was not under program’s control). As these redirected proposals were received in September 2022, this necessarily extended the review period for that call.

We focused on the comparison between planned versus actual time required for proposal reviews – the portion that is generally under the program’s control – rather than a comparison of overall timelines that include approvals. Our rationale was that the program has little control over timelines after it provides funding recommendations to the Minister.

Figure 3. Estimated and actual length of NTCF calls for proposals

Description

Figure 3 is a bar graph which illustrates estimated and actual timelines for Calls for Proposals from 2 to 6, as well as the number of applications received.

Call 2 received 31 proposals. The Expected Timeline was 134 days and the Actual Timeline was 62 days. This included a Program Review Period of 73 expected and 56 actual days.

Call 3 received 79 proposals. The Expected Timeline was 70 days and the Actual Timeline was 54 days. This included a Program Review Period of 28 expected and 45 actual days.

Call 4 received 33 proposals. The Expected Timeline was 119 days and the Actual Timeline was 123 days. This included a Program Review Period of 89 expected and 93 actual days.

Call 5 received 129 proposals. The Expected Timeline was 92 days and the Actual Timeline was 176 days. This included a Program Review Period of 92 expected and 151 actual days.

Call 6 received 21 proposals. The Expected Timeline was 64 days and the Actual Timeline was 111 days. This included a Program Review Period of 21 expected and 34 actual days.

Timeline analysis: Method and Limitations

The estimated lengths of time for CPP review and approval are based on timelines shared in launch memos to the Minister. In the launch memos for Calls 2, 4, and 5, estimated timeframes were presented as a month or season. For these calls, we selected dates which assumed the greatest number of days allocated (e.g., the last day of the specified month). The launch memo for Call 6 specified dates and Call 3 specified a number of weeks allocated for review and approvals for each CPP. Data was not available for Calls 1 or 7. In Figure 3, day 0 indicates the date the call was predicted to close and, if no date was specified, we used the actual date the call closed.

Actual times for review and approval are based on actual dates in recommendation memos to the Minister: the date the call closed, the date the program began routing recommendations for internal approvals, and the date the Minister signed. Call 3 differed from the other calls as CPPs were accepted on a continuous basis and thus were reviewed and approved in 19 batches. For this call, we used the dates the CPPs were received instead of the call closing date and calculated the average number of days for both review and approval stages of all CPPs. The timeline presented for this call in Figure 3 is therefore an average of all CPPs from batches 1 to 11 and excludes batches 12 to 19 due to insufficient data.

While each call has a unique purpose, structure, and context, our analysis shows that, for three out of five calls, once a call for proposals has closed, it has taken longer than anticipated for the program to review the applications. This may have contributed to uncertainty for stakeholders.

Recommendation 2:

The NTCF should improve communications with stakeholders to enhance predictability about the application and approval process.

2.1.3 Effectiveness

Finding 6: More time is needed for the impacts of NTCF-funded projects to materialize.

As of January 2024, 183 projects have been approved for funding, representing a commitment of $3.946 billion.^{Footnote 14} Of these, relatively few (30) have been completed at the time of writing, with the majority (22) falling under the first call for proposals (National Call to Address Capacity Constraints) (Table 6). These completed projects represent only about 5% ($184.54 million) of the total funding envelope. According to one interviewee, they were mostly small in scale, consisting of new infrastructure and upgrades or repairs to existing infrastructure, as well as studies and design projects.

Table 6. Number of proposals and funded projects per call

	# of EOI received	# of CPP/SPP received	# of projects funded	# of projects completed
Call 1	358	177	39	22
Call 2	47	31	11	1
Call 3	264	79	63	7
Call 4	41	33	16	-
Call 5	182	129	45	-
Call 6	N/A	21	9	-
Total:	892	470	183	30

The evaluation team reviewed a sample of final reports from the completed projects to get a sense of results reported by the recipients. Projects reviewed covered different regions in Canada and represent various transportation modes (i.e., rail, airports, marine ports, and roads). Results have included:

• Additional capacity (e.g., 60,000 feet of rail, 40,000 square feet of warehouse space)
• Reductions in wait times (e.g., decrease of 17% from previous year)
• Enabling the continued operation of ageing infrastructure (e.g., improvements to drainage, replacing rail ties)
• Improved safety (e.g., decrease in road corridor collision rate from previous year)

Along with actual results, proponents included forecasts of results as appropriate. Of the reports we reviewed, four included quantitative forecasts of project impacts, including:

• An increase of rail carload traffic by 30,000 in 5 years and 125,000 in 20 years
• An increase of 12 average vessel calls per year
• An increase in traffic volume by 3 650 carloads annually

The above shows a sample of results achieved through 30 projects completed to date. As more and more projects are completed and NTCF performance measurement is strengthened as discussed in the report, the program should be in a stronger position to demonstrate effectiveness.

2.2 TTIS

For TTIS, we report on the program:

• Relevance
• Delivery
• Effectiveness

2.2.1 Relevance

Finding 7: TTIS is well-aligned with TC’s mandate and priorities.

Reliable and detailed data helps both government and industry to understand their operating contexts and identify key risks and priorities. The 2015 Canada Transportation Act Review^{Footnote 15} found that accurate, real-time transportation data accessible to all stakeholders will strengthen decision-making and collaboration. In 2016, TC’s public consultations underscored this need. The 2022 Supply Chain Task Force reiterated the importance of timely data for supply chain decision-making, with the final report^{Footnote 16} highlighting that increasing efficiency in Canada’s supply chain requires collaboration between all stakeholders and a clear, real-time picture of each aspect of the transportation supply chain. TC acknowledges data as a priority in the strategic plan Transportation 2030^{Footnote 17} and subsequent Departmental Plans, which outline the department’s work on data collection, accessibility, and analytics for evidence-based decision-making in support of Mandate Letter commitments and program delivery.

2.2.2 Delivery

Finding 8: The implementation of the TTIS evolved as challenges were encountered.

The TTIS was implemented through the establishment of the Canadian Centre on Transportation Data (CCTD) and its Transportation Data and Information Hub (TDIH), a partnership between TC and Statistics Canada (StatCan). The two signed a Memorandum of Understanding in 2018 outlining roles, governance, data sharing, and a work plan. By jointly establishing the CCTD, this strategic partnership aimed to leverage the data expertise and infrastructure of StatCan and the transportation knowledge of TC, while ensuring a single federal strategy for addressing transportation data needs and management.

The first phase of the TTIS involved increasing access to existing data sets for external stakeholders. Available transportation data from both TC and StatCan were published through StatCan's website. However, the outbreak of the COVID-19 pandemic in 2020 and the ensuing transition of many federal departments and agencies to remote work had an impact on the partnership between TC and StatCan and the implementation of the TTIS. Interview participants noted that StatCan had to direct attention to other priorities while also establishing remote work capabilities.

Over time, TC prioritized the need for quick data updates, the use of third-party data, and more interactive features for the web platform, which was not compatible with data publication processes and structures at StatCan. The TDIH was therefore moved to TC’s infrastructure, allowing for the use of interactive dashboards and visualizations, intended to enhance user experience (Figure 4).

Figure 4. Example of dashboard available through TDIH 2.0

Description

Figure 4 is a screenshot from the Transportation Data and Information Hub (TDIH), demonstrating the interactive dashboards that are available. The example includes a pie chart, bar chart, tables and highlights of certain statistics outlining aircraft movements and passenger counts at certain airports.

Although the TDIH has transitioned to TC’s website, TC and StatCan continue to collaborate on the TTIS and other transportation data priorities.

2.2.3 Effectiveness

Finding 9: TTIS has succeeded in increasing the availability of transportation data and there is a clear understanding of information gaps and next steps.

Making data accessible to stakeholders is the intended immediate outcome of the TTIS. As of 2023, the TDIH provides public access to over 600 data sets, including traffic and volume data by mode, safety and security data (e.g., accidents, enforcement actions), and environmental data (e.g., greenhouse gases and pollutants emissions by mode). Performance metrics and various analyses are available as well, such as:

• Freight flows by commodity, geography, and mode
• End-to-end transit time, dwell time, and inventories of containers
• Freight forecasts
• Air travel performance (e.g., wait times at screening points, on-time performance by carrier/route)
• Rail performance (e.g., transit time)
• Economic impacts of disruptions

With the migration of the hub from StatCan to TC’s infrastructure and cloud-based environment, the TDIH incorporates near real-time updates, interactive maps and customizable data visualizations and dashboards.

Moving forward, TEA intends to address the remaining gaps in modal data (e.g., trucking, short line rail), enhance information-sharing on demand and capacity, and work towards real-time supply chain visibility. Improved analytics and forecasting for freight, air passenger outlook, and economic impacts will also be a priority. One interviewee highlighted that the TTIS has moved from data providing “hindsight” to “insight” and is now working towards “foresight” (i.e., being better able to forecast and anticipate future trends and issues). In addition, planned improvements to the TDIH platform itself, such as real-time reporting and the use of artificial intelligence, will advance information dissemination and usability for stakeholders.

The TTIS centralizes and disseminates data currently tracked by TC and StatCan. With more information being made available to stakeholders, the focus will shift to understanding whether and how that information is being accessed and used. The next stage will be to actively engage with stakeholders to increase their awareness of the data platform and to identify information needs. TEA intends to use web metrics and user feedback to track which types of stakeholders are accessing the site and what type of data they are looking for.

2.3 ACATS

For ACATS, we report on the program:

• Relevance
• Effectiveness

Under Effectiveness, we focus on the impacts of the program's five years of funding under the TTCI, though we note that ACATS has since been renewed through 2026-27.

2.3.1 Relevance

Finding 10: While ACATS is relatively well-aligned with TC’s strategic objectives, part of the funding it provides is not strongly linked with any legislated TC mandate.

Connected and automated vehicles have the potential to improve the safety, efficiency, and accessibility of transportation and, in this sense, the ACATS program aligns well with TC’s strategic objectives.

There is however a slight misalignment between some program investments and legislated TC mandate. In Canada, responsibility for road safety is divided between multiple levels of government: provinces, territories, and municipalities are responsible for road infrastructure and traffic management, while TC’s mandate is limited to vehicles, specifically regulating and overseeing the manufacture and import of motor vehicles and related equipment.

Some of the projects funded by the program extend beyond this mandate and into areas like Intelligent Transportation Systems. These are information and communications networks designed to address issues like congestion or traffic accidents and typically fall under provincial, territorial, or municipal mandates.

Despite the lack of a strong link to legislated TC mandate, there is a strong policy justification for ACATS involvement in Intelligent Transportation Systems or other similar approaches. Ensuring that connected and automated vehicles can be safely integrated into the road system – a key objective of ACATS – cannot reasonably be achieved by focusing solely on vehicles. The connection between roads and vehicles must be taken into account.

Most interviewees we spoke to about the program highlighted the need for a unifying body to support consistency at the provincial/territorial and municipal levels, as well as to enable knowledge-building and information-sharing amongst stakeholders. Key considerations raised in literature and in interviews include cybersecurity, clarity and coordination between different jurisdictions, capacity and training for personnel, and adaptability in infrastructure and planning. A program like ACATS is well-positioned to support these needs.

2.3.2 Effectiveness

Finding 11: Through its first call for proposals, ACATS contributed to awareness and knowledge-building for stakeholders and reached the targets for its immediate and intermediate expected outcomes.

Internal documents from the program track and outline progress towards intended outcomes. The immediate and intermediate outcomes of ACATS focus on (i) awareness building and (ii) capacity to develop policies, regulations, and plans related to the use and deployment of connected and automated vehicles (Table 7). Work done under ACATS in its initial five-year funding period clearly contributed to these goals. The focus to date has been on knowledge- and capacity-building, rather than operational deployment of connected and automated vehicle technologies.

Table 7. Intended outcomes of ACATS ^{Footnote 18}

	Outcome	Indicator	Target
Immediate	Canadian jurisdictions are more aware of the safety, security, environmental, social, and other considerations of connected and automated vehicles	Percentage of respondents reporting increased awareness	Reported/observed increased awareness as a result of ACATS initiatives (increasing trend)
Intermediate	Capacity to develop policies, regulations, guidelines, and strategies to support the safe and secure deployment of connected and automated vehicles is enhanced	Number of policies, regulations, guidelines, and strategies developed by Canadian jurisdictions	3-5 policies, guidelines, or strategies
Ultimate	A transportation system that supports innovation	Number of new transportation technology deployments in Canada	Increasing overall trend

Through one call for proposals in 2017-18, ACATS received 38 applications and ultimately funded 15 projects (Table 8).

Table 8. List of projects funded through ACATS

Recipient	Grant or Contribution	Funding Amount
Canadian Automobile Association	Grant	$25,000
City of Saskatoon	Grant	$25,000
Carleton University	Grant	$50,000
City of Calgary - Project #2	Grant	$50,000
Unmanned Systems Canada	Grant	$50,000
Canadian National Institute for the Blind	Grant	$50,000
Ministry of Transportation of Ontario	Contribution	$111,025
City of Ottawa	Contribution	$140,000
Canadian Urban Transit Research & Innovation Consortium	Contribution	$144,540
Intelligent Transportation System Society of Canada	Contribution	$247,275
City of Calgary - Project #1	Contribution	$290,000
City of Toronto	Contribution	$365,000
City of Vancouver	Contribution	$386,500
Canadian Standards Association	Contribution	$499,999
University of Alberta	Contribution	$500,000

A review of final reports from each project highlights that all aligned with the program’s desired outcomes, as the majority of projects focused on either awareness and understanding or pilot testing and feasibility studies. For example, the Canadian Automobile Association designed an online educational tool providing Canadians and stakeholders with information on connected and automated vehicles, and the Canadian National Institute for the Blind set out to understand and communicate to stakeholders how pedestrians who are blind or partially sighted will be impacted by the emergence of autonomous vehicles. Examples of pilot testing include projects by both the City of Ottawa and the City of Calgary, who installed and trialed Vehicle-to-Infrastructure (V2I) or Infrastructure-to-Vehicle (I2V) technology, while the City of Calgary also undertook a project to pilot an automated shuttle.^{Footnote 19} In addition, three projects supported the medium-term outcome of ACATS by developing guidelines, standards, and plans. For example, the Ontario Ministry of Transportation developed a Connected and Automated Vehicles Readiness Plan, and the Canadian Standards Association developed initial safety guidelines and a standardization roadmap.

Four external interviewees also highlighted that the impact of these projects is broader than the individual projects as they collectively increase public awareness through exposure to pilot projects and media coverage or by generating local political interest.

Most external interview participants (4 of 5) noted the importance of ACATS for enabling collaboration and connection between stakeholders, which they stated would not have occurred otherwise. Beyond funding projects, ACATS staff have worked to promote knowledge-sharing by hosting and supporting workshops, delivering presentations at various events and conferences, and collaborating with international and domestic partners (e.g., Transportation Association of Canada, United States Department of Transportation). The latter also builds internal expertise for TC.

External interviewees shared a few challenges, including:

• Lack of awareness or capacity among small municipalities to engage with connected and automated vehicles projects or seek project funding (3 of 5 interviewees)
• Lack of sustained funding beyond the pilot project phase, to enable implementation and operations (3 of 5 interviewees)
• Need for more follow-up following presentations and engagement, to turn awareness into action (2 of 5 interviewees)
• Unique challenges in rural areas that will require different solutions than urban use of connected and automated vehicles (3 of 5 interviewees)

Beyond the TTCI, ACATS was renewed through Budget 2022 and again in Budget 2023, with $7.4 million in funding over three years (2023-24 to 2026-27), including $1.7 million for grants and contributions. The next iteration of ACATS will continue to emphasize knowledge-building and information dissemination with particular focus on:

• Interoperability and cyber security;
• Helping road authorities build capacity and access qualified staff with the necessary skillsets for connected and automated vehicles and Intelligent Transportation Systems; and
• Technology research, testing, and trials.

Importantly, ACATS will place increased emphasis on rural or remote areas, and on accessibility and safety for persons with disabilities.

2.4 RPAS

For RPAS, we report on the program:

• Relevance
• Effectiveness

Under Effectiveness, we focus on the results of the RPAS Task Force^{Footnote 20} through its five years of funding under the TTCI. We note that since the completion of this work, additional funding has been allocated to RPAS.

2.4.1 Relevance

Finding 12: There is a clear need for TC involvement in RPAS and the program is well-aligned with TC’s mandate.

Since 2017, the RPAS sector has expanded quickly, from primarily recreational use to diverse and growing commercial operations. Drones now outnumber traditional aircraft in Canada’s skies.^{Footnote 21} As drones emerged into Canadian airspace, it became increasingly clear that TC regulation and oversight had to evolve to account for the risks and benefits of this technology.

Because RPAS are considered aircraft, TC’s role is clear: the department aims for safety and security in the air transportation system, and this requires accounting for the presence and use of drones. TC is thus directly responsible for regulating and overseeing the use of this technology. The department continues to be allocated funding for RPAS (e.g., Budget 2023) and incorporates this funding into its annual Departmental Plan. In addition, TC works to enable innovation in transportation, which is accomplished through understanding and supporting emerging technologies such as RPAS.

Three external interview participants mentioned needs related to the future of drone technology, such as further work on standards, dangerous goods, passenger use (e.g., air taxis), larger and more automated RPAS, and changing skill sets for operators. Literature also highlights the range of current and potential uses – from environmental monitoring to package delivery, recreational to academic – all of which need to be considered from the departmental lens of safety and security. As novel uses emerge and demand continues to increase, there is a continued need for TC to understand these potential uses and respond through appropriate regulation, oversight, and guidance.

2.4.2 Effectiveness

Finding 13: The RPAS Task Force has by and large advanced its objectives related to regulatory development, authorizations, and research and development.

TC’s RPAS immediate outcomes focus on capacity (to develop regulations and issue certifications/authorizations), and engagement (Table 9). In the medium term, the use of RPAS would increase and the industry would continue to expand. Work related to research and development is also key in supporting these outcomes.

Table 9. Intended RPAS outcomes^{Footnote 22}

	Outcome	Indicator	Target
Immediate	Capacity to develop policies, regulations, guidelines, and strategies to support the safe and secure deployment of RPAS is enhanced	Number of policies, regulations, guidelines, strategies that enhance Canada's readiness to accept new technology	Publication of three regulatory amendments to the Canadian Aviation Regulations
			Publication of ten standards and/or guidelines
	More predictable and timely services (authorizations) to the RPAS sector	Number of authorizations issued for RPAS operations	Not specified
			Authorizations meeting service standards 90% of the time
	TC engagement in RPAS fora is increased and more effective	Increased participation in RPAS-related meeting and panels	100% increase in participation (to be reassessed each fiscal year)
Intermediate	Canada's transportation system embraces innovative technologies	Barriers to RPAS deployments in Canada are identified and addressed as required	Not specified
		Percentage growth of Canada's RPAS industry	Not specified
Ultimate	(TTCI overall) Traditional and innovative investments in trade-related infrastructure help sustain and support environmental responsible economic growth and jobs for middle class Canadians	No indicator specific to RPAS

Regulatory development

The RPAS Task Force made progress in regulatory development but did not achieve its original target of three regulatory amendments by 2022. One amendment to the Canadian Aviation Regulations, covering RPAS less than 25kg within visual line of sight, came into force in 2019. In 2023, the next set of regulations was pre-published in Part I of the Canada Gazette. These will cover medium-sized RPAS and lower risk beyond visual line of sight (BVLOS) operations. Some parts will come into force upon publication in Part II of the Canada Gazette, with the remainder coming into force on April 1, 2025.

External interviewees (3 of 5) stated that the first set of regulations provides a good foundation. While the specific target for regulatory amendments was not met, the Task Force also worked on regulatory exemptions to support program implementation, and on guidance material for stakeholders.

RPAS brings together the fast-moving technology industry and traditional aviation. As a result, the RPAS Task Force needed to balance safety considerations, the length of the regulation process, and the pace of innovative developments. External interviewees had mixed opinions on the speed at which RPAS regulations are developed: one participant noted that the work on BVLOS was about two years behind schedule, while another felt the length of time was reasonable. There was general consensus that industry overall would want it to be faster, though some highlighted that those with experience in traditional aviation understand the amount of time the regulatory process can take.

Authorizations

From January 2019 until December 2023, over 96,000 RPAS have been registered^{Footnote 23} and over 108,000 pilot certificates^{Footnote 24} have been issued. Requests for authorizations and certifications can be quickly submitted and managed online via the Drone Management Portal, enhancing service to stakeholders. There is a service standard of 10 business days for:

• Issuing a certificate of registration
• The conduct of taking or retaking an examination for a pilot certificate or flight reviewer rating
• Issuance of a pilot certificate – small remotely piloted aircraft (VLOS) - advanced operations
• Endorsement of a pilot certificate – small remotely piloted aircraft (VLOS) - advanced operations with a flight reviewer rating

Program data indicates this service standard has been met 99.79% of the time.^{Footnote 25}

Research and development

To support data gathering and allow industry to operate RPAS in advance of specific regulations, the RPAS Task Force issues Special Flight Operations Certificates (SFOC), which are one-time operational approvals for commercial RPAS operations. SFOCs enable industry to trial new uses and gain operating experience. They also allow the RPAS Task Force to work with stakeholders to better understand risks and mitigations, providing evidence for regulatory development. Since 2019, following the implementation of the first phase of RPAS safety regulations, 2 596 SFOCs^{Footnote 26} have been issued. TC has also established two RPAS test ranges in Quebec and Alberta, providing a safe, controlled environment for operators to test new technologies and procedures.

The RPAS Task Force also collaborated with the National Research Council (NRC), which has technical expertise in RPAS through its previous Civil Unmanned Aerial Systems program and its ongoing Integrated Aerial Mobility Program. This work aimed to gather data to form the basis of regulations and to foster collaboration between government, industry, and academia. Each project had an Interdepartmental Financial Agreement to define scope and funding. Research topics included collision probability and severity (including human injury for different sexes, age, weights, etc.), the effect of various environmental factors (e.g., wind, icing), trials of detect-and-avoid systems, airspace modelling to better understand airspace risk, and the impact of noise from RPAS on humans, among other topics (see Appendix B).

Finding 14: While the RPAS Task Force has strong engagement with industry stakeholders and other governments, more awareness among recreational users is needed.

TC also aimed to increase its participation in RPAS-related meetings. The RPAS Task Force has been active in its engagement of industry, academia, and other governments:

• Stakeholder engagement through regulatory consultations. In 2019, consultations on future regulations were held in person. This involved a collaborative two-day workshop called Drone Talks, followed by the opportunity for those who could not attend to provide written comments. Virtual consultations were held during the COVID-19 pandemic, which internal interviewees noted as an effective approach to reach more stakeholders.
• Canadian Drone Advisory Council (CanaDAC). The RPAS Task Force established CanaDAC in 2021 for a two-year term, bringing together a variety of experts and stakeholders, including academics, manufacturers, operators, traditional aviation operators, first responders, and representatives from provinces and municipalities. Most (3 of 5) external interviewees stated that CanaDAC was valuable and brought together many people with diverse backgrounds. Two noted that at times it could be a challenge to get all participants to fully engage and contribute, though this was not due to TC.

• Engagement with international counterparts. One internal interviewee highlighted the value of engaging with international partners, noting that “everyone is struggling with the same regulatory problems”. TC has bilateral relationships with civil aviation authorities from the USA, Australia, and the European Union, and is involved with the International Civil Aviation Organization’s (ICAO) RPAS Panel, Unmanned Aircraft Systems Advisory Group, and Advanced Air Mobility Group as well as being co-chair of the Joint Authorities for Rulemaking on Unmanned Air Systems (JARUS).

  Given the shared border between Canada and the USA, the RPAS Task Force regularly meets with US representatives. For instance, they meet with the US Federal Aviation Administration twice a year and collaborate through the Cross-Border Working Group to discuss key issues, including harmonization and the potential for RPAS crossing the border. The RPAS Task Force has also established a close relationship with counterparts in Australia, given the two countries have similar operating contexts (i.e., relatively small populations across a large physical area, with both densely populated urban areas and open environments).

Engagement with the general public has been a challenge. The RPAS Task Force has undertaken various outreach and engagement activities, intended to increase knowledge of rules and best practices for drone use and to enhance public trust in the technology. This work has included:

• Annual Drone Safety Awareness campaigns, which have used outreach techniques such as a webinar series and a photography challenge
• Social media posts
• Educational materials and guidance
• Materials for retail partners to distribute
• Brief “Know Before You Go” videos on YouTube

Despite these efforts, program representatives indicate that success has been limited. Program documents and interviews noted the issue of relatively low engagement from the public with videos and social media. Some external interviewees (2 of 5) also flagged public awareness as a current weakness, sharing observations of misinformation and lack of clarity among recreational users. New methods and more direct engagement could be implemented: for instance, one suggestion was that outreach could be customized and targeted to specific groups (e.g., clubs, schools).

Recommendation 3:

The RPAS Task Force should identify impediments to effective engagement and information dissemination and develop a strategy to meet its public engagement objectives.

2.5 GBA Plus and Official Languages observations

As noted at the outset of the report, while specific GBA Plus and Official Languages considerations were not identified during the evaluation planning phase, these lenses were kept in mind during conduct. Although findings did not emerge, the evaluation team notes the following:

• New technologies can have positive and negative implications for certain groups, such as individuals with disabilities. This was raised in the context of ACATS. For instance, automated vehicles present an opportunity for increased mobility for those with visual impairments and other disabilities, as well as other groups such as the elderly. However, automated vehicles may also pose a risk to pedestrians with visual impairments. Research by the Canadian Institute for the Blind,^{Footnote 27} partially funded through ACATS, highlights a number of considerations, such as the need for consistent and predictable vehicle behaviour, algorithms which account for different needs and behaviours of vulnerable road users and react accordingly, and accessibility of onboard vehicle interfaces. The program is aware of needs such as these and intends to support research and testing which considers persons with disabilities as part of its second call for proposals.
• Project and administrative data from the NTCF do not reveal any major discrepancies in the regional distribution of funding (Figure 5). Given the priorities of the program, it does not aim to evenly distribute funds across regions, but rather target key needs and issues related to transportation corridors. Importantly, the NTCF did allocate a proportion of funding to support projects in the Arctic and Northern regions, recognizing the specific context and needs of the North. The North has a project count and amount of funding comparable to other major regions such as Ontario or Quebec.

Figure 5. Distribution of committed NTCF funding to date by region^{Footnote 28}

Description

Figure 5 is a pie chart that shows the distribution of NTCF funding across various regions in Canada. The financial allocations are as follows: Projects in the North received the largest share with $811M, constituting 26.45% of the total NTCF funds distributed to date. Pacific region follows with $540M, representing 17.62%, while projects in Quebec received $524M, accounting for 17.08%. The Prairie and Northern Region has been allocated $473M, comprising 15.41% of the funding. Atlantic projects received $342M, constituting 11.16%, and projects in Ontario have been allocated $337M, representing 11%. A smaller portion of $39M, or 1.28%, has been allocated to National projects.

3.0 Summary of Findings and Recommendations

The evaluation found that there continues to be a clear need and general alignment with TC’s mandate and federal government priorities for all components of the TTCI.

For the NTCF, we found:

• The NTCF has taken meaningful steps to improve its performance measurement practices by requiring all projects to report on common indicators. However, the utility of these indicators will depend on the quality of data provided by funding recipients.
• The expected contribution of NTCF project-level results to desired system-level changes is not clearly defined.
• The involvement of Transport Canada’s TEA group in NTCF decision-making processes is a key strength of the program.
• Uncertainty in timelines for NTCF project approval and funding has been identified as a concern for stakeholders.
• More time is needed for the impacts of NTCF-funded projects to materialize.

For the TTIS, we found:

• The implementation of the TTIS evolved as challenges were encountered.
• TTIS has succeeded in increasing the availability of transportation data and there is a clear understanding of information gaps and next steps.

For the measures to support transportation innovation, we found:

• Through its first call for proposals, ACATS contributed to awareness and knowledge-building for stakeholders and reached the targets for its immediate and intermediate expected outcomes.
• The RPAS Task Force has by and large advanced its objectives related to regulatory development, authorizations, and research and development.
• While the RPAS Task Force has strong engagement with industry stakeholders and other governments, more awareness among recreational users is needed.

With these findings in mind, we make the following recommendations:

Recommendation 1

The NTCF should:

• Map out the relationship between project outcomes and the overall system-level objectives.
• Work with TEA to assess the feasibility of accounting for impacts of funded projects in trade corridors and supply chains metrics.

Recommendation 2

The NTCF should improve communications with stakeholders to enhance predictability about the application and approval process.

Recommendation 3

The RPAS Task Force should identify impediments to effective engagement and information dissemination and develop a strategy to meet its public engagement objectives.

Appendix A. Management Action Plan

This Management Action Plan addresses the recommendations of the evaluation with proposed actions, forecasted completion dates and the Office of Primary Responsibility (OPI).

Recommendation	Proposed Actions	Forecast Completion Date	OPI
The NTCF should: Map out the relationship between project outcomes and the overall system-level objectives. Work with TEA to assess the feasibility of accounting for impacts of funded projects in trade corridors and supply chains metrics.	The Program accepts the recommendation.
	The Program has designed and implemented a results tracking tool to improve NTCF project data collection and outcomes reporting at the NTCF project level. The Program will compile and analyze the data and share the results with TEA to assess the feasibility of accounting for the impacts of NTCF funded projects in trade corridors and supply chains metrics, at the transportation system level, through outcome mapping and determine which tools can be used or developed for the mapping.	Fall 2025	ADM, Programs
The NTCF should improve communications with stakeholders to enhance predictability about the application and approval process.	The Program accepts the recommendation.
	The Program will improve communications with stakeholders through early engagement sessions. Call specific timelines will be communicated during technical webinars and included in the application guides posted to the Program website. The Program is also available to respond to stakeholder inquiries as well as questions on specific timelines through the Program’s generic email address. The Program will also look at how best to communicate to all stakeholders and applicants any unforeseen delays in the application assessment and approval process, including timing of notification for unfunded proposals.	In advance of the next call for proposals	ADM, Programs
The RPAS Task Force should identify impediments to effective engagement and information dissemination and develop a strategy to meet its public engagement objectives.	The RPAS Task Force accepts this recommendation and notes the success of public engagement requires collaboration with the Transport Canada (TC) Communications Group.
	Complete a review that assesses the effectiveness of previous efforts, identifies impediments to success, and evaluates what other jurisdictions are doing. The review will include input from all relevant stakeholders.	October 2024	ADM, Safety and Security
	Following the review, and in collaboration with the TC Communications Group, create a strategy – including performance indicators – for identifying and meeting public engagement objectives with respect to effective engagement and information dissemination.	March 2025	ADM, Safety and Security
	Implement the strategy and measure effectiveness against identified performance indicators. Identify future outreach needs and include activities and associated resources in proposal to extend RPAS activities beyond current funding (ends March 2026).	December 2025	ADM, Safety and Security

Appendix B. List of RPAS Research and Development Projects

Area of research	Specific project reports
Drone impact assessment – Collision severity	Part 25 drone impact assessment report
	Part 23 drone impact assessment report
	Drone impact damage assessment on a legacy transport aircraft structure: empennage testing
	Drone impact assessment on aircraft structure: flat plate testing and analysis
Canadian Airspace Modeling – Collision probability	Canadian Airspace Modeling – Phase 1 (PDF, 4.07 MB)
	Canadian airspace and aircraft equipment requirements (PDF, 2.92 MB)
Severe Weather - RPAS Icing	RPAS icing literature review
	Small RPAS propeller icing tolerance report (PDF, 4.25 MB)
	RPAS specific icing test rig development and calibration report
	Medium RPAS propeller icing tolerance report
	Investigation of tolerance for icing of UAV rotors/propellers (PDF, 6.56 MB)
	Report from work conducted in FY22-23 will be publicly available soon
Severe Weather – Urban Flow	Urban Air Flow Report (Operator guidance video)
	Urban Air Flow 1 – RPAS testing report (PDF, 4.26 MB) (inside wind tunnel test section)
	Urban Air Flow 2 – RPAS testing report (PDF, 4.53 MB) (outside using the outflow of wind tunnel)
Human injury severity	Investigating small RPAS ground impact injury severity criteria
	Modeling sRPAS system to head impact for investigating craniocerebral response
	Report from work conducted in FY21-22 and FY22-23 will be publicly available soon
RPAS Noise	Initial literature review on RPAS noise (PDF, 1 MB)
RTM trial support	State of the art in RTM technology for phase 2 trials (PDF, 2.46 MB)
Certification of Autonomy	Summary report (PDF, 1 MB)
Human Factor	Haptic/kinesthetic feedback in Remotely Piloted Aircraft Systems (RPAS) human-machine interaction: final report (PDF, 1.65 MB)
Detect and Avoid	LookNorth detect and avoid flight trials 2020/2021 (PDF, 6.08 MB)
	Airborne DAA flight trials 2022/2023 report
Social acceptance	Public Opinion Research Study on Drone Users’ Familiarity with the New Rules & the General Public’s Social Acceptance of Drones (PDF, 3.09 MB)
	Strategies to Improve the Social Acceptability of Drones | Max Bell School of Public Policy - McGill University
Yearly R&D program summary	Transport Canada RPAS R&D yearly progress report (2020-2021) - NRC Publications Archive - Canada.ca