Evaluation of the Grade Crossing Improvement Program

Table of Contents

• Executive Summary
  • Evaluation Scope and Approach
  • Major Findings, Conclusions and Recommendations
• Program Background and Profile
  • Program Management and Delivery
  • Program Resources
  • Evaluation Context
• About the Evaluation
  • Evaluation Rationale and Scope
  • Evaluation Methodology
• Evaluation Findings: Relevance
  • Continuing Need
  • Alignment with Federal Priorities and Departmental Strategic Outcomes
  • Alignment with Federal Roles and Responsibilities
• Evaluation Findings: Performance
  • Effectiveness of Key Program Activities
  • Effectiveness – Achievement of Expected Outcomes
  • Efficiency and Economy
• Conclusions and Recommendations
• Management Action Plan

Executive Summary

The evaluation of Transport Canada's Grade Crossing Improvement Program (GCIP) was conducted to meet the requirement under section 42.1 of the Financial Administration Act that all ongoing non-statutory programs of grants and contributions be reviewed every five years. The GCIP was last evaluated in 2009-10.

The GCIP is designed to improve safety through funding upgrades that will reduce the risk of accidents at high-risk road-railway (grade) crossings. The GCIP provides contributions of up to 50% of the eligible costs of crossing improvements completed by railways and road authorities, to a maximum contribution of $550,000 per project. Funded upgrades include improvements to warning signs, signals and systems, as well as roadway improvements at crossings. Since 2010-11, the GCIP has included a LED Special Contribution Program that funds the replacement of crossings' incandescent lights with light emitting diode (LED) units. Between 2009-10 and 2013-14, the program's contribution budget was between $10.9 million and $12.9 million a year.

The GCIP is delivered by the Transportation Infrastructure Programs directorate of Programs group. Prior to April 2012, the program was managed and delivered by the Rail Safety Directorate of the Safety and Security group. The Rail Safety Directorate continues to play a role in program delivery, including conducting outreach and prioritizing applications for funding.

Evaluation Approach and Scope

The evaluation included an analysis of program administrative and financial information, analysis of statistical data (including the application of a risk model to grade crossings), 21 interviews (with program management and staff, funding recipients, and other stakeholders), a review of program documents and academic literature, and four case studies of grade crossing improvements.

As per the Treasury Board's Policy on Evaluation the evaluation examined the relevance and performance of the GCIP, focusing on the years 2009-10 to 2013-14.

Major Findings, Conclusions and Recommendations

The GCIP remains relevant, as grade crossings are a continuing public safety issue and there remain a significant number of cost-effective upgrades that can be undertaken in the future. There is continued demand for the program from railways and road authorities, as demonstrated by the volume of applications for funding and the existence of a pending list of applications awaiting approval. Some of this demand is being driven by the new Grade Crossing Regulations, which include enforceable standards for grade crossings. The GCIP is unique in targeting Transport Canada funding to improving the safety of grade crossings. Infrastructure Canada funding is available for grade separations and short line railway upgrades, but its funding has not overlapped with the GCIP.

The GCIP aligns with federal priorities related to rail safety and support for infrastructure, and with Transport Canada's strategic outcome of a safe and secure transportation system. The GCIP aligns with Transport Canada's legislated authority as set out in the Railway Safety Act, which empowers the Minister of Transport to make contributions to improve the safety of grade crossings.

The GCIP continues to achieve its (short-term) outcome of improving grade crossings. Between 61 and 85 non-LED upgrades, and between 70 and 749 LED projects were approved for GCIP funding a year. The program most commonly funded improvements to Constant Warning Time circuits / train predictors, and additions of gates to crossings, in addition to the installation of LEDs. The GCIP has generally funded upgrades at the highest-risk crossings, according to risk modeling conducted for the evaluation.

The GCIP is increasing the safety of crossings. Based on the evaluation's risk model, over a 20-year lifespan, the GCIP improvements completed from 2009-10 to 2013-14 are predicted to result in 123 fewer accidents, 29 fewer fatalities, and 16 fewer serious injuries. Overall, the program has been cost-effective: the estimated benefits of the completed GCIP upgrades will exceed the costs of the upgrades by over $66 million over 20 years. However, accidents and fatalities at crossings persist, despite a long-term trend of reduced accidents. Several factors were said to be causing continuing crossing accidents, including increasing train and vehicle traffic. The disproportionate number of accidents and fatalities at crossings with active warning signals, which have the highest train and vehicle traffic, suggests the limits of this type of intervention in reducing crossing risk. At some high-risk crossings more costly interventions such as grade separations may be considered to further improve safety.

The cost to deliver the GCIP has decreased over the last five years. Less staff time is required for administration following changes to program delivery beginning in 2012-13. For example, post-project inspections by Rail Safety inspectors are no longer mandatory (with inspections carried out based on risk) and recipient auditing has replaced verification of project invoices by rail inspectors. Further administrative efficiencies could potentially be realized by the department through combining the GCIP with the Grade Crossing Closure Program (GCCP), which also provides transfer payments to increase rail safety at grade crossings.

Areas requiring attention were identified in the evaluation:

• Program communication and awareness – While railways appear to be aware and knowledgeable about the program, gaps remain in awareness and knowledge of the GCIP among road authorities. While Rail Safety has undertaken some outreach activities with municipalities, there is a continued need to build and sustain awareness of the program among all potential applicant groups.
• Project eligibility and prioritization – While Transport Canada's regional inspectors often work with railways and road authorities to identify potential GCIP projects, the eligibility criteria for program funding are broad and applicants may submit applications for safety improvements at any public grade crossings that have existed for at least three years. Traditionally, a surplus of applications for program funding has allowed the program to prioritize the highest-risk crossings for funding. The program as currently designed could see its ability to target funding to the highest-risk crossings diminish in the future should there be a reduction in the number of applications, or in the risk-targeting of applications.
• Headquarters' calculation of annual GCIP budgets for each region is not based on a rigorous assessment of crossing risk. Furthermore, across regions, there is no consistent, systematic risk-based approach to prioritization of applications for funding. The GradeX risk model and online application, developed for the department to undertake the identification of high-risk crossings, have not been fully operationalized.
• Program monitoring – Given recent changes in the design and delivery of the GCIP and in the program context, there is a particular need going forward to regularly monitor the GCIP to ensure it continues to remain effective. Program performance data should be collected and tracked related to project applications, completions and risk-targeting (by recipient type), as well as program expenditures, in order to facilitate ongoing program monitoring and reporting.

The evaluation includes the following four recommendations:

Program Background and Profile

Launched in 1988, Transport Canada's Grade Crossing Improvement Program (GCIP) is a contribution program that provides funding to railways and road authorities for upgrades at road-railway crossings. The objective of the GCIP is to improve the safety of public grade crossings on federally-regulated railways through the funding of upgrades that will reduce the risk of collisions, fatalities and injuries at crossings that represent the highest risk to the public.

Under the Railway Safety Act and related regulations, railway companies and road authorities are ultimately responsible for the maintenance and safety of grade crossings. Road authorities of public crossings include approximately 1,450 municipalities, almost 100 First Nation bands, as well as provinces and territories.^Footnote1 There are approximately 14,000 public grade crossings on federally regulated railways in Canada.^Footnote2

Program Activities and Expected Results

The GCIP provides a contribution of up to 50% of the eligible costs of a grade crossing improvement project, to a maximum contribution of $550,000. Eligible work under the program includes the improvement or relocation of a public crossing, in the interests of safety, on a line of a federally-regulated railway. Improvements can include upgrades to railway signals and systems (such as upgrading a passive Standard Reflectorized Crossing Signal to a flashing lights and bells (FLB) or flashing lights, bells and gates (FLBG) system) or road/civil upgrades (such as upgrades to approach roads, advance warning signs, pedestrian fences, and lighting).

Through providing funding to contribution recipients, the program is expected, in the immediate term, to improve/upgrade grade crossings. This is expected to contribute to safer crossings and, ultimately, to a safe rail transportation system. The logic model for the GCIP can be found in Annex A.

Program Management and Delivery

In accordance with Transport Canada's Policy-Program Continuum, the GCIP is primarily administered and delivered by Transportation Infrastructure Programs directorate within Programs group. Programs group receives and processes project applications; annually compiles a list of projects based on input from Rail Safety Directorate, and submits the list for ministerial approval; and drafts and manages contribution agreements, including reviewing invoices and making recommendations for payment. Programs group is also responsible for managing recipient audits.

The Rail Safety Directorate of Safety and Security group supports Programs group in program delivery. Railway Safety regional inspectors conduct meetings with railways and road authorities to discuss potential grade crossing improvement projects, and prioritize and recommend project applications in their regions up to the maximum regional allocation for that year.

Rail Safety Directorate at national headquarters calculates the GCIP funding allocation for each Transport Canada region each year based on each region's proportion of the rolling five-year average of fatal crossing accidents. It sends a call letter requesting each region prioritize applications for crossing improvements, reviews regions' prioritized projects lists, and helps to coordinate communication between regions and Programs group.

At both national headquarters and in the regions, Railway Safety also conducts railway safety promotion and communications activities, which include communicating with stakeholders about the GCIP at organized events.

A detailed description of program delivery roles, responsibilities and timelines can be found in Annex B.

Program Resources

Prior to 2009-10, the GCIP had an annual budget of $8.8 million, of which $7.1 million was contribution funding . As part of Budget 2009, the GCIP was allocated additional funding of $27.9 million over five years, including $24.8 million in additional contribution funding, as well as $5.8 million in additional ongoing contribution funding each year beyond 2013-14. As part of overall government deficit reduction, GCIP contribution funding was reduced to $10.9 million per year as of 2013-14.

Table 1 shows the GCIP's planned and actual spending from 2009-10 to 2013-14, as well as leveraged investment from railways and road authorities.

Table 1: GCIP Planned and Actual Spending, 2009-10 to 2013-14 ($ millions)

	2009-10		2010-11		2011-12		2012-13		2013-14		Total**
	Plan.	Act.	Plan.	Act.	Plan.	Act.	Plan.	Act.	Plan.	Act.	Plan.	Act.
Contributions	11.2	13.8	11.2	7.4	12.4	8.5	12.9	11.9	10.9	7.1	58.7	48.7
Salaries, EBP, Acc, OOC*	0.3	0.6	0.7	0.6	0.8	0.6	0.7	0.2	0.7	0.2	3.2	2.0
Total TC	11.5	14.3	11.9	8.0	13.2	9.1	13.7	12.1	11.7	7.3	61.9	50.8
Leveraged Investments	2.8	3.5	2.8	1.9	3.1	2.1	3.2	3.0	10.9***	7.1	22.8	17.5

*Employee Benefit Plan, Accommodations, Other Operating Costs

**Numbers may not add due to rounding.

*** Effective April 2013, stakeholders' share of project costs went from 20% to 50%.

Program Changes and Context

A number of changes to the GCIP were implemented over the period examined in the evaluation:

• The GCIP began funding the replacement at railway crossings of incandescent lights with light emitting diode (LED) units through the GCIP's LED Special Contribution Program. This new component of the GCIP, which began in 2010-11, followed the implementation of new standards for LED signal modules in grade crossings. Under this component of the GCIP, Transport Canada provides a contribution of 50% (formerly 80% prior to April 2013) of the total cost of eligible work or $600 per light unit, whichever is the lesser.
• Primary responsibility for program delivery within Transport Canada was transferred from Safety and Security group to Programs group as of April 2012.
• The Railway Safety Act was amended in spring 2013 to allow the Canadian Transportation Agency to enforce a 12.5% maximum share for road authorities for grade crossing improvement projects, in cases where disputes are brought to the agency for resolution. In those cases, railways would be responsible for the remaining 37.5% of eligible costs of the project. Prior to April 2013, railways were generally responsible for 7.5% of eligible project costs.
• The maximum federal contribution to projects under the GCIP was reduced from 80% of eligible costs to 50%, effective April 1, 2013. This change was made to bring the share of federal costs in line with other infrastructure programs.

In addition to these program changes, new Grade Crossing Regulations were approved by the Treasury Board in November 2014. The new regulations include standards for crossings that are enforceable, unlike the previous Canadian Railway-Roadway Grade Crossings Standards. Regulations require that all existing crossings comply with the standards within seven years.

About the Evaluation

Evaluation Rationale and Scope

The evaluation of the GCIP was conducted in 2014 to comply with the requirement under section 42.1 of the Financial Administration Act that the relevance and effectiveness of all ongoing non-statutory programs of grants and contributions be reviewed every five years. The previous evaluation of the GCIP was completed in 2009-10.

As per the Policy on Evaluation, the evaluation examined the relevance and performance of the GCIP. Specifically, in examining relevance, the evaluation assessed the continued need for the program, and the program's alignment with government priorities, departmental strategic outcomes, and federal roles and responsibilities. The assessment of performance examined the extent to which the program is effectively conducting program activities and achieving expected outcomes (effectiveness), and resource utilization (efficiency). The evaluation focused on the years 2009-10 to 2013-14.

Evaluation Methodology

The evaluation included the following lines of inquiry:

• Analysis of administrative and financial information was conducted to inform an assessment of continued need and program performance, including activities undertaken and resources utilized. Specifically, the evaluation team compiled and/or reviewed information on rail safety outreach activities, GCIP applications, projects recommended for approval, completed projects, cancelled or delayed projects, pending projects, and financial and staff resources.

• Statistical analysis was undertaken to examine information on continued need and program performance. The analysis examined the extent to which the program was targeting high-risk crossings, as well as the impact of the program in terms of reductions in fatalities, serious injuries, and damaged property. The analysis also examined the continued need for the program by calculating the number of cost-effective crossing upgrades that could be done in the future.

  Statistical analysis included the use of a risk model. The risk model was developed for the evaluation by Research & Traffic Group, drawing on work completed for Transport Canada by the University of Waterloo, as well as information from the Transportation Safety Board's Railway Occurrence Database System (RODS). The risk model includes two dimensions: collision frequency and collision severity. The evaluation applied the model to data on grade crossings contained in Transport Canada's Inventory of Rail Infrastructure System (IRIS) as well as GCIP project information contained in program documentation.

• A document review provided information on program design and delivery, and informed the assessment of program relevance. The evaluation reviewed program operational documents, foundational documents, previous evaluations, documentation on the new Grade Crossing Regulations, as well as contextual and background documents.
• A review of academic literature was also conducted to examine peer-reviewed articles and other documents related to the subject of grade crossing safety. A list of all reviewed documents and academic literature can be found in Annex C.
• Interviews were conducted with a sample of 21 key program stakeholders, including Transport Canada staff and managers (n=11), recipients of project funding (n=8), an academic expert in risk modelling, and a manager at the US Federal Highway Administration.
• Four case studies were conducted to examine specific examples of grade crossing improvements. The case studies were selected to target GCIP projects where issues identified in the interviews could be examined in more detail. Case studies examined GCIP-funded grade crossing improvements in two small municipalities (including one where a GCIP project was cancelled) and a First Nation community, in order to examine how the program was working for these types of stakeholders. A case study of a grade crossing improvement that was conducted at a crossing that had a low relative risk ranking according to the risk model was also conducted to better understand program risk-targeting and project selection. Case studies included a review of program files and documents, and additional consultations with recipients and staff as necessary.

Finally, the evaluation manager participated in a workshop of Transport Canada railway signals inspectors, held in Ottawa on November 20, 2014. The workshop included a discussion of the GCIP and the GradeX risk model.

Evaluation Findings: Relevance

To assess the relevance of the GCIP, the evaluation examined the continuing need for the program, and alignment with federal priorities, departmental strategic outcomes, and federal roles and responsibilities.

Continuing Need

In 2013, there were 175 accidents at federally-regulated public crossings, resulting in 31 fatalities and 26 serious injuries.^Footnote3 Twenty percent of crossing accidents in that year resulted in either fatal or serious injuries. Both 2012 and 2013 saw a year-over-year increase in the numbers of crossing accidents and fatalities, as shown in Figure 1.^Footnote4

While accidents remain a safety issue, there has been an overall trend of declining accidents and fatalities over the long term. There was a 17% reduction in the number of accidents for the five-year period of 2009-2013 compared with the previous five-year period of 2004-2008. For the same periods, fatalities at grade crossings declined by 6%.

Figure 1: Number of Crossing Accidents and Fatalities, 2004 to 2013, Federally-regulated Crossings

 

Source: Transportation Safety Board of Canada, Statistical Summary – Railway Occurrences 2013.

Crossing safety can be improved through upgrading the physical structure and surroundings of crossings.^Footnote5 Research has shown that improvements to warning signs, for example, can improve safety through modifying driver behavior at crossings.^Footnote6 However, not all potential crossing improvements generate benefits that justify the cost of upgrades.

The evaluation calculated the number of cost-effective upgrades to crossings that could be undertaken through the GCIP, based on the current population of public, federally-regulated grade crossings. This included calculating the estimated cost of upgrading each crossing to the next level of safety (such as adding flashing lights and bells to a crossing with only a warning/stop sign) and the estimated social costs avoided by conducting the upgrade (including monetized lives lost and injuries sustained, and damage to vehicles and railways), over a lifecycle of 20 years. In other words, the evaluation calculated the number of crossings for which there would be a net benefit from an upgrade, based on the risk of each crossing and the cost of upgrades.

Based on this analysis, at the time of the evaluation there were 1,210 public crossings under federal jurisdiction that could be upgraded cost-effectively. These projects were distributed across all regions of the country. At the rate of 75 improved projects a year^Footnote7, it would take approximately 16 years to complete all of these cost-effective upgrades through the GCIP.

As indicated by the number and dollar value of applications submitted to the program, there has continued to be demand for the GCIP among railways and road authorities. As shown in Table 2, the number of (non-LED Special Contribution Program) GCIP applications decreased during and immediately after the program's period of transition to the new funding formula (2012-13 and 2013-14).^Footnote8 The number rose again in 2014-15, however; as of November 2014, there were 58 (non-LED Special Contribution Program) applications received in that fiscal year, for a total requested Transport Canada contribution of $8.1 million.

Table 2: GCIP Applications (Non-LED Special Contribution Program) Received and Dollars Required to Meet Transport Canada's Funding Share of Applications, 2011-12 to November 2014*

Fiscal Year Application Received by Transport Canada	Railway Applications		Road Authority Applications		Total Applications
	# of Applications	Requested TC Share ($ millions)	# of Applications	Requested TC Share ($ millions)	# of Applications	Requested TC Share ($ millions)**
2011-12	55	10.0	10	1.8	65	11.7
2012-13	28	2.7	6	0.3	34	3.1
2013-14	30	2.9	11	2.0	41	4.9
2014-15 (as of Nov. 2014)	38	4.4	20	3.7	58	8.1

* Number and value of applications was not able to be determined for 2009-10 and 2010-11 as this information was not tracked by the program.

**Numbers may not add due to rounding.

There was also a significant demand during the evaluation period for funding under GCIP's LED Special Contribution Program. From the commencement of this component in 2010-11 to November 2014, the total requested funding was $21.5 million. Applications under this component were often submitted as part of multi-year applications at the beginning of the program, and so cannot be used to show a trend in demand over time.

The level of demand for the program can also be seen in the number of GCIP applications received by Transport Canada that have not been prioritized and submitted for ministerial approval due to other projects being selected as higher priorities for funding. As of December 2014, there were 61 projects on the program's pending list, representing $7.0 million in requested Transport Canada funding.^Footnote9 This constitutes a decrease as the number of pending projects was 102 in 2004 and 125 in 2009. There was also a pending list of $4.0 million in LED Special Contribution projects as of November 2014.

The new Grade Crossing Regulations are a major driver of demand for the GCIP. According to interviews, some railways and road authorities view the GCIP as a source of funding that can assist them in preparing to meet the requirements of the regulations' standards. According to the cost-benefit analysis undertaken for the Grade Crossing Regulations, the expected costs of the new regulations to railways, road authorities and private authorities will total an estimated $126.2 million, much of which is due to necessary upgrades to warning systems and sightline improvements.^Footnote10

Conversely, it was stated in interviews that the change in the funding formula reduced the number of applications for funding submitted, as railways, and in some cases road authorities, are covering a higher share of project costs under the new formula.

Given these divergent factors, the level of demand for the program should be closely monitored going forward. Currently, the annual number, value and source (railways, municipalities, First Nations and provinces) of applications are not systematically tracked by the program.

The GCIP is the only Transport Canada program funding upgrades at high-risk grade crossings to improve safety. Two complementary Transport Canada programs were identified:

• The Grade Crossing Closure Program (GCCP) provides grants to compensate public road authorities and private land owners for the relinquishment of their right to cross a federally-regulated railroad track, in order that the crossing can be closed to improve safety. The GCCP is also managed within Transportation Infrastructure Programs with assistance from Rail Safety Directorate.
• The Asia-Pacific Gateway and Corridor Initiative (APGCI), which has funded transportation initiatives focused on maximizing trade with the Asia-Pacific region, has funded grade separations (e.g., building overpasses to separate vehicle traffic from railway lines). While grade separations were eligible projects under the GCIP, the program has not been used for these projects due to their high costs, which exceed the maximum project contribution of $550,000. Furthermore, unlike the GCIP's objective of improving safety, the APGCI funds projects based on their ability to enhance the efficiency of the transportation system. Finally, unlike the GCIP, the APGCI does not have a national reach.

Infrastructure Canada funds some grade separations as part of highway or major road projects through the Building Canada Fund (BCF) and New Building Canada Fund (NBCF). In addition, funding has been available under the BCF and the NBCF for short line rail infrastructure.^Footnote11 At the time of the evaluation, no funding was committed to short line projects under the NBCF, and funding was committed under the BCF for two short line projects, which included large-scale rehabilitation of major portions of the Huron Central Railway and the Vancouver Island Rail Corridor rail-lines.

No significant overlap was therefore noted among federal programs.

Alignment with Federal Priorities and Departmental Strategic Outcomes

Through funding improvements to the safety of the rail transportation system, the GCIP aligns with the federal government priority of "Supporting and Protecting Canadian Families" identified in the 2013 Speech from the Throne.^Footnote12 While not mentioning grade crossing improvements specifically, the Speech discussed the importance of federal efforts to support rail safety under the theme of "Safeguarding Families and Communities". The GCIP aligned with federal priorities articulated in the Government of Canada's Economic Action Plan, which increased contribution funding from $7.1 million a year to $12.9M a year.^Footnote13

The program also aligns with the Government of Canada's commitment to funding infrastructure. For example, in November 2014, $5.8 billion in federal funding was announced as part of the government's support for infrastructure, including transportation infrastructure.^Footnote14

The GCIP's objective of improving the safety of grade crossings in order to contribute to a safe rail transportation system is directly aligned with TC's Strategic Outcome #3 – a safe and secure transportation system.

Alignment with Federal Roles and Responsibilities

The Railway Safety Act, which came into effect in 1989, is the main legislation governing the safety of federally-regulated railways. The Act gives the Minister of Transport responsibility for the development and regulation of matters related to the safety and security of railways, and the ability to promote railway safety and security through appropriate means to carry out that responsibility.^Footnote15

Section 12 of the Act empowers the Minister to make a contribution towards the costs of railway work (including the relocation of a public road) that would improve the safety of a grade crossing that has been in existence for public use for at least three years. The contribution is made through an agreement with a proponent that has submitted an application for funding. The GCIP aligns with this legislated authority.

Evaluation Findings: Performance

The performance of the GCIP was assessed through an examination of key program activities and the achievement of outcomes (effectiveness), and program resource utilization (efficiency).

Effectiveness of Key Program Activities

The evaluation examined the effectiveness of GCIP promotion/communication and project selection/prioritization.

The GCIP appears to be well known and understood among railways, as it is a longstanding program that has been continually accessed by railways. Communication between program staff and members of Canadian Pacific Railway and Canadian National Railway has been especially frequent, with regular conference calls to discuss projects. Rail inspectors have also frequently discussed the GCIP with railways as part of their ongoing communication during inspections.

The GCIP has been promoted by Railway Safety staff to municipalities at various railway safety events across the country through the directorate's outreach and education activities. Of the 207 events at which outreach was conducted by Rail Safety in 2012-13 and 2013-14, 13 events were specifically noted in the directorate's events log as having included GCIP and/or grade crossings improvements as one of the major topics discussed. These included events organized by the Federation of Canadian Municipalities and provincial municipal associations. In addition, Railway Safety staff noted that the GCIP was sometimes being promoted to municipalities by regional rail inspectors as part of their routine inspections. No evidence was found that the program was promoted to First Nations communities, however, which are also eligible recipients.

Continuing promotion and outreach is required in order to build and sustain awareness of the GCIP among road authorities. Interviews and case studies suggested that there continue to be gaps in awareness and knowledge of the program among this group. Reasons include the large number of municipalities, the infrequent need to upgrade crossings in some municipalities, staff turnover and retirements, and road authorities' past reliance on railway companies for program information. It was also noted in interviews that road authorities have not consistently felt they have had timely information related to project funding decisions. Finally, it was suggested that there had been a lack of clarity and consistency in communications regarding how the GCIP can be used to assist railways and road authorities to meet the requirements of the new Grade Crossing Regulations.

Finding 8: Overall, the GCIP appears to have funded improvements at crossings that were among the highest risk to Canadians.

In order to assess the effectiveness of project selection and prioritization, the evaluation examined the extent to which the GCIP was upgrading the highest-risk crossings, based on the risk model. The evaluation calculated a risk value (in dollars) for each federally-regulated public crossing, based on the frequency and consequences of accidents that would be expected at each crossing over 20 years. The average risk values were then calculated for different sets of interest (i.e., GCIP-improved crossings by region, by signal type, and overall). Finally, these averages were compared to the risk values of all federally-regulated public crossings in Canada, to determine the percentile risk ranking for each crossing group of interest.

Overall, the GCIP was found to generally have funded upgrades at the highest-risk crossings during the five-year period examined. The risk ranking of all GCIP-upgraded crossings was at the 97th percentile; only 3% of grade crossings ranked higher in risk than the average GCIP-upgraded crossing. This indicates that the GCIP has been used to upgrade crossings that have presented a high risk relative to the entire pool of eligible crossings. The relative risk ranking of the GCIP-upgraded crossings was comparable (and even a little higher) than for the GCIP-improved crossings examined in the 2009 and 2005 evaluations, which ranked in the 95th percentile in both previous evaluations.

Finding 9: There is a risk that the program may see its ability to target the highest-risk crossings diminish in the future.

Since the program is application-based, the GCIP relies on railways and road authorities submitting applications for funding to improve the highest-risk crossings. Railways and road authorities can submit applications for funding for upgrades to any public crossings that have been in operation for at least three years. While Transport Canada regional inspectors have, in some cases, identified and discussed potential projects for applications with railways and road authorities, there has been no comprehensive and systematic national effort to identify and conduct outreach with stakeholders specifically to target the highest-risk crossings across Canada.

It was suggested in interviews that the department may see future GCIP applications that are less consistently targeted to the highest-risk crossings and are more focused on railways' and road authorities' priorities related to preparing to meet the requirements of the new regulations. While upgrades undertaken to meet the Grade Crossing Standards of the new regulations would have safety benefits, they may not consistently target the highest-risk crossings.

Some Transport Canada interviewees similarly noted that the applications being submitted to the program by the railways in recent years were less consistently targeted to the highest-risk crossings. The analysis conducted for the evaluation using the risk model saw a small decline over time in the risk ranking of completed GCIP upgrades: while the 2009-10 to 2011-12 period saw completed GCIP projects at crossings within the 98th percentile in terms of risk, this decreased to the 96th percentile and the 95th percentile in 2012-13 and 2013-14, respectively.

However, at the time of the evaluation, the program did not limit eligibility to the highest-risk crossings, nor did it define "highest risk".

Finding 10: Headquarters' calculation of annual GCIP budgets for each region is not based on a rigorous assessment of crossing risk.

In addition, across regions, there is a lack of a consistent, systematic approach to the prioritization of projects for funding. The GradeX risk model and online application developed for the department to help identify high risk crossings have not been fully operationalized for this purpose.

The share of the GCIP budget allocated annually to each region is not calculated based on a rigorous assessment of grade crossing risk in each region. Headquarters calculates regional GCIP funding allocations each year based on one variable: each region's proportion of the rolling five-year average of fatal crossing accidents. However, this approach does not include indicators of crossing risk identified in the literature and commonly used in crossing risk models, such as crossing traffic volumes and the characteristics of the crossings (e.g., signs, signals). A more rigorous approach would be based on a systematic assessment of the risk presented by crossings in each region (for example, each region's proportion of high-risk crossings as determined by a crossing risk model), or on the level of crossing risk targeted by the applications submitted for GCIP funding in each region.

In addition, across regions, there is no consistent, systematic risk-based approach to prioritizing GCIP applications for funding. Each of the regions uses a different approach to prioritizing applications for funding, with varying degrees of methodological rigour, including employing different risk models and/or inspectors' judgment. This has contributed to a lack of transparency and predictability in the selection of projects for funding approval for applicants. It also does not ensure that applications for upgrades at the highest-risk crossings across the country are consistently given first priority for funding.

A tool to assist the regions in crossing risk-targeting and project prioritization has been developed for the department, but it is not operational. Transport Canada engaged the University of Waterloo to develop a risk model for identifying high-risk crossings, and a web-based application to support use of the model. Despite recent efforts by Rail Safety to strengthen the online tool, and to build awareness of GradeX among inspectors, the system was not operational at the time of the evaluation. More timely and reliable data in the online system, and more capacity building within the department on the risk model, appear to be necessary if GradeX is to be operationalized. An operational GradeX tool would help with proactive identification of potential GCIP projects with railways and road authorities, as well as increasing the rigour and transparency of project prioritization.

Effectiveness – Achievement of Expected Outcomes

The evaluation assessed the achievement of expected outcomes, which include the improvement of the safety of grade crossings in order to contribute to a safe rail transportation system.

Finding 11: The GCIP approved funding for between 61 and 85 regular GCIP projects, and between 70 and 749 LED projects, each year.

Between 61 and 85 regular GCIP projects, and between 70 and 749 LED Special Contribution Program projects, were approved for GCIP funding each year from 2009-10 to 2013-14. No new projects were approved in 2012-13 in advance of the new funding formula, in order to complete in-progress projects prior to the change. Table 3 shows the number of GCIP projects approved between 2009-10 and 2014-15.

Table 3: GCIP Projects Approved, 2009-10 to 2014-15

Year	GCIP (non-LED)	LED Projects
2009-10	85	70
2010-11	76	282
2011-12	61	749
2012-13	--	--
2013-14	70	453
2014-15	73	568

From 2009-10 to 2013-14, there were 243 crossings upgraded through the GCIP, according to project completion dates in the IRIS database. It should be noted that a single crossing can be the subject of more than one GCIP project (such as where there is an improvement to both the signal work and the road approaching the crossing), so this number does not reflect the number of projects completed. Nor does it reflect the timing of program approvals, as, once an application is submitted, projects can be completed before they are approved and still be eligible for GCIP funding.

Crossings upgraded were most often in Ontario (32%) and Prairie and Northern Region (32%), with a smaller proportion in Quebec (17%), Pacific (13%) and Atlantic (7%) regions. Not including LEDs, upgrades were most often either Constant Warning Time circuits or train predictors (25%), or additions of gates to crossings with flashing lights and bells (23%). Other upgrades included roadside active advance warnings or advanced pre-emption of nearby highway signals (12%), and upgrades from standard reflectorized crossing signs to flashing lights, bells and gates (8%).

GCIP funding recipients have largely been railways. For projects approved from 2009-10 to 2013-14, 97% of projects' funding recipients were railways, 2% were municipalities, and 1% were provinces. During the evaluation period, three projects were approved for First Nations' funding recipients. Civil/roadwork projects were, on average, higher cost projects than other GCIP projects, so the share of approved GCIP funding dollars for municipalities was higher, at 7%, than their share of the number of projects.

Finding 12: GCIP-funded upgrades completed from 2009-10 to 2013-14 are predicted to result in 123 fewer accidents, 29 fewer fatalities and 16 fewer serious injuries over 20 years. The calculated benefits of the upgrades exceed the costs by $66.4 million.

The GCIP is funding grade crossing improvements that are contributing to increased safety. Based on estimates undertaken through risk modeling, GCIP-funded upgrades completed from 2009-10 to 2013-14 are predicted to result in 123 fewer accidents, 29 fewer fatalities and 16 fewer serious injuries, over a lifespan of 20 years.

The value that the program was providing to Canadians through increased safety was also demonstrated through a cost-benefit assessment. The net benefit of the GCIP-funded improvements completed from 2009-10 to 2013-14 is estimated to be $66.4 million over 20 years. This number is the difference between the costs associated with upgrades and the estimated monetized value of avoided deaths, serious injuries, and vehicle and railway damage as a result of the upgrades. Based on the modeling, 73% of the completed GCIP upgrades were calculated to show a net benefit.^Footnote16

Finding 13: The disproportionate number of accidents and fatalities at crossings with the most advanced warning signals suggests the limits of the GCIP in improving rail safety.

Although declining over the long term, there remains a significant number of crossing accidents and fatalities each year. Reasons provided in interviews included increasing road and rail traffic, increased train speeds on core lines, and heavier trains. Between 2009 and 2012, freight transportation increased 13%, based on kilometers travelled.^Footnote17

Crossings that already have the most sophisticated types of warning signals are most likely to see accidents and fatalities. As shown in Table 4, crossings with flashing lights and bells (FLB) or flashing lights, bells and gates (FLBGs) are disproportionately the sites of accidents, compared with crossings with (passive) warning signals like stop signs or cross bucks. Furthermore, 40% of crossing fatalities occurred at FLBG crossings, although these crossings made up only 17% of crossings. The over-representation of FLB and FLBG crossings as sites of accidents and deaths is a result of their higher exposure levels (in terms of train and road traffic); while they represent only 40% of all crossings, they account for 97% of exposure.

Table 4: Accidents at Public Crossings, by Type of Warning System, 2004-2013, Average and Distribution

Warning Signal Type	Average Annual Number of Accidents	Average Annual Number of Fatalities	Distribution of...
			Accidents	Fatalities	Public Crossings	Exposure (vehicles/day X trains/day)
Passive	67.2	6.2	39%	25%	59.5%	3.2%
FLB	65.7	8.8	38%	35%	23.6%	21.8%
FLGB	41	10.1	24%	40%	16.8%	74.7%
Other	3.2	0.1	2%	0%	0.1%	0.3%
Total	177.1	25.2	100%	100%	100%	100%

Source: Transportation Safety Board Statistical Summary of Railway Accidents and Transport Canada's IRIS database.

Due to this high exposure, based on the risk model, the majority of the highest-risk crossings have automated warning signals, with 45% of the 500 highest-risk crossings already having full FLBG signals.

The number of accidents and fatalities at automated crossings suggests the limits to upgrading crossings as an intervention to reduce risk. While some active crossings can be further upgraded (such as through civil upgrades, Constant Warning Time circuits, and other improvements), in cases where all other appropriate safety features are already present, only full grade separation would further increase safety through physical improvements.

While full grade separations are eligible under the GCIP, the program has not been used to fund these projects due to the high costs associated with these improvements. For example, a grade separation completed at Smithers, British Columbia cost $6.0 million, while others have cost much more.^Footnote18 As previously discussed, other available Transport Canada funding for grade separations is limited to projects that facilitate the increased efficiency of the Asia-Pacific Gateway and Corridor.

Efficiency and Economy

To assess the efficiency and economy of the GCIP, the evaluation examined the extent to which the initiative used resources as planned, and the cost of administering the contribution funding.

Finding 14: The GCIP has spent less than its planned contribution funding.

From 2009-10 to 2013-14, the GCIP spent $48.7 million of its $58.7 million contribution budget. The program under-spent by $10.0 million, or 17% of its contribution budget during this five-year period. The program under-spent in four of the five years examined.

One of the reasons that the contribution budget has not been fully expended is that planned project costs have been lower than expected. This was due to:

• Lower than estimated project costs – Among completed non-LED GCIP projects approved in 2013-14, actual costs were 12% lower than originally estimated.
• Project delays – Reasons for project delays indicated in interviews included slower-than-planned construction and short construction seasons due to weather issues. Delays affected 12% of approved funding in 2013-14.

In addition, cancelled or withdrawn projects amounted to 7% of funding for approved projects in 2013-14, the year following the change in the funding formula.

The other major reason for under-spending has been that the program has not consistently approved sufficient projects to expend its full budget. Rail Safety indicated that, when it managed the delivery of the program, it over-committed its annual contribution budget to take into consideration delays and over-estimates of project costs. Programs group has not continued this practice. The total value of projects approved has, in fact, been less than the annual contribution budgets in recent years. With a contribution budget of $10.9 million a year, the total value of projects approved for funding was $9.4 million in 2013-14 and $9.3 million in 2014-15.

Given recent under-spending, program expenditures (as well as other program activities and results) should be tracked and reported on a regular basis to ensure effective performance monitoring.

Finding 15: Delivery costs have declined since program changes began in 2012-13.

The estimated administrative costs of the program comprised 4% of total program costs for the period of 2009-10 to 2013-14.^Footnote19 Based on estimates of salary costs provided by the program, administrative costs comprised just 2% of total program costs for 2012-13 to 2013-14, compared with 5% for the three previous years.

Reduced administrative time was a result of changes to program delivery following 2012-13, including:

• Inspection of crossing by regional rail inspectors after project completion is no longer mandatory. Inspectors decide when to carry out inspections based on level of risk. Funding recipients sign affidavits that the projects were completed as agreed.
• Programs group has taken over responsibility for managing project invoice verification, with recipient audits replacing verification of invoices by regional staff.

Administrative costs may see further reductions going forward, as Programs group continues to streamline delivery processes, such as through completing the transition of project management to the Surface Infrastructure System application and, potentially, undertaking multi-year contribution agreements.

Finding 16: Combining the GCIP and the GCCP could further reduce departmental administrative costs.

While having the same overall objective of increasing grade crossing safety, the GCIP and the GCCP currently have separate Terms & Conditions and administrative processes. Folding the GCCP into the GCIP could help to streamline administration within the department by eliminating the need for a separate call for applications and separate approval processes, and would reduce the number of transfer payment programs. Combining the two programs would also simplify program promotion and awareness building with stakeholders.

Combining the programs would also reduce the number of agreements, and administrative effort, in cases where the GCIP and GCCP are used in a complementary manner. One of the case studies conducted for the evaluation examined a project that accessed GCCP funding to relinquish the rights to close a grade crossing and GCIP funding to construct a new access road to divert traffic to a safer nearby crossing. Although the GCCP grant provided a relatively small complement of $20,000 to the GCIP contribution that was in excess of $300,000, this project required two sets of applications and agreements.

Conclusions and Recommendations

The GCIP remains relevant, as grade crossings are a continuing public safety issue, there remains a significant number of cost-effective upgrades that can be undertaken in the future, and there is a continued demand for the program from railways and road authorities. Some of this demand is being driven by the new Grade Crossing Regulations, which include enforceable standards for grade crossings. The GCIP aligns with federal government priorities related to safety, and with Transport Canada's strategic outcome of a safe and secure transportation system. The GCIP aligns with Transport Canada's legislated authority as set out in the Railway Safety Act. No areas of overlap with other federal programs were identified.

The GCIP continues to achieve its short-term outcome of improving crossings, with the program approving funding for between 61 and 85 non-LED Special Contribution Program projects, and between 70 and 749 LED projects, a year. In addition to LED installations, the program commonly funded improvements to Constant Warning Time circuits / train predictors, and additions of gates to crossing signals. Overall, the GCIP has generally funded upgrades at the highest-risk crossings.

The GCIP is increasing the safety of crossings. Based on the risk model, over a 20-year lifespan, the GCIP improvements completed from 2009-10 to 2013-14 are predicted to result in 123 fewer accidents, 29 fewer fatalities, and 16 fewer serious injuries. Overall, when comparing the costs of upgrades to the monetized value of the expected benefits of the crossing upgrades completed, the program has been cost-effective. The estimated benefits of the GCIP upgrades will exceed the costs of the upgrades by over $66 million over 20 years. However, as noted, accidents and fatalities at crossings persist, despite a long-term trend of overall reduction; the disproportionate number of accidents and fatalities at crossings with active warning signals, which have the highest volume of traffic, suggests the limits of this type of intervention. At some high-risk crossings, more costly interventions such as grade separations may be considered to further improve safety.

The cost to deliver the program has decreased over the last five years as less staff time is required for program delivery due to administrative changes. Further administrative efficiencies may be realized by the department through combining the GCIP with the GCCP.

The evaluation identified four recommendations, included in the table below with supporting findings.

Recommendation #1: Transport Canada should implement a strategy to promote further awareness and knowledge of the GCIP among road authorities.

Supporting findings:

• Interviews and case studies suggest limited awareness/knowledge of the GCIP among road authorities. Some road authorities indicated that they do not feel communication with respect to GCIP and funded projects has been sufficient, and some have relied on railways for program information.
• Use of the GCIP by road authorities has been limited.
• The program did not spend its entire contribution budget in four of the five years examined.

Recommendation #2: Transport Canada should implement a consistent and systematic approach to targeting the highest-risk crossings in Canada through the GCIP.

Supporting findings:

• The GCIP is not designed to limit project eligibility to the highest-risk crossings, which is a risk should the program not receive applications that address the highest-risk crossings in the future. Furthermore, there has been no comprehensive, national effort to identify and conduct outreach with railways and road authorities to target the highest-risk crossings across the country.
  • The program has relied on a surplus of applications from which to select the highest-risk projects. There is a risk the program may received fewer future applications and/or applications for lower risk crossings from which to select its priorities, which would diminish the effectiveness of the program.
• Across regions, there is no consistent, systematic risk-based approach to prioritizing GCIP applications for funding.
  • The regional funding allocation model is based on only one variable (past fatal crossing accidents in each region) rather than a more rigorous assessment of the risk of each region's crossings and/or the risk ranking of crossings applications received by the GCIP.
  • The regions are prioritizing GCIP projects using different approaches, including risk models and informal approaches based on inspectors' expertise. This has contributed to a lack of transparency and predictability for applicants, as well as varying degrees of rigor in prioritization.
  • The GradeX risk model and online application developed for Transport Canada by the University of Waterloo are not being used for prioritization of applications, primarily due to data issues.

Recommendation #3: Transport Canada should develop and implement a system for regularly monitoring and reporting to senior management on GCIP performance, including project applications, completions and project risk-targeting, by recipient type (railways, municipalities, First Nations, provinces) as well as program expenditures.

Supporting findings:

• Several factors are influencing the program, each of which may have an impact on future performance: changes in the program funding formula, new Grade Crossing Regulations, ongoing outreach with road authorities, and changes to Transport Canada delivery/administrative processes.
• Recent data and stakeholder opinions suggest that there is a risk the program could see fewer applications targeted to higher risk crossings in the future.
• The program has a history of under-spending its contribution funding.
• The program has not been tracking all performance information, including annual applications by applicant type, etc.

Recommendation #4: Transport Canada should consider combining the GCIP and the GCCP.

Supporting findings:

• The GCIP and the GCCP have separate Terms & Conditions and are delivered separately, though they are designed to achieve the same objective of improved rail safety.
• A case study showed how the two programs can and have worked in a complementary way.
• Combining programs would create efficiencies in administration, would rationalize transfer payment programs in the department, and could simplify program promotion.

Management Action Plan

#	Recommendation	Proposed Actions	Forecast Completion Date	OPI
1	Transport Canada should implement a strategy to promote further awareness and knowledge of the GCIP among road authorities.	Rail Safety will develop a strategy to promote further awareness and knowledge of the GCIP among road authorities by: Prioritizing Education and Awareness (E&A) events in locations where road authorities can be reached; Providing technical expertise at E&A events and meetings with road authorities; Developing guidance material on the program to be distributed at E&A events and meetings; Delivering presentations to promote the program at key events where road authorities (FCM, Provinces, First Nations, etc.) are present.	March 2016	Rail Safety Operations
2	Transport Canada should implement a consistent and systematic approach to targeting the highest-risk crossings in Canada through the GCIP.	Rail Safety will implement a consistent and systematic approach to targeting highest-risk crossings in Canada by: Reviewing its regional funding allocation model to incorporate science-based risk assessment criteria in the initial allocation of funds; Providing functional guidance to regions to increase consistency of project prioritization in each region and across regions ; Using the Grade X software tool to support the prioritization decision making process.	October 2015	Rail Safety Operations
3	Transport Canada should develop and implement a system for regularly monitoring and reporting to senior management on GCIP performance, including project applications, completions and project risk-targeting, by recipient type (railways, municipalities, First Nations, provinces) as well as program expenditures.	Programs Group currently has a number of tracking tools in place and will harmonize these tools into a single report that can be shared with senior management on a quarterly basis.	May 2015	Programs Group
4	Transport Canada should consider combining the GCIP and the GCCP.	The programs are currently under review and this recommendation will be carefully considered when renewing the programs.	October 2015	Programs Group