7. 2021 Reports of the Auditor General of Canada to the Parliament of Canada Report 5—Follow-up Audit on Rail Safety—Transport Canada

Introduction

Background

5.1 Canada has a large and well-developed railway network that transports more than 88 million passengers and more than $328 billion worth of goods across the country each year.

5.2 Of all the railway networks in the world, Canada’s is one of the longest and transports the fifth largest volume of goods. Each year, Canadian railways move 70% of the country’s surface goods (including 40% of its exports). Railway tracks converge with motorists and pedestrians at thousands of public and private crossings.

5.3 Safety risks are inherent to all modes of transportation, and rail transportation is no exception. In 2019, there were 1,245 rail accidents (Exhibit 5.1), up from 1,169 in 2018. Total accidents in 2019 were 17% higher than the previous 10-year average of 1,064. The trend in accident occurrences over the past 10 years should be considered in the context of increasing rail traffic, which would suggest some improvement in the rate of accidents relative to rail traffic volume. Rail accidents include derailments and collisions, with the majority happening in train yards at low speeds and not on main-line railway tracks. Accidents may result in the unintended release of dangerous goods, serious injuries, or even deaths (Exhibit 5.2).

5.4 Rail traffic has been increasing. Freight tonnage increased to more than 328 million tonnes in 2018, up from 312 million tonnes in 2017. In terms of goods transported, fuel oils and crude petroleum recorded a significant increase by weight from 2017 to 2018 of more than 45%. This surge in train traffic means more wear and tear on tracks, which can pose additional safety risks.

5.5 Safety risks are greater as more land is developed close to railway operations and as rail tracks expand into urban areas with road and pedestrian traffic. Rail safety can also be affected by extreme weather conditions and climate change—that can result in coastal storm surges, landslides and avalanches, and extremes in temperatures and precipitation.

5.6 All these factors make it even more important that Transport Canada, railway companies, and stakeholders work together to make rail transport as safe as possible for all Canadians.

5.7 Railways. Railway companies are responsible for the safety of their own operations pursuant to the Railway Safety Act and the accompanying Railway Safety Management System Regulations adopted in 2001. These were repealed and replaced in 2015. Each railway company is required to develop and follow its own safety management system in its day-to-day operations. These systems generally include a safety policy, safety targets, a risk-assessment process, and processes for continuous improvement.

5.8 Safety management systems complement and do not eliminate or replace other regulatory requirements. They are intended to enable railway companies to manage risks before Transport Canada intervenes and before major safety issues emerge.

5.9 Transport Canada. Under the Railway Safety Act, Transport Canada is responsible for overseeing safety requirements for federally regulated railway companies. The department’s oversight role includes monitoring compliance and enforcement activities.

5.10 In all, 75 railway companies and more than 45,000 kilometres of railway tracks in Canada are subject to Transport Canada’s oversight. These railway companies include both federal companies operating across provincial or international borders and local railway companies operating on federally regulated tracks.

5.11 To mitigate safety threats, Transport Canada oversees rail safety from its headquarters in Ottawa and its 5 regional offices through activities such as

• inspections
• safety management system audits
• data analytics

5.12 Stakeholders. While railway companies are responsible for rail safety, municipal road authorities are responsible for maintaining the approaches to public crossings for motorists and pedestrians. Stakeholders include provinces and territories, rail safety associations, labour unions, and the general public.

5.13 This audit followed up on selected recommendations from our 2013 Fall Report of the Auditor General of Canada, Chapter 7—Oversight of Rail Safety—Transport Canada. In that audit, we concluded that Transport Canada needed to address significant weaknesses in its oversight of safety management systems implemented by federal railway companies to manage safety risks on a day-to-day basis. We recommended that the department

• improve its risk-based planning of oversight of railway companies, including collecting relevant risk and safety information from railways
• conduct timely follow-up on deficiencies affecting the safety of railway companies’ operations, including assessing the effectiveness of railway companies’ safety management systems

Focus of the audit

5.14 This audit focused on whether Transport Canada implemented selected recommendations from our 2013 audit regarding the department’s oversight of the safe transportation of people and goods on federally regulated tracks. In particular, we examined the risk-based planning process for rail safety oversight and whether the department confirmed that railway companies corrected deficiencies affecting the safety of their operations.

5.15 This audit is important because rail safety accidents can have serious consequences, including the loss of life and damage to property, goods, and the environment.

5.16 We did not examine the inspections of facilities and sites that handle, offer for transport, transport, or import dangerous goods. We also did not examine the emergency response assistance plans for companies regulated by the Transportation of Dangerous Goods Act, 1992. On these topics, we published a separate follow-up audit report—see our 2020 Fall Reports of the Commissioner of the Environment and Sustainable Development to the Parliament of Canada, Report 1—Follow-up Audit on the Transportation of Dangerous Goods. In addition, we did not examine the department’s enforcement activities, the Transportation Safety Board of Canada, railway companies, provinces, territories, or municipalities. Finally, our audit was not designed to conclude on whether individual railways in Canada are safe.

5.17 More details about the audit objective, scope, approach, and criteria are in About the Audit at the end of this report.

Findings, Recommendations, and Responses

Overall message

5.18 Overall, Transport Canada made progress in addressing recommendations from our 2013 audit in the areas we followed up on; however, we also found that the department still had important improvements to make. The department strengthened its approach to planning and prioritizing activities to oversee rail safety and in following up on companies’ plans and actions to address deficiencies noted in inspections.

5.19 However, Transport Canada was unable to show whether departmental oversight activities have contributed to improved rail safety. In addition, the department did not assess the effectiveness of the railways’ safety management systems—despite the many reports over the last 14 years recommending that Transport Canada audit and assess these systems.

Risk-based planning for oversight

Transport Canada could not demonstrate that its oversight planning activities improved railway companies’ regulatory compliance

5.20 We found that Transport Canada improved its risk-based planning for oversight. In particular, the department significantly increased the number of its planned risk-based inspections. We also found that Transport Canada made progress in conducting more audits of railway companies’ safety management systems.

5.21 However, we found that the department did not measure the overall effectiveness of its oversight activities. Properly assessing the effectiveness of the department’s oversight activities is complex because other factors, such as new technology, can also improve rail safety. Nonetheless, it remains crucial to determine whether the department’s oversight is having a positive impact. We found that the department could not demonstrate the extent to which its oversight activities have improved the railway companies’ compliance with regulations that mitigate key safety risks. This is an important element of measuring its oversight effectiveness.

5.22 We also found that although the department began to implement our recommendation to collect more information from railway companies to support risk-based decision making, it did not provide sufficiently detailed guidance to the railway companies on how to report their safety data.

5.23 We found that although the department completed inspections and collected data from multiple sources, including the Transportation Safety Board of Canada, it did not adjust the selection of its planned inspections to take into account the results from safety management system audits.

5.24 The analysis supporting this finding discusses the following topics:

• Improved risk-based planning process
• Overall effectiveness of oversight activities not measured
• Inadequate information to plan for inspections
• Audit results of safety management systems not considered

5.25 This finding matters because incorporating better data into its planning activities will help Transport Canada address the areas of highest risk and better support the safe transportation of people and goods.

5.26 Transport Canada’s rail oversight program is planned annually and adjusted as needed. The department monitors compliance through inspections and audits (Exhibit 5.3) and takes necessary enforcement action.

5.27 Because the department cannot oversee each railway location, kilometre of track, piece of equipment, or train crew, it relies on risk-based oversight planning. The department develops these plans to determine the audits and inspections to conduct in order to assess whether railway companies manage safety risks appropriately and comply with regulations.

5.28 Transport Canada’s risk-based planning process identifies and prioritizes where inspections and audits may be required. The department’s annual planning process starts with data collection from a variety of sources:

• Transportation Safety Board of Canada incident reports and accident reports
• Transportation Information Regulations data provided by the railway companies, which includes information on the condition and maintenance of locomotives and train cars, and information relevant to railway infrastructure, including bridges, tracks, culverts, signalling, and grade crossings
• railway companies’ past inspection results, enforcement actions, and public complaints and inquiries

5.29 Our 2013 Fall Report of the Auditor General of Canada, Chapter 7—Oversight of Rail Safety—Transport Canada recommended that Transport Canada improve its planning for risk-based oversight activities to

• collect relevant risk and safety performance information from federal railways and assess its completeness and reliability (recommendation 7.42)
• reassess the number of its planned railway audits and inspections to take into account the safety management system environment (recommendation 7.49)

5.30 Our recommendations in this area of examination appear at paragraphs 5.37, 5.41, and 5.45.

Improved risk-based planning process

5.31 We found that since our 2013 audit, Transport Canada made progress in improving its risk-based approach to identifying and overseeing safety issues. In the 2016–17 fiscal year, 40% of inspections were risk-based, and this proportion increased to 72% in the 2019–20 fiscal year. The department also conducted random inspections (15%) to measure general levels of compliance and reactive inspections (13%) to assess emerging issues. The proportion of risk-based inspections relative to random inspections was determined by the department.

5.32 We also found that each regional office followed different planning practices to prioritize their oversight activities. Although Transport Canada identified risks for rail safety at both the national and regional levels, the department’s risk assessment was inconsistent across regions and was not always adequately documented, even though the department provided guidance to regional offices on how to identify areas of highest risk. The department was aware of this situation and acknowledged that it needed to work with the regional offices to assess risk consistently.

Overall effectiveness of oversight activities not measured

5.33 We found that Transport Canada did not measure the overall effectiveness of its rail safety oversight. It had not identified whether rail safety has improved as a result of its inspections and audits of safety management systems.

5.34 The department reported publicly on a 10-year rail accident rate and a 10-year rail fatality rate. Although the department measured the number of accidents and fatalities, these measures did not specifically assess the impact of the department’s oversight on the safety of rail transportation. Moreover, reductions in accidents and fatality rates depend on collaboration among many parties, including railway companies and municipal road authorities, which share responsibility to help reduce these rates. Transport Canada also developed internal performance indicators, such as 5-year average compliance rate trends in key risk areas, but it did not consistently measure performance against them.

5.35 We also found that Transport Canada senior management received only partial information on the results of oversight activities to make informed risk-based planning decisions. Although the department’s senior management received year-end reports, the following information to support decision making was not included:

• sufficient data on railways’ compliance with regulations
• results of departmental audits of railways’ safety management systems
• inspection results—departmental data focused instead on the number of inspections performed

5.36 Without clear information about the outcomes and impacts of the department’s safety oversight activities, the department has difficulty making risk-based planning decisions so that inspectors and auditors focus on the highest-risk areas that can improve rail safety.

5.37 Recommendation. Transport Canada should determine the extent to which its inspections and audits have improved the railway companies’ compliance with regulations that mitigate key safety risks. The department should integrate these results into its risk-based planning decisions, including determining which and how many inspections to perform.

The department’s response. Agreed. Transport Canada will significantly improve the ongoing performance measurement of the rail safety program by improving the Rail Safety Oversight Performance Information Profile. This effort will be informed and supplemented by other work in the department, including the development of an outcome measurement system and a safety and security data strategy (February 2022).

Transport Canada will also conduct an evaluation to assess the impact of oversight activities on the railway companies’ compliance with key regulations (December 2022).

These results will be used to improve risk-based planning (June 2023).

Inadequate information to plan for inspections

5.38 We found that Transport Canada began to implement our 2013 recommendation to collect relevant risk and safety performance information from federal railways. The department amended the Transportation Information Regulations requiring railway companies to submit specific safety data so the department could proactively identify risk areas—and incorporate this information into its planning.

5.39 However, we also found that the department did not provide sufficiently detailed guidance to the railway companies on how to report their safety data. As a result, the information was sometimes late, incomplete, or of varying quality.

5.40 In particular, the department did not obtain all the required information needed to plan inspections for areas of highest risk. Departmental staff identified several concerns, such as limited information on how a signal malfunction was resolved, and varying interpretations of the content required for some indicators. Staff also noted that the annual deadline, as specified in the regulations for railway companies’ data submissions, was not always respected and that some submissions were received later than the due date.

5.41 Recommendation. Transport Canada should improve guidance to railway companies so that safety data submissions are complete, reliable, and timely.

The department’s response. Agreed. Transport Canada will develop an information request template (September 2021) and issue guidelines to railway companies and stakeholders (October 2021) to facilitate data analysis and streamline reporting.

Audit results of safety management systems not considered

5.42 In the 8 years since we recommended that Transport Canada reassess the number of its planned audits, we found that the department made progress in conducting more audits of railways’ safety management systems. According to the department, it audited all the railway companies in operation at least once between 1 April 2016 and 31 March 2020.

5.43 However, we also found that Transport Canada’s oversight planning process did not consider findings from departmental audits of the railways’ safety management systems. Our examination of these audits identified results that the department could have used to focus its inspection planning on areas of highest risk. For example, one audit revealed that a rail company did not collaborate with its employees when developing a hazard management program and a policy on whistleblowing. Another audit discovered that a railway company did not have procedures or documentation to show that it provided employees with the required skills, knowledge, and qualifications to conduct their work.

5.44 If safety management system audit findings had been included in Transport Canada’s planning oversight process, the department’s annual inspection focus may have shifted. In our view, it is important that the department integrate the results of its audit activities into the risk-based planning for rail safety oversight, so that it considers all available information.

5.45 Recommendation. Transport Canada should integrate the findings from audits of safety management systems when planning its oversight activities and adjust its selection of planned inspections to address areas of highest risk.

The department’s response. Agreed. Transport Canada will identify and adapt best practices to integrate safety management system audits into the selection of oversight activities for risk-based planning purposes. The risk-based business planning procedure will be updated to incorporate a review of audit findings by April 2021.

Auditors and inspectors who take part in safety management system audits will also participate in risk-based business planning activities for the various functions to highlight and explain audit findings. These measures will be completed by May 2021.

Conducting audits and inspections

Transport Canada made progress in assessing and following up on railway companies’ corrective actions but did not measure the effectiveness of their safety management systems

5.46 We found that since our 2013 audit, Transport Canada made progress in assessing whether railway companies corrected safety issues identified in departmental inspections and audits of safety management systems.

5.47 However, we also found that Transport Canada did not establish standard time frames for how long the department should take to assess whether railway companies corrected the identified safety issues.

5.48 We also found that the department did not assess the effectiveness of the safety management systems that each railway implemented. Departmental audits of these systems determined only whether railway companies complied with specific regulations and not whether the overall systems themselves resulted in greater safety.

5.49 The analysis supporting this finding discusses the following topics:

• Good progress on assessing railway companies’ corrective actions
• No standard time frame for assessing whether railway companies completed corrective actions
• Improved information management for inspections but not for safety management system audits
• Effectiveness of safety management systems not assessed

5.50 This finding matters because safety management systems are designed to achieve the highest level of safety in day-to-day rail operations. Unless Transport Canada assesses the effectiveness of these systems and follows up within a standard time frame to determine whether railway companies have corrected the identified safety issues, it cannot demonstrate that the likelihood of accidents is minimized.

5.51 As a key component of its oversight activities, Transport Canada assesses whether the railway companies’ corrective actions resolve safety issues. It is critical that these assessments be timely so that high-risk issues are prioritized and resolved before accidents can occur.

5.52 Measuring the effectiveness of safety management systems is another important component of the department’s oversight responsibilities. Auditing effectiveness gives Transport Canada assurance that railway companies are integrating safety into their day-to-day operations.

5.53 This audit included follow-up on recommendation 7.58 from our 2013 audit of Transport Canada to

• require railway companies to make the necessary changes to correct deficiencies that affect the safety of their operations
• conduct timely follow-up on deficiencies affecting the safety of railway companies’ operations to assess whether they were corrected
• improve departmental oversight of railway companies’ safety management systems by assessing their quality and effectiveness

5.54 Our recommendations in this area of examination appear at paragraphs 5.61, 5.64, and 5.68.

Good progress on assessing railway companies’ corrective actions

5.55 We found that for most of the inspections and audits we examined, Transport Canada assessed whether railway companies took corrective actions when non-compliances were identified. In our view, Transport Canada’s process to validate the railway companies’ plans to address problems was reasonable.

5.56 We used representative sampling to examine 66 inspections conducted between September 2018 and August 2020 that required a railway to take follow-up action. We found that the department confirmed that railway companies took corrective actions in 61 (92%) of those inspections. For the remaining 5 (8%), Transport Canada determined that the railway companies’ plans to address minor issues, such as trimming grasses and shrubs near crossings, would satisfy regulatory requirements.

5.57 We also randomly selected 24 of Transport Canada’s safety management system audits requiring a railway company to correct safety issues between September 2018 and August 2020. As of December 2020, the department was still in the process of assessing corrective action plans for 7 of these audits.

5.58 For the remaining 17 audits, the department assessed that railways had corrected all identified issues for 10, and it was in the process of confirming that corrective actions were implemented for the remaining 7. In our opinion, Transport Canada’s assessments of corrective actions taken were reasonable. Compared with inspections, safety management system audits are broader in scope, and the identified issues may be more complex. Exhibit 5.4 presents an overview of how Transport Canada conducts rail safety inspections and safety management system audits.

No standard time frame for assessing whether railway companies completed corrective actions

5.59 To determine whether Transport Canada fulfilled its requirement to provide timely follow-up on safety issues, we examined samples of the department’s inspections and audits conducted between September 2018 and August 2020. Although Transport Canada performed follow-up activities, we found that the department had no standard time frame for how long it should take to assess whether railway companies completed corrective actions.

5.60 The department’s standards for on-site audits and inspections focused primarily on how quickly the railway companies needed to provide a plan to correct the issues—and not on confirming that railway companies made timely corrections to the deficiencies affecting safety.

5.61 Recommendation. Transport Canada should set standards for the time it should take to assess whether railway companies corrected deficiencies identified in inspections and audits of safety management systems.

The department’s response. Agreed. By June 2021, Transport Canada will identify best practices, including those in other modes of transportation, to establish standards for following up on inspections and safety management system audits. The department will establish a standard by December 2021 on how long it should take to assess whether railway companies corrected deficiencies resulting from inspections and audits of safety management systems.

Improved information management for inspections but not for safety management system audits

5.62 We found that Transport Canada updated its information management system to better manage inspections. Refinements included tracking railway companies’ corrective actions, inspection results, and departmental communication with railway companies. The system also generated reports for management.

5.63 However, we found that the department did not integrate information and results from its safety management system audits into its information database, nor did it have another way to track this data. Furthermore, Transport Canada did not adhere consistently to its documentation standards, which made it difficult to determine whether the department uniformly assessed the railway companies’ safety management systems. For example, the department used a standard checklist in only 6 of the 24 audits we examined.

5.64 Recommendation. Transport Canada should improve data management for its safety management system audits by

• adhering to its documentation standards
• integrating the results of its safety management system audits into a data management system

The department’s response. Agreed. Transport Canada has confirmed that the audit teams in every region are using the recently implemented electronic working papers system to manage audit documentation (December 2020). In addition, the department will conduct a review of data management practices for audits by April 2021 to ensure consistent use across regions. A data management system will be developed to facilitate the tracking of audit findings and identify compliance trends. This system will be in place by December 2021.

Effectiveness of safety management systems not assessed

5.65 Safety management systems emerged in the 1990s as a proactive way to manage safety. Starting in 2001, the Railway Safety Management System Regulations required that railway companies implement and maintain a safety management system that included, among other things, goals, risk assessments, and monitoring and evaluation processes. The regulations also required that railway companies collect and provide the department with data annually for assessing their safety performance. In 2015, the department repealed and replaced the regulations. Collectively, these legislative provisions supported our view that Transport Canada’s oversight role included assessment of the effectiveness of these safety management systems.

5.66 Over the past 14 years, several reports advised Transport Canada to assess the effectiveness of railway companies’ safety management systems (Exhibit 5.5). In particular, the 2007 review of the Railway Safety Act recommended that the department integrate the assessment of safety management systems into its oversight activities. Furthermore, the department agreed to audit safety management system effectiveness in response to the Transportation Safety Board of Canada’s Lac-Mégantic investigation report in 2014 on runaway trains and main track derailments.

5.67 We found that the department’s audits continued to focus on whether safety management systems complied with regulatory requirements and not whether these systems were effective and actually improved rail safety. Determining whether the supporting processes are effective is an important next step for the department in assessing the overall effectiveness of railway companies’ safety management systems.

5.68 Recommendation. Transport Canada should make it an immediate priority to regularly assess the effectiveness of railway companies’ safety management system processes.

The department’s response. Agreed. Transport Canada will undertake program and regulatory action to assess the effectiveness of safety management system processes:

• Develop and launch a targeted audit framework and supporting implementation methodology, which will be used as a guide to identify issues that may be a result of ineffectiveness of a particular railway company’s safety management system processes (April 2021).
• Finalize a risk-based process for determining when targeted audits are required (June 2021).
• Develop tools to measure effectiveness and identify baseline and targets for each safety management system process (November 2021).
• Undertake the audits of effectiveness that leverage the tools developed (December 2021).
• Launch consultations in April 2022 on potential amendments to the safety management system regulations to strengthen rail safety requirements, including measures to improve the overall performance of safety management systems.

Conclusion

5.69 We concluded that Transport Canada had not fully implemented all of the selected recommendations from our 2013 audit report on rail safety regarding the department’s oversight of the safe transportation of people and goods.

5.70 The department made important improvements to both its safety oversight planning and inspection follow-up, but it did not assess whether its oversight activities improved rail safety. Also, the department did not measure the effectiveness of the railway companies’ safety management systems or integrate the results of those audits into its rail safety oversight planning.