9. AG 2021 REPORT: MANAGEMENT ACTION PLAN

1

Transport Canada should determine the extent to which its inspections and audits have improved the railway companies’ compliance with regulations that mitigate key safety risks.  The department should integrate these results into its risk-based planning decisions, including determining which and how many inspections to perform.

• Agreed. Transport Canada will significantly improve the ongoing performance measurement of the Rail Safety program by improving the Rail Safety Oversight Performance Information Profile (PIP). This effort will be informed and supplemented by other work in the department, including the development of an outcome measurement system and a safety and security data strategy. (February 2022)
• Transport Canada will also conduct an evaluation to assess the impact of oversight activities on the railway companies’ compliance with key regulations. (December 2022) 
• These results will be used to improve risk-based planning. (June 2023)

The action plan is two stages,

• At the end of stage 1 Rail safety will have better performance indicators that can be used on a regular basis to monitor performance. These will be consistent with those used by other modes at Transport Canada.
• Stage two will include the program evaluation that will determine the extent to which oversight activities are improving rail safety.

September 2021

December 2022

Performance Indicators

• Gap analysis to be completed  (April 2021)
• Develop action plan to address gaps (June 2021)
• Implement action plan to create more accurate performance indicators (September 2021)
• Implement consistent calculations for compliance. (November 2021)
• Monitor compliance rates on a regular basis. (January 2022)

Evaluation

• The evaluation is currently on the Internal Audit and Evaluation work plan for 2022-23.
• The evaluation will start in approximately January 2022 and take place over that year. It will provide recommendations for improvement/changes.
• Rail Safety will provide a detailed action plan to the evaluation recommendations outlining activities necessary to better measure program effectiveness on an ongoing basis. (February 2023)

Rail Safety

Anne Marie Muldoon, Director Safety Management Systems and Risk / Stephanie Lines Director Operations

Anne Marie Muldoon, Director Safety Management Systems and Risk

Claude Blanchette, Executive Director Evaluation, Fraud, Practice Management

2

Transport Canada should improve guidance to railway companies so that safety data submissions are complete, reliable and timely.

• Agreed. Transport Canada will develop an information request template (September 2021) and issue guidelines to railway companies and stakeholders to facilitate data analysis and streamline reporting. These measures will be completed by October 2021. 

The guidelines and information request template will provide clear direction to railway companies specifically with respect to the format of data received.

October 2021

• Develop additional guidance and information request templates to provide industry improved tools to make their data submissions (September 2021)
• Conduct information sessions for railways on the revised guidance and templates.(October 2021)
• Publish updated data submission guidance and templates on the Transport Canada Web site. (October 2021)

Rail Safety,

Stephanie Lines Director Operations

3

Transport Canada should integrate the findings from audits of safety management systems when planning its oversight activities and adjust its selection of planned inspections to address areas of highest risk.

• Agreed. Transport Canada will identify and adapt best practices to integrate safety management system audits into the selection of oversight activities for risk-based planning purposes. The risk-based business planning procedure will be updated to incorporate a review of audit findings.
• Auditors and inspectors who take part in safety management system audits will also participate in risk-based business planning activities for the various functions to highlight and explain audit findings. These measures will be completed by May 2021.

Revised risk-based planning procedures will require that staff from Safety Management Systems participate in oversight planning with each of the functional groups. This participation will include bringing forward audit results from prior years to be considered when selecting oversight locations.

May 2021

• Review the existing risk-based planning procedure and update to include participation of Safety Management System Staff in functional risk-based planning meetings;(April 2021)
• Establish guidelines to Safety Management System Staff with respect to the information they will need to bring to the functional meetings for planning purposes. (May 2021)

Rail Safety

Anne Marie Muldoon, Director Safety Management Systems and Risk

4

Transport Canada should set standards for the time it should take to assess whether railway companies’ corrected deficiencies identified in inspections and audits of safety management systems.

• Agreed. By June 2021, Transport Canada will identify best practices, including those in other modes of transportation, to establish standards for following-up on inspections and safety management system audits. The Department will establish a standard by December 2021 on how long it should take to assess whether railway companies corrected deficiencies resulting from inspections and audits of safety management systems.

A consistent approach to follow-up will be established for SMS audits that will guide all regions. Benchmarks for timelines will be established and monitored for both audit and inspection follow-up.

December 2021

• Establish guidelines for follow up of safety management system audits. These will include consultation with other modes as well as approaches used internationally. (March 2021)
• Complete analysis of existing time required for follow up of both audits and inspections including consultation with other modes and benchmarks they have in place. (October 2021)
• Establish benchmarks for Rail Safety and a monitoring program to measure actuals against benchmarks. (December 2021)

Rail Safety

Anne Marie Muldoon, Director Safety Management Systems and Risk / Stephanie Lines Executive Director Operations

5

Transport Canada should improve data management for its safety management systems audits by
• adhering to its documentation standards;  and
• integrating the results of safety management system audits into a data management system.

• Agreed. Transport Canada has confirmed that the audit teams in every region are using the recently implemented electronic working papers system to manage audit documentation (December 2020). In addition, the department will conduct a review of data management practices for audits to ensure consistent use across regions (April 2021).
• A data management system will be developed to facilitate the tracking of audit findings and identification of compliance trends. This system will be in place by December 2021.

A review of audit files will ensure consistent documentation standards. Data management system will capture audit results that can be more readily accessible.

December 2021

• A review of a sample of audit files from across headquarters and the regions for compliance with documentation standards will be completed. (April 2021).
• Recommendations will be made and regions and headquarter will have to respond with an action plan to correct.(June 2021)
• A data management system to track audit findings is currently under development and will be piloted to ensure accuracy and usability. (June 2021)
• Results from pilot will be evaluated and changes made as needed. (September 2021)
• The data management system will be fully rolled out in December 2021.

Rail Safety

Anne Marie Muldoon, Director Safety Management Systems and Risk

6

Transport Canada should make it an immediate priority to regularly assess the effectiveness of railway companies’ safety management system processes.

• Agreed. Transport Canada will undertake program and regulatory action to assess the effectiveness of safety management system processes:
  • Develop and launch a Targeted Audit Framework, and supporting implementation methodology, which will be used as a guide to identify issues that may be a result of ineffectiveness of a particular railway companies' safety management system processes (April  2021);
  • Finalize a risk-based process for determining when targeted audits are required (June 2021);
  • Develop tools to measure effectiveness and identify baseline and targets for each SMS process (November 2021);
  • Undertake  the audits of effectiveness that leverage the tools developed (December 2021);

• Launch consultations in April 2022 on potential amendments to the Safety Management System Regulations to strengthen rail safety requirements, including measures to improve the overall performance of safety management systems.

Transport Canada will have a system in place that assesses the effectiveness of a railway company’s safety management system’s processes.

Consultation will also take place to determine if changes are required to the Safety Management System Regulation.

December 2021

• A framework for measuring effectiveness of safety management system processes was established in June 2020.
• A methodology to support the Framework was established in September 2020 and was piloted for 2021-2022 planning. Results of the pilot will be reviewed and the methodology will be finalized in June 2021.
• Work to develop tools to measure baselines and targets for measuring the effectiveness of the various processes will be done in consultation with the regions and the Human Factors Specialist. (November 2021)
• Audits that measure effectiveness of the various processes and using the new targets and benchmarks will start no later than December 2021.
• Benchmarks and targets will be reviewed and refined on an ongoing basis to ensure they accurately reflect effectiveness. (Ongoing)

• Consultations will be launched in April 2022 with the rail industry and with the public. Consultations will commence to identify potential amendments to the Safety Management System Regulation.

Rail Safety

Anne Marie Muldoon, Director Safety Management Systems and Risk

Sean Rogers Director of Regulatory Affairs