Audit of Transport Canada’s Transformation Governance and Oversight

Governance

An effective governance framework is in place to oversee the implementation of TC Transformation, which has a clearly defined vision and scope.

Governance committees are in place with clear objectives, defined roles and responsibilities that help ensure effective coordination, integration and oversight of Transformation initiatives and projects.

Risk management: risks to the overall success of TC Transformation are tracked, managed and reported.

Integrated planning

An effective planning approach has been developed to set clear direction and priorities on the implementation of TC Transformation.

Change management

Change management processes and practices are established to facilitate the implementation of TC Transformation.

Savings and reinvestments

The department defined specific saving and reinvestment initiatives under the Comprehensive Review. The progress of these initiatives is effectively tracked and reported. Senior management and governance bodies receive relevant, accurate, complete and timely information for their timely decision making.

An effective framework is in place to oversee the implementation of TC Transformation, which has a clearly defined vision and scope.

TC Transformation has a clearly defined vision and scope.

A survey of TMX members to seek input on clarifying the vision for Transformation was completed in January.

A Logic Model, and a high level Transformation Integrated Implementation Plan was presented at the December 4, 2018 TMX meeting for feedback and approval.

Prior to the recent survey of TMX members there was ambiguity and differing views re the vision and scope for TC Transformation.

• Safety and Security only/primarily?
• How does TC’s environmental and efficient transportation system mandates fit?
• World Class; Best in Class etc. and what are their requirements and implications?
• What specifically needs to transform to become a “modern” TC (Regulator)?

TMX should assure itself that the vision and scope for Transformation are sufficiently defined to gain the level of management and staff support needed to help ensure successful implementation.

An effective framework is in place to oversee the implementation of TC Transformation, which has a clearly defined vision and scope.

Governance Committees are in place with clear objectives, defined roles and responsibilities to help ensure effective coordination, integration and oversight of Transformation initiatives and projects.

The current governance framework which includes existing senior level departmental committees, along with Transformation specific committees at the director and director general level is being revised with the recent change in Transformation leadership.

Several of the committees identified in the governance framework have terms of reference (ToRs) (including authorities, roles and responsibilities) and maintain records of decision (RoDs).

There is confusion regarding roles and responsibilities and how the various committees should interact and coordinate with each other which can lead to missed opportunities.

The exact role and responsibilities of the Co-Champions is still unclear. There is no all inclusive Co-Champions committee/forum/process to help ensure integration and to fully leverage their knowledge and experience.

There is a clear willingness to collaborate but the level and consistency of coordination between Safety and Security Management Committee, including the regions and the leads of the various Transformation initiatives could be strengthened, since it is critical to ensuring alignment between short term incremental improvements and the longer-term modernization work.

Other than the ADM Policy committee, it is not clear how the Policy group is integrated into the governance or planning processes to provide advice and weigh in on safety and security policy questions which have environmental and economic impacts.

Not all working level committees are identified in the current governance structure despite the significant support they provide. Including them in the formal governance structure may lead to opportunities for collaboration and better coordination.

Effective risk management processes are in place to track, monitor and report risks to support the successful implementation of TC Transformation.

Responsibility has been assigned for identifying, monitoring and mitigating risks to the success of TC’s Transformation.

There are procedures documented and communicated to ensure the risks are consistently tracked, monitored, and reported.

There is a defined risk escalation process to ensure risks are mitigated at the appropriate level and reported to senior management.

A unit under the former ADM Transformation Team consolidates risks and issues self- identified by the programs and Transformation initiative leads.

The programs use TC Transformation Strategy – Initiative Status Reports to monitor and report the initiative status including reporting the “risks and issues”.

TMX is briefed quarterly on the high risks to individual initiatives/projects through the dashboard reporting.

Currently the primary focus is on individual initiatives and project risks but given the linkages and dependences between the various initiatives/projects the overall risks to the success of TC Transformation may not be fully understood and managed.

As identified in past audit work and acknowledged by the department, project management is a general area of weakness which impacts the quality of project status and risk reporting. The real level of risk for each initiative may not be fully understood and mitigated in a timely manner.

There is no clearly defined risk escalation process in place to ensure risks are escalated or addressed at the appropriate governance level.

Adequate planning is in place to define clear priorities that are sequenced to optimize resource use, minimize burden on programs and support management oversight.

A comprehensive TC Transformation project plan that integrates all initiatives to support management oversight and decision making. The plan and the planning process is consistent with best practices such as. PMBOK.

A TC Transformation High Level Integrated Implementation Plan has been developed and presented to TMX December 4, 2018.

Plans exist for “Signature Initiatives” and other Wave 1 priorities.

The current integrated plan could better describe the critical path to achieving the goals of TC Transformation by highlighting the priorities, interdependencies, required sequencing, risks, and clarifying accountabilities.

The common project planning and reporting templates are not yet being used by all Signature Initiatives and Wave 1 priority projects.

Programs have limited capacity to manage the new initiatives and effectively manage change while maintaining their core activities. There is a perception that core program activities could be sacrificed to shift resources to focus on the Transformation initiatives.

An integrated planning process in place to clearly define and manage priorities, the sequence of activities, and that helps ensure dependencies are well understood.

A prioritization tool has been designed to support sequencing decisions for implementing changes within priority program areas. The tool helps to assess:

• A program’s ability to implement changes.
• The relative strategic value to the department.
• Other timing factors

The criteria for assessing a Program’s general project management readiness does not ensure that a Program has answered key fundamental policy questions, which are critical to ensuring TC focuses its efforts on core issues and reduces the risk of changing direction and requiring re-work. For example:

• Is there a clear understanding of the public transportation risks that are being addressed by the program?
• Has the most cost-effective intervention approaches been selected (regulatory versus non-regulatory)?
• Has the most cost-effective program delivery method been selected (TC resources versus outsourced to delegates)?
• Have the program delivery processes (for example, certification; plan approvals; inspections; audits etc.) been reviewed to ensure they are streamline and efficient; before automating/digitalizing or increasing/introducing fees (cost recovery)?

There are change management processes and practices established to facilitate the implementation of TC Transformation.

The change management plans and processes are consistent with best practices (for example, PROSCI’s awareness, desire, knowledge, ability and reinforcement (ADKAR) model).

Change Management training has occurred at both the executive and manager levels. Approximately 200 TC managers were invited to attend the training in 2017 to 2018, and more one day change management training sessions have been scheduled. Feedback from both the executives and managers that have taken the training, indicates that the majority found the training “very useful”.

The TC Transformation High Level Integrated Implementation Plan includes a change impact assessment.

Defining the full scale of the changes required to achieve the goals of TC Transformation.

Although the Initiative Status Report template has a Change Management section where project Offices of Primary Interest (OPIs) can provide their change management plan, there is no specific guidance on how to develop a change management plan, or any indication, as to when such a plan should be developed.

Only a few plans have been developed.

A general awareness of the change management processes and practices.

TC Transformation has held multiple events to spread awareness of their signature initiatives and change management.

Although the Change Management team continues to work with project leads to provide advice and guidance on how to integrate change management best practices into their work, they have not yet engaged with working level staff, and based on our work to date the messaging may not be permeating the working levels. As noted in McKinsey’s report “Delivering for citizens: How to triple the success rate of government transformations”; nearly 90% of survey respondents said success would have been enhanced by engaging more with frontline employees.

Reallocation (savings and reinvestments) initiatives are effectively managed.

Savings and reinvestments are effectively tracked and reported.

Senior management and governance bodies receive relevant, accurate, complete, and timely information for decision-making.

TC Corporate Finance reduced programs’ notional budgets based on the {ATIP REMOVED} subsequent RMC decisions.

Programs are to use the TC Transformation Strategy – Initiative Status Report template to track, monitor, and report the actual saving and reinvestment initiatives.

Transformation Team collects and reports the actual savings based on programs’ reports

The status of the reallocations is reported to TMX quarterly and Treasury Board Secretariat in an annual report including the overall achievements (for example numbers of initiatives completed, on track, or require attention).

There is no assurance that Program activities have been “stopped” or estimated “productivity gains” have been realized.

There is no explicit requirement as part of the Integrated Planning and Reporting (IPR) process for Program managers to explain any budget pressures that they face {ATIP REMOVED}

The actual achievements of savings are not centrally tracked and reported.

{ATIP REMOVED} The savings estimates and project statuses are based on self-reporting and are not subject to further validation.

Without close tracking of actual savings that documents the key decisions and changes that occur over time the department will have difficulty telling a comprehensive story and learning valuable lessons.

The Audit of Risk-based Business Planning in Safety and Security made 5 recommendations. The first 3 recommended fundamental change and improvements to S&S’s risk assessment and management capabilities:

1. Develop a consistent approach to modal program level risk assessments, in other words, a common modal risk assessment methodology and process to identify, assess and rank transportation safety and security risks.
2. Ensure assessments of alternative risk intervention/mitigation strategies (for example, education versus regulation) consider costs and benefits before selecting a preferred strategy.
3. Define risk tolerance levels to provide the basis for choosing oversight interventions that will reduce risks to an acceptable level for each mode; and provide the rationale for differences between the modes.

Given the significant level of change required to fully address the recommendations, Safety and Security committed to making both incremental improvements and transformative changes via TC’s Transformation. S&S is making progress introducing incremental improvements such as:

• Developing a Regulatory and Oversight Framework and Continuum of the transportation system.
• Making a clearer distinction between internal process risks and external public safety/security transportation risks in program planning documents.
• Creating an S&S risk registry.
• Consolidating data to support comparative analysis of programs to support senior management decision making (for example, program/modal oversight inspection plans; regulatory development action plans).

To fully address our recommendations one of the key initiatives of the department’s Transformation is focused on achieving the following outcomes:

• A safety and security public risk management methodology and oversight framework to enable the management of risks to the transportation system as a whole.
• A reliable data collection process and repository that will feed the oversight framework to measure risks and outcomes in a consistent and objective manner.
• An outcome measurement system capable of assessing the influence of safety and security activities on the safety, security, efficiency, and environmental sustainability of Canada’s transportation system over time.

This initiative is underway and it is currently forecasted to be completed by 2022. Given this timeline, it is critical to continue to make incremental improvements aligned with the ultimate methodology and process being developed

One of the final two audit recommendations focused on S&S’s ability to cost its activities to support senior management’s resource allocation decisions. The other focused on the important role of Corporate Services to support program managers need for complete, accurate and timely financial information to help their analysis and decision making.

4. Safety and Security:
   1. In the shorter term, modal programs should use existing data to analyze their current activities to identify areas of inconsistency and opportunities for improvement.
   2. In the longer term, Safety and Security should design and adopt a consistent standard activity costing methodology for oversight activities that supports risk-based business planning.

5. Corporate Services – Corporate Financial Management should be made responsible for:
   1. Gathering program financial information (Headquarters (HQ) and Regional inputs); and

Providing programs with a complete financial picture they can use for decision making purposes.

At the S&S group level, broad activity definitions are used for planning financial resource requests but there is no ability to report actual versus planned expenditures. This limits management’s ability to compare activity costs between regions within a program and between programs to support resource allocation decisions. It is important to note, however, that a few programs are further ahead. And as part of the department’s Fee Modernization initiative, under Transformation, a project to develop a longer term solution for activities subject to cost recovery is underway since the ability to track and report activity costs is fundamental to any cost recovery regime.

As well, the Chief Financial Officer (CFO) continues to take steps to improve the level and quality of support provided to program managers throughout the department. One recent measure involved training all departmental managers with a particular focus on helping managers to improve their financial forecasting.

Despite this progress, the capability to cost activities (not just cost recoverable activities) to enable comparative analysis, show trends over time, measure performance and ultimately assess the cost effectiveness of programs continues to be a significant gap and currently there is no comprehensive plan in place to address this gap.

An effective governance framework is in place to oversee the implementation of TC Transformation, which has a clearly defined vision and scope

TC Transformation has a clearly defined vision and scope.

Prior to the recent survey of TMX members there was ambiguity and differing views re the vision and scope for TC Transformation.

• Safety and Security only/primarily?
• How does TC’s environmental and efficient transportation system mandates fit?
• World Class; Best in Class etc. and what are their requirements and implications?
• What specifically needs to transform to become a “modern” TC (Regulator)?

TMX should assure itself that the vision and scope for Transformation are sufficiently defined to gain the level of management and staff support needed to help ensure successful implementation.

Transformation and Digital Agenda (April to June)

To broaden scope of transformation, both the Transformation Plan (based on {ATIP REMOVED}, five signature initiatives, S & S only) and the Digital Roadmap are being integrated.

Update storyline (April to June)

The new Transformation “Storyline” deck presentation to DAC on March 28, 2019 is reflective of the expanded departmental scope.

Adjust vision to reflect TMX input (April to June) and update Logic Model (July to September)

Present a proposed vision of “Agile. Smart. Trusted.” to TMX in April 2019 for discussion and decision. Vision will be defined and validated by the Logic Model and results framework in development.

An effective governance framework is in place to oversee the implementation of TC Transformation, which has a clearly defined vision and scope.

Governance Committees are in place with clear objectives, defined roles and responsibilities to help ensure effective coordination, integration and oversight of Transformation initiatives and projects.

Confusion regarding roles and responsibilities and how the various committees should interact/coordinate with each other.

Role and responsibilities of the Co-Champions is still unclear. There is no all-inclusive Co-Champions committee/forum to ensure integration.

Clear willingness to collaborate but the level and consistency of coordination between SSMC, regions and leads of Transformation initiatives could be strengthened.

Unclear how the Policy Group is integrated into the governance/planning processes to provide advice/input on policy questions having environmental and economic impacts.

Working level committees are not all identified in the current governance structure

Establish simplified governance (April to June)

New governance structure will be in place with clear mandates, defined roles and responsibilities towards better coordination, integration, collaboration and representation (including Policy Group).

Effective risk management processes are in place to track, monitor and report risks to support the successful implementation of TC Transformation.

Responsibility has been assigned for identifying, monitoring and mitigating risks to the success of TC’s Transformation.

There are procedures documented and communicated to ensure the risks are consistently tracked, monitored, and reported.

There is a defined risk escalation process to ensure risks are mitigated at the appropriate level and reported to senior management.

Currently the primary focus is on individual initiatives and project risks but given the linkages and dependences between the various initiatives/projects the overall risks to the success of TC Transformation may not be fully understood and managed.

As identified in past audit work and acknowledged by the department, project management is a general area of weakness which impacts the quality of project status and risk reporting. The real level of risk for each initiative may not be fully understood and mitigated in a timely manner.

There is no clearly defined risk escalation process in place to ensure risks are escalated or addressed at the appropriate governance level.

Establishing capacity (April to June)

Create capacity to identify, assess and manage risk holistically across Transformation initiatives (including project management risks)

Identify accountable program risk leads to manage and report on risks on a per-initiative basis

Processes and procedures (April to June)

Build on and use IM/IT Project Management Framework to ensure consistent application in tracking, monitoring and managing the Transformation initiatives.

Governance integration (April to June)

Establish new governance processes for escalating risks to ensure mitigation and intervention by appropriate authorities

Adequate planning is in place to define clear priorities that are sequenced to optimize resource use, minimize burden on programs and support management oversight.

A comprehensive TC Transformation project plan that integrates all initiatives to support management oversight and decision making. The plan and the planning process is consistent with best practices (for example Project Management Book of Knowledge (PMBOK)).

The current integrated plan could better describe the critical path to achieving the goals of TC Transformation by highlighting the priorities, interdependencies, required sequencing, risks, and clarifying accountabilities.

Common project planning and reporting templates are not yet being used by all Signature Initiatives and priority programs.

Programs have limited capacity to manage the new initiatives and effectively manage change while maintaining their core activities.

There is a perception that core program activities could be sacrificed to shift resources to focus on the Transformation initiatives.

Establish a Transformation Portfolio Office (April to June)

Lead for Transformation strategic direction, risk management, communication, planning and results reporting

Task Team to be created with resources from various program areas to address changing/diverse needs (including support development of common project planning tools).

Develop high level Transformation Integrated Plan (April to June)

Comprehensive map of key Transformation initiatives and timelines across priority programs

Identification of major linkages, dependencies, cross-cutting issues and risks.

‘How We Work’ Playbook (April to June)

Develop a Playbook (principles, tools and guidance) to consider and embed all aspects of Transformation

Raise awareness for Playbook importance, use and application to programs (highlight initiatives, approaches, new ways of working, change management).

Adequate planning is in place to define clear priorities that are sequenced to optimize resource use, minimize burden on Programs and support Management oversight.

An integrated planning process in place to clearly define and manage priorities, the sequence of activities, and that helps ensure dependencies are well understood.

The criteria for assessing a Program’s general project management readiness does not ensure that a Program has answered key fundamental policy questions, which are critical to ensuring TC focuses its efforts on core issues and reduces the risk of changing direction:

• Is there a clear understanding of the public transportation risks that are being addressed by the program?
• Has the most cost-effective intervention approaches been selected (regulatory versus non-regulatory)?
• Has the most cost-effective program delivery method been selected (TC resources versus outsourced to delegates)?
• Have the program delivery processes (for example, certification; plan approvals; inspections) been reviewed to ensure they are streamlined and efficient; before automating/digitalizing or increasing/introducing fees (cost recovery)?

Moving forward on priority programs (April to June)

Review priority programs to:

Define scope of change (requires a clear understanding of the program’s public policy objectives, program mandate/context, program delivery approach, processes, program risks, etc.).

Assess readiness (will be informed by the above considerations)

Determine impact of change.

Program design (July to September)

Develop Transformation Implementation Plans for each priority program in accordance to the ‘How We Work’ Playbook.

Continue building momentum through priority programs underway:

Navigation Protection Program (NPP); Transportation of Dangerous Goods (TDG); Security Screening (Intermodal Security).

There are change management processes and practices established to facilitate the implementation of TC Transformation.

The change management plans and processes are consistent with best practices (for example, PROSCI’s ADKAR model).

A general awareness of the change management processes and practices.

Defining the full scale of the changes required to achieve the goals of TC Transformation.

Although the Initiative Status Report template has a Change management section where project OPIs can provide their change management plan, there is no specific guidance on how to develop a change management plan, or any indication, as to when such a plan should be developed.

Only a few plans have been developed.

Although the Change Management team continues to work with project leads to provide advice and guidance on how to integrate change management best practices into their work, they have not yet engaged with working level staff, and messaging may not be permeating the working levels. As noted in Delivering For Citizens: How To Triple The Success Rate Of Government Transformations”; nearly 90% of survey respondents said success would have been enhanced by engaging with frontline employees.

Change Management Toolkit (April to June)

Launch change management training, guidance material and toolkit, incorporating best practices from PROSCI.

Impact and readiness assessments (April to June)

Support programs in determining the full scale of change needed to meet transformation objectives and establishing change management strategies and plans.

Building awareness (ongoing)

Monthly engagement on Transformation themes

Special Events, including EX Think Tank Series (for ex.: leadership in culture change) and TC Talks.

Engaging employees (ongoing)

Tailored employee engagement activities for programs, regions.

Engage change leaders:

• Digital Ambassadors
• Inspectorate Board
  Young Professionals Network
• Middle Managers
• Joint Union-Management Consultation Committees

Reallocation (savings and reinvestments) initiatives are effectively managed.

Savings and reinvestments are effectively tracked and reported.

Senior management and governance bodies receive relevant, accurate, complete, and timely information for decision-making

There is no assurance that Program activities have been “stopped” or estimated “productivity gains” have been realized.

There is no explicit requirement as part of the IPR process for Program managers to explain any budget pressures that they face due to unrealized Comprehensive Review saving estimates.

The actual achievements of savings are not centrally tracked and reported.

{ATIP REMOVED} The savings estimates and project statuses are based on self-reporting and are not subject to further validation.

Without close tracking of actual savings that documents the key decisions and changes that occur over time the department will have difficulty telling a comprehensive story and learning valuable lessons.

ADM Accountability (April to June)

Reporting will shift from OPIs to ADMs, including on pressures resulting from continuing activities for which budgets have been adjusted for {ATIP REMOVED}.

IPR Process (April to June)

Reporting via the IPR process and ADM-level reporting on a quarterly basis on savings and pressures to ensure senior management and governance bodies receive relevant, accurate, complete, and timely information for decision-making.

Activity Costing (ongoing)

Continue to enhance our activity costing capability to enable complete, accurate and timeline financial information for decision making.

Follow-up status of 2017 Audit of Risk-Based Business Planning in Safety and Security

Activity Costing – Capability. Two final audit recommendations:

Safety and Security’s ability to cost its activities to support senior management’s resource allocation decisions.

Role of Corporate Services to support Program managers need for complete, accurate and timely financial information to help their analysis and decision making.

Despite this progress, the capability to cost activities (not just cost recoverable activities) to enable comparative analysis, show trends over time, measure performance and ultimately assess the cost effectiveness of Programs continues to be a significant gap and currently there is no comprehensive plan in place to address this gap.

See above