Horizontal evaluation of the world class tanker safety system: Lessons learned

From: Transport Canada

Evaluation report outlining results of the assessment of the horizontal evaluation of the world-class tanker safety system.

On this page

Executive summary

The Horizontal Evaluation of the World Class Tanker Safety System (WCTSS): Lessons Learned covers the five-year period from 2015-2016 to 2019-2020. The scope includes all activities/initiatives of the WCTSS partner departments. The evaluation was conducted to address commitments made in the 2013 WCTSS funding document. In addition, as the WCTSS was the predecessor to the Oceans Protection Plan (OPP), this report will provide context for the upcoming OPP horizontal evaluation.

Given that the WCTSS has already sunset, no recommendations are made and a Management Response and Action plan is not required.

Successes and challenges

Successes and challenges were noted under the following three themes:

Program management and results delivery

The WCTSS contributed to stronger marine safety measures and protections, through actions such as increased surveillance and bolstering pollution prevention requirements. The WCTSS enhanced Canada’s response capacity by expanding key programs.

Challenges included resource capacity, time delays, and monitoring resource use and performance.

Collaboration within government

Through the WCTSS, federal partners communicated on shared priorities related to marine safety. International collaborations on oil spill science were also initiated, enhancing Canada’s science/research reputation internationally.

However, governance was an ongoing challenge, with the distinct roles and responsibilities of partners not always clear. Further coordination would have been valuable.

Stakeholder engagement, communication and inclusiveness

Although GBA+ was not a requirement when the WCTSS was designed, engagement activities did involve diverse populations. Importantly, through the WCTSS, the federal government began engaging Indigenous groups in conversations about marine safety. In addition, WCTSS supported the integration of science and policy in the context of ship-source oil spills.

Inconsistencies in communication and engagement practices contributed to engagement fatigue and confusion amongst stakeholders.

The implementation of the WCTSS was the federal government’s first horizontal undertaking aimed at creating a world-class regime for the prevention of, and preparedness and response to, ship-source oil spills. The WCTSS was key in establishing the relationships and governance protocols that were ultimately leveraged by subsequent horizontal initiatives, like the OPP, and has facilitated the development and implementation of certain marine safety interventions and activities. It is within this context that the WCTSS lessons learned are presented.

Lessons learned

Program management and results delivery

  1. The intended results of the WCTSS were not well understood during program design, which ultimately made it difficult to assess effectiveness.
  2. The implementation of the WCTSS suffered due to challenges with staffing, the underestimation of the time required to draft key policy/funding documents, and the volume of distinct projects.

Collaboration within government

  1. Collaboration within the federal government for WCTSS initiatives could have been strengthened through a shared lexicon and an enhanced strategic perspective.
  2. The design and implementation of the WCTSS could have been improved by the continuous and active involvement of interdepartmental groups from regions and national headquarters, to ensure that multiple perspectives were incorporated and key issues addressed.

Stakeholder engagement, communication and inclusiveness

  1. Partnerships with Indigenous communities could have been improved through a more coordinated and strategic approach to ensure meaningful engagement.
  2. Partnerships with industry could have been improved through formalized engagement practices and clear communication on plans and relevant changes.
  3. Some of the factors highlighted to foster an effective partnership include having a common understanding of (1) shared objectives and alignment with individual organizational objectives, (2) decision-making processes, and (3) the roles of each partner.

Context and current status

Evaluation context

Context and scope

The Horizontal Evaluation of the World Class Tanker Safety System (WCTSS): Lessons Learned was conducted to address commitments made in the 2013 WCTSS funding document. The WCTSS preceded the Oceans Protection Plan (OPP) and influenced the identification and design of certain OPP initiatives. The results of this evaluation will provide useful context for the upcoming Horizontal Evaluation of the OPP.

This evaluation covers the five-year period from 2015-2016 to 2019-2020. The scope includes all activities/initiatives of the WCTSS partner departments, with a particular focus on monitoring recommendations from an interim evaluation (i.e. the WCTSS implementation review; 2017), and identifying key lessons learned.

Given that the WCTSS has already sunset, no recommendations are made and a Management Response and Action plan is not required.

Methods

Transport Canada conducted the WCTSS evaluation in partnership with Fisheries and Oceans Canada-Canadian Coast Guard (DFO-CCG), Environment and Climate Change Canada (ECCC), and Natural Resources Canada (NRCan).

Evaluation methods included a document review and interviews. Key questions during interviews related to the achievement of outcomes, challenges, lessons learned, future implications, and inclusiveness.

Limitations and considerations

  1. Given COVID-19 restrictions, all interviews were held virtually.
  2. Some key informants were also interviewed for another evaluation, the OPP Case Studies, at the same time. Because of these concurrent conversations and the fact that the WCTSS transitioned into the OPP, respondents spoke from the OPP perspective, presenting issues as reflections on the OPP rather than solely as WCTSS lessons learned.
  3. Due to the rapid transition from WCTSS to OPP, a financial/efficiency analysis of WCTSS was neither possible nor practical. This topic will be explored in the upcoming horizontal evaluation of the OPP

Program background: Evolution of Canada’s ship-source oil spill response regime

Canada’s ship-source oil spill response regime has evolved over time, particularly in the 2010s, with the launch of the WCTSS.

1970s-1980s: Early incidents and action

Oil spills, such as the one from the SS Arrow (February 4, 1970), were milestone incidents that influenced Canada’s early marine oil spill response regimes. Environment Canada established its environmental emergencies program in 1971, followed by an R&D program.

1990s: Recognition of risk and public-private partnership

Following the Exxon Valdez spill (1989), the Public Review Panel on Tanker Safety and Marine Spills Response Capability was formed. The Panel submitted their final report, Protecting Our Waters: Final Report (the Brander-Smith Report) the following year, outlining actions to improve Canada’s prevention and response regime.

From 1991 to 1993, the Canadian Coast Guard and Environment Canada undertook work on private-sector funded response capability.

Since 1995, Transport Canada has been the lead policy and regulatory department for Canada’s ship-source oil spill preparedness and response regime. The Canadian Coast Guard (CCG) is the lead federal response agency (i.e., operational).

2010s: Expansion of partnership for responsible resources development

In 2013, the federal government launched the World Class Tanker Safety System (WCTSS). Stakeholder groups were consulted to create a world-class regime for the prevention of, and preparedness and response to, ship-source oil spills.

Also in 2013, the government announced the creation of the Tanker Safety Expert Panel to review Canada's tanker safety system and propose further measures to strengthen it.

The government later announced and launched the 5-year $1.5 billion Oceans Protection Plan (OPP), covering the period of 2017-18 to 2021-22. The OPP built upon the WCTSS, with marine safety, marine ecosystem protection, Indigenous partnerships, and a stronger evidence-base as pillars.

2020s: Exploration of options

The OPP is expected to end on March 31, 2022. The Government of Canada is exploring options for the renewal of the OPP.

Program background: Objectives, pillars, and key players

The WCTSS consisted of 32 initiatives intended to strengthen Canada’s oil spill response regime and to minimize the potential impacts of a ship-source oil spill, under three pillars with a variety of key players involved. With the exception of one initiative (namely, Science and Technology for Clean-up), WCTSS initiatives relied on B-base funding with a specific end date.

Prevention: Avoidance of ship-source oil spills

  1. TC plays a regulatory and oversight role, which includes regulatory development and compliance monitoring, including enforcement actions administered by marine safety inspectors. DFO provides key data (e.g., hydrographical information), which contributes to the identification of high-risk areas. Not-for-profit organizations, such as the Clear Seas Centre, undertake research and engagement activities. Indigenous and coastal communities collaborate with federal partners to share knowledge and resources to increase prevention and response capacity.

Response: Responding to ship-source oil spills

  1. The Canadian Coast Guard (CCG) leads marine oil spill response and works with partners to ensure appropriate incident response. DFO supports partners in managing response efforts and identifying high risk areas through maps and radar information. TC’s National Aerial Surveillance Program (NASP) detects oil spills and maps clean-up progress. Industry-funded and TC-certified response organizations provide spill clean-up services to the shipping industry. Environmental Emergency Officers, wildlife biologists, modelling experts, analysts, and other specialists with ECCC, NRCan, and DFO provide scientific advice on clean-up, which includes information on the path of the oil, weather conditions, and marine and coastal wildlife. Various response partners and Indigenous and coastal communities execute the response and clean-up, including actions such as debris removal and deflecting the spill.

Compensation: Ensuring that polluters pay for losses or damages

  1. Ship owners and insurers, Canada’s Ship-source Oil Pollution Fund (SOPF), and the International Oil Pollution Compensation Funds (IOPC) pay out claims and provide compensation.

Current status of WCTSS initiatives

A total of 32 initiatives were implemented through the WCTSS.

The interim 2017 WCTSS Implementation Review indicated that over half (17 of 32) of WCTSS initiatives were experiencing delays in comparison to their forecasted completion dates. By that time, two initiatives had been cancelled: Transport Canada Centre in Kitimat (Initiative #4) was cancelled because the Northern Gateway Pipeline was stopped, and the Team of international experts to assess tanker safety (Initiative #6) was rolled into another initiative (Panel Review of Canada's Oil Spill Response Capacity).

As of 2021, excluding the above-noted cancelled initiatives, all WCTSS initiatives have been completed. Of the 30 completed initiatives, the key informants confirmed that 28 met their planned objectives. Due to changing context and various challenges, two initiatives – Alternative Response Measures (ARMs; Initiative #26 - TC, ECCC) and Area Response Planning (ARP; Initiative #27 - TC, CCG, DFO, ECCC) – could not achieve their original intended objectives.

Crosswalk with OPP

Some WCTSS work has continued under the OPP, while other aspects have not.

  • 19 WCTSS initiatives were carried forward under OPP (with some variations in scope and focus)
  • 2 WCTSS initiatives were carried forward with A-base funding (distinct from OPP)
  • 9 WCTSS initiatives were not carried forward (work completed, no further funding /work required)
  • 2 WCTSS initiatives were not carried forward (cancelled due to changing context during the WCTSS)

In 2015, while the WCTSS was being implemented, there was a change in government. The new government launched the OPP, which, with a $1.5 billion budget, was significantly larger than the WCTSS ($394.8 million).

The work of 19 of 32 WCTSS initiatives was carried forward into the OPP. Others were not. Not all 19 WCTSS initiatives and OPP sub-initiatives can be linked one-to-one, as work under the OPP sub-initiatives have expanded or evolved in some cases. For some, while the names may sound similar, the scope and nature of work are different.

Follow-up on the 2017 WCTSS Implementation Review Management Action Plan

2017 WCTSS Review Recommendations

“When developing a tracking scheme to monitor the implementation of Oceans Protection Plan (OPP) initiatives, it is recommended that TC:

  1. Ensure that it is able to track the WCTSS initiatives distinctly within that scheme; and
  2. Apply lessons learned from monitoring WCTSS implementation. Specifically, it is recommended that TC track and request from the partner departments the following information:
    • A list of all expected and completed milestones and timelines by fiscal year for each initiative based on what is described in foundational documents and performance measurement strategies; and
    • Interdepartmental expenditure information, specifically, the total dollars allocated and the percentage of dollars spent by fiscal year for each initiative.”

Action

To address this recommendation, all OPP initiatives were organized so that they could be tracked through new comprehensive tools and dashboards, to support the OPP Deputy Minister Committee that oversees the initiatives. The deadline for this action was April 2017.

The Management Action Plan Update conducted in October 2017 indicated that this action was completed. Evidence gathered by the evaluation team found that the OPP has been tracking its initiatives through:

  • tracking sheets (also known as input sheets), which were used to track project milestones/results for all 57 OPP sub-initiatives
  • interdepartmental strategic dashboards (also known as the “DM Dashboard”) that are presented to the OPP Deputy Minister Committee. These provide a summary of the information included in the tracking sheets and highlight key achievements and issues/risks.

Observations and lessons learned

Considerations

The following provides an overview of WCTSS achievements, challenges, and lessons learned. When reviewing these observations, it is important to consider the context surrounding the WCTSS:

Speed: The quick transition between the WCTSS and the OPP;

Scope: The shift toward more inclusive and collaborative approaches with partners and stakeholders, including Indigenous groups; and

Novelty: The lack of previous experience and tools to support this novel approach.

Program management and results delivery: Observations

The WCTSS enhanced Canada’s response capacity

The National Aerial Surveillance Program (NASP) expanded its coverage, using WCTSS funds, for an additional 1500 patrol hours for NASP’s two Dash-8 aircraft, one on each coast. NASP takes on a prevention role by patrolling commercial shipping lanes, which acts as a deterrent for illegal activities and facilitates rapid responses to spills. NASP works collaboratively with TC’s Marine Safety & Security, DFO, CCG, and ECCC to investigate anomalies, monitor spills, and provide tactical information to response programs.

ECCC’s Integrated Satellite Tracking of Pollution program (ISTOP) also expanded its coverage through WCTSS, with an additional 8 hours per day for a total of 18 hours per day, 7 days per week. The program uses satellite imagery to spot anomalies in the water and forwards that information to NASP for further investigation.

The Incident Command System (ICS) has led to a greater awareness of the roles in the management of transport-related incidents and an enhanced surge capacity for government response efforts. The ICS is considered a key WCTSS success, as its methodology enhanced collaboration across federal departments in incident response. Further, its implementation across the CCG ensures national coordination, consistency, and predictability in incident management.

The Canadian Coast Guard Auxiliary (CCGA) in the Arctic was expanded, which enabled further participation of non-governmental organizations in response. The CCGA consists of nearly 4000 volunteers across Canada who support marine safety education and rescue. There are now 24 CCGA units across the Arctic, including 420 members and more than 45 vessels.

The WCTSS also invested in environmental response through Environmental Response Program Modernization. This work involved amendments to the Canada Shipping Act, 2001 and the development of a new management framework with clear work descriptions and policies. This enhanced departmental expertise and strengthened oversight capacity.

The WCTSS contributed to stronger marine safety measures to protect the public and the environment. It supported the government’s plan for Responsible Resource Development

The WCTSS was implemented to strengthen ship-source oil spill prevention, preparedness and response, and to advance the polluter pays principle. The highlights of the WCTSS included:

  • Funding to strengthen the marine safety system and measures such as increased tanker inspections, the expansion of aerial surveillance, a review of pilotage and tug-escort requirements, expanded research on non-conventional petroleum products, and traffic control measures at various ports.
  • Requirements for pollution prevention and response at oil-handling facilities were bolstered through amendments to the Canada Shipping Act, 2001, which was enacted in the Safeguarding Canada’s Seas and Skies Act. Amendments also introduced new offences for contraventions of the Act and extended pollution penalties.
  • The first tanker safety expert panel in more than three decades was also created.

Challenges for program delivery included resource capacity, time delays, and monitoring resource use and performance

Interviewees highlighted that insufficient new financial and human resources presented a challenge, particularly for engagement and policy development activities. For example, the Environmental Response Program had gaps in human resources under WCTSS. It improved after the completion of the WCTSS when the program started receiving A-base (ongoing) funding.

Numerous delays were encountered, including difficulties accessing funding, issues with procurement and contract design, and long staffing processes. In addition, staff time was frequently redirected and strained. Because the WCTSS had multiple policy and funding documents to prepare, the time of certain staff members was divided between program delivery and the preparation of funding documents. It also took a significant amount of time and operational experience before certain initiatives hit their stride and were able to overcome obstacles and operate efficiently.

Monitoring and tracking was also a challenge in some cases:

  • Defining appropriate indicators for specific initiatives was challenging. Under the three scientific research initiatives, counting the number of publications was not an ideal metric to account for the outcomes that were enabled or achieved by the availability of research information. Furthermore, due to the nature of research initiatives, it is natural to experience a time delay in the observation of outcomes related to the research conducted.
  • There were insufficient tools for monitoring and tracking the performance of all of the initiatives, especially those with B-base (time-limited) funding. As a result, project managers devised their own tracking tools, which resulted in inconsistent practices.

Program management and results delivery: Lessons learned

Through interviews with key informants and reviews of pertinent documents, it was learned and observed that:

  1. The intended results of the WCTSS were not well understood during program design, which ultimately made it difficult to assess program effectiveness. In general, there are frequent difficulties in identifying precise outputs, outcomes, and indicators at the program design stage, particularly when the program is large, complex, and does not have a clear precedent. This was the case for WCTSS, particularly in the context of Indigenous engagement and environmental protection in the transportation of resources. High expectations were set without a clear plan to measure and track results.

  2. The implementation of the WCTSS suffered due to challenges with staffing, the under estimation of the time required to draft key policy/funding documents, and the volume of distinct projects. The WCTSS was implemented in a phased approach to align with the publication of Tanker Safety Expert Panel reports and the schedules of central agencies, in terms of their ability to accept policy recommendations and funding proposals. This implementation process, which required the preparation of multiple policy and funding documents, along with the challenges encountered with staffing certain positions, resulted in implementation delays. In contrast, the OPP approach, with one comprehensive policy document and funding document, was more efficient, as time could be spent on implementation activities rather than drafting additional policy or funding documents. In addition, the OPP launched shortly after the completion of WCTSS, with little time allocated for reflection and identifying areas for improvement. WCTSS results and lessons learned could not be fully incorporated into the planning and implementation of OPP.

    Considerations of workload and project interdependencies could improve project design and delivery. Interviewees highlighted that the volume of distinct projects presented a challenge, as some projects were clearly linked but could not advance collectively due to the design of the initiatives.

Collaboration within government: Observations

The WCTSS enabled government-wide discussions of marine safety

The WCTSS brought together key federal partners responsible for planning and undertaking marine safety activities. Through the WCTSS initiative, federal partners communicated and discussed how to support shared priorities. For instance, pre-WCTSS, NRCan’s CanMET Devon lab had developed expertise in petroleum science but was not well connected with federal partners on oil spill research. Under WCTSS, the CanMET Devon lab began meeting with other government partners, which led to its focus being extended to include oil spill research. It has since become a recognized contributor to marine safety.

Research under the WCTSS spurred international collaborations

As WCTSS-related research papers were published, the international community began to take notice and to collaborate with Canadian researchers. For example, researchers from ECCC have been contacted by US federal agencies such as the National Oceanic and Atmospheric Administration (NOAA), the Environmental Protection Agency (EPA), and the United States Coast Guard (USCG), as well as numerous state agencies. International researchers who are trying to improve oil spill modelling have expressed interest in working with Canadian scientists. WCTSS activities have contributed to maintaining Canada’s status in the global community.

Governance of horizontal initiatives is an ongoing challenge. ‘Silos’ persist, and roles and responsibilities are not always clear

Interview evidence highlighted that there was a lack of clarity of roles and responsibilities, which presented challenges, particularly in the context of engagement with Indigenous communities and stakeholders. Within departments as well, the mixture of A-based programs and B-based WCTSS initiatives meant that WCTSS initiatives were managed separately from regular programming. This created challenges in understanding the links between activities and coordinating their implementation.

It was also noted that further coordination of science activities could be valuable. One interviewee suggested there could be a formalized interdepartmental committee for oil spill science, similar to what is currently in place for energy, to establish a coordinated approach to conducting this research. This would build on informal interdepartmental committees and collaborations already in place.

Governance structures in place under the WCTSS have not necessarily continued under the OPP. However, working-level cooperation continues through these pre-existing relationships.

Collaboration within government: Lessons learned

Through interviews with key informants and reviews of pertinent documents, it was learned and observed that:

  1. Collaboration within the federal government for WCTSS initiatives could have been strengthened through a shared lexicon and an enhanced strategic perspective.

Partnerships among federal departments would be enhanced through establishing common terminology to ensure a shared understanding of key issues and responsibilities.

Two interviewees also noted that an interdepartmental ADM committee for WCTSS would have enabled strategic thinking. This improvement was made during the transition from WCTSS to OPP.

  1. The design and implementation of the WCTSS could have been improved by the continuous and active involvement of interdepartmental groups from regions and national headquarters, to ensure that multiple perspectives were incorporated and key issues addressed.

This involves ongoing communication and coordinated governance throughout all stages of the horizontal initiative.

Stakeholder engagement, communication, and inclusiveness: Observations

While GBA+ was not a requirement at the time the WCTSS was designed, engagement involved diverse groups and was central to the effectiveness of WCTSS initiatives

The Government of Canada committed to using gender-based analysis (GBA) to advance gender equality in Canada in 1995. GBA was rebranded as GBA+ in 2011, and the government recommitted to GBA+ through the Action Plan on GBA (2016-2020) in 2016. At the time of designing the WCTSS, GBA+ considerations were not a program design requirement, like they are currently. WCTSS engagement activities did involve diverse populations, particularly through Initiative #18 –Private, Public, and Community Partnership.

WCTSS partners were able to conduct outreach and increase their presence through grassroots engagement. This was particularly instrumental for the CCG’s Auxiliary, a network of volunteers which augments Canada’s national maritime search and research response capacity.

Importantly, through the WCTSS, the federal government began to engage with Indigenous groups and include them in conversations about the future of Canada’s marine safety system.

Interviewees noted that WCTSS engagement experiences informed OPP practices.

The WCTSS supported knowledge-sharing and the integration of science and policy

The role of science in policy and decision-making related to marine safety, particularly for TC and CCG, advanced through the WCTSS. Following the implementation of WCTSS, improvements were noted by interviewees in the way science, as it relates to oil spills, is integrated into policy decisions.

Research under the WCTSS focused on oil and oil spills. Focused experiments in the area of geoscience, marine oil spill response, and clean-up have produced a large amount of published, peer-reviewed research. The WCTSS also coordinated scientific research among federal partners, namely ECCC, DFO, and NRCan.

Some interviewees suggested that the sharing of knowledge between Indigenous communities, the scientific community, the response community, and other stakeholders improved over time.

Engagement approaches and communication were inconsistent, which in some instances led to engagement fatigue and confusion

The WCTSS lacked a clear governance structure for engagement. Consultations with Indigenous communities were variably led by the national headquarters, regional offices, and different departments. This contributed to engagement fatigue among community partners. Interviewees noted this was a particular challenge in the Area Response Planning initiative.

A lack of communication regarding engagement priorities and program activities was sometimes noted. For instance, some industry partners were found to be unaware of the government’s response to certain Tanker Safety Expert Panel recommendations.

Based on the recommendation of the Expert Panel report, the Regional Advisory Councils were cancelled. This left a gap in engagement practices and led to challenges and inconsistencies with provincial involvement. Under the OPP, TC is working to refine engagement practices and governance to address this gap.

Stakeholder engagement, communication, and inclusiveness: Lessons learned

Through interviews with key informants and reviews of pertinent documents, it was learned and observed that:

  1. Partnerships with Indigenous communities could have been improved through a more coordinated and strategic approach to ensure meaningful engagement.

    WCTSS engagement practices could have been improved by:

    • Budgeting enough time to build meaningful relationships with Indigenous communities;
    • Developing strategies to enhance capacity (both external and internal to the federal government, in terms of human resources and subject matter expertise) to enable effective participation and engagement;
    • Strategizing engagement activities to meet the needs of different communities, and maintaining awareness that communities face a broad range of issues beyond one specific subject at a given time; and
    • Building better data collection and coordination to improve engagement planning and follow-up.
  1. Partnership with industry could have been improved through formalized engagement practices and clear communication on plans and relevant changes.
  1. Some of the factors highlighted to foster an effective partnership include having a common understanding of: (1) shared objectives and alignment with individual organizational objectives, (2) decision-making processes, and (3) the roles of each partner.

Appendix A: Complete list of lessons learned

Program management and results delivery

  1. The intended results of the WCTSS were not well understood during program design, which ultimately made it difficult to assess program effectiveness.
  2. The implementation of the WCTSS suffered due to challenges with staffing, the under estimation of the time required to draft key policy/funding documents, and the volume of distinct projects.

Collaboration within government

  1. Collaboration within the federal government for WCTSS initiatives could have been strengthened through a shared lexicon and an enhanced strategic perspective.
  2. The design and implementation of the WCTSS could have been improved by the continuous and active involvement of interdepartmental groups from regions and national headquarters, to ensure that multiple perspectives were incorporated and key issues addressed.

Stakeholder engagement, communication, and inclusiveness

  1. Partnerships with Indigenous communities could have been improved through a more coordinated and strategic approach to ensure meaningful engagement.
  2. Partnership with industry could have been improved through formalized engagement practices and clear communication on plans and relevant changes.
  3. Some of the factors highlighted to foster an effective partnership include having a common understanding of: (1) shared objectives and alignment with individual organizational objectives, (2) decision-making processes, and (3) the roles of each partner.

Report a problem with this page

Date modified: