Internal audit report outlining results of the assessment of the Follow-up audit of Oceans Protection Plan at Transport Canada.
On this page
- Purpose
- Impact of COVID-19 on OPP
- September 2019 DM Strategic Dashboard
- Fencing guidelines for OPP funds
- Status update of management action plan
Purpose
In the 2019-2020 Risk-based Audit and Evaluation Plan, Internal Audit committed to following-up on the recommendations made in the 2018 Internal Audit of Oceans Protection Plan (OPP) Governance and Oversight.
The purpose of this report is to provide:
- 1) An update on how COVID-19 has affected OPP
- 2) The results of the September 2019 OPP Strategic Dashboard Analysis
- 3) An update on the fencing guidelines for OPP Funds
- 4) A status update on the implementation of the recommendations made in 2018 (Annex B)
1. Impact of COVID-19 on OPP
{ATIP REMOVED}
Since March 2020, the OPP, like the rest of the Department, has been affected by COVID-19. However, with the end date of OPP less than 21 months away, it is likely that some of the OPP initiatives will not be completed on time. Some of the major impacts of COVID-19 include:
- Pandemic-related restrictions on travel, site access, and disruption to the supply chains caused delays in several capital projects under OPP involving the procurement, construction, installation and/or testing of new assets or infrastructure.
- Temporary reduction of staff in the OPP Secretariat – Staff were loaned out to other areas within the Department (ex: National Emergency Coordination Centre) to support high priority tasks.
- Indigenous Engagement – Travel restrictions and public health directives have made consultations difficult as Indigenous communities were completely locked down. While OPP has been able to continue engaging virtually, the capacity of Indigenous communities to deal with the crisis, as well as engage digitally, is inconsistent across the country.
- The collection of OPP and related initiatives that are {ATIP REMOVED}
- In order to address these issues, adequate time will be required to enable all OPP partners to reassess their capacity, their priorities and their needs, while working in a COVID-19 context where there are continued restrictions.
- OPP is looking at options moving forward to provide additional time for initiatives to be completed.
2. September 2019 DM Strategic Dashboard
2.1 Decisions related to DM Strategic Dashboards should be documented
Context
The Deputy Minister (DM) Strategic Dashboard is prepared twice a year, every April and September, to provide the DM Steering Committee with a status update on the OPP sub-initiatives. The members of this steering committee are the deputy heads from the departments that are involved with OPP (Transport Canada, Department of Fisheries and Oceans/Canadian Coast Guard, Natural Resources Canada and Environment and Climate Change Canada). Transport Canada is the lead department and TC’s DM is the Chair of the DM Interdepartmental Committee.
The September 2019 Strategic Dashboard, which was presented at the December 2019 Departmental Audit Committee (DAC), was verified using Transport Canada (TC) and Department of Fisheries and Oceans/Canadian Coast Guard (DFO/CCG) input sheets.
In the 2018 Audit of OPP, a sample of 14 sub-initiatives was analyzed based on risk. The progress of the 10 TC-led sub-initiatives (the remaining 4 sub-initiatives are led by DFO/CCG) were analyzed using TC input sheets from September 2019. {ATIP REMOVED}
In order to prepare the DM Strategic Dashboard, TC collects financial and non-financial information, which is also used for public reporting purposes, from within and from partner departments. This information is collected through input sheets that are prepared by the managers involved with the sub-initiatives. Along with the status of the milestones associated with each sub-initiative, the input sheets also provide a status on the:
- {ATIP REMOVED}
- Sub-initiative health – overall rating on the health of the sub-initiative, taking into account milestones, financials, and hiring process. This status communicates the likelihood of successful project delivery with/without additional intervention.
- Engagement relationship health - overall rating of the health of the relationship with the participants of engagement sessions, taking into account milestone completion and points or issues raised during the sessions. This status communicates the likelihood of successful project delivery with/without additional intervention.
Findings
Out of the 58 sub-initiatives that were reported in the September 2019 DM Dashboard, only three had different results reported in the input sheets:
| Sub-Initiatives (Lead Department) |
Strategic Dashboard Status | {ATIP REMOVED} | CCG Summary Sheet Sub-initiative Health | Project Scope Status | Strategic Outcome Health | Sub-Initiative HealthSub-Initiative Health | Engagement Health |
| Additional Radar Sites (CCG) | On Track | {ATIP REMOVED} | Remediation Required | On Track | Remediation Required | Remediation Required | On Track |
| Places of Refuge (TC) | On Track | {ATIP REMOVED} | N/A | N/A | Attention Required | Attention Required | Attention Required |
| Risk Analysis of Maritime Search and Rescue Delivery (CCG) | On Track | {ATIP REMOVED} | On Track | On Track | Attention Required | Attention Required | Attention Required |
Caption: The table provides a description of the three inconsistencies found when comparing the results of the input sheets to the results of the dashboard.
OPP Secretariat confirmed that the discrepancy with the Additional Radar Sites’ sub-initiative was as a result of a discussion that took place within the OPP Interdepartmental Secretariat, where a decision was made not to repeatedly inform the DM OPP Committee of an issue that had already been identified several times in previous dashboards. The issues related to Places of Refuge and Risk Analysis of Maritime Search and Rescue Delivery were deemed to be manageable without the DM OPP Committee’s involvement (DGs and ADMs were informed) and therefore were not identified in the DM Dashboard.
A similar analysis of the June 2020 DM Dashboard, which was conducted using TC input sheets and partner departments’ summary sheets identified similar findings. Although the Additional Radar Sites and Risk Analysis of Maritime Search Rescue Delivery were updated to ‘Remediation Required’ and ‘Attention Required’ respectively and were in line with CCG’s summary sheets, two other sub-initiatives (Increase Emergency Tow Capacity and Alternative Response Measures) had different results reported in the DM Dashboard compared to the summary sheets.
While preparing the DM Dashboard, the OPP Secretariat engages with all partner departments, to challenge and validate the sub-initiatives statuses provided by partner departments. Based on this discussion, a decision is made on the final status of the sub-initiatives presented in the DM Dashboard. These decisions should be documented in order to demonstrate support for the decisions.
2.2 The Interdepartmental OPP Secretariat employs a flexible approach to monitoring the progress of sub-initiatives.
Context
Guidance on completing input sheets states that sub-initiatives should be assigned “Attention Required”, “Remediation Required” or “Off Track” if milestones are facing financial or Human Resource related issues. Ratings are assigned based on the level of intervention required. {ATIP REMOVED}
Findings
For the 2019-2020 fiscal year, input sheets for the 10 sampled TC sub-initiatives that were part of the audit’s scope were examined (see Annex A for details), and the original and revised target dates were analyzed for 176 milestones (total number of milestones for the 10 TC sub-initiatives). The assessment found:
- 16 milestones’ target dates were delayed between 7 to 12 months
- 8 milestones’ target dates were delayed between 13 to 24 months
- 5 milestones’ target dates were delayed between 25 to 36 months
- 1 milestone’s target date has been delayed by 48 months
- 3 milestones were marked “Attention Required”. All others were marked “On track”.
As milestones are delayed, the risk of sub-initiatives not meeting their deadline is increased. {ATIP REMOVED} OPIs are responsible for their respective deliverables and the OPP Secretariat believes OPIs must have the flexibility to manage their sub-initiatives without undo oversight of specific interim milestones.
2.3 The Interdepartmental OPP Secretariat’s risk-based approach to its oversight function should be documented.
Context
During the first year of the OPP, the Interdepartmental OPP Secretariat was receiving detailed input sheets from all partner organizations. In the second year, DFO/CCG replaced their detailed input sheets with summary sheets that include the overall status of strategic outcomes, overall health of the sub-initiative, and any issues and corresponding mitigating strategies.
Findings
A decision was made by the Interdepartmental OPP Secretariat for partner departments to only provide a summary on the status of the sub-initiatives due to the quantity of detailed information that was recorded in DFO/CCG’s input sheets. This decision was not documented. Internal Audit was informed that each Departmental OPP Secretariat is responsible for reviewing and challenging the information provided by their OPIs to reduce the risk of inaccurate reporting. However, this Departmental OPP Secretariat challenge function should be documented in the OPP Secretariat Roles and Responsibilities document.
DM Strategic Dashboard - Conclusion(s)
As milestones are delayed, the risk of not meeting target dates is increased. As the OPP nears the end of its life, it will be critical for the Interdepartmental OPP Secretariat to closely monitor the sub-initiatives’ progress.
Given there were only three inconsistencies between the results reported in the Dashboard and the input sheets, the risk associated with the level of accuracy and completeness of the Dashboard is low. However, changing the statuses of projects, based on the fact that issues/delays were already communicated to the DM OPP Committee, creates a misrepresentation of the status of the projects since it may be considered that the issues/delays facing the sub-initiatives have been addressed.
Furthermore, in order to create an audit trail related to management of OPP, it is important to document the decisions related to OPP.
3. Fencing guidelines for OPP funds
Context
The 2018 Audit of OPP Governance and Oversight recommended that TC’s Chief Financial Officer, in consultation with ADM, Transformation (responsible for OPP at the time) should clearly document the rules around OPP fencing.
Findings
{ATIP REMOVED} There are no requirements that restrict the use of any surplus funding on other Departmental activities. Initially, at the time that the Ocean Protection Plan program had been implemented, emphasis was placed on the importance to focus the use of the funding allocated to this program. There were no specified rules in place around the use of OPP surplus but it was agreed that OPP surplus should whenever possible go towards bolstering other ocean related initiatives.
Internal Audit was informed that, through a discussion at Resource Management Committee, it was decided that if the ADM, OPP confirmed surplus funds, efforts would be made to use those surpluses to enhance funding to other marine and ocean related initiatives. Corporate Financial Management documents all OPP reallocations by highlighting surpluses and recording where they have been allocated.
{ATIP REMOVED} TC’s Financial Management indicated that each sub-initiative received the amount of funding that had been requested to complete the activities as part of the Department’s annual Integrated Planning process. It is clear that the assumptions used to cost many of the activities were not fully in line, evidenced by the actual costs being below the estimates.
At the end of the fiscal year, six of the 10 sub-initiatives had surpluses greater than 10%, which were explained and documented by Financial Management. {ATIP REMOVED}
Fencing Guidelines for OPP Funds – Conclusion
As there is no requirement to use OPP funds solely for OPP purposes, the Department does have the flexibility to manage the Treasury Board funding. Efforts are made to align this spending with additional ocean related activities and key decisions are clearly documented to demonstrate stewardship of public funds.
For future programs, should it be necessary to place restrictions on the use of program funding it would be beneficial to document the agreed use of a program’s funds in order to reinforce good stewardship practices and help ensure a consistent understanding in case there is a change in the department’s staff and management.
{ATIP REMOVED}
4. Status update of the implementation of the management action plan
The 2018 Audit of OPP Governance and Oversight made 19 recommendations. Out of the 19 recommendations:
- 15 (79%) are complete – recommendations were fully implemented
- 3 (16%) are complete with opportunities for improvements – recommendations were partially implemented
- 1 (5%) recommendation is related to the delay in the deadline for co-developing performance indicators for the participation of Indigenous groups in the marine sector from April 2018 to April 2022, which has been approved by the OPP DM Steering Committee.
Additional details on the implementation of the recommendations can be found in Annex B.
Annex A – Analysis of milestones target date {ATIP removed} (as of September 2019)
| Sub-initiatives (Total number of Milestones) |
Number of target dates delayed between 6 and 12 months (%) |
Number of target dates delayed between 13 and 24 months (%) |
Number of target dates delayed between 25 and 36 months (%) |
Number of target dates delayed between 37 and 48 months (%) |
Incomplete milestones with past target dates (%) |
|---|---|---|---|---|---|
| 1. Enhanced Maritime Situational Awareness (25) | 9 (36%) |
1 (4%) |
0 | 0 | 0 |
| 2. Regional Response Planning (12) | 1 (8%) |
2 (17%) |
1 (8%) |
0 | 0 |
| 3. Technical Review Process of Marine Terminal Systems and Transhipment Sites (TERMPOL) (18) | 0 | 0 | 0 | 0 | 2 (11%) |
| 4. 24/7 Emergency Response Capacity to Effectively Manage Marine Incidents (5) | 0 | 0 | 0 | 0 | 5 (100%) |
| 5. Increase On-Scene Environmental Response Capacity (10) | 0 | 0 | 0 | 0 | 0 |
| 6. Cumulative Effects of Marine Shipping (36) | 1 (3%) |
2 (6%) |
4 (11%) |
0 | 0 |
| 7. Vessels of Concern New Legislation (14) | 2 (14%) |
1 (7%) |
0 | 0 | 1 (7%) |
| 8. Establishing Marine Environmental Regulatory and Non-Regulatory Measures (41) | 2 (5%) |
1 (2%) |
0 | 0 | 0 |
| 9. Building Meaningful Relationships with Indigenous Groups in Marine Safety (10) | 1 (10%) |
0 | 0 | 1 (10%) |
3 (30%) |
| 10. Increase Emergency Tow Capacity (5) | 0 | 1 (20%) |
0 | 0 | 1 (20%) |
Caption: The table provides information on the types of delays faced by the 10 sampled sub-initiatives.
Annex B – Status update of MRAP
| Audit recommendation | Management action plan | Internal audit’s status update | |
|---|---|---|---|
| Governance | |||
| a) | Collaborating with partner departments to clarify the responsibilities of lead department and partner departments. | Will make adjustments to Governance Committee Terms of Reference to clarify the responsibilities of lead and partner departments. These adjustments will align with the TBS Guide to Departments on the Management and Reporting of Horizontal Initiatives as well as the Roles and Responsibilities outlined and committed to in the OPP funding document. | The Terms of Reference for the OPP DM Executive Steering Committee has been updated to reflect TC’s lead role in providing whole-of-government executive leadership and direction on the implementation of OPP. TC will also lead the horizontal reporting related to OPP. COMPLETE |
| b) | Defining the roles of the committees including decision-making authority. | Will update OPP Governance Committee ToRs to indicate decision making authority of each of the committees. | Decision making has been clarified in the DM Executive Steering Committee and ADM/DG Steering Committee Terms of Reference. The ADM/DG Steering Committee is responsible for decision making based on the principle of consensus-building. If no consensus is achieved on a given item, the committee will bring forward options to the DM Committee as required. COMPLETE |
| c) | Defining issues escalation / disagreement resolution processes | The Interdepartmental DM OPP Committee is the final body should disputes at other levels not be decided. Will update OPP Governance Committee ToRs to indicate process for issues escalation and disagreement resolution. | The DM Executive Steering Committee Terms of Reference was updated to reflect that it is responsible for resolving disputes or any other issues that may rise among horizontal initiatives partners that cannot be resolved at the ADM/DG levels of bilaterally between partners departments. COMPLETE |
| d) | Clarifying the roles and responsibilities of the OPP Secretariats. | Will update Secretariat Roles and Responsibilities document to better clarify roles and responsibilities in line with audit expectations and ensure they are agreed to and approved. | The Terms of Reference of the Interdepartmental OPP Secretariat has been updated to state that it will follow-up on items as appropriate. The TC OPP Secretariat mentioned that if anomalies affecting the outcomes of the sub-initiatives are identified, they will follow-up with the managers of the sub-initiatives. The TC OPP Secretariat also mentioned that each department’s OPP secretariat is responsible for playing a challenge role within their own department. This individual challenge function played by each department should be clarified in the OPP Secretariats Roles and Responsibilities in order to identify the levels of control established to accurately report on sub-initiatives. COMPLETE WITH OPPORTUNITIES FOR IMPROVEMENT |
| e) | Improving the communication of guidance documents and processes. | Will issue a reminder by email and at governance meetings of guidance documents that are available and how they can be accessed. | Project Tracking Sheet Guidance Document, OPP Process for the Management of Joint Initiatives and the Change Request form were shared at the ADM/DG Steering Committee and also shared with OPP managers. OPP Managers also informed that they are receiving the guidance they need for the OPP Secretariat. COMPLETE |
| f) | Regularly monitoring and reporting on the status of committee decisions/ action items. | Will create action item tracker for each of the governance committees and include as a standing item on each committee agenda. | The Interdepartmental OPP Secretariat has created an OPP Committee Action Item Tracker. The Tracker identifies the action items mentioned at each committee meeting. However, it does not identify whether the action items have been addressed. COMPLETE |
| Information for Decision Making | |||
| a) | Establish and implement the data strategy to support performance indicators. | Data strategy is being implemented through the departmental Performance Information Profiles (PIP) process – data sources can be added to the dashboard input sheets where applicable. | A PIP has been created for OPP. The PIP identifies long, medium, and short-terms outcomes, for which indicators, data sources and targets have been identified. COMPLETE |
| b) | Continue to improve the reporting of OPP performance results based on the earlier audit recommendation. | Results reporting has been implemented with the updated dashboard process and will be presented to DMs in October. OPP Secretariats have also implemented a joint review process where dashboards including emerging issues and health rating of each sub-initiative are reviewed and agreed upon before reporting to governance committees (this includes joint review at the working level, Director, DG and ADM levels). | Please refer to Section 2 of the report. COMPLETE WITH OPPORTUNITIES FOR IMPROVEMENT |
| c) | Collaborate with partner departments to develop a process to track, document and report the variances from the approved funding, e.g. 1) reallocation 2) re-profile or 3) carry-forward, or 4) lapsed. | Recommendations with respect to tracking financial information, as outlined in the TBS Guide to Departments on the Management and Reporting of Horizontal Initiatives, will be adopted to ensure effective reporting on the overall performance of the horizontal initiative. The robust tracking of horizontal issue financial information will include reconciliations to the approved funding and identify the costing of internal services, employee benefit plans and accommodation costs, as well as the indication of any use of unspent funds as a reallocation, re-profile, carry forward or lapse. | Documentation provided by the Corporate Finance indicates the surpluses and the reasons for the surpluses in 2017-2018 and 2018-2019. It also provides a financial strategy, which describes how funds were reallocated to other areas within marine to bolster oceans related initiatives. For 2017-2018 and 2018-2019, the Interdepartmental OPP Secretariat maintained a database where variances between the annual budgets and year-end actuals for TC lead/supported sub-initiatives were identified and explained. COMPLETE |
| Engaging Stakeholders and Indigenous Partners | |||
| a) | Develop outcomes and indicators for measuring the success of engaging all stakeholders, including Indigenous partners. | A collaborative process is being developed to allow for co-development of indicators, which would measure the success of relationship building between the GoC and Indigenous communities. Preliminary approach could be finalized by November 2018, with finalization and implementation of approach in the remaining Fiscal Years (FY) of the OPP. An engagement strategy is also being developed for marine stakeholders. |
The deadline for co-developing target for participation of Indigenous groups in the marine sector has been delayed from April 2018 to April 2022. Community Engagement and Indigenous Partnerships, who is responsible for engagement, believe it will allow for:
The decision to push back the development of the outcomes and indicators was approved by the OPP DM Steering Committee. |
| b) | Clearly define the roles and responsibilities of national hubs, regions, and sub-initiatives. | Finalize the National Integration Model document which will flesh out the roles and responsibilities of hubs, partner departments and sub-initiatives leads. | The roles and responsibilities of national hubs, regions and sub-initiatives have been defined in the National Integration Model, which was shared with OPP ADM/DG Steering Committee. COMPLETE |
| c) | Ensure that all sub-initiatives have submitted their engagement plans, or rationale for why one is not required. | A national call-out will occur, reminding initiative leads to submit their engagement plans. Only 13 initiatives have submitted plans to date. A tracker will be created and follow-ups will be undertaken monthly until fully completed. |
The National OPP Initiatives Engagement Planning Matrix only tracks 28 sub-initiatives:
COMPLETE |
| d) | Continue to analyze and monitor the resource needs to ensure sufficient resources for carrying out the interdepartmental OPP engagement function. | Work with Regional Hubs to identify resource requirements for engagement for the remaining years of OPP, with the goal to have plans fleshed out for the Integrated Planning and Reporting process for FY 19-20. | The need for additional resources to carry out the interdepartmental OPP engagement function and its associated risks have been identified in the 2019-2020 Integrated Planning and Reporting. COMPLETE |
| Project Management | |||
| a) | Monitor and report the compliance with the project charter guidance. A rationale justifying the reasons for any exceptions to the guidance should be documented and reported. | Will develop an overall list of sub-initiatives indicating which sub-initiatives have charters in place and which sub-initiatives are relying on alternative project management tools. Will confirm the parameters of this exercise with TC Audit and Evaluation. | The Interdepartmental OPP Secretariat is monitoring the compliance with project charter guidance on the 32 sub-initiatives for which TC is lead or participant. Seven initiatives were identified as not requiring a charter, all except one provided a justification for not having a charter. COMPLETE |
| b) | Provide clear guidance on when and how to use the change control form, and monitor and report its use. | Will adjust guidance currently imbedded in change request form to better align with audit expectations. Change request form will be re-issued in line with recommendation e) under Governance. | The Interdepartmental OPP Secretariat has developed guidance on how to use the Initiative/Sub-initiative Change Submission Form. The guidance was shared with the OPP ADM/DG Steering Committee. Interviews with OPIs indicated that the guidance on the Form was communicated. All changes for which an Initiative/Sub-Initiative Change Submission Form is submitted, is presented to the OPP ADM/DG Steering Committee and the OPP DM Steering Committee, when appropriate. COMPLETE |
| c) | Ensure actions are taken to improve records management. | A new RDIMS folder structure has been created for OPP Secretariat. All Secretariat files have been moved over to the new structure and it is being used by Secretariat Staff on a go forward basis. Protected files have been removed from the shared drive. OPP Secretariat has also engaged IM/IT to advance IM practices and collaborative tools across OPP. | A structured folder has been created for the Interdepartmental OPP Secretariat. The folder is being updated as documents are being developed and gathered. {ATIP REMOVED} According Transport Canada Information Management Policy, all electronic Secret documents are required to be stored in a removable media and locked in a security-approved container. COMPLETE WITH OPPORTUNITIES FOR IMPROVEMENT |
| Corporate Support | |||
| a) | The procurement plan is better aligned with the IPR process and monitored and revised periodically to reflect changes throughout the year. | Procurement plan will be moved to align with the Integrated Planning and Reporting Process and be tracked on a monthly basis by the Program Business Committee. | The 2019-2020 OPP Procurement Plan was presented to the departmental Senior Procurement Review Committee in March 2020. Plan described the types of contracts, which organizations utilized the contracts, dollar amounts and the level of risk associated with the contracts. The 2020-2021 OPP Procurement Plan, which was developed alongside the OPP IPR/IBD to ensure alignment, was also presented to the Committee. COMPLETE |
| b) | The Financial Reporting Guidelines provide sufficient detail describing the interdepartmental reporting process. | Will work with Audit to make improvements were required to guidance information that has already been developed. | The TC OPP Activity, Financial and HR Reporting Procedures provide a description of the Financial Reporting Process. It describes how the information should be extracted, analyzed and reported. According to the guidelines, variances between annual budgets and year-end expenses for each sub-initiative should be explained using the template provided in the guideline. Explanations on variances were provided for 2017-2018, 2018-2019 and 2019-2020. COMPLETE |
| c) | The CFO, in consultation with ADM, Transformation, should clearly document the rules around OPP fencing. | There is no requirement to fence funding for OPP. As a result, it is at the discretion of each department how OPP funding is used and allocated. At Transport Canada a decision was made at Resource Management Committee (RMC) that any reallocation of OPP surpluses must come back to RMC for approval. Corporate Finance and the OPP team will develop and bring to RMC a strategy on how any lapsed funding could be returned to OPP, should the lapsed funding be available. | Please refer to Section 3 of the report. COMPLETE |
Caption: The table provides information on the original audit’s recommendations and management action plan, as well as an Internal Audit status update on the progress of the implementation of the recommendations.