Internal audit report outlining results of the assessment of the Oceans Protection Plan Governance and Oversight at Transport Canada as well as management’s action plan in response to recommendations identified in the audit.
On this page
- Background
- Audit objective, scope and statement of conformance
- Audit criteria
- Overall conclusions
- Observations
- Recommendations and management action plan
Background
Oceans Protection Plan
The Oceans Protection Plan (OPP) is a horizontal initiative that involves four federal government departments, with $1.5B in funding over five years. It has a mandate to improve marine safety, protect the marine environment, and offer new opportunities for Indigenous and coastal communities. It is; therefore, at the nexus of sensitive and high priority issues, such as the Trans Mountain Pipeline, Government of Canada’s (GoC) reconciliation agenda, and concerns related to marine safety and marine mammals. As well, the success of OPP hinges on the collaboration of multiple departments, and meaningful consultation with a variety of external stakeholders, including Indigenous partners, marine industries, provinces and territories.
As the lead department for OPP, Transport Canada (TC) works with its partner departments to address sensitive issues and has been establishing processes to support collaboration and consultation with stakeholders.
OPP is one of the first horizontal initiatives with this level of complexity, where TC is the lead department; thus, the department does not have a lot of experience to draw on.
Given OPP’s unique nature and its importance, we began this audit in year one in order to identify good practices and lessons learned at this point in time and to allow opportunities for early adjustments.
OPP interdepartmental governance structure
OPP interdepartmental governance structure - Text version
Three types of committees make up the OPP’s governance structure:
- Interdepartmental governance
- Joint Sub-initiatives
- Policy sub-committee
There are three Interdepartmental Governance Committees:
- The first is the OPP Interdepartmental DM (Deputy Minister) Committee
- The second is the OPP Interdepartmental ADM (Assistant Deputy Minister)/DG Committee,
- The third is the OPP Interdepartmental DG (Director General) Committee
The OPP Interdepartmental Policy Sub-Committee supports the OPP Interdepartmental DG Committee.
- The Director of Transport Canada’s OPP Policy chairs this sub-committee
- As required, the OPP Interdepartmental Policy Sub-Committee may form:
- OPP policy working groups (Chair: TBD)
- OPP legislative/regulatory working groups (Chair: TBD)
The OPP Interdepartmental DG Committee and the OPP Interdepartmental Policy Sub-Committee can form a joint initiative steering committee at the Director and DG level.
- a Director or DG from the lead department would chair the steering committee
- the joint initiative steering committee can form a joint initiative working group, chaired by a representative of the lead department
Audit approach
In 2017, we worked with internal audit of OPP partner departments to develop a multi-year OPP audit plan, which provides assurance throughout the life cycle of OPP, starting at year one.
After completing the OPP program level Risk Assessment and Sub-initiative Risk Ranking in fall 2017, TC’s audit and evaluation teams begun conducting this audit. Using a “project under development” approach, the teams shared results with programs on a “real-time” basis, to allow opportunities for early adjustment. Observations have led to early management action plans throughout the audit.
Audit objective, scope and statement of conformance
Objective:
Audit of OPP governance and oversight: To assess the design and effectiveness of the governance structure and related processes put in place to oversee and manage the implementation of the OPP.
OPP strategic dashboard validation: To examine and assess the information in the OPP strategic dashboards to determine if it is relevant, timely, accurate and complete.
Scope:
An assessment of the governance structure, processes and controls in place to provide oversight on the OPP, both at the interdepartmental level and within TC, for the period from April 2017 to August 2018. Specifically, the assessment includes new structures and processes that TC put in place for the OPP (committee structure, engagement, and secretariat) as well as how the existing corporate functions (communications, finance, procurement, and human resources) enable and support the plan.
Statement of conformance
This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of an external assessment of our quality assurance and improvement program.
Audit criteria
We used the following five criteria based on the principles in the Project Management Body of Knowledge (PMBOK), along with the Secretariat Horizontal Initiatives Guide for the audit and the validation exercise.
| Criteria | Description |
|---|---|
| Governance | There is an effective governance framework to facilitate and oversee the delivery of results. |
| Information for decision making | Information in the reporting dashboards to support management decision-making is relevant, accurate, complete, and timely. |
| Engaging stakeholders and Indigenous partners | Effective communication takes place to engage stakeholders/ partners and to manage their expectations. |
| Project management | Project management processes are in place to monitor the cost, schedule and scope of the OPP and its sub-initiatives. |
| Corporate support for OPP | Program support functions (communications, finance, procurement, and human resources) are effective in enabling the success of the OPP. |
Overall conclusions
As OPP is a high profile, priority government program that spans five years, it is important to ensure early on that the appropriate governance and oversight processes are in place and working as intended. Using a “project under development” approach since the start of the OPP implementation we have been providing management “real-time” observations and making recommendations to help them to continue to strengthen governance and oversight. We have seen steady progress establishing the foundation for effective oversight of this multi-department horizontal initiative.
Summarized below are elements of governance and oversight, at this stage of OPP’s implementation, requiring further strengthening.
- since year one, as the lead department, TC has established an interdepartmental governance committee structure and dashboard reporting process to govern and oversee OPP implementation. To further strengthen governance and oversight, it is necessary to clarify decision-making authority and the roles and responsibilities of lead and partner departments, including each department’s OPP secretariat.
- strengthening the process to track OPP budgets and spending against approved funding is needed to ensure a complete accounting of budget and spending decisions over the life of the program.
- TC and its partner departments clearly recognize the importance of engagement especially with Indigenous groups and have implemented an interdepartmental process to collaborate internally with partner departments and to consult externally with OPP stakeholders and Indigenous partners. To further strengthen the processes put in place, some improvements are needed related to engagement outcomes and indicators, roles and responsibilities of hubs and regions, engagement plans, and resourcing of the engagement team.
- though the interdepartmental OPP secretariat established templates and guidance to support effective project management which is especially vital for a high profile, complex horizontal government initiative such as OPP, some gaps and weaknesses need to be addressed related to project charters, tracking of project changes, and records management.
- specific to TC, OPP sub-initiative managers faced procurement and financial forecasting challenges in year one. Measures are being taken by TC to address these department-wide challenges. In addition, how surplus OPP funds are to be managed, that is “fenced or not fenced”, should be further clarified and clearly communicated since it is important to ensure all players understand the ground rules and senior management’s expectations. This will also facilitate reporting a coherent and comprehensive performance story at the end of OPP implementation.
The following provides a high level summary of the key observations in each of the five criteria listed above and includes progress and achievements, lessons learned or areas for improvement, and recommendations.
Observations
Observations: governance
| Progress and achievements | Lessons learned or areas for improvement |
|---|---|
|
Interdepartmental governance committee structure has been put in place; Terms of Reference (TOR) have been created for majority of the interdepartmental committees (the TOR for the Director General Committee is under development); regular meetings are taking place. Interdepartmental OPP secretariat and engagement team have created guidance documents for sub-initiatives to promote consistency across partner departments. |
Internal audit has not found a document that clearly outlines the roles and responsibilities of the lead department and partner departments. It is not clear what the expectation is for TC as the lead department to monitor and challenge other department’s OPP performance and practices. The following areas could be better defined and documented in order to reduce confusion:
Not all the OPP sub-initiative managers are aware of the guidance documents and processes provided by the interdepartmental OPP secretariat such as the change control form, which signals a need for further communication and education. The administrative process to support committees could be improved. We could not verify who attended meetings as the records of decision do not record the names and ranks of participants, and there was no formal tracking of the status of action items raised at committees to demonstrate whether they have been addressed. |
Recommendation:
The Executive Head, Oceans Protection Plan should strengthen OPP governance by:
- collaborating with partner departments to clarify the responsibilities of the lead department and partner departments
- defining the roles of the committees including decision-making authority
- defining issues escalation / disagreement resolution processes
- clarifying the roles and responsibilities of the OPP secretariats
- improving the communication of guidance documents and processes
- regularly monitoring and reporting on the status of committee decisions/ action items
Observations: Information for decision making
| Progress and achievements | Lessons learned or areas for improvement |
|---|---|
|
Interdepartmental OPP secretariat, in collaboration with partner departments, established a process in year one to collect and report financial, human resources and performance information of all sub-initiatives by using input sheets and strategic dashboards. Based on an early audit recommendation, the interdepartmental OPP secretariat has held workshops and interdepartmental discussions to improve reporting of OPP achievements and performance results, and is currently transitioning to new dashboards. The new design will capture outcomes, indicators, targets and annual outputs from the Submission. |
The reporting of information to reflect the health and milestones of the sub-initiatives needs improvement: Milestones: Some sub-initiatives’ input sheets show sub-initiatives’ milestones as being “on track” while completion dates for these milestones have been postponed for longer than six months (23 of 134, or 17%, sampled sub-initiative milestones between October 2017 and May 2018). Sub-initiative Health: Each department is reporting their health separately for joint sub-initiatives on their respective input sheet, and it is not clear how the health status is consolidated for the strategic dashboard. Overall, there is no clear documentation to describe the rationale for changes from input sheet to strategic dashboard. The sources and the strategy to collect data needed to support performance indicators are unclear. This raises the risk that appropriate data may not be collected. The financial budgets reported in the strategic dashboard showed fluctuations throughout fiscal year 2017 to 2018, and the reported budget throughout the year did not reconcile with the approved funding until the year end. As well, the fluctuations and the rationale were not highlighted in the reporting to the interdepartmental deputy minister committee. As the lead department, TC has not developed a process to track, document and report variances. |
Recommendation:
The Executive Head, Oceans Protection Plan should:
- establish and implement the data strategy to support performance indicators
- continue to improve the reporting of OPP performance results based on an earlier audit recommendation
- collaborate with partner departments to develop a process to track, document and report the variances such as reallocation, re-profile, carry-forward or lapsed
Observations: Engaging stakeholders and Indigenous partners
| Progress and achievements | Lessons learned or areas for improvement |
|---|---|
|
In year one of the implementation, TC in collaboration with partner departments established an interdepartmental structure that consists of a national hub and five regional hubs to coordinate OPP engagement activities. The interdepartmental engagement team established many tools and processes to facilitate collaboration and consultation such as an OPP engagement calendar and engagement sessions. The interdepartmental OPP engagement team recognized the need to help with capacity building for Indigenous partners, and have started to introduce processes including reconciliation framework agreements and contribution agreements that will allow more flexibility in providing funding via grants and contributions for this purpose. |
Outcome and performance indicators for OPP engagement: The Program has a requirement to co-develop performance indicators with Indigenous partners for measuring the success of engaging with Indigenous groups. This process is complex as it involves seeking agreements from multiple internal and external stakeholders; thus, the original deadline of April 2018 has been postponed, and a new deadline has not been identified. The interdepartmental engagement team is leading an interdepartmental process to consult with Indigenous communities in order to co-develop the required performance indicators. In addition, the outcome and performance indicators for engaging stakeholders other than Indigenous partners like the marine industry, for example, have not been defined. Engagement Plans: Many sub-initiatives have not submitted their engagement plans. In May 2018, we found that eight of 14 (57%) sampled sub-initiatives did not provide an engagement plan. An engagement plan should contain key details such as stakeholders, and short- and long-term engagement needs. Roles and Responsibilities: Some sub-initiative managers do not clearly understand the roles and responsibilities of national engagement hubs, regions and sub-initiatives such as booking facilities and contacting participants for engagement sessions as well as collecting and responding to feedback. Resource Planning: The resource requirements for having an engagement team capable of coordinating all interdepartmental OPP engagement activities was not completely envisioned during the initial stages of the Program. Based on interviews, the resources that are dedicated to engagement have significantly increased due to the complexity and scope of activities. |
Recommendation:
The Executive Head, Oceans Protection Plan should:
- develop outcomes and indicators for measuring the success of engaging all stakeholders including Indigenous partners
- clearly define the roles and responsibilities of Office of Primary Interests (OPIs) national hubs, regions, and sub-initiative
- ensure that all sub-initiative OPIs have submitted their engagement plans or rationale for why one is not required
- continue to analyze and monitor the resource needs to ensure sufficient resources for carrying out the interdepartmental OPP engagement function
Observations: Project management
| Progress and achievements | Lessons learned or areas for improvement |
|---|---|
|
TC, as the lead department, has set out requirements for all sub-initiatives to prepare a project charter and implementation plan (later referred to as project charter) as a baseline to monitor performance. In December 2017, TC also established a process, using a change control form, to track and approve sub-initiative changes. |
Project Charters: Some sub-initiative managers do not see the value in having a project charter, which is a key project management control recognized by PMBOK. As of August 2018, we found 28 out of 58 (48%) sub-initiatives have not submitted a project charter or a consolidated charter for joint sub-initiatives. Ten of these are for TC led sub-initiatives. There is also one additional project charter from TC that is considered incomplete because it is missing key information. Out of the 58 sub-initiatives, 18 are conducted jointly between two or more departments. In the OPP process for the management of joint initiatives, it is expected that the lead department prepare a high level joint project charter. Out of the 18 joint sub-initiatives, 11 sub-initiatives (61%) do not have a joint project charter. Two of these are TC- led sub-initiatives. In the absence of a project charter, managers rely on a range of informal tools and processes including their own knowledge and experience, work plans, and other documents that govern the Program. Change Control Form: Managers expressed that it would be beneficial to have more detailed guidance and better communication on the criteria on when the form is required, for example, budge threshold, number of days delayed. Records Management: We have previously recommended to the interdepartmental OPP secretariat that records management needs improvement. The inconsistent use of TC’s Records Document Information Management System (RDIMS) limits document version control and the safeguarding of secret documents. Information management is especially important for a high profile program such as OPP. Management’s actions to address this risk have yet to be implemented. |
Recommendation:
The Executive Head, Oceans Protection Plan should:
- monitor and report the compliance with the project charter guidance. A rationale justifying the reasons for any exceptions to the guidance should be documented and reported
- provide clear guidance on when and how to use the change control form, and monitor and report its use
- ensure actions are taken to improve records management
Observations: TC corporate support
| Progress and achievements | Lessons learned or areas for improvement |
|---|---|
|
TC is currently focused on training managers department wide in order to improve budgeting and forecasting practices in the department. This could have a positive impact on the OPP budgeting and forecasting practices. TC human resources and communications have established plans and processes in their respective areas to support OPP activities. |
Procurement: TC OPP procurement planning: The procurement plan for fiscal year 2017 to 2018 was overestimated ($21.1M) when compared to the total Other Operating Cost (OOC) {ATIP REMOVED} in the same year. Procurement planning was not fully aligned with the department’s integrated planning process. Procurement challenges department wide: At the September 5, 2018 senior TMX meeting, the Chief Financial Officer (CFO) presented a plan to address systemic issues with the procurement function in TC. The procurement issues that TC OPP managers raised were consistent with the ones identified by the CFO. The difficulty recruiting and retaining procurement officers is a challenge across all government departments and likely one of the root causes of the procurement process issues. Given that the department has a plan in place to improve the procurement process we are not making any specific recommendations. As part of our future monitoring of OPP implementation we will assess the extent to which OPP procurement needs are being met. Financial Management: OPP financial reporting guidelines: Financial reporting procedures do not contain sufficient detail to support knowledge transfer in the event of staff turnover. This is especially important given OPP’s complexity and high profile. Fencing of OPP funds: Need greater clarity and communication of how OPP funds are to be fenced. Budget and forecast accuracy in TC: the accuracy of OPP’s year one financial budgets and forecasts needs improvement. For a sample of 10 OPP sub-initiatives we found the following:
|
Recommendation:
The Executive Head, Oceans Protection Plan should ensure that:
- the procurement plan is better aligned with the integrated planning and reporting process and monitored and revised periodically to reflect changes throughout the year
- the financial reporting guidelines provide sufficient detail describing the interdepartmental reporting process
The CFO, in consultation with the Executive Head, Oceans Protection Plan, should clearly document the rules around OPP fencing.
Recommendations and management action plan
Governance
| Recommendation | Management Action Plan | Timelines |
|---|---|---|
| Collaborate with partner departments to clarify the responsibilities of lead department and partner departments. | Will make adjustments to the governance committee terms of reference to clarify the responsibilities of lead and partner departments. These adjustments will align with the Secretariat guide to departments on the management and reporting of horizontal initiatives as well as the roles and responsibilities outlined and committed to in the OPP funding document. | Updated terms of reference will be presented to committees for their approval at their next scheduled meeting – December 2018. |
| Define the roles of the committees including decision-making authority. | Will update OPP Governance Committee terms of reference to indicate decision-making authority of each of the committees. | Updated terms of reference will be presented to committees for their approval at their next scheduled meeting – December 2018. |
| Define issues escalation / disagreement resolution processes | The Interdepartmental Deputy Minister OPP Committee is the final body should disputes at other levels not be decided. Will update OPP Governance Committee terms of reference to indicate process for issues escalation and disagreement resolution. | Updated terms of reference will be presented to committees for their approval at their next scheduled meeting – December 2018. |
| Clarify the roles and responsibilities of the OPP secretariats. | Will update secretariat roles and responsibilities document to better clarify roles and responsibilities in line with audit expectations and ensure they are agreed to and approved. | November 2018 |
| Improve the communication of guidance documents and processes. | Will issue a reminder by email and at governance meetings of guidance documents that are available and how they can be accessed. | Will be done to align with presentation of audit findings at OPP governance committee meetings – December 2018. |
| Regularly monitor and report on the status of committee decisions/ action items. | Will create action item tracker for each of the governance committees and include as a standing item on each committee agenda. | Will be included as part of the next round of governance committee meetings – December 2018. |
Information for decision making
| Recommendation | Management Action Plan | Timelines |
|---|---|---|
| Establish and implement the data strategy to support performance indicators. | Data strategy is being implemented through the departmental Performance Information Profiles (PIP) process. Data sources can be added to the dashboard input sheets where applicable. | November 2018 |
| Continue to improve the reporting of OPP performance results based on the earlier audit recommendation. | Results reporting has been implemented with the updated dashboard process and will be presented to deputy ministers in October. OPP secretariats have also implemented a joint review process where dashboards including emerging issues and health rating of each sub-initiative are reviewed and agreed upon before reporting to governance committees (this includes joint review at the working level, Director, Director General and Assistant Deputy Minister levels) | Completed |
| Collaborate with partner departments to develop a process to track, document and report the variances from the approved funding such as reallocation, re-profile, carry-forward or lapsed. | Recommendations with respect to tracking financial information, as outlined in the Secretariat guide to departments on the management and reporting of horizontal initiatives, will be adopted to ensure effective reporting on the overall performance of the horizontal initiative. The robust tracking of horizontal issue financial information will include reconciliations to the approved funding and identify the costing of internal services, employee benefit plans and accommodation costs as well as the indication of any use of unspent funds as a reallocation, re-profile, carry forward or lapse. | November 2018 |
Engaging stakeholders and Indigenous partners
| Recommendation | Management Action Plan | Timelines |
|---|---|---|
| Develop outcomes and indicators for measuring the success of engaging all stakeholders including Indigenous partners. | A collaborative process is being developed to allow for co-development of indicators, which would measure the success of relationship building between the Government of Canada and Indigenous communities. Preliminary approach could be finalized by November 2018 with finalization and implementation of approach in the remaining fiscal years of the OPP. An engagement strategy is also being developed for marine stakeholders. | Work is underway to finalize a stakeholder strategy - November 2018. |
| Clearly define the roles and responsibilities of national hubs, regions, and sub-initiatives. | Finalize the national integration model document which will flesh out the roles and responsibilities of hubs, partner departments and sub-initiatives leads. | November 2018 |
| Ensure that all sub-initiatives have submitted their engagement plans or rationale for why one is not required. | A national call-out will occur, reminding initiative leads to submit their engagement plans. Only 13 initiatives have submitted plans to date. A tracker will be created and follow ups will be undertaken monthly until fully completed. | November 2018 |
| Continue to analyze and monitor the resource needs to ensure sufficient resources for carrying out the interdepartmental OPP engagement function. | Work with regional hubs to identify resource requirements for engagement for the remaining years of OPP, with the goal to have plans fleshed out for the integrated planning and reporting process for fiscal year 2019 to 2020. | December 2018 |
Project management
| Recommendation | Management Action Plan | Timelines |
|---|---|---|
| Monitor and report the compliance with the project charter guidance. A rationale justifying the reasons for any exceptions to the guidance should be documented and reported. | Will develop an overall list of sub-initiatives indicating which sub-initiatives have charters in place and which sub-initiatives are relying on alternative project management tools. Will confirm the parameters of this exercise with TC audit and evaluation. | November 2018 |
| Provide clear guidance on when and how to use the change control form, and monitor and report its use. | Will adjust guidance currently imbedded in change request form to better align with audit expectations. Change request form will be re-issued in line with recommendation under governance. | Will be done to align with presentation of audit findings at OPP governance committee meetings – December 2018 |
| Ensure actions are taken to improve records management. | A new RDIMS folder structure has been created for OPP secretariat. All secretariat files have been moved over to the new structure and it is being used by secretariat staff on a go forward basis. Protected files have been removed from the shared drive. OPP secretariat has also engaged information management/information technology to advance information management practices and collaborative tools across OPP. | Completed |
Corporate Support
| Recommendation | Management Action Plan | Timelines |
|---|---|---|
| Better align the procurement plan with the integrated planning and reporting process and monitor and revise it periodically to reflect changes throughout the year. | Procurement plan will be moved to align with the integrated planning and reporting process and be tracked on a monthly basis by the program business committee. | Will align with integrated planning and reporting process which begins in November 2018. |
| Ensure the financial reporting guidelines provide sufficient detail describing the interdepartmental reporting process. | Will work with audit to make improvements where required to guidance information that has already been developed. | November 2018 |
| The Chief Financial Officer in consultation with the Assistant Deputy Minister, Transformation, should clearly document the rules around OPP fencing. | There is no requirement to fence funding for OPP. As a result, it is at the discretion of each department how OPP funding is used and allocated. At Transport Canada a decision was made at Resource Management Committee (RMC) that any reallocation of OPP surpluses must come back to RMC for approval. Corporate finance and the OPP team will develop and bring to RMC a strategy on how any lapsed funding could be returned to OPP, should the lapsed funding be available. | To be determined |