Audit of Integrated Planning, Budgeting and Forecasting

• EXECUTIVE SUMMARY
  • Introduction
  • Audit Objectives and Scope
  • Conclusions
  • Statement of Conformance
• 1. INTRODUCTION
  • 1.1. Context
  • 1.2. Background
  • 1.3. Audit Objective, Approach, Scope and Criteria
    • 1.3.1. Audit Objective
    • 1.3.2. Audit Approach, Scope and Criteria
  • 1.4. Report Structure
• 2. FINDINGS
  • 2.1. Governance
    • 2.1.1. Tone at the Top
    • 2.1.2. Priority Setting
  • 2.2. Building Management Capacity
    • 2.2.1. Management Capacity for IPBF
    • 2.2.2. Guidance, Tools and Training
    • 2.2.3. Support from Planners/Analysts, HR Advisors, and FMAs
  • 2.3. Monitoring and Reporting
    • 2.3.1. Performance Measurement
    • 2.3.2. Monitoring and Reporting
    • 2.3.3. Review and Challenge Functions
• 3. CONCLUSIONS
• 4. RECOMMENDATIONS AND MANAGEMENT ACTION PLAN

EXECUTIVE SUMMARY

INTRODUCTION

Transport Canada introduced the Integrated Planning and Reporting (IPR) process in fiscal year 2011-12 to: better align the activities of the Department with its mandate; improve departmental priority setting, decision-making, and resource allocation and utilization; reduce duplication of effort in planning and reporting; and clearly communicate the Department’s priorities and direction to employees.

For the last five years, Transport Canada (TC) has been in a surplus situation at the end of each fiscal year, raising concerns that forecasts are not being adjusted in a timely manner to reflect actual business needs and to allow the reallocation of surplus funds to more pressing needs, priorities, or initiatives that support strategic outcomes.

This Audit addresses the Deputy Minister’s interest in improving planning, budgeting and forecasting as one of the top priorities for the Department in fiscal year 2014-15.

AUDIT OBJECTIVES AND SCOPE

The Audit of Integrated Planning, Budgeting, and Forecasting (IPBF) was included in Transport Canada’s (TC) 2014-15 – 2016-17 Risk-Based Audit Plan (RBAP). Its inclusion stems from the annual risk based planning process that identifies higher risk areas to target internal audit attention and resources. The objective of this audit was to assess whether: the governance over the integrated planning, budgeting, and forecasting processes was effective in supporting the achievement of departmental strategic outcomes and priorities: and integrated planning, budgeting, and forecasting processes, including corporate oversight, monitoring, and reporting, are effective in supporting sound decision-making.

CONCLUSIONS

TC ’s Executive Management Committee (TMX) has provided a consistent “Tone at the Top” in support of improved planning, budgeting and forecasting. TMX has been directly involved in approval of the IPR process, in annual priority setting, and in monitoring quarterly results.

Recent changes to the IPBF processes designed to increase the rigor and reinforce managers’ accountability for planning and budgeting have been supported with efforts to increase management’s capacity in these areas. However, there is still a need for targeted training and consultation with managers to improve tools for monthly monitoring and the use of available information on workforce and demographics analysis.

With quarterly dashboards, more structure and rigor has been introduced to monitoring delivery of program and staffing plans and relating this progress to financial forecasts. Still there are opportunities to improve program monitoring by developing performance measures that consider both the quality and the quantity of activities carried-out, and by providing more specific direction on how to best demonstrate the links between program delivery, budget utilization and mitigation steps being taken to reduce the risk of budget surpluses/deficits. We also noted that Executive Performance Agreements could more fully support the IPBF processes by clearly defining how executives would be evaluated on delivery of their plans and managing their budgets. Finally, the Department would benefit by improving the efficiency and effectiveness of the challenge function by the Strategic Outcome Management Boards to ensure plans, budgets and forecasts have been sufficiently vetted prior to presentation to TMX .

Overall, we found that progress is being made to improve the integrated planning, budgeting and forecasting processes.

STATEMENT OF CONFORMANCE

This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of an external assessment of Internal Audit’s quality assurance and improvement program.

Signatures

INTRODUCTION

1.1. CONTEXT

Canadians expect the federal government to manage public funds well by conducting realistic planning and making appropriate resource allocation and utilization decisions based on sound analysis of reliable and timely information. Effective Integrated Planning, Budgeting, and Forecasting (IPBF) activities should be supported by structured processes with clearly defined roles and responsibilities for all staff involved, consistent timelines and guidance, appropriate training of program staff and enabling functions^Footnote1, relevant performance measures for identifying progress on planned activities, periodic monitoring, timely and reliable reporting, and mechanisms for timely corrective actions.

For the last five years, Transport Canada (TC) has been in a surplus situation at the end of each fiscal year, raising concerns that forecasts might not be adjusted in a timely manner to reflect actual business needs and to allow a timely reallocation of surplus funds to more pressing needs, priorities, or initiatives that support strategic outcomes.

The Audit of Integrated Planning, Budgeting, and Forecasting was included in Transport Canada’s (TC) 2014-15 – 2016-17 Risk-Based Audit Plan. The Audit addresses the Deputy Minister’s interest in improving planning, budgeting and forecasting as one of the top priorities for the Department in fiscal year 2014-15.

1.2. BACKGROUND

The Treasury Board Secretariat expects departments to develop integrated plans in order to reduce risk, to align activities to the departments’ mandates and priorities, and to track performance to ensure that expected results are achieved in an efficient manner. Management of Integrated Risk, Planning and Performance was introduced as a new core Area of Management in the 2015-18 Management Accountability Framework (MAF), with the objective of strengthening integration and alignment of planning in departments and agencies.

Transport Canada introduced the Integrated Planning and Reporting (IPR) process in 2011-12, for the 2012-13 fiscal year, to better align the activities of the Department with its mandate, improve departmental priority setting, decision making, and resource allocation and utilization, reduce duplication of effort in planning and reporting, and clearly communicate the Department’s priorities and direction to employees. Since we are not examining the external reporting components of the IPR process, we use Integrated Planning Budgeting and Forecasting (IPBF) rather than IPR to refer to TC ’s processes.

Under this the new integrated planning process, Transport Canada Executive Management Committee (TMX) identifies departmental priorities in June. Under the leadership of Program Accountable Executives (PAEs), Directorate Work Plans are drafted in October and then rolled up into National Program Integrated Plans (NPIP) by the end of November, which outline all activities taken by Programs at Headquarters and in the Regions to achieve the departmental Strategic Outcomes (SOs). These plans are challenged and approved by TC ’s governance committees. Based on the approved activities, TMX reviews and approves the Initial Budget Delegation (IBD) in February. Once the fiscal year begins, financial forecasts are monitored and updated monthly by Responsibility Centre Managers (RCMs) and are rolled up and reviewed at the Program, SO and departmental levels. Quarterly progress on activity plans, staffing and financial forecasts are reported using dashboards at the Program level and rolled up to the departmental level for review by TMX .

Recognizing that managers have struggled in the past to develop plans and budgets that match the actual pattern of expenditures, the Department has taken several relatively recent steps to support managers and improve their capacity, including the following:

• approval of a needs-based forecast of resources associated with activities in the planning cycle instead of budgeting based on Annual Reference Level Update allocation (ARLU);
• agreement to risk manage at the departmental level where funds are approved in excess of ARLU ;
• rigorous challenge roles of Financial Management Advisors (FMAs), Human Resources (HR) Advisors and SO planners/analysts to support managers on IPBF ;
• implementation of a national portfolio approach by FMAs and HR Advisors to increase strategic/horizontal support and collaboration between SOs and Regions;
• introduction of quarterly dashboard performance reporting to closely monitor the progress on planned activities, staffing and financial status; and
• provision of more IPBF training sessions/tools to managers and staff involved in IPBF in HQ and the Regions.

1.3. AUDIT OBJECTIVE, APPROACH, SCOPE AND CRITERIA

1.3.1. Audit Objective

The objective of this audit was to assess the effectiveness of TC ’s integrated planning, budgeting, and forecasting processes based on the 2014-15 processes in place and taking into account the improvements currently being implemented. Specifically, the audit objectives were to determine whether:

1. The governance over the integrated planning, budgeting, and forecasting processes is effective in supporting the achievement of departmental strategic outcomes and priorities.
2. Integrated planning, budgeting, and forecasting processes, including corporate oversight, monitoring, and reporting, are effective in supporting sound decision-making.

1.3.2. Audit Approach, Scope and Criteria

Audit Approach

Our audit work included:

• conducting interviews, and focus groups with staff and managers involved in the IPBF processes as they relate to budgeting and forecasting;
• reviewing relevant documents such as policies, guidelines, and frameworks;
• assessing adherence to the planning process;
• reviewing mechanisms established to measure and report financial and human resource staffing performance and progress;
• conducting walkthroughs of the financial management systems and tools;
• reviewing variance analysis at different periods within the fiscal year and actions taken to address issues; and,
• examining analysis conducted to compare the implementation status of work plans and the actual resources used.

Scope

The audit focused on the following areas:

• the planning process for developing departmental work plans that set the basis for determining budget requirements;
• financial aspects of the IPBF processes and their relationship to the Department’s budgeting and forecasting activities;
• impact of the planning and implementation of HR staffing plans in relation to financial planning, budgeting; and
• analysis and reporting on the implementation status of departmental work plans in relation to actual resource use.

The audit focused on the planning, budgeting and forecasting activities related to Operations and Maintenance expenditures (which include both salary and Other Operating Costs) up to Period 6 of fiscal year 2014-15. We examined compliance with the planning process steps but did not assess the quality of the resulting plans. We excluded Capital Expenditures and Grants and Contributions because of significant differences in their management processes and the Department’s dependence on third parties for program delivery.

The audit did not examine whether the overarching departmental governance structure which supports integrated planning, budgeting and forecasting is appropriate since it was assessed as part of the recent internal Audit of Governance. However, we did review whether the mandate of the committees and individuals under the existing structure are providing the necessary oversight and support to help ensure effective planning, budgeting and forecasting.

Finally, in recognition of significant changes to IPBF processes in the last year as well as the commitment to continuous improvement, we looked for opportunities to further advance improvements.

Criteria

We drew our findings based on an assessment of the Department against the following criteria:

Governance

• The governance structure provides adequate direction and oversight of the planning, budgeting and forecasting activities.
• Clear accountabilities, roles and responsibilities are established, documented, and communicated for employees involved in IPBF in Headquarters and in Regions.
• Employees involved with the IPBF activities (e.g. FMAs , HR Advisors, Strategic Planners / Analysts, RCMs , etc.) are trained, understand the IPBF processes, and have access to effective tools to support their roles and responsibilities.

Integrated Planning, Budgeting, and Forecasting Processes

• Guidelines, policies, and functional guidance are in place to support the IPBF processes.
• Risks associated with IPBF processes are identified, evaluated and mitigated in a timely manner.
• Standardized IPBF processes, procedures, and tools are used to support consistent IPBF activities across the Department, and to provide the necessary information for decision making in order to achieve the program objectives and financial targets.
• Monitoring and reporting mechanisms are in place to review results of IPBF activities and to ensure that appropriate and timely action was taken to address any significant variance from the approved departmental integrated plan.

1.4. REPORT STRUCTURE

The Introduction of this audit report is followed by findings grouped into three categories: “Governance” (Tone at the Top, Priority Setting), “Building Management Capacity” (Management Capacity for IPBF , Guidance, Tools and Training, Support from Planners/Analysts, HR Advisors, and FMAs ), and “Monitoring and Reporting” (Performance Measurement, Monitoring and Reporting, Review and Challenge Functions). Recommendations are made at the end of each finding category. A table of all recommendations and management’s action plan to address these is included at the end of the report.

2. FINDINGS

2.1. GOVERNANCE

2.1.1. Tone at the Top

As the most senior decision-making body, TMX determines strategic direction, sets priorities for the Department and leads the annual departmental integrated planning process. We expected to find that this governance body provided adequate direction and oversight of the integrated planning, budgeting and forecasting activities.

TMX has provided a very consistent message in support of the importance of improving the planning process, developing more accurate budgets and forecasts, and ensuring diligent monitoring of performance throughout the fiscal year. TMX endorsed the processes for the 2014-15 and 2015-16 integrated departmental plans and for quarterly dashboard reporting. TMX approved the TC Integrated Plan for 2014-15 and the resulting Initial Budget Delegation (IBD) was approved by the Deputy Minister, which were based on programs’ identified needs as opposed to past spending. This resulted in a projected deficit that would be risk-managed throughout the year at the departmental level. Managers were provided with clear direction that they were not required to budget for unexpected urgencies as these situations would be reviewed as they arise and consideration would be given to separate funding as appropriate. It should be noted that since the implementation of IPBF processes, initial budgets have been delegated earlier than ever to managers.

2.1.2. Priority Setting

One of the anticipated benefits of the integrated planning process was to improve the alignment of the departmental activities with TC ’s mandate and priorities. We expected to find that departmental priorities would be clearly articulated, communicated and understood prior to the planning of program activities.

TMX identifies departmental priorities at the outset of the planning process. These priorities reflect TC ’s mandate and the overall Government of Canada priorities, as well as TC ’s environment and feedback/input from Programs. Through Program Business Committees (PBCs), Program Accountable Executives (PAEs) determine the key activities that are required in support of the departmental priorities, as well as the corresponding performance measures. It is anticipated that the core program activities to meet the departmental mandate, and thus the majority of the budget, would be relatively consistent from year to year.

Activities and projects that would allow expected results to be accelerated are defined as “parking lot” items (potential priorities for future consideration) and are not to be included in the budget as they will only be undertaken if funds become available.

“Parking lot” items were identified during the integrated planning process and through the fiscal year. However, there is no clear process to prioritize “parking lot” items at the departmental level; should funds become available after risk managing the current year deficit. As a result, there is a possibility that funds could be reallocated within a Program or SO when there are “parking lot items” in other areas of the Department that are of higher priority to meet the overall departmental priorities.

Recommendation 1

Set clear expectations on prioritization of “parking lot” items at the departmental or the SO level and adjust the existing process and provide guidance, as required.

2.2. BUILDING MANAGEMENT CAPACITY

2.2.1. Management Capacity for IPBF

Recent changes to the IPBF processes were designed to increase the rigor of the planning and budgeting processes and to reinforce managers’ accountability for developing and delivering on plans and budgets. Given these changes, we expected to see an improvement in resource allocation and utilization of budgets, including a reduction in surpluses.

Based on data analysis and interviews, some managers are struggling with planning, budgeting and forecasting. In part, this may be because many of the recent changes challenge managers’ traditional approach to planning, budgeting, and forecasting.

Moving Towards “Needs-based” Budgets

Before the IPBF processes were revised, planning and budgeting processes were quite separate. RCMs were assigned budgets and developed their activity plans to fit within the budget. Now managers are tasked with planning the activities required to deliver their programs and meet the Department’s priorities. In simple terms, they must define their “needs” first and then estimate the resources needed to deliver their plans. This requires managers to make judgments about the right level of activity to address risks and priorities. It also challenges them to determine what can be accomplished and identify when resources, such as new staff, will be required in the fiscal year.

In our interviews with managers, HR Advisors and FMAs it was clear that the current IPBF processes have generated more awareness of the linkages between effective planning and accurate budgets, and the importance of realistic planning and forecasting assumptions. For example, to establish accurate salary budgets, the salary of any new staff should be based on a reasonable expected start date rather than on a full year’s salary. There were 1,150 staffing actions planned for the 2014-2015 year. Slippage on the start dates for these can significantly impact the $465 million salary budget. Indeed, at Q 2, Programs reported that 180 staffing actions had not been completed by mid year and estimated the end of year impact of delayed hiring and unanticipated departures could reduce salary budgets by $32M (7%). For its part, Corporate Services anticipated this reduction could be $37M (8%) to $42M (9%) with 150 to 175 staffing actions lapsing into the new fiscal year.

Although it is specified in the IPR guidance that PBCs are responsible to identify activities, and what should inform these discussions, the mechanisms to identify Program activities to develop National Program Integrated Plans are not clear and not applied consistently among Programs. This creates difficulties and confusion for related Programs, regions, and enabling functions to plan their individual activities. For instance, some PAEs were very prescriptive in defining Program-level activities to be followed by Regions, some left the identification of Program activities entirely to the Regions, while others collaborated with the Regions in planning the Program activities.

Some interviewees raised concerns as to whether the level of effort for the new planning, budgeting and forecasting processes is sustainable. Others commented that all managers need to accept their IPBF -related responsibilities and respond by expending greater effort in developing their IPBF capacity and competencies.

Further Work is Required to Achieve Goal of Effective and Efficient IPBF Processes

A history of surplus and lapsed funds from budgets not spent in the fiscal year forced an increased focus on IPBF . Changes were introduced to improve the planning of activities, develop accurate needs-based budgets and implement monitoring so that surpluses or deficits could be identified early and reallocated to other priorities if needed.

In our discussions with managers, many view this year's projected $4M surplus (as of October 2014) as an example of effective budget management and believe that they are meeting TMX ’s goal. However, as per the exhibit below, this does not take into account that half way through the year, the projected surplus is a $55.9M variance from the forecast spending by the Department at the start of the year. This variance shows that overall there has not been an improvement for Programs/Directorates in developing accurate needs-based budgets to support the activities that can actually be executed in the fiscal year.

Exhibit 1
Transport Canada
Operating Funds
Comparison of Deficit / Surplus over 2014-2015

Original Deficit based on program budget requests for 2014-2015	$(41.1) M
Adjusted Deficit at Quarter 1	$(51.8) M
Deficit based on Programs’ forecast at Quarter 2	$(20.9) M
Surplus based on Financial Planning and Resource Management forecast at Quarter 2	$4.1 M
Variance from Adjusted Deficit at Quarter 1 to Financial Planning and Resource Management forecast at Quarter 2	$(55.9) M

As this has been a transition year for IPBF , neither TMX nor Corporate Services were surprised that the needs-based budget has not been fully utilized. However, in reviewing and approving the 2015-16 plans and budgets, they will be looking for clear justification for budget requests in excess of actual 2014-15 spending. IPBF can only be considered a success when budgets are precise enough that actual spending closely mirrors budgets (notwithstanding any in-year course changes), managers deliver on their planned activities and budgets, and overall departmental resources are being adjusted on a timely basis to meet annual needs.

Equating Forecasting a Surplus to Releasing of Funds

Another challenge is manager’s perception of their control over identified surpluses in their budgets. They do not equate forecasting a surplus with the releasing of funds.

TMX has provided direction for managers to focus on delivering planned activities with the funds requested. If a Program will be able to carry-out all its planned activities, any excess funds are to be “released” making them available for other departmental priorities or reduce risk taken at IBD .

It would appear to the audit team that many managers do not always equate their planned activities to their planned spending. Many managers expect that they can continue to access their initial delegated budget throughout the year, regardless of whether or not they are delivering on their planned activities.

Several factors contribute to this disconnect between managers’ practices and TMX expectations:

• In the TC Financial Policy and Procedures manual, the authority to reallocate surplus funds to other priorities is given to the lowest level of the PAA (that would be both giving and receiving the funds). It does not identify any restrictions with respect to reallocation where activities are not being delivered.
• Because of the administrative burden to book reallocations in the financial system, TC currently does not transfer in-year identified surpluses from RCMs ’ budgets. Therefore, according to the official records of the Department, RCMs still see their full budget as available for their use. RCMs are not required to explicitly release these funds and so may perceive that they can still use this budget.

2.2.2. Guidance, Tools and Training

The IPBF processes at TC have undergone significant changes in recent years. We expected to find guidelines, policies and departmental tools in place to support TC staff through the transition.

Formal guidelines and policies exist for planning, budgeting and forecasting and the current process is in keeping with the spirit of the existing guidelines with the exception of the authority to reallocate forecasted surplus funds.

The CPR Directorate has prepared extensive documentation and updated templates to support Directorate and Program areas in developing their plans and such information is readily available on the internal departmental website. The Directorate also offers a range of training options on how to use the templates based on the extent of managers’ involvement and level of experience with the IPBF processes. There were information sessions provided to the PAEs and their management staff (including the planners/analysts, HR Advisors and FMAs ), formal training sessions, open-door sessions, etc. As well, CPR staff are available on an ongoing basis to respond to technical and process questions. Guidance on the preparation of the quarterly dashboards has been improved as a result of Corporate Services’ review of lessons learned.

Despite the level of documentation and training available, we still found gaps in managers’ understanding of the evolving processes. As an example, managers widely expressed concern that they are locked into the plans and budgets that they submit in November for the coming year and that many factors can affect their ability to deliver on those plans in the subsequent 16 months. Although CPR confirmed the current process allows for updates of plans and budgets up to February if a significant change occurs, most managers interviewed were not aware of this step in the process. Other managers indicated that they found the process complicated and did not fully understand the staffing worksheet. Many managers do not even attempt to complete a Year 2 or Year 3 budget.

There are departmental tools available to support the monitoring phase of the IPBF activities and they are being used by managers, planners, HR Advisors and FMAs . However, most managers were not comfortable using the tools on their own, therefore relying heavily on the planners and/or FMAs to provide the required information, and generally they have difficulty interpreting and analyzing the information presented. In some cases, managers created their own reports (often not using departmental reporting tools) to capture the information they feel is necessary to better understand and analyze their financial and HR status. Moreover, FMAs and HR Advisors use the departmental tools to collect information but do not always use the departmental reporting tools to present the information to managers. This is partly due to their level of experience and to requests they receive from managers to provide more customized reports. Although there are training courses available that provide technical training to use tools, stronger emphasis should be placed on standardizing and requesting users’ feedback on the use of Business Objects reporting tools and Salary Management System. We believe the current suite of tools needs to be enhanced so that managers and advisors are well supported and can be confident that the standard departmental tools are useful and the information from them is accurate, sufficient to their needs, and easy to understand.

Although reviewing the controls in place for ensuring the data integrity from the Department’s financial reporting tools was not in the audit’s scope, it should be noted that managers did raise some data integrity and reliability concerns^Footnote2.

2.2.3. Support from Planners/Analysts, HR Advisors, and FMAs

TC ’s new approach to IPBF brings together staff with expertise in specific components of the IPBF processes to support PAEs . We expected to find processes that ensure managers are provided with direct support from FMAs , HR and integrated planning advisors.

The role of Program planners/analysts has been strengthened to better support PAEs in developing their integrated plans and to review, analyze, challenge, and coordinate dashboard reporting with CPR and other TC governance bodies. We noted that in some Programs, senior managers have essentially delegated their IPBF activities to their planners/analysts.

HR and Financial Planning and Resource Management have recently adopted a national portfolio structure that mirrors the Department's Program Alignment Architecture (PAA). Each PAE is assigned a senior HR Advisor and FMA to provide advice, analysis and challenge on a quarterly basis. Through interviews and the focus group discussion, managers indicated that the value added of the HR Advisors and FMAs has increased as these advisors become more knowledgeable about their assigned business areas. HR Advisors, in particular, felt that the national focus enabled them to support the business areas at a strategic level and identify opportunities such as streamlining the recruitment process by running national competitions where there are similar needs across regions.

Managers from the focus group felt the monthly meetings with HR Advisors and FMAs were worthwhile but they did not see the need to involve these advisors in their initial planning processes as they rely on their Program planners/analysts to lead and play a challenge role during that stage. We believe that it is essential to have the HR Advisors and FMAs directly involved in the planning process in order to improve their understanding of the Program area and to support the development of realistic and well-founded plans. From interviews, we noted the quality and consistency of HR Advisors’ and FMAs ’ support and advice during the monitoring and reporting phases varies depending on the capacity and experience of the individual advisors.

Efficient and timely staffing to maintain or expand the workforce is a key concern for many Program managers. There is high turnover, including retirements, within the existing workforce. Despite some recent efforts to streamline the staffing process, it remains a lengthy process that does not always prove successful. In order to devise strategies to better address on-going recruitment needs, managers need to better understand both the internal and external factors that influence staffing. HR is proactively sharing workforce and demographic analysis with planners and managers but this information is not being used consistently to promote better forecasting of staffing requirements and realistic sustainable staffing strategies.

In our opinion, the process changes have been positive as they now put the onus on managers at all levels to exercise greater leadership and work collaboratively in the planning of activities and stewardship of resources. However, to maximize the benefits of the new processes, we believe managers’ capacity and the support they receive should be further strengthened.

Recommendation 2

To continue to improve guidance and the quality of support to build management capacity and improve planning, budgeting and forecasting:

1. Revise the Financial Policy and Procedures Manual to clearly reflect the expectations of TMX on allocation of in-year surpluses.
2. Provide targeted training and support to PAEs , managers and corporate advisors (e.g., FMAs and HR Advisors) who need to improve their IPBF capacity and competencies.
3. Continue to consult with managers to identify and implement changes needed to improve the departmental planning and reporting tools including additional standardized tools and processes that can be used by FMAs and HR Advisors to support managers.
4. Consult with managers to identify and implement more active/ direct support to enable managers to better understand and apply available workforce and demographics analysis to improve planning and forecasting of salary budgets.

2.3. MONITORING AND REPORTING

2.3.1. Performance Measurement

Performance measures are a key part of the IPBF processes to track whether planned activities are being undertaken and whether the level of performance is considered adequate in meeting the mandate and priorities of the organization. We expected to find performance measures in place that provide timely, accurate and complete information to support senior management's decision making.

The planning process for 2014-15 included the identification of mid-year and year-end milestones to track progress throughout the year. However, it did not provide sufficient direction nor sufficient challenge and/or analysis of the performance measures. As a result, the comprehensiveness and quality of performance measures identified ranged significantly amongst Programs. Some Program plans included specific, measurable milestones while others were not specific enough to monitor if activities were progressing at an acceptable rate and to measure success in meeting mandates, objectives, priorities and standards.

Following the review of the Quarter 1 results, TMX directed Programs to provide more information to support the Program’s self assessment of progress against plans in future quarterly dashboards. In interviews, PAEs acknowledged performance measures are limited and that selected measures may not fully reflect significant elements of a Program considering financial materiality, risk, and the breadth of their Program’s activities. As well, although there are indicators to measure the quantity of activities completed, there are seldom indicators to measure the quality of completed activities or the potential impacts of activities not being completed as planned.

To ensure that managers are being held accountable for delivery on their approved activities; progress on their planned activities, planned staffing, and financial management are incorporated in their Executive Performance Management Agreement (PMA). However, we noted that for fiscal year 2014-15, it has not been determined how managers’ achievement in completing their activity and staffing plans will be measured. This uncertainty late in the fiscal year creates confusion, does not promote consistent behaviour, and limits the effectiveness of this key accountability mechanism.

Recommendation 3

Provide sufficient guidance to help managers define comprehensive quantitative and qualitative performance indicators to support senior management decision making and more broadly to monitor and report on Program progress.

Recommendation 4

As part of the Executive Performance Management Process (PMP), define, by the beginning of the Performance Management process cycle how executives will be measured on their staffing, financial and program delivery performance.

2.3.2. Monitoring and Reporting

Monitoring, reporting and taking corrective action (when needed) are necessary to assess whether plans, objectives and priorities have been achieved, and for informing future planning cycles. We expected to find monitoring and reporting mechanisms in place to review results of IPBF activities and to ensure that appropriate and timely action is taken to address any significant variances from the approved departmental integrated plan.

In fiscal year 2014-15, TC introduced a dashboard approach to monitor quarterly activities in three areas: Program activities, staffing and financial performance. The dashboard provides the Programs with a mechanism to report their progress against their overall plan, their indeterminate staffing plan, and their revised financial forecasts. Corporate Services continuous efforts to improve the direction and guidance provided to managers on dashboard reporting has resulted in a noticeable improvement in the information provided between the first and second quarter reports.

One of the new monitoring practices introduced this year is to track staffing plans for indeterminate positions on a monthly basis. The planned staffing information for each position, as agreed with managers, is tracked in the Workforce Planning Tool by HR Advisors and is adjusted in the Department's Salary Management System to calculate the salary forecast.

Financial forecasts are updated monthly at the RCM level through the Resource Utilization Status Highlights (RUSH) report with input from the FMAs , HR Advisors and Planners. The financial forecast is rolled up to the departmental level in the Resource Overview report. This process is intended to identify changes in funding needs as early as possible in the fiscal year.

As part of the dashboard reporting process, in addition to the assessment provided by the Programs, HR Advisors and FMAs , Corporate Services provides an overall assessment of each Program's performance. We observed, however, that the linkages between the performance in each of the three categories as well as the overall performance assessment are not always apparent. As well, it was not always clear how a Program that is falling behind in its staffing targets could continue to meet its Program-related activity targets or how two Programs with identical performance ratings could end up with different overall assessments. In addition, the dashboard report provides a snap-shot of a Program as it stands at the end of a quarter but does not necessarily predict where the Program will be by the end of the fiscal year.

While there is general support for these detailed monitoring and reporting processes, all managers expressed concerns over the tight reporting timelines for the dashboards, particularly for Programs that were required to gather information from the regions or other Program contributors. Likewise, Corporate Services has only a few days to consolidate the information provided by the Programs and synthesize this with other information to provide an overall analysis for discussion at TMX . At the second quarter of fiscal year 2014-15, TC gathered, analyzed, reviewed, and presented the information to TMX in 18 working days (27 calendar days). In comparison, the Office of Comptroller General’s Horizontal Internal Audit of Financial Forecasting in Large and Small Departments^Footnote3 found that, on average, departments were completing quarterly reporting in 35 calendar days. We believe that adding a few days for Programs and Corporate Services to analyze results could provide better and still timely information for decision making.

Recommendation 5

Revise the IPBF guidance to include dashboard reporting requirements to:

1. Clearly explain the logical linkages between the performance on Program-related activities, staffing, and financial forecasting performance;
2. Describe any actions they intend to take to mitigate risk and to describe the level of residual risk; and
3. Allow more time for analysis of information at the Program and National level.

2.3.3. Review and Challenge Functions

The Treasury Board Policy on Financial Resource Management, Information and Reporting^Footnote4 calls for effective oversight of the Department's financial plan, budget, and related resource allocations through monitoring and reporting on the use and performance of financial resources. We expected to find clear accountabilities, roles and responsibilities at various levels of management including a robust challenge function to ensure complete, accurate, and timely information to support decision making.

The IPBF processes clearly define roles, responsibilities and accountabilities for the timely review of plans and quarterly reporting at the Directorate, Program, and SO levels prior to TMX review.

In our interviews, PAEs expressed concern that while they understand their accountability within a matrix organization^Footnote5, they are reluctant to question the decisions of their fellow Directors General (DGs) that fall within their Program. Therefore, PAEs may not be as actively managing the Program areas which report operationally to other DGs .

Integrated plans, budgets and forecasts are reviewed by Strategic Outcome / Internal Services (SO/IS) Management Boards (MB) composed of Assistant Deputy Ministers and Regional Directors General. However, when quarterly dashboard reports are being reviewed all PAEs , Directors General, and Corporate Services attend as optional attendees. Each PAE is required to present their dashboard information and identify what mitigating actions they are taking if necessary to achieve their plans. Both members and optional attendees have the opportunity to challenge the information presented. We observed at the SO/IS MB meetings to review the second quarter dashboard reports that the challenge role fell primarily to Corporate Services and there was little input from the other attendees. We believe there is a need to strengthen the SO/IS MB challenge role to ensure plans, budgets and forecasts are effectively assessed and SO/IS MB Chairs’ exercise their accountability, reducing the burden on TMX to ensure effective oversight.

Recommendation 6

The Chairs of SO/IS MBs should work with Corporate Services to improve the efficiency and effectiveness of the challenge function to ensure plans, budgets and forecasts have been sufficiently vetted prior to presentation to TMX .

3. CONCLUSIONS

Overall, we found that progress is being made to improve the integrated planning, budgeting and forecasting processes.

In the spirit of continuous improvement, we have highlighted the following areas to help management target its on-going improvement efforts.

• Confirm whether parking lot items should be prioritized at the SO or departmental level;
• Reinforce TMX direction on allocation of in-year surpluses;
• Enhance support to managers in areas such as use of HR staffing and demographics analysis, improved utilization of tools and provide targeted training for those who need to improve their competencies;
• Enhance dashboard reporting to describe the logical linkages between assessments of progress on Program activities, staffing, and financial forecasting performance and better assess the level of residual risk;
• Define and consistently report on meaningful performance measures that go beyond financial indicators to monitor progress against program objectives;
• Increase the efficiency and effectiveness of the SO/IS MB challenge function;

Although the audit makes specific recommendations, sustained improvement to planning, budgeting and forecasting is dependent on continued cultural change within TC . To facilitate and encourage a wider understanding of the key issues and how improvement directly depends on the actions of managers, the Chief Audit and Evaluation Executive will support Corporate Services in presenting these findings and lessons learned to executive management teams throughout the Department.

4. RECOMMENDATIONS AND MANAGEMENT ACTION PLAN

#	Recommendation	Detailed Management Action Plan	Completion Date (for each action)	OPI for each specific action
1	Set clear expectations on prioritization of “parking lot” items at the departmental or the SO level and adjust the existing process and provide guidance, as required.	For the current planning cycle (2015-16), parking lot items will be prioritized at the SO level and consolidated for discussion at the departmental level. For future planning cycle, the process will be adjusted, including the guidance material, taking into account the lessons learned from the 2015-16 approach.	June 2015	Corporate Services
2	To continue to improve guidance and the quality of support to build management capacity and improve planning, budgeting and forecasting: Revise the Financial Policy and Procedures Manual to clearly reflect the expectations of TMX on allocation of in-year surpluses. Provide targeted training and support to PAEs , managers and corporate advisors (e.g., FMAs and HR Advisors) who need to improve their IPBF capacity and competencies. Continue to consult with managers to identify and implement changes needed to improve the departmental planning and reporting tools including additional standardized tools and processes that can be used by FMAs and HR Advisors to support managers. Consult with managers to identify and implement more active/direct support to enable managers to better understand and apply available workforce and demographics analysis to improve planning and forecasting of salary budgets.	The appropriate revisions will be made to reflect the process for handling in-year surpluses, and details will be included in the annual Initial Budget Delegation (IBD) memo from the Deputy Minister. Actions in support of recommendation 2b, c, and d: The annual Integrated Planning and Reporting process post-mortem will be reviewed to ensure adequate feedback (i.e., specific and from a sufficiently large sample) is received from managers, support staff and functional specialists (e.g., HR Advisors). Further to feedback from the community, changes will be made to the existing IPR process and related guidance for the next planning cycle, and to the Quarterly Reporting for 2015-16, to the extent possible. Existing training methods and tools will be modified to reflect feedback received, and be more targeted to the specific audience (managers, support staff and functional specialists) and/or the specific topic, including workforce and demographic analysis. A list of PAEs and their staff that could benefit from targeted support will be identified (using SO/IS MB chairs input, past results and feedback). This may include suggesting external training (e.g. CSPS courses) Note: Improved procedures, tools and training will need to be supported by clear delineation of roles and responsibilities and accountability, which are elements addressed as part of the MAP related to the audit on governance.	September 2015	Corporate Services
3	Provide sufficient guidance to help managers define comprehensive quantitative and qualitative performance indicators to support senior management decision making and more broadly to monitor and report on Program progress.	The post-mortem mentioned in the previous recommendation will seek feedback to be used in: Improving the guidance for the 2016-17 IPR cycle; Developing a longer-term action plan to improve performance measurement (subject to the results of a presentation to TMX on the PAA / PMF and performance information for decision-making in mid-January 2015).	June 2015	Corporate Services
4	As part of the Executive Performance Management Process (PMP), define, by the beginning of the Performance Management Process cycle, how executives will be measured on their staffing, financial and program delivery performance.	A recommend approach will be presented at the TMX planning retreat – where approval of the TC Integrated Plan and Initial Budget Delegation (IBD) will be sought – and communicated as part of the IBD memo. Formal incorporation in the PMP will follow the normal timelines.	June 2015	Corporate Services
5	Revise the IPBF guidance to include dashboard reporting requirements to: Clearly explain the logical linkages between the performance on Program-related activities, staffing, and financial forecasting performance; Describe any actions they intend to take to mitigate risk and to describe the level of residual risk; and Allow more time for analysis of information at the Program and National level.	Actions in support of recommendation 5a and b: Revised dashboard that includes an executive summary and examples to draw linkages between the various elements of the dashboard and a focus on impacts and mitigation measures and residual risks. Further feedback on the new format will be used to make additional amendments if/as required. Actions to support recommendation 5c: The various due dates associated with the quarterly reporting will be reviewed for 2015-16, and communicated to PAEs and functional specialists. The changes will also be included in the guidance for the 2016-17 IPR cycle.	June 2015	Corporate Services
6	The Chairs of SO / IS MBs should work with Corporate Services to improve the efficiency and effectiveness of the challenge function to ensure plans, budgets and forecasts have been sufficiently vetted prior to presentation to TMX .	As part of the IPR 2015-16 post-mortem, meet with ADMs and RDGs to identify tools to help SO / IS members exercise their responsibility.	March 2015	Corporate Services

Appendix A: Acronyms