Update Evaluation of Transport Canada’s Road Safety Transfer Payments Program (RSTPP)

| Print Version |

Table of Contents

• List of Abbreviations
• Executive Summary
• Introduction
  • Profile
  • Funding and Delivery
  • Program Logic Model
  • Evaluation Approach
• Findings on Relevance
  • Alignment with Federal Roles and Responsibilities and Priorities
  • Ongoing Rationale for the Program
• Findings on Performance
  • Direct outcomes
  • Increased levels of motor vehicle safety
  • Economy and Efficiency
  • Conclusions
• Management Action Plan
• Annex 1: Road Safety Transfer Payments Program Resources

List of Abbreviations

Executive Summary

Responsibility for motor carrier and commercial driver safety is shared between the federal government and the Provinces and Territories. The federal government is responsible for extra-provincial truck and bus carriers (i.e. those that cross a provincial, territorial or international boundary).

Through contributions to Provinces and Territories and the CCMTA , Transport Canada supports the implementation of the Safety Fitness Framework - a subset of four of the fifteen National Safety Code standards.

Expenditures are set at $4.4 million annually with $50,000 allocated to the CCMTA for the educational component. Since the program began in 1987 TC has provided over $105 million in funding support to Provinces and Territories for the implementation of the National Safety Code.

The evaluation found that the Program supports the Motor Vehicle Transport Act (MVTA) objective of ensuring consistency in extra-provincial motor vehicle safety oversight. The evaluation also found that the Provinces and Territories have fully adopted the Safety Fitness Framework and that they were continuing to exchange collision, inspection and conviction information. An apparent decrease in the exchange of conviction since 2006-2007 was noted, which may indicate issues regarding consistency in application of the Safety Fitness Framework.

{ ATIP Removed}

Introduction

The Road Safety Transfer Payments Program includes three components: the Motor Carrier Safety Program (funded previously under the National Safety Code contribution program); the Canadian Council of Motor Transport Administrators Education and Quality Assurance Team Programs; and the Road Safety Outreach program.

This evaluation covers the first two components, but not the outreach component, which was not funded by TC during the current agreement period.  

Profile

The National Safety Code is a comprehensive code of performance standards for the safe operation of commercial vehicles (motor carriers and buses). Fifteen standards make up the Code with the goal of encouraging truck and bus safety, promoting efficiency in the motor carrier industry, and ensuring the implementation of consistent safety standards across Canada. The standards continue to be developed through the Canadian Council of Motor Transport Administrators, an umbrella organization of federal, provincial, and territorial governments, industry and public interest representatives.

Since 2001, Transport Canada has had contribution agreements with each of the Provinces and Territories and the CCMTA to support implementation of the Safety Fitness Framework - a subset of four of the fifteen National Safety Code standards ( # 7, # 12, # 14.^Footnote1 and # 15). Under the current Terms and Conditions, the recipients conduct facility audits and inspections of extra-provincial commercial carriers and report back on the four standards that make up the Safety Fitness Framework. Under a different contribution agreement, the CCMTA ’s Education and Quality Assurance Team is responsible for managing the national program for the certification of instructors and inspectors, including the development and distribution of Commercial Vehicle Safety Alliance (CVSA) educational material.  

Funding and Delivery

The current contribution agreement period is from 2009-2010 to 2014-2015. Expenditures are set at $4.4 million annually with $50,000 allocated to the CCMTA for the educational component. Since the program began in 1987 TC has provided over $105 million in funding support through Contribution Agreements to Provinces and Territories.

Within Transport Canada, the Program was previously delivered through the Motor Carrier Safety Directorate of the Safety and Security Group. Following the centralization of Transport Canada’s delivery of its grants and contributions programs in 2011, the processing and approval of claims were transferred to the Surface Infrastructure Programs Directorate while the responsibility for the provision of technical advice and guidance remained with the Motor Carrier Safety Directorate^Footnote2.

Program Logic Model

Figure 1: Logic Model for the NSC Contribution Program

ACTIVITY AREAS / OUTPUTS	PROGRAM REACH	IMMEDIATE OUTCOME	INTERMEDIATE OUTCOMES	ULTIMATE OUTCOMES
Contribution Agreements with jurisdictions Manage and assess program performance in relation to the Contribution Agreements Provide leadership to provinces and territories in: a) harmonizing the implementation of safety fitness framework across Canada b) simplifying the regulation of extra- provincial/territorial motor carriers	Provincial Territorial Governments or their agent or instrumentality Motor Carrier Industry Canadian Public	Jurisdictions adopt and enforce the safety fitness framework	Increased jurisdictional cooperation Motor Carriers in all jurisdictions operate under uniform safety standards	Increased levels of national motor carrier transportation safety Increased levels of national motor carrier transportation efficiency

Evaluation Approach

In 2009, a comprehensive evaluation was conducted that resulted in the renewal of the program. Since then, the primary components of the program have not undergone any changes. Given the program’s low materiality and low risk, the scope of the study is therefore limited to reviewing the continued relevance of the program and updating the key findings on performance of the 2009 evaluation.

To conduct the evaluation, relevant documents, such as contribution agreements and annual reports from recipients were reviewed. Program staff were consulted to validate updated figures and findings. Representatives from the Provinces and Territories were not interviewed as they were in 2009. A review of various program documents confirmed that their views on the program have not changed since the 2009 evaluation.

Findings on Relevance

Evaluators examined TC ’s role with respect to inter-provincial motor vehicle safety, the program’s alignment with government priorities and with TC ’s strategic outcomes, including alignment with relevant policy frameworks, and the on-going need for the program.

Alignment with Federal Roles and Responsibilities and Priorities

Responsibility for motor carrier and commercial driver safety is shared between the federal government and the provinces/territories. The federal government is responsible for extra-provincial truck and bus carriers (i.e. those that cross a provincial, territorial or international boundary), while the provinces/territories have sole responsibility for intra-provincial truck and bus carriers, those whose operations are entirely within a province or territory^Footnote3.

The Program continues to align with TC ’s strategic outcome of a safe and secure transportation system and contributes to TC ’s objective of “highest possible safety and security of life and property ...” In the current TC Program Alignment Architecture, the Program falls under Motor Vehicle Safety (P 3.4).

While motor carrier safety has not been mentioned in recent Speeches from the Throne, the safety of the overall transportation system has been mentioned several times and is clearly a priority for the Government.

Ongoing Rationale for the Program

Consistency with respect to standards that apply to extra-provincial motor vehicle activities is an objective set out in the Motor Vehicle Transport Act (MVTA). It is expected that the adoption and enforcement of the Safety Fitness Framework by the Provinces and Territories will contribute to this objective.

Previous evaluations (2009, 2004) noted a strong consensus amongst stakeholders and in TC regarding the involvement of the federal government in the development of all NSC safety regulations and enforcement because of the national scope of the program. However, the 2009 evaluation also noted a growing dissatisfaction amongst the recipients regarding the level of funding provided by the federal government as representatives from the Provinces and Territories expressed the opinion that the federal government did not contribute its appropriate share of the costs for implementing the regulations.

A review of program documents summarizing ongoing feedback from recipients shows that these views have persisted. { ATIP Removed}

However, an examination of the activities conducted by Provinces and Territories on TC ’s behalf for the period assessed did not reveal significant evidence or indications that this reported discontent had a negative impact on the implementation of the current contribution agreements. Provinces and Territories have completed the timely implementation of the four NSC standards that make up the SFF (e.g. facility audits, information sharing, etc.), as per the agreements.

Therefore, despite dissatisfaction amongst recipients regarding funding levels, TC has been able to advance the national consistency objective by virtue of the fact that Provinces and Territories have conducted the activities they have committed to in the contribution agreements.

Findings on Performance

This section presents findings on short term and long-term outcomes of the program. It also provides information regarding economy and efficiency being achieved in delivering the program.

Direct outcomes

The immediate outcome expected from this program is that jurisdictions adopt and enforce the Safety Fitness Framework.

The adoption and enforcement of the SFF was measured through an assessment of information and data included in the Annual Performance Reports submitted to TC between 2009 and 2013 by each jurisdiction as required by the contribution agreements. In table 1 below, a “yes” indicates that a jurisdiction has fully implemented and reported on the requirements for the standard in question for that year.

The 2009 evaluation reported ten of the twelve jurisdictions had fully adopted the SFF and the territories were in the process of doing so. The current evaluation found that all twelve jurisdictions have fully adopted the SFF and they are reporting annually on:

• the number of convictions, accidents and inspections for the period ( NSC # 7)
• the number of CVSA roadside inspections conducted on extra-provincial motor carriers ( NSC # 12)
• the number of new safety certificates issued to extra-provincial trucks and buses ( NSC # 14)
• the number of facility audits conducted on extra-provincial trucks and buses ( NSC # 15)

Only three of the 48 data points were missing over the four years for the twelve jurisdictions.

Table 1: Submission of Safety Fitness Framework Data by Jurisdiction,
2009-10 to 2012-13

Year	NSC Standard	Jurisdiction												# Provinces and Territories who reported fully
		BC	AB	SK	MB	ON	QC	NB	NS	PEI	NL	NWT	YK
2009-10
	NSC # 7	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 12	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 14	yesY	yesY	yesY	yesY	yesY	yesY	yesY	noN	yesY	yesY	yesY	yesY	11
	NSC # 15	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
2010-11
	NSC # 7	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 12	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 14	yesY	noN	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	11
	NSC # 15	yesY	noN	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	11
2011-12
	NSC # 7	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 12	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 14	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 15	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
2012-13
	NSC # 7	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 12	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 14	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12
	NSC # 15	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	yesY	12

Standard # 14 primarily requires each jurisdiction to issue safety fitness certificates to their inter-provincial motor vehicles (buses and trucks). Jurisdictional performance reports state that this was done (see table 2 below). Quebec issued the largest number of certificates, followed closely by Ontario, then British Columbia (BC) and Alberta.

Table 2: New Safety Certificates Issued to Extra-provincial carriers by Year and Jurisdiction, 2009-10 to 2012-13

Jurisdiction	Total New Safety Certificates
	2009-10	2010-11	2011-12	2012-13
BC	357	402	356	632
AB	564	328	392	399
SK	133	230	141	119
MB	87	92	130	135
ON	887	846	847	764
QC	767	811	1038	1154
NB	73	78	49	54
NS	51	13	15	33
PEI	15	24	19	21
NL	526*	7	8	5
NWT	11	7	22	8
YK	19	15	16	9
Total	2985	2853	3033	3333

*Includes both intra and extra provincial carriers

Standard # 15 requires jurisdictions to carry out Facility Audits. Jurisdictional performance reports state that this was done (see table 3 below), although jurisdictions reported inspections of all carriers and not only extra-provincial carriers.

Table 3: Facility Audits & CVSA Inspections by Jurisdiction and Year,
2009-10 to 2012-13

	2009-10		2010-11		2011-12		2012-13
Jurisdiction	Extra- prov audits	Total Inspections*	Extra- prov audits	Total inspections	Extra- prov audits	Total inspection	Extra- prov audits	Total inspection	Extra- prov audits	All CVSA inspections
BC	205	27382	251	26089	253	27762	190	31865	899	113091
AB	182	32013	n/a	36,720	377	32119	368	32771	927	133623
SK	129	17860	30	15218	13	13052	49	9943	221	56073
MB	48	7494	56	6189	58	4837	62	3541	224	22061
ON	237	104120	200	95,513	211	102807	269	102651	917	382858
QC	92	87540	229	77126	99	10654	69	65204	489	240524
NB	65	28991	94	29808	98	26452	155	25729	412	111070
NS	31	7502	12	10545	25	10618	7	7987	75	36652
PEI	13	2160	13	3837	13	1759	13	1521	52	9277
NL	28	1748	20	1986	22	1765	20	1636	90	7135
NWT	0	892	3	535	2	635	1	1021	6	3283
YK	3	909	4	782	7	892	7	801	21	3384
Total	1033	318611	912	304348	1178	233352	1210	284670	4333	1118748

*Audits reported only for extra provincial carriers but inspections reported on all carriers.

{ ATIP Removed}

The NSC component of the program is also expected to result in jurisdictional cooperation. Exchange of information between jurisdictions is an indicator of such cooperation. All twelve jurisdictions have been reporting annually on the number of collisions, convictions and inspections transmitted to and received from other jurisdictions^Footnote4 (see table 5 below), although the level of conviction data exchange appears to have decreased.

For the period assessed, jurisdictions transmitted on average 211,463 reportable events to other Canadian and American jurisdictions, with a peak in 2011-12 of 243,395 and a low of 128,522 in 2012-13. On average, 122,611 reportable events were received by jurisdictions, with a high in 2010-11 of 134,046 and a low in 2012-13 of 110,635^Footnote5.

Table 4: Number of events transmitted and received by Jurisdiction and Year,
2009-10 to 2012-13

	Total events transmitted 2009-10	Total events transmitted 2010-11	Total events transmitted 2011-12	Total events transmitted 2012-13	Total events received 2009-10	Total events received 2010-11	Total events received 2011-12	Total events received 2012-13
BC	12010	10414	11367	10824	20147	19297	15992	18511
AB	62544	60,525	60641	38310	18646	23,238	23453	20301
SK	12140	11112	16038	11423	21096	15048	15798	15755
MB	9805	1943	2431	1320	14604	15498	12993	13854
ON	21352	18949	20847	25415	14445	14213	16928	7452
QC	9625	24804	9934	14095	9580	26300	17228	15490
NB	15843	13519	17230	14652	14653	14763	11991	11616
NS	8645	1866	12094	10110	6008	2250	5938	4917
PEI	363	298	237	228	701	1227	946	813
NL	1016	732	760	649	1484	1354	1216	932
NWT	777	1017	476	781	782	690	659	796
YK	652	519	689	715	262	168	212	198
Total for year	154772	145698	152744	128522	122408	134046	123354	110635

However, there seems to be a decline in the number of convictions being exchanged. The 2009 evaluation noted the significant increase in this area - nearly doubling between 2003-2004 and 2005-2006 from 68,825 to 137,263, as further evidence that funding provided by TC had facilitated exchange of information between jurisdictions. The revised figures show a decrease in the number of convictions being exchanged. In 2012-2013, 62,607 convictions were transmitted and 31,865 were received. The program managers could not identify an explanation for the decrease but were of the opinion that it was not indicative of lesser cooperation between jurisdictions. We do note that the way the exchange of conviction data is measured for the 2009 evaluation and the current evaluation may differ. The 2009 evaluation counts “recorded and exchanged” conviction data. The data available for the current evaluation is tabulated as “number of convictions transmitted” and “number of convictions received”. However, even the total of these two categories would indicate a decrease from the 2009 levels.

Increased levels of motor vehicle safety

In the long-term, the RSTPP is expected to contribute to the safety of the motor vehicle sector. Since 2007, the number of commercial vehicles involved in reportable traffic collisions that resulted in at least one fatality has been decreasing. There were 557 such collisions in 2007. By 2011^Footnote6, that number was down to 422, a decrease of 24.2%. Similarly, there was a 16% decrease in the number of commercial vehicles involved in reportable traffic collisions that resulted in personal injury; however there was no change in the number of such collisions between 2009 and 2011.

Table 5: Number of commercial vehicles involved in reportable traffic collisions, by year and collision severity 2007-2011

Collision Severity	2007	2008	2009	2010	2011	Variance 2007 - 2011
Fatal	557	478	423	418	422	-24.2%
Personal Injury	11,480	10,715	9,215	9,546	9,645	-16%

As noted in previous evaluations, it is virtually impossible to link the NSC and the SFF to the safety performance of commercial vehicles. The direct contribution of the NSC to carrier safety is difficult to ascertain because many variables (e.g. road conditions, weather, vehicle maintenance, new technologies, etc.) influence the state of road safety in Canada. It is difficult, if not impossible, to assess the impact of one particular element. The decrease in the numbers of collisions involving commercial vehicles with serious consequences has to be viewed in the context of the overall improvement of motor vehicle safety in Canada in the same period (see table 7 below). 2011 marked the first year that the fatality rate per 10,000 registered vehicles (of 0.90) fell below 1.0^Footnote7. It is therefore challenging to measure the extent to which the advances made in the implementation of the SFF contributed to improving motor vehicle safety performance.

Table 6: Collisions involving motor vehicles 2007-2011

Year	Fatal	Personal Injury
2007	2,462	138,612
2008	2,192	127,678
2009	2,011	123,516
2010	2,026	123,141
2011	1,834	121,159

Source: TC National Collision Database (NCDB)

Economy and Efficiency

Since the 2009 evaluation, TC has streamlined its administrative processes in the delivery of its grants and contributions programs by centralizing program management and delivery, including claims approval and processing.

Although no official analysis has been undertaken to quantify the savings achieved, the centralization of G&C delivery to the Programs Group has generally allowed for gains in efficiency due to greater consistency, increased compliance, experienced personnel, and established business processes. Many of the tools and processes required to deliver a contribution program have already been developed and applied in the Programs Group and are based on best practices that are supported by a significant pool of knowledge and experience.

Conclusions

While there is a view that the perceived inadequacy of funding levels has been making it increasingly challenging for TC to exercise influence in driving consistency in extra-provincial motor vehicle safety oversight, TC has been able to advance this objective through contribution agreements with Provinces and Territories.

The contribution programs contributed to all jurisdictions adopting and enforcing the Safety Fitness Framework. The evaluation found that all twelve jurisdictions had fully adopted the Safety Fitness Framework. Although all jurisdictions were exchanging collision, inspection and conviction information, the exchange of conviction data appears to have decreased over the last five years, which may indicate issues regarding the consistent application of SFF .

{ ATIP Removed}

Management Action Plan

To address the recommendation presented in this evaluation, the following action plan will be implemented:

{ ATIP Removed}

Annex 1: Road Safety Transfer Payments Program Resources

	2009-2010*		2010-2011		2011-2012		2012-2013		2013-2014		2014-15
	Planned	Actual	Planned	Actual	Planned	Actual	Planned	Actual	Planned	Actual	Planned
Total O&M	10,000	10,000	10,000	10,000	10,000	10,000	10,000	3,377	10,000	3,751	42,975
Salary	10,000	10,000	10,000	10,000	10,000	10,000	0	3,377	0	3,751	33,850
OOC	0	0	0	0	0	0	10,000	0	10,000	0	9,125
G&C	250,000	239,203	9,136,000	9,086,218	4,692,681	4,686,789	4,442,681	4,442,284	4,442,681	4,440,264	4,442,681
Total	260,000	249,203	9,146,000	9,096,218	4,702,681	4,696,789	4,452,681	4,445,661	4,452,681	4,444,015	4,485,656
	S&S	S&S	S&S	S&S	S&S	S&S	PROGRAMS	PROGRAMS	PROGRAMS	PROGRAMS	PROGRAMS