| Print Version |
Table of Contents
- List of Tables and Figures
- List of Abbreviations
- Main Findings and Conclusions
- Introduction
- Evaluation Approach
- Detailed Evaluation Findings
- Annex 1: Logic Model of the Boating Safety Class Contribution Program
- Annex 2: Evaluation Framework
- Annex 3: Estimate of Program Reach
List of Tables and Figures
- Table 1: Expenditures of the Boating Safety Class Contribution Program, 2008-2009 to 2012-2013
- Table 2: Average Annual Number of Boating Fatalities per 100,000 People
- Table 3: Organizations Referenced in Newspaper Articles Relating to Boating Safety
- Table 4: Attitudes and Intended Behaviours of Boaters, 2008 to 2012: Percentage Who Agree (Top 2 Box)
- Table 5: Number and Percentage of Projects that Produced a Given Output Type, 2009-2010 to 2012-2013
- Table 6: Geographic Distribution of BSCCP Funds from 2009-2010 to 2012-2013 and Boating Fatalities from 1991 to 2008
- Table 7: Approved BSCCP Funding by Type of Boating Activity and Region, 2009-2010 to 2012-2013
- Table 8: Results of Efficiency Analysis for Six BSCCP -funded Projects
- Figure 1: Recreational and Other Boating Fatalities in Canada, 1991 to 2015
- Figure 2: Average Annual Number of Boating Fatalities by Province, 1991 to 2008
List of Abbreviations
Main Findings and Conclusions
This report presents the findings and conclusions regarding the relevance and performance of the Boating Safety Class Contribution Program (BSCCP) over the five years spanning 2008-2009 to 2012-2013, to fulfill the requirements of section 42.1 of the Financial Administration Act.
The BSCCP provides funding to organizations involved in boating safety or recreational boating, to increase boating safety awareness in Canada. Its objective is to influence attitudes and behaviour in a manner that would ultimately reduce injuries, fatalities and property damage caused by recreational boating accidents. Over the time frame of the evaluation, $1,394,806 was paid-out to 16 organizations in support of 32 projects.
Relevance
There is a large recreational boating population in Canada. The most recent estimate available at the time of the evaluation was 11.8 million adult Canadians who boat at least occasionally or 10.5 million Canadians who boated at least once in the past year. Considering the size of this population, boating related fatalities are relatively rare events. The most recent published data (2013) put the average number of deaths per year at 127 between 2006 and 2010. This represents a 7% decline in the number of fatalities recorded over the 2001 to 2005 time period. Evaluators estimate that the boating fatality rate in Canada is similar to what has been observed in the United States, Australia and New Zealand.
Boating safety is a federal responsibility, although the Office of Boating Safety (OBS) relies largely on federal, provincial and municipal police forces to enforce compliance with Transport Canada’s (TC) regulations.
Evaluation findings show that were it not for the funding provided by the BSCCP (and the OBS ), there would be little information on or promotion of boating safety in Canada. Although the program is not a large one in terms of dollars, it has been the single most important source of funding for boating safety promotion in Canada. Its relative importance would have increased since the time of the evaluation since all boating safety awareness activities in the Department are now delivered solely through the BSCP and its annual funding amount has doubled since 2013-2014.Footnote1
Evaluation findings suggest that the program constitutes an appropriate response to the issue.
Impact
It was estimated that the BSCCP ’s reach in 2012 was approximately 4.8 million boaters who boat at least occasionally (or 41% of such boaters).
The level of boating safety awareness in the Canadian boating population has been stable since 2008 but new boaters are constantly entering the boating population, suggesting that the BSCCP might be contributing to the maintenance of the level of boating safety awareness. Evaluation findings also suggest that the program’s impact might be improved by further aligning the distribution of funds with the distribution of risks in the boating population (e.g. by region, type of activity and age group).
The BSCCP has funded primarily outreach projects and non-government organizations (NGOs). Expanding the types of projects and recipients that are funded could help fill research and data gaps that would provide a better understanding of boating risks, of the types of interventions needed and which interventions are most effective.
The BSCCP is an economical and cost-efficient program because it funds organizations that are often able to leverage the support of volunteer workers in its delivery and often generate free media impressions. However, with a majority of the funded organizations developing their own awareness materials, rather than using existing ones, some duplication of effort may have occurred.
Conclusions
Preliminary findings and conclusions of this evaluation were shared with the program to inform the development of the new terms and conditions for the program in 2012. Based on these, program managers proceeded to:
- Expand the reach of the program by sending the call for proposals to a broader list of stakeholders.
- Set priorities based on risk, geographical coverage and project categories. These priorities are reviewed annually and adjusted prior to the release of new calls for proposals in the spring of each year.
- Create a working group of stakeholders with a mandate to provide options for the development and funding of a data collection strategy. The first meeting was held on September 18, 2013.
- Include the development of data as a priority in the first two calls for proposals. The program is currently funding four recipients who, as part of their project, will conduct research and data collection. The program, regional OBS and Québec enforcement agencies have been working together on a project to create a model for collecting and developing data in a consistent manner. If the project is successful, this model will be shared with other regional OBS across the country
- Encourage stakeholders to work together towards larger scale projects to optimize the recipients’ capacity and the impact of the projects.
Introduction
This report presents the results of an evaluation of the Boating Safety Class Contribution Program (BSCCP) for five years from 2008-2009 to 2012-2013. The evaluation was conducted to fulfill the requirements of section 42.1 of the Financial Administration Act, which states that departments must conduct a review of the relevance and effectiveness of ongoing grant and contribution programs every five years.
Background
The federal government’s support for recreational boating safety has a long history, going back to 1979. Initially, the Canadian Coast Guard led public outreach activities for boating safety awareness as part of its search and rescue operations, with the cooperation of volunteer-based organizations. In support of such activities, the federal government provided financial assistance to community groups.
In 1995, the Office of Boating Safety (OBS) was created as a mandated function within the Canadian Coast Guard. The OBS delivered activities such as dock-side safety checks and safety education directly to the boating public in marine and designated waters of the Canadian Coast Guard’s jurisdiction, until 1999. Following changes to the Canada Shipping Act, 2001 and its regulations, the mandate of the boating safety program was expanded to include all waters in all areas, including rivers, lakes, and other in-land waters. In 2003, in part because of this change to the Canada Shipping Act, 2001, the OBS was transferred to Transport Canada under Marine Safety, along with two contribution programs for which it became responsible.
The two contributions were tailored to specific organizations and to specific activities. They were made annually to two organizations, the Canadian Red Cross (since 1993) and the Canadian Safe Boating Council (since 2002). An evaluation of these contributions covering a period from 1999 to 2005 was completed by Transport Canada in 2006. The evaluation questioned whether the safety problems in recreational boating were large enough to warrant a prevention-based program and highlighted the need to determine the types of interventions required.
Program Profile
The authority to undertake the BSCCP is established in the Canada Transportation Act. The program, created in 2008, provides financial contributions to external organizations to carry out projects that are selected through a ‘call-for-proposals’ process. In 2012, the BSCCP was placed under the joint responsibility of Marine Safety in the Safety and Security Group and Stewardship and Sustainable Transportation in the Programs Group. Under this arrangement, Marine Safety focuses on the policy matters of the program while the Programs Group leads program planning and implementation. Starting in 2013-2014, the program will be resourced as an on-going contribution program with an annual budget of approximately $1 million. Key components of the BSCP (outlined in foundational documents) are as follows:
Program objectives
- Raise public awareness of boating safety issues,
- Increase the level of pleasure craft operators following safe boating practices,
- Improve national boating incident data quality and collection systems to support evidence-based awareness and education initiatives, and ultimately
- Contribute to a reduction in loss of life, injuries and property damage, due to boating accidents.
Program target groups
- Power-boating,
- Anglers and hunters,
- Paddling,
- Sailing, and
- Other small vessels.
Eligible recipients
- Not-for-profit non-government organizations,
- Public safety organizations,
- Educational institutions,
- Healthcare institutes,
- Entities associated with facilities specializing in safety and medical research,
- Enforcement services, and
- Provincial, territorial and municipal governments.
Eligible projects
- Conduct outreach activities to encourage safe boating practices and compliance with regulations through education, awareness and information, i.e. promotional awareness/public education campaigns;
- Conduct research, undertake studies and analysis, which contribute to a better understanding of boater behaviour and attitudes in an effort to develop and advance evidence-based education and awareness initiatives;
- Collect and analyze boating-related incident data, i.e. injury, fatality and enforcement activities, review trends and compile information for correlation with data obtained from other repositories, prepare associated reports with recommendations for strategies that offer a better understanding of boating-related risks and safety issues; and
- Conduct polls/surveys and establish focus groups to determine more effective ways of promoting boating safety awareness programs.
Program Resources
Over the time frame of the evaluation, $1,394,806 was paid-out to 16 organizations in support of 32 approved projects.
Table 1: Expenditures of the Boating Safety Class Contribution Program, 2008-2009 to 2012-2013
The dash ( - ) indicates that the specified entry is not applicable.
| 2008-2009 | 2009-2010 | 2010-2011 | 2011-2012 | 2012-2013 | Total | |
|---|---|---|---|---|---|---|
| Total Approved Budget | $245,000 | $251,000 | $245,000 | $260,000 | $275,000 | $1,276,000 |
| Operating Expenditure | $45,000 | $51,000 | $45,000 | $60,000 | $75,000 | $276,000 |
| Grants and Contributions | $200,000 | $200,000 | $200,000 | $200,000 | $200,000 | $1,000,000 |
| Full Time Employees | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | - |
| Approved Projects (#) | 0 | 6 | 12 | 8 | 6 | 32 |
| Actual Contribution | 0Footnote2 | $226,164 | $384,884 | $350.579 | $433,179 | $1,394,806Footnote3 |
| Total Expenditure | 0 | $277,164 | $429,884 | $410,579 | $508,179 | $1,625,806 |
Source: 2008 foundational document and Program files.
Evaluation Approach
The evaluation examined the relevance and performance of the BSCCP , including the extent to which it has achieved its expected outcomes, as presented in the logic model that can be found in Annex 1.
Scope and Issues
While funding for the BSCCP was approved in 2008, funding agreements were not established until 2009. Due to this fact, the evaluation focused mostly on the latter four fiscal years of the program (2009-2010 to 2012-2013).
The evaluation addressed the core issues laid out in Treasury Board Secretariat’s 2009 Directive on the Evaluation Function:
- Continued need/rationale for the program,
- Alignment with government priorities and departmental strategic outcomes,
- Alignment with federal roles and responsibilities,
- Achievement of expected outcomes, and
- Efficiency and economy in achieving expected outcomes.
The BSCCP is a small contribution program compared to TC’s other contribution programs. It was redesigned in 2008, in part as a response to the 2006 evaluation. Because the 2006 evaluation raised questions about the need for the two contributions in place at that time (the BSCCP ’s predecessor) and the lack of data to demonstrate program effectiveness, this evaluation focused on program relevance and impact.
In terms of expected outcomes, the evaluation assessed the program’s achievement of the following:
- Increased awareness of the importance of following safe boating practices,
- Increased number of recreational boaters following safe boating practices, and
- Reduction in loss of life and injury (ultimate outcome)
The evaluation issues, outcomes and indicators used in the evaluation are outlined in Annex 2.
Lines of Inquiry
The evaluation team used multiple lines of enquiry to inform the overall evaluation. Interviews were used to develop information that was not otherwise available, to provide context to outcome data obtained through other methodologies, and to identify and clarify evaluation issues. Other lines of inquiry such as a review and analysis of statistical data, a newspaper scan and an internet scan were used to assess questions of ongoing need and impact. In all, nine lines of enquiry were used in this evaluation:
Documents Review
Policy, strategic planning and reporting documents that are related to the program context were reviewed. Some of the key documents include strategic planning documents, statutes and regulations, Speeches from the Throne, federal budgets, departmental reports on plans and priorities and departmental reports on performance.
File Review
Contribution agreements, final project reports produced by recipients, and financial records were reviewed to assess outputs, impacts, and the utilization of resources.
Review and Analysis of Statistical Information
Statistical data available from Transport Canada, Statistics Canada, BSCCP or OBS funded projects, and other countries were compiled and analyzed to understand trends, patterns and causes of recreational boating accidents; trends in compliance with safety regulations; the size of the boating population in Canada; and trends in boating safety awareness. This includes data on boating fatalities from the Canadian Red Cross, data on serious injuries from the National Trauma Registry, data on Criminal Code violations from the Canadian Centre for Justice Statistics (Statistics Canada), and data on boaters’ attitudes/intended behaviours from the Canadian Safe Boating Council.
Interviews
Fourteen interviews were conducted. This included 9 funding recipients, 1 program staff at headquarters and 4 in the regions.
Counterfactual Analysis
In order to assess need, a portion of the funding recipients were asked the extent to which their boating safety activities would change, in the absence of BSCCP funding. Counterfactual analysis was also used to assess need by ascertaining the extent to which there would be promotional materials and information in the print media and on the internet, without BSCCP funding.
Literature Review
Reports from federal, provincial and foreign governments, publications of the boating industry, academic literature and research, and best practices were reviewed.
Media Scan
A newspaper media scan was conducted to assess the prominence of the issue of boating safety in the media and to develop an understanding of program need and reach.
Internet Scan
An internet and website search was conducted to assess the amount of recreational boating safety awareness materials available to Canadians online and to ascertain whether there was duplication of effort or responsibilities.
Case Studies
Case studies were undertaken to investigate questions about efficiency and economy and the quality of project performance measurement.
Limitations
Available data was insufficient to assess the impact of the BSCCP on expected results. Few of the recipients conducted polls, surveys or research to measure the effectiveness or even the reach of their projects as per the contribution agreements. To compensate for this, the evaluators estimated program reach through newspaper scans and survey data (on the understanding that in the case of awareness, the first outcome in the results chain, reach and impact can be conflated). Counterfactual analysis and interviews were also used to compensate for the paucity of usable performance data.
Detailed Evaluation Findings
This section presents the detailed findings on relevance, performance, economy and efficiency.
Relevance
Relevance is assessed by examining the program’s alignment with federal roles and responsibilities, the extent to which it addresses a demonstrable need and its linkages with federal government priorities and departmental strategic outcomes.
Alignment with Federal Roles and Responsibilities
Finding 1: Boating safety is a federal responsibility, although the Office of Boating Safety relies on federal, provincial and municipal police forces to enforce compliance with TC’s regulations.
The Constitution Act of 1867, Section 91 (10), assigns navigation and shipping to the exclusive Legislative Authority of the Parliament of Canada. The Canada Shipping Act, 2001 is the legislative framework that provides the Minister of Transport with the authority to oversee recreational boating safety. Specifically, the Act provides the Minister of Transport with the authority to: (a) “promote safety in marine transportation and recreational boating,” and to (b) “develop a regulatory scheme that encourages the viable, effective, and economical use of Canadian waters by recreational boaters” (pp. 4-5). It applies to Canadian vessels operating in all waters and to all vessels operating in Canadian waters. The Act and its regulations apply to pleasure craft and recreational boating in four major ways: certification of operators, registration of pleasure craft, safety equipment to be carried aboard, and operation of pleasure craft.
While overseeing and promoting recreational boating safety is clearly a federal responsibility, the Canada Shipping Act, 2001 permits any person designated by the Minister of Transport to enforce the Act and its regulations. The Department relies on federal, provincial and municipal police forces to do so. Violations to the boating regulations are processed under the Contraventions Act.Footnote4 Under this Act, enforcement authorities can ticket offenders on the spot. Tickets can be issued for offences such as not having the required safety equipment on board, disobeying speed limits or careless operation.
The Canada Transportation Act, 1996 establishes TC’s responsibility for instituting policies and programs to achieve safety objectives in transportation. The BSCCP has been approved under the authority of the Canada Transportation Act (section 48).
Ongoing need
Previous evaluation of TC’s contributions to the Canadian Red Cross Society and the Canadian Safe Boating Council (CSBC) indicated that while there was some evidence to support the need for a prevention-based program, the need for the awareness program in place at the time was not clear. To assess the ongoing need for the BSCCP in the current evaluation, the evaluators examined statistics on the size of the recreational boater population, hospitalizations for major injuries and recreational boating fatalities in Canada and comparable countries. The evaluators also undertook an analysis of the boating safety information and promotional materials available on the Internet and in newspapers to get an understanding of the availability of these materials, who was producing/funding them and whether there was duplication or overlap with other organizations.
Finding 2: The recreational boating population in Canada is large. In 2012, 11.8 million adult Canadians boated at least occasionally and the average annual number of fatalities over the period of the evaluation was between 121 and 127.
The National Marine Manufacturers Association of Canada estimates, based on a 2011 survey of 2,000 adult respondents, that 38% of Canadian adults, or 10.5 million, went boating at least once in the past twelve months. The CSBC estimates, based on a 2012 survey of 1,016 Canadian adults, that 43% of adult Canadians (or 11.8 million) participate in recreational boating at least occasionally. These numbers provide an indication of the size of the population at risk of a boating injury or fatality.
Recent (2013) Canadian data reveal that of the 15,186 hospitalizations for major injuries in 2010-2011, only 39 (0.26%) were caused by water transport and only 26 (0.17%) were caused by drowning.Footnote5 By contrast, 5,948 of these hospitalizations were the result of unintentional falls and a similar number were due to motor vehicle collisions. Looking at injuries due to sports and recreation activities, boating is not even listed. A study on Ontario injuries suggests that boating injuries seen at emergency departments are generally not severe, since over 90% of individuals who visited an emergency department as a result of a boating-related injury were discharged to their place of residence rather than admitted to hospital.Footnote6
The average annual number of boating fatalities fell from 220 to 127 between 1991-1995 and 2006-2010 (see Figure 1). The majority (86%) of these fatalities occurred as a result of recreational boating incidents. The evaluators estimate that the average annual number of boating fatalities in Canada is currently about 121 people.Footnote7
Figure 1: Recreational and Other Boating Fatalities in Canada, 1991 to 2015Footnote8
Source: Lifesaving Society Canada (2013) and Canadian Red Cross (2011).Footnote9
Text version of Figure 1: Recreational and Other Boating Fatalities in Canada, 1991 to 2015
Figure 1 displays a bar graph with an overlain line plot. The bar graph illustrates the average amount of recreational and other boating fatalities in relation to the population, from 1991 projected into 2015. The bar segment of the graph represents boating deaths while the line overlay represents the population. Each bin of the graph spans a five year period. 1991-95 had the highest fatality to population ratio at 220 fatalities to a population of approximately 27.3 million. The death ratio declines each year as 1996-00 had a ratio of 179 to 29.8 million followed by 2001-05 with a ratio of 136 to 30.0 million and 2006-10 has a ratio of 127 to 31.6 million. This pattern is predicted for the period 2011-15 to continue with the projected death ratio at 121 to 33.7 million.
As Figure 2 demonstrates, most of the boating fatalities have occurred in the three largest provinces.
Figure 2: Average Annual Number of Boating Fatalities by Province, 1991 to 2008
Source: Canadian Red Cross (2012).
Text version of Figure 2: Average Annual Number of Boating Fatalities by Province, 1991 to 2008
Figure 2 is a bar graph which displays in decreasing order the average annual amount of boating related deaths by province / territory from 1991 to 2008. Most deaths occur in Ontario with an average of 47.6 deaths followed by British Columbia with an average of 31.3 deaths, Quebec with an average of 28.0 deaths, Nova Scotia with an average of 11.3 deaths, Newfoundland with an average of 10.9 deaths, Manitoba with an average of 8.1 deaths, Alberta with an average of 8.0 deaths, Saskatchewan with an average of 6.8 deaths, New Brunswick with an average of 6.1 deaths, Northwest Territories with an average of 4.9 deaths, Yukon with an average of 2.0 deaths, Prince Edward Island with an average of 1.4 death, and last Nunavut with an average of 0.5.
Finding 3: Recreational boating fatalities per 100,000 are similar in Canada to the rates in the United States, Australia and New Zealand.
Data on boating fatalities per 100,000 suggest that the boating fatality rate for Canada is similar to what has been observed in Australia, New Zealand, and the United States. Although international comparisons are complicated by different data collection methods and time frames, Table 2 shows that recreational boating fatality rates in Canada, the United States, Australia and New Zealand vary by a maximum of 0.23 deaths per 100,000 people. More precise comparisons would require comparative data on exposure to risk (e.g. the size of the registered boating population and the frequency of boating).Footnote10
Table 2: Average Annual Number of Boating Fatalities per 100,000 People
| Country | Average Fatalities per 100,000 people | Details |
|---|---|---|
| Australia | 0.26 | Statistics from 1999-2004. Includes recreational and small commercial boating fatalities. The percentage of fatalities that were from recreational boating is not known but 81% of vessels involved in fatal accidents were recreational vessels.Footnote11 |
| Canada | 0.36 | Statistics from 2006-2008. Recreational fatalities only.Footnote12 |
| United States | 0.23 | Statistics from 2008. Recreational fatalities only.Footnote13 |
| New Zealand | 0.46 | Statistics from 2000-2005. Recreational fatalities only.Footnote14 |
Finding 4: The BSCCP is the primary source of funding for boating safety awareness in Canada. Were it not for the funding provided by the BSCCP (and the OBS ), there would be little Canadian boating safety information and promotion of boating safety on the Internet.
While BSCCP -funded projects generally do not have a high Internet profile, the Office of Boating Safety (OBS) does. The OBS was the first search result when the search terms “safe boating,” “boating safely,” and “boating regulations” were used. Were it not for materials funded through the OBS , and often replicated in whole or in part at other sites, there would be a much smaller amount of Canadian material available. Examples of some of these products are Safe Boating Guide, Surviving in Cold Waters, Kayaking Safety Guide and Boating Immersion and Trauma Deaths in Canada. The Safe Boating Guide contains, among other things, information on Canadian regulations. This is important to the present evaluation because in the near future, all of the OBS ’ awareness materials will be produced through the BSCCP .
Most of the websites having to do with Canadian boating or boating safety provide a link to this OBS website, including some provincial government departments (e.g. the Ontario Ministry of Transport and the B.C. Ministry of Forests, Land and Natural Resources Operations) along with Fisheries and Oceans Canada and Parks Canada. While these aforementioned sites do provide some original boating safety information within the context of their own mandates (e.g. drinking and boating legislation in Ontario), they clearly defer to Transport Canada as the authority on recreational boating safety by providing a link to the OBS website.Footnote15
After the OBS , the most prominent search results were web sites from the United States, Pleasure Craft Operator Card (PCOC) distributors, and organizations funded by the BSCCP , including the Canadian Safe Boating Council (CSBC), the Canadian Power and Sail Squadrons (CPSS), the Canadian Red Cross (CRS), the Conseil québécois du nautisme (CQN), and Mothers Against Drunk Driving (MADD).
PCOC distributors often only provide information on how to get accredited and do not usually provide any substantial information on boating safety. American sites are also not very useful to learn about boating safety due to the fact that safe boating requirements in the two countries are different. The OBS and organizations such as the CSBC , CPSS , CRS , and CQN provided the most useful information (The CSBC , CQN , along with MADD received funding from the BSCCP specifically for website development in 2010-2011 or 2011-2012).
The most salient BSCCP -funded project on the Internet is the Safe Boating Awareness Week (SBAW)/North American Safe Boating Awareness Week (NASBAW) campaign on the CSBC ’s website. The materials on this website include the 2012 Safe Boating Awareness Survey results, the campaign announcements and safe boating messages, among other things. The noticeable presence of BSCCP -funded safety awareness materials will probably increase in the future, since all OBS awareness materials will be produced through BSCCP funding in the future.
If the boating safety awareness materials and information evident on the Internet which have been produced through the OBS / BSCCP were deleted, there would be very little remaining Canadian boating safety information and promotion. This counterfactual was given further weight from interview data, which revealed that it is unlikely that the organizations that the BSCCP funds to produce and promote the boating safety information available on the Internet (and elsewhere) would be able to find alternative sources of funding to continue their work.Footnote16
Their view was that no such alternative sources exist (e.g. from either provincial or municipal governments or the private sector). The evaluators did not find any evidence to the contrary. A number of interviewees similarly stated that they would not be able to provide the boating safety services and products they have provided, without BSCCP funding, because they are largely volunteer-based with no means of raising revenue. As for private sector funding, some interviewees raised the point that corporations are more likely to want to promote the fun and sport component of boating rather than the safety component. Thus, if there is a need for Canadian boating safety awareness materials to be on the Internet, where more and more Canadians are turning for their information, the BSCCP is contributing to the fulfillment of this need.
Evaluators also found that there is very little being done in the area of boating safety awareness by other federal departments or levels of government, although police forces, notably the marine squads of the Ontario Provincial Police and the RCMP do some boating safety promotion.
Finding 5: Boating safety awareness does not get much coverage in the print media.
The evaluators undertook a newspaper scan to determine the extent to which boating safety was an important public issue and how much boating safety awareness materials or messaging was available in newspapers. Since the CSBC had undertaken a media scan in 2012, the evaluators built a scan around theirs.Footnote17
Over the 67 days to which the scan refers, there were on average 3.0 community newspapers and 0.9 daily newspapers per day that published an article on boating safety in Canada. Most of the dailies that published such an article were small in terms of distribution, though the evaluators did find some medium sized ones that did so, such as the Times-Colonist and the London Free Press. Looking at Canada’s top five daily newspapers in terms of circulation, there were no articles about boating safety in the Globe and Mail, the National Post or the Toronto Star over the time frame of the scan, though there were 2 in the Gazette and 1 in the Vancouver Sun.
The evaluators estimate, based on the scan, that 7.7 million people were potentially reached through newspapers with boating safety awareness content over the boating season, which translates into 3.3 million boaters (see Annex 3). Although newspapers are not the only media used to promote boating safety over the boating season, most media content, especially important media content, is duplicated across media modes. As such, newspapers can be used as a proxy measure of the salience of boating safety in the media overall. Results suggest that the overall salience of boating safety in the media is low.
Finding 6: Although the issue of boating safety is not prominent in newspapers, its presence would be lower were it not for the CSBC and the other organizations the BSCCP funds.
Table 3 presents the number of newspaper articles covering a boating safety awareness issue between May 1 and July 6 of 2012, along with the organizations that were mentioned in them, if any. A total of 122 community newspapers published one or more articles (206) with boating safety content over the time frame of the scan. Twenty-four daily newspapers published 57 articles. This means that on average, each newspaper that did publish an article, also published a second one (2.2 and 1.7 articles, respectively) over the time frame of the scan.
The italicized organizations in Table 3 are ones that have received BSCCP funding. As the data show, the majority of newspaper articles about boating safety are associated with organizations the BSCCP funds. Thus, the presence of the issue of boating safety in the newspapers would be much lower were it not for the CSBC and the other organizations the BSCCP funds.
Table 3: Organizations Referenced in Newspaper Articles Relating to Boating Safety
The dash ( - ) indicates that no articles were found on the specified topic.
| Organization | # of Articles in
Community Newspapers |
# of Articles in
Daily Newspapers |
|---|---|---|
| Canadian Safe Boating Council | 142 | 3 |
| Ontario Provincial Police | 18 | 5 |
| Canadian Power and Sail Squadrons | 13 | 1 |
| Transport Canada | 10 | - |
| Boatsmart! (website of the CSBC) | 6 | 7 |
| Ontario Conservation | 5 | - |
| Royal Canadian Mounted Police | 4 | 5 |
| Local sailing club | 3 | - |
| No organization | 2 | 9 |
| Local police | 1 | 10 |
| Mustang Survival | 1 | - |
| Quebec Boating Council | 1 | - |
| Emergency services | - | 4 |
| Red Cross | - | 3 |
| Lifesaving Society of Canada | - | 3 |
| Provincial government | - | 2 |
| United States Coast Guard | - | 1 |
| Industry | - | 1 |
| Insurance Bureau of Canada | - | 1 |
| Local government | - | 1 |
| Mothers Against Drunk Driving | - | 1 |
| Grand Total | 206 | 57 |
Alignment with Government and Departmental Priorities and Departmental Strategic Objectives
Finding 7: The Government of Canada has shown its continued support for boating safety and the objectives of the BSCCP are clearly aligned with the Department’s strategic objectives.
The government has communicated its continued support for boating safety through program funding renewal and public education. In 2012, the government renewed funding for the BSCCP for 2014 and beyond. Also, the government has led nation-wide public-education campaigns for boating safety, such as Safe Boating Awareness Week, each year for a number of years.Footnote18
The objectives of the BSCCP are clearly aligned with the strategic objectives of Transport Canada ( SO 3 – a safe and secure transportation system). Recreational boating safety is a component of marine transportation safety ( PAA 3.2).
Performance
Program Impact
The BSCCP ’s impact over the evaluation time frame has likely been limited by the fact that it is a relatively small program, even though it makes use of organizations that can tap into community resources. The BSCCP ’s level of funding amounted to only one cent for every Canadian resident (approximately $350,000 annually). A comparable federal program in the United States for boating safety awareness was twice as large, amounting to two cents for every U.S. resident ($5,366,550).
Finding 8: Evaluators estimate program reach in 2012 to be 4.8 million boaters who boat at least occasionally, or 41% of such boaters.
Evaluators attempted to assess the reach of the BSCCP . They did so through a combination of the estimates they produced for newspapers carrying boating safety awareness content and survey measures (see Annex 3). They estimate program reach to be around 4.8 million boaters who boat at least occasionally, or 41% of them, which is considerable for a small program.
Finding 9: Evaluators were not able to find any measurable evidence of program impact on boating safety attitudes, behaviour or overall awareness in the aggregate.
In its performance measurement strategy, the BSCCP identified surveys of recreational boaters as a data source for measuring the extent to which it contributed to the increased awareness of the importance of following safe boating practices and an increased number of boaters following safe boating practices. The Canadian Safe Boating Council’s (CSBC) Safe Boating Awareness Surveys on attitudes and intended behaviours of boaters provide a measure of the overall impact of BSCCP -funded projects over time.
Results in Table 4 point to no change or small unfavourable changes in the attitudes or intended behaviours of boaters over the time frame of the evaluation.
Table 4: Attitudes and Intended Behaviours of Boaters, 2008 to 2012: Percentage Who Agree (Top 2 Box)
| Question: Please indicate how much you agree or disagree that each of these statements applies to you, i.e. describe things you are doing or intend to do this year. | Year of Survey | |||
|---|---|---|---|---|
| 2008 | 2009 | 2010 | 2012 | |
| N=487 | N=442 | N=437 | N=436 | |
| %Footnote19 | ||||
| Drinking & boating:
I am not going to drink any alcoholic beverages while out on the water in a boat this season. |
65 | 70 | 66 | 61 |
| Wearing your
PFD
:
I will wear my PFD or lifejacket all the time when I’m out on the water in a boat this season. |
66 | 64 | 59 | 54 |
| I am going to strongly encourage everyone else who is out in a boat with me this season to wear their PFD or lifejacket, no matter what their age or swimming ability may be. | 68 | 66 | 62 | 55 |
| Preparedness:
I will always check my boat over every time before I go out on the water this season; including making sure I have enough PFD s/lifejackets on board. |
n/a | 56 | 55 | 50 |
| I’m going to review my pre-departure checklist, every time I go out on the water this season. | 40 | 33 | 34 | 29 |
| I have a pre-departure checklist written down that identifies the things to check every time before I head out on the water. | n/a | 22 | 25 | 25 |
| Cold water:
I’m going to make a point of being better prepared for the possibility of falling into cold water this season, by wearing my lifejacket. |
59 | 50 | 53 | 43 |
| I don’t worry about the temperature of the water, as I don’t boat during what I consider to be the cold water season. | n/a | 30 | 30 | 34 |
| I feel that I am well prepared for the possibility of cold water immersion/falling into cold water while out in a boat. | n/a | 32 | 36 | 24 |
For example, there was a steady decrease since 2009 in those who did not intend to drink any alcoholic beverages while boating. Similarly, there was a steady decrease since 2008 in the percentage of boaters who strongly agreed that they would wear their PFD or lifejacket at all times while boating and who reported that they would encourage others to wear their PFD or lifejacket at all times. Other unfavourable trends were observed in the percentage of people who agreed that they would check their boat for safety equipment every time they went boating, would review pre-departure checklists, would make a point of being better prepared for the possibility of falling into cold water and who feel they are well prepared for the possibility of falling into cold water. In fact, there are no statistically significant favourable trends.
In addition to these trends, according to the same CSBC data, there has been a decline in the percentage of people who said that they always wear a PFD or lifejacket when out in a boat, from 56% in 2008 and 2009, to 52% in 2010, to 47% in 2012. Further, Pleasure Craft Courtesy Check data indicate that there was no change in the percentage of checked boaters that had an awareness of cold water (72% for both 2009 and 2012).
Given these trends in attitudes and intended and reported behaviours between 2008 and 2012, it would be difficult to argue that the NASBAW / SBAW campaigns or the BSCCP have had an impact on the attitudes or behaviours of Canadian boaters over time, at least not in the aggregate.Footnote20 It is still possible, however, that the NASBAW / SBAW , other campaigns and the BSCCP have contributed to maintaining the level of awareness and appropriate boating safety attitudes and behaviours in the Canadian boater population. This is because the size of the boating population has grown over time; and new people are always entering it, e.g. immigrants, teenagers, young adults, adults in general and retirees. Furthermore, these data do not provide a perfect test of the question of impact. For example, the demographic composition of the samples used has changed over time and this compositional change could very well have affected survey results as regards boating safety attitudes, behaviour and awareness. The sample sizes for any given year were not large enough to control for the impact of these compositional changes in the samples on results.
Program Delivery
The BSCCP uses a competitive funding process. After the Call for Proposals, a Marine Safety Proposal Review Committee convenes to review project applications to determine if they meet mandatory eligibility criteria (now called eligibility assessment criteria). To be eligible, a proposed project must address at least 1 of the criteria in each of a) program objectives, of which there are four, b) program target groups, of which there are five, and c) eligible project categories, of which there are four. In addition, applicants must be one of seven eligible types of program recipients. The evaluators examined the outputs resulting from these selection criteria.
The most common outputs from BSCCP funding, as demonstrated in Table 5, were media campaigns/releases and brochures posters, flyers, signs and pamphlets (both 42%); followed by events, seminars and presentations (39% of projects); and newsletters/publications or advertisements in trade or boating magazines (32%). As was evident in Table 3 in the case of newspapers, media releases tend to be local in nature.
Table 5: Number and Percentage of Projects that Produced a Given Output Type, 2009-2010 to 2012-2013
The dash ( - ) indicates that the specified entry is not applicable.
| Major Outputs | Number of projects | % |
|---|---|---|
| Media campaigns/releases/public service announcements | 13 | 42 |
| Brochures/posters/flyers/signs/pamplets | 13 | 42 |
| Events/seminars/presentations | 12 | 39 |
| Newsletters/publications/advertisements in trade or boating magazine/newsletter | 10 | 32 |
| Survey, poll, focus group | 9 | 29 |
| Safety guides | 9 | 29 |
| Promotion products – e.g. key chains, pins | 9 | 29 |
| Web information/website | 8 | 26 |
| Courtesy checks | 6 | 19 |
| Clinics/courses in aquatics, paddling, etc. | 6 | 19 |
| Behavioural research | 6 | 19 |
| Volunteers trained and engaged | 5 | 16 |
| Data ( PCCC checks) | 4 | 13 |
| Video production | 3 | 10 |
| Total projects | 31 | - |
Note: A single project typically has multiple outputs.
Finding 10: The BSCCP has funded a narrow range of project and recipient types, relative to what is allowed in its Terms and Conditions.
As Table 5 demonstrates, the projects funded by the BSCCP over the evaluation time period were primarily outreach projects (project category 1). Most of the polls and surveys (project category 3) undertaken with BSCCP funding were designed to try and establish the reach of a particular project’s boating safety campaign, though a few were designed to establish whether a particular presentation or video was effective. None helped “establish more effective ways of promoting boating safety awareness campaigns,” for which the project category of polls and surveys was intended. No projects involved the collection and analysis of boating incident data (project category 4), listed as a BSCCP objective.
Although the BSCCP has seven possible recipient types, most of the applications for funding came from not-for-profit non-government organizations (NGOs), most of which are volunteer-based and often do not have the capacity to do quality research or data collection. Only one educational institution applied and received funding over the time period of the evaluation. Three enforcement agencies applied but did not receive funding under the program. No public safety organization, healthcare institution, or organization specialized in safety and/or medical research applied for or received funding. Being relatively new and having a small applicant base may have contributed to the fact that a narrow range of project and recipient types were funded. This is a recognized challenge, and one best practice is to expand the applicant pool by marketing the program to potential applicants.
Finding 11: There are some research and data gaps regarding the risks in the Canadian boating population that have not yet been addressed.
Although the BSCCP , as a small program, cannot be expected to address all of the data or research gaps that could improve the effectiveness and efficiency of boating safety awareness initiatives in Canada, it has the mandate to address these gaps. The BSCCP has made some gains in funding the CSBC to benchmark and track boating safety awareness and attitudes/intended boating behaviour, and to a lesser extent, reported behaviour between 2008 and 2012.Footnote21 The evaluators found no survey data on what boaters in the Canadian population actually know about TC regulations, including drinking and boating legislation. As for boating incident data, the most recent national data on boating fatalities is only for 2010;Footnote22 the most recent data on the provincial distribution of boating fatalities is for 2008, the start of the evaluation time period; and there is very little data available on boating related injuries. At the time of drafting this report, the U.S. had published fatality and injury data that is as recent as 2012.Footnote23
The most significant boating safety related data gap, judging from a literature review undertaken by the evaluators, is the absence of Canadian life-jacket/ PFD wear rates from observational studies. Trend data on life jacket/ PFD wear rates from observational studies are available in the United States and Australia.Footnote24 In the absence of these Canadian data, especially over time, it is difficult to understand trends in boating fatalities and their causes, as well as whether certain intervention strategies are working (e.g. public awareness campaigns regarding the wearing of PFD ’s). For example, it is known that some 80% of Canadian boaters who drowned were not wearing a life jacket/ PFD . However, this percentage has remained the same overtime even though boating related fatalities have declined substantially.Footnote25 Is a greater percentage of the boating population wearing life jackets/ PFD s? According to survey data on reported behaviour, there has actually been a 12 percentage point decline over the evaluation time period in the percentage of boaters who say they always wear a life jacket/ PFD when out in a boat.Footnote26
Other types of data available in the United States and Australia but not in Canada are the number and type of boats in use, hours of participation by boat type, the number and percentage of children involved in recreational boating, the distribution of recreational boaters across the country, the experience level of boaters by type of boating activity,Footnote27 and the number of fatalities per registered vesselFootnote28 all of which could be useful to understand exposure to risk, trends in boating fatalities and the types of interventions that are needed.Footnote29
Finding 12: The applicant base does not appear to be large or diverse enough to expand the types of projects and recipients that receive funding, to direct funding into targeted priority areas or meet all program objectives.
A 2010 review undertaken by the BSCCP stated that the program was expecting less than ten organizations to apply for funding that year and that over the course of its three years of implementation, it was averaging 15 organizations. Except for 2009-2010 when 6 different organizations were funded, the BSCCP has funded 11 different organizations annually. Since the BSCCP has only funded 16 different organizations over the time period of the evaluation, this means the BSCCP is funding the same organizations repeatedly.Footnote30 The fact that the overwhelming majority of successful applicants are NGO s with outreach projects further suggests that the applicant base is neither large nor diverse. In order to address some of the research and data gaps, the BSCCP might have to actively market the program to expand the range of eligible recipients that apply, along with the types of proposals that are submitted. The same applies to directing funds to meet all policy objectives (i.e. to improve national boating incident data quality and collection systems) and priority areas (program target groups); less than one-third of program funding went to program target groups and even then it went to only two of the five program target groups (paddlers, and anglers and hunters ).
A small applicant base might also explain, in part, why, out of the 32 projects funded by the BSCCP ,Footnote31 there were 19 outputs that were either not completed or for which the recipient gave no evidence of completing, i.e. a larger base would allow for the selection of better quality recipients. This does not include the many cases for which outputs were only partially completed, e.g. not as many PCCC s, posters, or brochures delivered as promised; press releases or symposiums for only 1 initiative when 2 or more were promised; promising data that was impossible to obtain; and not reaching as many people or the types of people (e.g. Aboriginal peoples) promised. The size of the funding agreement was also a factor in project completion. Recipients of small funding agreements (under $20,000) completed 81.0% of the activities in their proposals, at least partially, while recipients of funding agreements for over $20,000 completed 92.5% of them.
Finding 13: While the BSCCP has made some progress in the area of performance measurement, its expectations might not have been in line with its recipients’ capacity.
The BSCCP ’s performance measurement strategy relies on recipient organizations’ measuring the performance of their BSCCP -funded projects. However, the output that was most often not delivered at all was “survey, poll or focus group” for the purposes of performance measurement. Of the 15 funding agreements that had this type of output as a commitment (16 did not require this type of output), only 9 resulted in the output being delivered. Of the 12 funding agreements that were for an average of $20,000 a year or less, three involved proposals to do surveys or polls to measure project performance and none of them were conducted. Recipients of all three funding agreements valued at $100,000 per year or more conducted polls or surveys to assess their performance. Most of the funding agreements approved by the BSCCP were in the low dollar range. Almost 70% of the funding agreements were under $50,000, 42% were under $30,000 and 29% were less than $20,000.
Few recipients were even able to provide a reasonable measure of the reach of their project and fewer still were able to demonstrate the impact of their project on attitudes or behaviour. For the most part, only large national organizations with large funding agreements were able to give a reasonable account of their project’s performance.
Efficiency and Economy
Finding 14: The BSCCP could improve its efficiency by aligning the distribution of funds more closely with the distribution of specific risks in the boating population.
Geographic Distribution
Table 6 shows that from 2009-10 to 2012-2013, 57.31% of BSCCP funds were approved for projects that were national in scope while 42.69% were approved for projects targeting a specific region (e.g. Atlantic), province or small locality within a province (e.g. Regional District of Alberni-Clayoquot).
Based on the distribution of boating fatalities between 1991 and 2008, the most recent data for which provincial break-downs are available, Quebec received a much larger share of the BSCCP ’s regional distribution of funds relative to the percentage of boating fatalities that occurred in that province (38.86% compared to 16.78%, respectively). British Columbia and Ontario were under-funded according to their share of fatalities, and to a lesser extent, so were the other provinces and territories, except for Newfoundland and the Northwest Territories, which were over-funded.Footnote32
Table 6: Geographic Distribution of BSCCP Funds from 2009-2010 to 2012-2013 and Boating Fatalities from 1991 to 2008
The dash ( - ) indicates that the specified entry is either non-applicable or did not receive any funding.
| Geographical Scope | Approved SpendingFootnote33 | Distribution of All Approved Spending | Distribution of Regionally Targeted Spending | Distribution of Boating Fatalities (1991-2008) |
|---|---|---|---|---|
| $ | % | % | % | |
| National | 939,787 | 57.31 | - | - |
| Quebec | 272,000 | 16.59 | 38.86 | 16.78 |
| Newfoundland | 115,625 | 7.05 | 16.52 | 6.56 |
| Northwest Territories | 95,000 | 5.79 | 13.57 | 2.96 |
| Ontario | 87,860 | 5.36 | 12.55 | 28.53 |
| British Columbia | 54,500 | 3.32 | 7.79 | 18.74 |
| New Brunswick | 25,000 | 1.52 | 3.57 | 3.63 |
| Nova Scotia | 25,000 | 1.52 | 3.57 | 6.76 |
| Prince Edward Island | 25,000 | 1.52 | 3.57 | 0.87 |
| Atlantic ProvincesFootnote34 (including Newfoundland) | 190,625 | 11.62 | 27.23 | 17.82 |
| Alberta | - | - | - | 4.79 |
| Manitoba | - | - | - | 4.83 |
| Saskatchewan | - | - | - | 4.06 |
| Nunavut | - | - | - | 0.30 |
| Yukon | - | - | - | 1.20 |
| Total | 1,639,772 | 100.00 | 42.69 | 100.00 |
Source: Canadian Red Cross (2011).
Distribution by Type of Boating Activity
The BSCCP identified the five program target groups in Table 7, “in order to advance boating safety awareness and education in those areas most often associated with boating-related fatalities.” This decision was based on the results of 10 years of research on drowning and water-related injuries by the Canadian Red Cross.Footnote35
Table 7: Approved BSCCP Funding by Type of Boating Activity and Region, 2009-2010 to 2012-2013
The dash ( - ) indicates that the specified entry did not receive any funding.
| Type of Boating Activity | Percentage of Approved
FundingFootnotea |
Percentage of Recreational
Boating Fatalities (1991-2008) |
|
|---|---|---|---|
| Program Target Groups | 26.1% | Footnoteb | |
| 1 | Power Boaters | - | 54.6% |
| 2 | Anglers & Hunters Using a Boat | 16% | 41.6% |
| 3 | Paddling Community (Canoe, Kayak, Raft, Rowboat) | 10.1% | 30.5% |
| 4 | Sail Boats | - | 3.5% |
| 5 | Other Small Vessels | - | 5.2% |
| All Boaters | 74% | 100% | |
| Small Powerboats | - | 35%Footnotec | |
Footnotes
- Footnote a
-
Some projects proposed to reach other target groups, but were in fact designed to reach all boaters.
- Footnote b
-
Not mutually exclusive and cannot be added up.
- Footnote c
-
Represents at least 62.6% of all power boating fatalities. Likely represents closer to 80% of all power boating fatalities.
No projects targeted the priority group involved in the most fatalities, power boaters, or in particular, users of small open power boats, which represent the greatest risk according to three reports on boating fatalities, published by the Canadian Red Cross Society over the last seven years. The most recent report states that “small open powerboats are the most frequent type of boat associated with recreational boating fatalities.” Footnote36 The report also states that 62.6% of those who died between 1991 and 2008 in recreational boating accidents on powerboats were on small powerboats, and the figure would be closer to 80% if a large portion of powerboats of unknown size were included.
Finding 15: Large scale public media campaigns are more efficient at reaching boaters than direct interventions such as courses, events, or Pleasure Craft Courtesy Checks (PCCCs).
The efficiency of five BSCCP -funded projects of different sizes that had sufficient data on reach was assessed. As Table 8 demonstrates, projects that attempt to reach boaters directly through courses, events, or Pleasure Craft Courtesy Checks (PCCCs) are much more costly per boater than those that reach boaters through public media campaigns.
Table 8: Results of Efficiency Analysis for Six BSCCP -funded Projects
| Name | BSCCP Funding | Direct or
Media-based Intervention |
Boaters Reached | Annual Cost per
Boater |
|---|---|---|---|---|
| Project A | $101,250 | Media-based | 1.8 million (low estimate) to 6.5 million (high estimate) | $0.02 to $0.06 |
| Project B | $54,110 for two year program | Media-based | 2.6 million (low estimate) | $0.01 |
| Project C | Estimated that $90,000 spend reaching boaters directly | Direct (with small media-based portion) | 17,480 | $5.1 |
| Project D | $17,949 for two year program | Direct | 612 (low estimate) to 2,612 (high estimate) | $6.9 to $29.3 |
| Project E | $17,000 (estimated that 25-50% spent reaching children in schools) | Direct | 964 (children – not known how many boat) | $4.41 (low estimate) to $8.82 (likely estimate) |
Public Media Campaigns
In 2009-2010, an organization that received a contribution of $101,250 distributed media material to newspapers and radio stations across the country, printed 26,500 posters, distributed 10,851 of them, purchased 572 outdoor advertisements, and produced television Public Service Announcements to be distributed in future years (see project A in Table 8). Survey results for the project estimated that 15% of adult boaters recognized at least one of the organization’s campaign posters and 55% of adult boaters recognized at least one of the organization’s campaign messages, although the use of aided awareness prompts and other methodological issues would inflate this estimate of campaign reach. Using the high estimate of project reach, 6.5 million adult boaters (55% of adult boaters) were reached by the campaign. Using the low estimate of reach, based on the reach of the poster campaign, 1.8 million adult boaters were reached. The reality is likely somewhere in the middle, but in either scenario, the cost per boater reached is very low: $0.02 to $0.06.
Project B in Table 8 targeted a large number of boaters through media campaigns. It received $54,110 from the BSCCP over two years; an average of $27,055 per year. The organization’s survey research suggests that 54% of adult boaters were aware of the organization’s campaign in 2012-2013. Due to the same methodological issues as project A, this estimate of campaign reach is also inflated. However, even taking this into account, the project’s reach was nonetheless high. Assuming that the organization actually reached 54% of adult boaters, this would amount to 6.4 million boaters. Other estimates of 40%, 30%, and 20% of adult boaters would amount to 4.7 million, 3.5 million, and 2.6 million boaters, respectively. Even if only 20% of boaters were reached, the organization would have reached boaters at a cost of one cent ($0.01) each.
Projects that Reach Boaters Directly
Funding of $100,000 for project C in Table 8 led to 16,170 people being reached at 197 events, demonstrations, and presentations; 1,200 PFD s being loaned; 110 Pleasure Craft Courtesy Checks being conducted; and 104 media outlets being contacted, though it is unknown how many articles or news stories appeared in the media as a result. Assuming that 10% of project costs went toward contacting the media,Footnote37 the remaining 90% was spent to directly reach 17,480 adult boaters, which amounts to $5.1 per boater. The organization’s outputs were significant, but it would cost between $9.2 million and $13.3 million to reach 15% to 20% of adult boaters in this way.
Project D in Table 8 was a two-year project that funded a small kayaking club with 170 active members. With the project, the club reached 212 kayakers in safety courses and events and distributed six editions of 400 copies of its newsletter to members and local businesses. The cost per boater reached can be estimated with two different assumptions. If the six editions of the newsletter reached the same people each time, 612 kayakers were reached at a cost of $29.3 per boater. If each edition of the newsletters reached a new boater, 2,612 kayakers were reached at a cost of $6.9 per boater. It would cost between $17.3 and $73.3 million to reach all adult Canadian kayakers at the same cost per kayaker. Furthermore, interviews revealed that a large majority of kayakers do not belong to clubs and cannot be reached directly by a club. In fact, most boaters do not belong to clubs. Paddle Canada, the largest paddling organization in Canada, has a membership of 1,800 instructors and individual members, which is only 0.03% of all individuals who paddle at least occasionally. As a result, a mass media campaign by these organizations would seem to make more sense to reach boaters.
Another example of the high cost of reaching boaters directly is project E, a small project in 2012-2013 that delivered presentations to students aged 5-14 in schools, reaching 964 students. It likely spent approximately $8.82 per student reached, while a low cost estimate would be that it spent $4.41 per student reached.Footnote38 Thus, less direct forms of intervention would be a more efficient way of reaching them.
Finding 16: The BSCCP is economical due to the fact that it funds organizations that operate at leverage the support of volunteer workers in its delivery, and often generate free media impressions.
An important success identified in the 2006 evaluation of the BSCCP ’s predecessors, Transport Canada’s contributions to the Canadian Safe Boating Council (CSBC) and the Canadian Red Cross (CRC), was that the TC had leveraged its resources by funding organizations that were able to make use of volunteers to advance their TC -funded projects. Of the 16 organizations that the BSCCP funded over the evaluation time period, 12 were volunteer-based. Some of these organizations were asked in interviews about the use of volunteer workers and free media for BSCCP -funded projects. Three of these organizations estimated that they made use of 1,600, 1,000, and 260 volunteer-hours for projects funded by the BSCCP , assigning values of $12,000, $16,000, and $4,200, respectively, to that volunteer work. Several organizations provided estimates of the dollar value of the free media coverage generated by their projects, one of which was as high as $5 million.
Finding 17: Organizations usually developed their own awareness materials instead of using resources to distribute existing materials and maximize their reach.
Six of the eight projects that received funding worth less than $20,000 per year and 18 of the 32 projects overall were involved in the creation of new pamphlets, brochures, press releases, and radio or television public service announcements. Except when a campaign is trying to reach a specific population such as Aboriginal boaters, existing material could likely be used instead, especially by small organizations. TC ’s Safe Boating Guides are available online for use by other organizations and the CSBC makes its awareness material available to other organizations as well. By using these materials, organizations could focus their resources on distribution and promotion. An example of a project that did make use of existing material was a 2012-2013 project that produced and distributed TC ’s Safe Boating Guides and other TC materials.
Annex 1: Logic Model of the Boating Safety Class Contribution Program
| Activity Areas | Outputs | Target Audience | Immediate outcomes (General Public) | Intermediate outcomes (General Boating Public) | Ultimate outcome (General Boating Public) |
|---|---|---|---|---|---|
TC
Marine Safety/
OBS
|
|
|
|
|
|
Outreach (Recipient)
|
|
|
|||
Research (Recipient)
|
|
|
Source: Integrated Results-based Management Accountability Framework and Risk-based Audit Framework, BSCCP , March 2008
Annex 2: Evaluation Framework
| Core Evaluation Issues | Indicators | Data Sources | Lines of Inquiry |
|---|---|---|---|
| Relevance | |||
|
|
|
|
|
|
|
|
|
|
|
|
| Achievement of expected outcomes | |||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Demonstration of efficiency and economy | |||
|
|
|
|
Annex 3: Estimate of Program Reach
The evaluators decided to estimate the potential reach of program activities through newspapers carrying boating safety content during the 67 days of the NASBAW / SBAW campaign on boating safety (the campaign involves multiple partners), to extend this number over the entire boating season, and then average this estimate with the estimate obtained from survey data (which is on the high side). The reach of newspapers can be used as a proxy for media reach in general because content is duplicated across media modes. The goal of the evaluators was to develop a measure of the number of people reached at least once (media impression methodology assumes that people reached multiple times represent multiple people). The evaluators’ methodology generously assumes that each person who received a newspaper read it, and read all of it; that there is no overlap between daily and community newspaper readerships; and that boating safety articles are just as prevalent in the second half of the summer as the first.Footnote39 This was then averaged with the CSBC survey results in an attempt to remove the upward bias from their survey estimate.Footnote40
The total circulation of the community newspapers that carried an article with boating safety content over the 67 days was 1.3 million. For the daily newspapers, it was 1.9 million. This gives a total potential reach of 3.2 million people. Adjusting this number to include the francophone population would give a total potential reach of 4.2 million.Footnote41 If we extend the same potential reach figures to cover the four month period from May 1 to August 31, 2012, we can assume that 7.7 million people were potentially reached through newspapers.
Accepting the CSBC ’s 2012 estimate that 43% of people over 18 participate in boating activities at least occasionally, this would imply that 3.3 million boaters were potentially reached through newspapers in the four month stretch in 2012, based on the assumption that boaters and non-boaters have the same readership habits. Yet the size of the adult boating public (those who boat at least occasionally) is estimated to be 11.8 million. Considering that the time frame of the two newspaper scans includes and surrounds the largest boating safety awareness campaign in Canada, if not also North America, the presence of the issue of boating safety in Canadian newspapers over the boating season is rather small.
While the CSBC ’s Safe Boating Awareness Survey might have aimed to assess the reach of the Safe Boating Awareness Week (SBAW) campaign, the survey actually provides an estimate of the potential reach of all boat safety awareness activities. This is because the survey asks if people recall having heard or seen nine boating safety messages, many of which are generic and could have been encountered anywhere.Footnote42 Examples of generic boating safety messages include:
- “Don’t drink alcoholic beverages while boating… it’s that simple. If you drink, don’t drive;” 47% of respondents recalled this message, which captures the popular “If you drink, don’t drive” message.
- “Everyone operating a motorized boat or water craft should get their Pleasure Craft Operator Card;” 27% of respondents recalled this message.
- “Make the smart choice. Don’t just carry your lifejacket aboard your boat. Wear It;” 27% of respondents recalled this message.
An upward bias in the results is to be expected since the messages were presented to respondents, they were quite long, and respondents would not be selective about exact wording or exactly when they heard/saw them.
According to the 2012 survey, 54% of boaters reported that they could recall having heard or seen at least one of the boating safety messages with which they were presented.Footnote43 The CSBC ’s estimate translates into 6.4 million boaters (based on CSBC estimate of the percentage of the population that boats at least occasionally). The actual number of boaters reached is likely somewhere in between the evaluators’ estimate of 3.3 million boaters reached in 2012 through newspapers, which provides a proxy measure of all media activity due to duplication of content across media modes, and the CSBC ’s estimate of 6.4 million boaters. The average of the two estimates is 4.8 million or 41% of boaters who boat at least occasionally.
Note that this estimate would still overestimate the reach of BSCCP awareness activities because there are other organizations that promote boating safety (e.g. police forces) that are not funded by the BSCCP . However, as was evident from the evaluators’ newspaper scan, most of the messaging is promoted by BSCCP -funded organizations.
The following document is available for downloading or viewing:
- Evaluation of the Boating Safety Class Contribution Program ( Version, 356 kb )
To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet: