Airport Wildlife Management Bulletin - TP 8240 - No. 40

Airport Wildlife Management Performance Measurement

Bulletin No. 40 - Spring, 2008

 

 

SMS Performance Measurement

Performance measurement is one of the cornerstones of SMS (safety management systems), now the foundation of Canada's civil-aviation regulatory program. In the case of airport wildlife management, ongoing progress is achieved only when airport authorities effectively monitor and adjust mitigation to ensure wildlife management activities meet objectives and reduce wildlife-related risks. The only way to determine whether these objectives and risk reductions are being achieved is to set out performance indicators against which the effectiveness of wildlife management plans and activities can be measured. Historically, these indicators have included the annual number of wildlife strikes, the strike rate per 10 000 aircraft movements and damage associated with strikes.

Eyes on the future

SMS marks a fundamental shift in the management of aviation safety partly because it demands that stakeholders look forward in attempts to analyze and understand hazards, and assess and forecast future risk.

An SMS is an explicit and comprehensive way to manage safety-risks proactively. It builds on well established methods for business and business-risk management. SMS requires that goals and objectives be set, that resources be allocated to conduct associated activities purposefully, that the activities be coordinated (throughout the organization, and sometimes with stakeholders) and that the results be measured in the short- and medium-term. Measurement indicates strengths and deficiencies in performance, and helps guide appropriate actions to optimize performance or prevent excessive losses. No one would consider running a business by setting out good business practices (the equivalent of meeting the minimum requirements specified in CARs) and then waiting for statements of financial loss to guide corporate decisions. Similarly, an airport, for example, should not simply comply with the CARs and wait for audit findings, incidents or worse, accidents, to determine where it should invest its "safety dollar". Safety must be designed into the airport's changing operations, the risks managed, and the results measured. And all this must be done proactively, just as a business, or even a household, plans for the future and manages its financial risks.

The SMS approach opposes current, reflective safety analyses that look back to compare trends from past occurrences. Based on the performance indicators described in CAR 302 and noted above, for example, a safety analysis would only highlight risks that have already been realized, rather than reveal new risks that may impact the safety of future operations.

These traditional performance indicators are considered to be lagging; they provide only after-the-fact information about performance. Unarguably, they are useful in tracking general trends over time; but, as descriptive rather than predictive indicators, they result in measurement programs that are founded on event-based data and guided by reactive decision-making.

Non-performance indicators

As a measure, wildlife strikes, for example, are negative indicators. Whether categorized by species, type of aircraft or damage to aircraft, strikes actually indicate non-performance. They show that an element of a wildlife management plan may have failed to achieve its objective. They point vaguely in the direction of mitigation that may not be working, rather than indicating clearly what actions prove effective. In SMS, you manage what you measure. It can be argued that managing wildlife strikes is essentially a futile, indirect approach, whereas managing activities to reduce strikes should yield results.

The flaw in the reflective and reactive approach to safety assessment is captured in the traditional wisdom that concludes safety exists because no accidents have occurred. This is a perspective that SMS discredits. The science is clear: where there are hazards, there are risks. The greater the risk, the more likely that a strike will occur. Reducing the risk to the lowest level practicable requires thorough knowledge not only of the hazards, but also the efficacy of efforts to counter them.

Examining a new approach

In implementing SMS, Transport Canada has recognized the need to collect, categorize, store and analyze data that enable the identification not only of safety-risk management weaknesses, but also its strengths. Wildlife measurement practices must provide the specific, timely information needed to appropriately guide decision-making. For this reason, the department is exploring the creation of a wildlife-management performance-measurement program (WMMP) that:

• Identifies the measurement requirements of each category of stakeholder regarding wildlife hazard management;
• Determines how those requirements can be met (i.e., the data that need to be collected; how they can be collected and analyzed; and how they will be used); and
• Establishes how information might be shared among stakeholders.

Transport Canada has prepared a working document that provides preliminary details about the WMMP. Acceptable Risk: Developing a Wildlife Management Measurement Program for Canadian Airports is introduced in this bulletin, and will be made available to key aviation-industry stakeholders for consultative review during 2008.

A range of requirements

Acceptable Risk acknowledges and seeks to accommodate the range of stakeholder needs for the measurement of airport wildlife management. Airport managers, for example, need guidance to inform short- and long-term decisions related to resource allocation for wildlife management. Transport Canada needs, among other things, information to guide development and ongoing administration of regulatory programs.

Acceptable Risk also sets out an extensive list of preliminary and generic measurement-program performance requirements that would be addressed in a WMMP. The program will have to be compatible with current SMS at airports in Canada. It should be easy to use for both tactical, day-to-day wildlife management activities and long-term strategic planning. A WMMP will also be required to integrate with other components of airport regulatory oversight, and deliver comprehensive data consistently from sources across the country.

Measurement program description

Wildlife management programs generally flow from and to their goals. In other words, direction is set by a program goal such as "reduce the risk of wildlife strikes in the airport environment to the lowest practical level." The program's outcomes (number of strikes, presence of wildlife on the airport, etc.) indicate whether the goal is being met. The circular nature of this flow is significant, as it reinforces wildlife-risk management as a continuous, integrated, self-informing loop (see Figure 1).

Figure 1. Wildlife Management Program Component Flow

 

The space between a program's goal and its outcomes is the central focus of Transport Canada's performance measurement initiative, which will identify and chart the various wildlife-management components and establish measurement points, or ‘performance nodes', throughout.¹

These nodes will be assessed according to three indicators: the soundness, appropriateness and effectiveness of each safety management activity.

• Soundness evaluates the foundations of an airport's wildlife management program.
• Appropriateness determines whether wildlife management activities are correctly targeted to achieve results.
• Effectiveness establishes whether wildlife management activities are actually achieving the desired results consistently and effectively.

Acceptable safety performance is achieved only when management activities are sound and appropriate and effective.

Soundness + appropriateness + effectiveness = optimized performance

For example, an airport may be experiencing a rash of strikes. In trying to determine the reasons, it is revealed that the airport set long-term performance goals and some short-term objectives regarding wildlife management (a sound foundation for the program). The goals and objectives are appropriate because they address the wildlife hazards and risks at that airport. However, the goals and objectives were not disseminated to the board-or to the managers responsible for airside operations. As a consequence, the necessary funding was not approved by the board (not effective), nor the correct mitigating activities conducted by the manager (not effective). Importantly, there need not be a "rash of strikes" to indicate ineffective safety-risk management. Simple proactive "triggers" can suffice, such as an observation that a hazardous species has been seen more often at an airport than is customarily the case.

Consider another example: A wildlife-program evaluation identifies a significant lack of reporting of bird strikes at an airport. If it is determined that there is no reporting policy or process, the problem would be foundational and indicate a lack of soundness. The ‘fix' would be clear. But assume that there are well documented policies, procedures and processes-that the program is sound in this respect-and that another line of inquiry reveals that bird-strike reporting at the airport is web based only, and that computer and internet access in the area are limited. This indicates the reporting aspect of the program is inappropriate. If the reporting system is appropriate, but no one in the airport community knows about it, then there is a clear deficiency in communication-as a result, the reporting system is not effective.

Delivering clear direction

The presence of wildlife contributes significantly to a dynamic airport operating environment where daily, weekly and seasonal fluctuations are continual. Proactive measurement enables detection or anticipation of ongoing variations in safety management over time, assesses their significance, and directs appropriate corrections. By identifying deficiencies in performance-and categorizing them in terms of soundness, appropriateness or effectiveness-the WMMP will permit airport operators to discern the causes of weaknesses and plan improvements. Without such indicators, mitigation is likely to be ill-informed, ill-conceived, wasteful in terms of its draw on resources and confusing in terms of results.

Data-driven management

Acceptable Risk proposes that measurement-program data would be regularly obtained from various sources that proactively provide information regarding safety management:

• periodic internal assessments of the wildlife management program,
• management, safety, operational, technical or community-based committee meetings,
• audit or inspections by various regulatory bodies (such as Transport Canada or a government environment ministry),
• avifauna studies, and
• internal or external hazard reporting programs.

Transport Canada envisions a system in which these data would do more than guide day-to-day wildlife risk management and strategic decision-making by airport executives. Data from individual airports could feed repositories that Transport Canada could use to:

• identify and evaluate regional and national trends in effective or less-than-effective wildlife management program elements or activities,
• advise departmental headquarters and regional management regarding risk-based priorities for audit schedules, or the development or modification of policies or rules, and
• develop risk-based strategies for wildlife-management awareness or safety promotional products.

The way forward

The proposed measurement program will address a longstanding deficiency in aviation safety measurement. In fact, research suggests that no such wildlife management measurement program currently exists anywhere in the world. Consequently, the program's development must be guided carefully to ensure that it is sound, and to foster understanding among stakeholders-namely the airport community and Transport Canada.

Transport Canada proposes that phase two of WMMP development will, among other things, validate the contents of Acceptable Risk and create the necessary measurement tools or protocols. Phase two will begin with a consultative review of Acceptable Risk by a representative sample of regulatory personnel, airport community members, biologists, wildlife- and safety-management experts. These and other aviation industry stakeholders will be asked to help refine and validate the WMMP logic model.

Interested parties can download Acceptable Risk and are invited to contact the department at the address below if they are interested in being part of the consultative team.

Based on the consultative review, Transport Canada will prepare a measurement protocol and convene a group of aviation industry subject-matter experts to validate the protocol prior to testing and implementation.

The goal: truly effective safety-risk measurement

Traditional wildlife measures in aviation focus on strikes: their numbers, rates, locations, the damage they inflict on aircraft, etc. While this kind of strike data will remain essential, Transport Canada's new measurement program will recognize that the primary focus must be management, and the goal must be to determine the effectiveness of efforts to reduce the risk of wildlife strikes.

Consider the analogy of a flood. The traditional, reactive focus comes after the deluge, measuring the amount of water, the rate at which it flowed, the locations where breeches occurred. The new proactive focus would be on the defenses themselves. Are they designed according to predicted, worst-case weather conditions and the amount of water? Are they well constructed and maintained? Are flood-management authorities properly trained? Are efforts ongoing to reduce the likelihood of flooding?

Rather than looking back from the outcomes-an approach that could have dire ramifications-progressive measurement forecasts future risks and examines the elements that lead to the outcomes. The resulting knowledge is key to the ongoing enhancement of effective, proactive airport wildlife management, and the assurance of greater aviation safety.

Bird-remains Identification Service

In past years, airport operators were able to collect and send feather remains from bird strikes to the Smithsonian Institution in Washington for species identification. This information proved helpful in determining bird species involved in strikes, and enabled operators to verify that mitigation activities were targeting hazardous birds. Recent border-security measures, however, severely complicated the process of transporting animal remains into the U.S.

Thanks to the Smithsonian, this valuable-and free-service can now resume. The institution has graciously made available the permits and procedures that airlines and airport operators can use to send bird feathers across the border. There are four documents, three of which can be downloaded to complete and include with each shipment:

1. A completed copy of the certificate of origin, which is a simple letter that indicates where the shipment is coming from.
2. A completed copy of the certificate of treatment. Be sure to circle the method used to treat the feather specimen-and note that the Smithsonian prefers the alcohol method, which ensures better DNA preservation.
3. The final document is a copy of the Smithsonian's USDA-APHIS permit, which must accompany your shipment.

Samples should be sent directly to:

Smithsonian Institution
Feather Identification Lab
P.O. Box 37012
NHB E-600, MRC 116
Washington, DC 20013-7012

The Smithsonian feather identification lab will respond with notification of results as soon as possible.

For more information on other issues in this bulletin, please contact:

Civil Aviation
Communication Centre (AARCB)
330 Sparks Street
Ottawa ON
K1A 0N8
North America: 1-800-305-2059
Fax: 613-957-4208
Civil Aviation Communications Centre contact form