6.2 Data Management

From: Transport Canada

Commercial & Business Aviation

6.2.1 Overview

In AQP, data management is a continual process of data collection, entry, submission and analysis.

6.2.2 Data Collection

AQP data collection is required in all curricula. The specifics are detailed in the air operator’s AQP Data Collection and Analysis Plan, which are contained in the Implementation and Operations Plan (I & O Plan). Data is collected at each validation or evaluation gate. This data consists of graded proficiency objectives using a rating scale with associated reason codes (if applicable). Data collection requirements for the AQP will vary with the curriculum, the type of curriculum activity (training, validation, or evaluation), the type of participant (flight crewmember, instructor, or evaluator), and the overall management objectives for use of the data. Figure 6-1 illustrates the AQP data collection fields required for submission to Transport Canada. All performance data collected on each proficiency objective must be relative to the applicable Qualification Standards defined for the training and evaluation activities.

For each flight crewmember in a Qualification or Continuing Qualification Curriculum, Transport Canada must be able to associate the data records applicable for that flight crewmember in that curriculum through logical grouping of the records, or linkage by a common de-identified index number (but not by name).

6.2.3 Data Entry

All performance/proficiency data collected throughout the AQP is entered into the air operator’s PPDB. Typically, this is an electronic database for ease in analysis, comparison and reporting purposes. Considerations for data entry include the method, the hardware/software required for data input, and the hardware/software required for data storage and utilization. Distinct advantages, disadvantages and costs are associated with any method of data entry. Database design is at the discretion of the air operator, provided that the design can generate the required report table specified, in a manner acceptable to Transport Canada.

6.2.4 Data Submission

For the purposes of program oversight, Transport Canada has established the minimal requirements for the submission of de-identified data by curriculum. Figures 6-1 through 6-4 summarize the submission requirements in three tables. These tables are the Performance Data Report Table (PDRT ), the Skill Reason Table (SklRsn) and the Training Objectives Report Table (TORT). The information in these tables is downloaded from the air operator’s PPDB. The air operator is responsible to prepare de-identified data reports and check them for data integrity, and then to submit them to Transport Canada in 1 calendar-month blocks within 2 months of collection. Reports can be forwarded electronically to Transport Canada or made available by convenient means such as through web access. Transport Canada will analyze the de-identified data using standard automated queries and reports to identify AQP performance trends. Figures 6-2, 6-3, and 6-4 contain specifications for each field in the three tables.

  1. PDRT
     
    The PDRT contains a listing of 23 fields that are reported for every measured item, providing a separate record for each. A measured item is a manoeuvre, task, procedure, or event set, and is the main component for data analysis. These fields provide a record of the results of the performance of each measured item along with supporting data for reporting and analysis. Certain supporting data fields (air operator designator, curriculum, etc.) repeat for each record and can be automatically generated from a query/software routine. Each field in the PDRT must contain either an alpha/numeric, numeric, or text entry. Figure 6-2 contains the specifications for each field, defines the meaning of each field item, and provides examples of the field values. 
     
  2. SklRsn
     
    With respect to the SklRsn table, an unsatisfactory rating (Mrate) of a measured item requires that a reason be entered to explain the rating. All reasons are entered in the SklRsn table in field No. 24, Skill Reason Text. This table allows the carriers to report more than one Skill Reason Text for an unsatisfactory rating of a measured item without having to enter multiple records. There are only two fields in this table.
     
    1. The first field is Field No. 5 (SklRsn), which is the same as in the PDRT and provides the linkage between the two tables. For each record in the PDRT where one or more Skill Reasons need to be reported, an identifier is entered in the SklRsn field that is unique to the measured item. This identifier can be a concatenation of the fields that make it unique. For instance: Fleet+EvalType+CmID+Mitem. As long as the identifier is unique to the record, it is the carrier’s discretion to determine which fields are used.

      NOTE: If this approach is used, care must be taken to ensure that none of the component field contains an NA value. 
       
    2. The second field in the SklRsn Table is the Skill Reason Text, the field that will contain descriptive text for each reason relevant to a particular measured item. For example: Systems Knowledge, Procedural, CRM, Technical Knowledge, etc. 
       
  3. TORT
     
    The TORT table is comprised of training objective(s) associated with each measured item. It provides a means of auditing the relationship between the measured item and the task analysis, qualification standards, and the curriculum. There are four fields in this table. The TORT links to the PDRT through the first field, Measured Item ID (MItemID). The second field, Objective ID (ObjID) is an identifier assigned to each terminal proficiency objective (TPO) or supporting proficiency objective. The Objective ID may be the same as the Measured ID if it is a single TPO or SPO. However, in the case of a measured item that is an event set, there may be several training objective identifiers associated to the Measured Item ID. The third field, Objective Title (ObjTitle) is a text description of the training objective. The last field in this table is the Objective Type (ObjType) that identifies the training objective as a TPO, SPO, EO or as required by the air operator’s qualification standards structure.

    Due to the unique features of each operator’s AQP, TC in consultation with the air operator may require the collection of additional data as deemed appropriate. 
     
  4. File Naming Convention
     
    Data submissions must use the file-naming format (xyzmmyya.mdb) as shown in the example below:

     

    On the following pages, Figure 6-1 lists the minimum AQP data submission requirements. Figure 6-2 provides the specification for each field in the PDRT. Figure 6-3 discusses the skill reason codes and supporting text applicable to the SklRsn table while Figure 6-4 on the topic of the TORT links training objectives with measured items.

Figure 6-1: DATA SUBMISSION REQUIREMENTS

Training Program Advanced Qualification Program
Curriculum Qualification Continuing Qualification
Table # Field Short Name SV PV MV LOE OE FL MV LOE OE
Proficiency
Data
Report
Table 		1 Measured Item MItem Des Des Req Req Req Req Req Req Req
2 Measured Item ID MitemID Des Des Req Req Req Req Req Req Req
3 Measured Item Rating* Mrate Des Des Req Req Req Req Req Req Req
4 Rating Meaning Rmean Des Des Req Req Req Req Req Req Req
5 Skill/Reason SklRsn N/A N/A Req Req Req Req Req Req Req
6 PF/PNF PFPNF N/A N/A DES** Req Req Req** DES** Req Req
7 Is Currency Item? Currcy N/A N/A Req N/A N/A Req Req N/A N/A
8 Is Critical Item Crit N/A N/A Req N/A N/A Req Req N/A N/A
9 Crew ID CrewID Req*** Req.*** Req Req Req Req Req Req Req
10 Evaluation Date EvalDate Req Req Req Req Req Req Req Req Req
11 Air operator Designator AirDsgn Req Req Req Req Req Req Req Req Req
12 TC Fleet Designator Fleet Req Req Req Req Req Req Req Req Req
13 Training Program TrPgm Req Req Req Req Req Req Req Req Req
14 Curriculum Curr Req Req Req Req Req Req Req Req Req
15 Evaluation Type EvalType Req Req Req Req Req Req Req Req Req
16 TC Simulator ID SimID N/A N/A Req Req N/A Req Req Req N/A
17 Evaluator ID EvaltrID Req Req Req Req Req Req Req Req Req
18 TC Inspector ID TCID Req Req Req Req Req Req Req Req Req
19 OE Geo. Area GeoArea N/A N/A N/A N/A N/A N/A N/A N/A Req
20 Comments Comment Des Des Des Des Des Des Des Des Des
21 Crewmember ID  CmID Req Req Req Req Req Req Req Req Req
22 Crew Position CrewPos Req Req Req Req Req Req Req Req Req
23 Evaluation Rating Type EvalRtg Req Req Req Req Req N/A Req Req Req
 
Skill Reason
Table 		5 Skill/Reason SklRsn N/A N/A Req Req Req Req N/A Req Req
24 Skill Reason Text SklRsnText N/A N/A Req Req Req Req N/A Req Req
 
Training
Objectives
Report
Table 		2 Measured Item ID MItemID Des Des Req Req Req Req Req Req Req
25 Objective ID ObjID Des Des Req Req Req Req Req Req Req
26 Objective Title ObjTitle Des Des Req Req Req Req Req Req Req
27 Objective Type ObjType Des Des Req Req Req Req Req Req Req

 

 Req - A required entry.  Qual - AQP Qualification Curriculum  PV - Procedures        Validation
 N/A - If no value is to be entered, use N/A. Leave no blank fields.  CQ - AQP Continuing Qualification Curriculum  MV - Manoeuvres Validation
 Des - Desired. This is optional information that is not required, but desired for submission. If not submitted, then N/A must be entered as the value.  FL - First Look Manoeuvres  LOE - Line Operational
Evaluation
 */** - See Notes in Data Reporting Format - Figure 6-2.  SV - Systems Validation   OE - Online Evaluation
 *** - If there is no crew pairing for Systems or Procedures Val, submit the single CmID (No. 21) for the crewmember.    

 

 

 

Figure 6–2: PERFORMANCE DATA REPORT TABLE (PDRT)

 

 

Table No Field Description Short
Name 		Type Field
Size 		Example(s) Notes
PDRT 1 Measured Item The task, manoeuvre, procedure, or event set statement. Mitem Text 80 Perform Engine Failure Procedure
  1. This field will be N/A for Qual. SV and PV because these validations are usually graded pass/fail and recorded in field No. 23.
2 Measured Item ID An alphanumeric used to identify the task, manoeuvre, procedure or event set being graded. MitemID Text 12 1.2.1.3
or
KK73456
  1. This value is associated with a TPO, SPO, or event set. When it is a TPO or SPO, use the related number from the hierarchical numbering system. When it is an event set, enter the event set identifier.
  2. This field will be N/A for Qual. SV and PV because these validations are normally graded in aggregate (pass/fail), and recorded in field No. 23.
3 Measured Item Rating The numeric rating assigned on the first attempt for each measured item. Mrate Numeric 1 3
  1. Values depend on rating scale.
  2. Do not provide records that are missing MRate: Provide only performance related information. Do not provide “Incomplete”.
  3. Reported rating must refer only to the first attempt of the measured Item, not the terminal performance.
  4. * Use 9 instead of N/A for Qual. SV and PV. This is a numeric only field.
4 Rating Meaning A description of the Measured Item Rating Rmean Text 40 Satisfactory
  1. The text meaning of each number in the rating scale.
5 Skill/Reason Link between the PDRT and SKLRSN tables SklRsn Text 90 B-737-LOE-5555-Taxi
  1. This field is a unique record identifier that must be provided for Unsatisfactory Measured Item ratings (MRate) to provide a link to the Skill Reason table for single or multiple reasons for a failure.
  2. This identifier can be a combination of the fields that make it unique. For example: Fleet+EvalType+CmID+MItem. As long as the identifier is unique to the record, it is the carrier’s discretion to determine which fields are used.
  3. If Mitem is satisfactory, insert N/A in this field.
6 PF/PNF Indicates whether the pilot performing the Measured Item was flying or not flying. PFPNF Text 3 PF or PNF
  1. No entry requirement for FE.
  2. N/A for Qualification Curriculum Systems and Procedures Validation
  3. ** For FL, it is desired and encouraged that PNF data be submitted along with PF data. However, if significant grade sheet changes are necessary to accommodate this, only the standard PF items need be identified. For these air operators, the value entered for all FL Measured Items will be “PF”.
  4. ** This field is desired for Qual. and CQ Manoeuvres Validation; however, if PNF is not submitted then the value will be “PF”.
7 Is currency Item? Indicates whether the Measured Item is a currency item. Currcy Text 3 Yes, No or N/A
  1. N/A is used when the Measured Item is an Event Set.
8 Is Critical Item Indicates whether the Measured Item is a critical item. Crit Text 3 Yes, No or N/A
  1. N/A is used when the Measured Item is an Event Set.
PDRT 9 Crew ID The de-identified alphanumeric assigned to a specific pairing of crewmembers that will be maintained for the duration of a curriculum, unless the crew make-up changes or a seat substitute is present. CrewID Text 40 P21234 SIC F12312

 
  1. CrewID is the concatenation of the CmID’s (#21) of all crewmembers present. Order of ID’s: PIC SIC RP FE .  
  2. If two PICs, SICs or RPs are trained together use the structure PIC PIC FE, or SIC SIC FE or RP RP, as appropriate, separated by spaces.
  3. If there is no crew pairing for Systems or Procedures Val, submit the single CmID (#21) for the crewmember.
  4. Maintain CrewID in exact form and order, except for seat substitutes. For seat subs, use Crew Position (No. 22) values relative to the seat position in the CrewID sequence and add the CmID number. For example, P21234 SIC S17521 F12312 would indicate that the first officer position was occupied by a First Officer seat substitute with CmID number S17521.
  5. Do not submit data collected on non-employees (e.g., contract instructors) performing their required validations/evaluations.
10 (MM/YYYY) The month and the year the Measured Item data is collected. Date Date 7 10/2001
  1. Set the date field format in Access to month/year; use full century: MM/YYYY.
  2. The day value will default to 01.
11 Air operator
Designator 		The air operator’s four-character designator reporting the measured items. AirDsgn Text 4 XYZA  
12 TC Fleet Designator The TC designation of the fleet or equipment type relevant to the Measured Item. Fleet Text 20 B-737
  1. See Air operator/Fleet Designator Menu
13 Training Program The training program relevant to the Measured Item. TrPgm Text 4 AQP  
14 Curriculum The AQP curriculum in which the Measured Item is being validated or evaluated. Curr Text 4 Qual, CQ or N/A  
15 Evaluation Type The evaluation type in which the Measured Item is accomplished. EvalType Text 4  FL, SV, PV, MV, LOE, OE  
16 TC Simulator ID The TC simulator ID number of the simulator where the Measured Item is performed. SimID Text 4 1234 or N/A
  1. If a simulator is not used, enter N/A
17 Evaluator ID The identification number of the evaluator who graded the Measured Item. EvaltrID Text 15 123456
  1. If an TC inspector is the evaluator, use the TCID in this field and in the TC Inspector ID field, No. 18.
PDRT 18 TC Inspector ID The Form 110A number of the aviation safety inspector who observed the Measured Item. TCID Text 4 1234
  1. Report the TC ID Number (110A) of a TC inspector who is there as an observer and/or evaluator (#17). If no TC inspector is present, enter N/A.
19 OE Geographic Area The geographic area where the Measured Item is collected for OE . GeoArea Text 30 Pacific
  1. Use geographic descriptors as defined by the air operator.
  2. This field is only required for AQP operators intending to apply or already authorized for deviations from regulatory OE interval requirements; otherwise, enter N/A.
20 Comments Additional comments submitted by evaluator. Comment Memo    
  1. The comment field provides further explanation of unsatisfactory or outstanding measured Item Rating.
  2. Evaluator comments are provided at the discretion of the air operator, unless otherwise required in accordance with the air operator’s AQP.
  3. If there are no comments, enter N/A.
21 Crewmember ID The de-identified alphanumeric assigned for the duration of a curriculum to the crewmember performing the measured item. CmID Text 10 P123456
  1. Begin all PIC IDs with P, SIC IDs with S, Cruise Relief Pilot (CRP) IDs with R and FE IDs with F.
  2. The P, S, R or F refer to the position for which the pilot in training is being qualified, not necessarily the seat occupied.
  3. Do not submit data collected on non-employees (e.g., contract instructors) performing their required validations/evaluations.
22 Crew Position The “seat” occupied by the person performing the Measured Item. CrewPos Text 4 PIC PIC = Pilot in Command
SIC = Second in Command
FE = Flight Engineer
E.g., a SIC or CRP in the left seat would be entered as PIC.
23 Evaluation Type Rating The overall rating the pilot performing the Measured Item received for the evaluation type. EvalRtg Text 5 Sat or
Unsat
  1. Rating must refer to the first execution of the evaluation type. Repeated sessions are not reported in this table.

Figure 6–3:  SKILL/REASON TABLE (SKLRSN)

  # Field Description Short
Name 		Type Field
Size 		Example(s) Notes
SklRsn 5 Skill/Reason Link between the PDRT and SKLRSN tables SklRsn Text 90 B-737-LOE-5555-Taxi
  1. This field is a unique record identifier that must be provided for Unsatisfactory Measured Item ratings (MRate) to provide a link to the Skill Reason table for single or multiple reasons for a failure.
  2. This identifier can be a combination of the fields that make it unique. For example: Fleet+EvalType+CmID+ MItem.
  3. As long as the identifier is unique to the record, it is the carrier’s discretion to determine which fields are used.
  4. None of the component fields can contain a N/A value.
  24 Skill reason Text A description of the skill or reason for assigning an unsatisfactory Measured Item rating. SklRsn Text Text 40 Technical
CRM
Procedural

 
  1. Provided for Unsatisfactory Measured Item Ratings from the carrier’s Reason Codes.
  2. If more than one skill or reason is relevant, allow one field for each skill or reason.
  3. Enabling Objectives (EOs) are acceptable as skill descriptors.

Figure 6–4:  TRAINING OBJECTIVES REPORT TABLE (TORT)

  # Field Description Short
Name 		Type Field
Size 		Example(s) Notes
TORT 2 Measured Item ID An alphanumeric used to identify the task, manoeuvre, procedure or event set being graded. MitemID Text 12 1.2.1.3 or

KK73456
  1. This value is associated with a TPO, SPO, or event set. When it is a TPO or SPO, use the related number from the hierarchical numbering system. When it is an event set, enter the event set identifier.
  2. This field will be N/A for Qual. SV and PV because these validations are normally graded in aggregate (pass/fail), and recorded in field No. 23 .
25 Objective ID An alphanumeric assigned to each terminal or supporting objective tested by the Measured Item. ObjID Text 12 3.2
  1. List all the TPOs, SPOs or other high level objective grouping that apply to the Measured Item.
26 Objective Title A description of the objective ID ObjTitle Text 80 Perform Engine Fire Procedures
  1. A description of the objective ID in field No. 25.
27 Objective Type A description of the objective type ObjType Text 5 TPO or
SPO or
EO
  1. Identify the training objective as a TPO, SPO, or EO.

6.2.5 Data Analysis

The primary users of data reports are air operator personnel and Transport Canada.

  1. Air Operator  
     
    AQP requires that the data collection conducted by the air operator for its own use in monitoring curricula will support more analytical detail and diagnostic functions than the data collected for submission to Transport Canada. Transport Canada expects the air operator to do an in-depth analysis of the effectiveness of the training. Reporting of data is based on the analysis of the PPDB to provide information on the curriculum and participant groups (flight crewmembers, instructors/evaluators). Once the data is collected and entered into the data management system (PPDB), an analysis should be performed on the aggregate information. Statistical analysis of the proficiency data enables air operators to make an internal assessment of their performance. Air operators should tailor these processes and techniques to suit their own requirements. Each air operator’s data collection and performance assessment processes should be refined over time, based on their own practical experience. That is, the measures and processes should be optimized on an iterative basis to provide the degree of discrimination in flight crewmember performance needed to establish effective quality control over AQP curricula. 
     
  2. Transport Canada 
     
    The data submissions to Transport Canada are primarily ratings and reason codes associated with performance measures taken at validation and evaluation gates and supporting data. The data, presented to Transport Canada in the tables previously discussed is analyzed using the Canadian Data Analysis Reporting Tool (CDART) to allow a POI and other Transport Canada personnel to conduct trend analysis to monitor overall program effectiveness.

Report a problem with this page

Date modified: