Decisions

TATC File No. H-4113-27
MoT File No. 5802-379061

TRANSPORTATION APPEAL TRIBUNAL OF CANADA

BETWEEN:

James Christopher Beckstead, Applicant

- and -

Minister of Transport, Respondent

LEGISLATION:
Paragraph 6.71(1)(b) of the Aeronautics Act, R.S.C., 1985, c. A-2


Review Determination
Franco Pietracupa


Decision: December 17, 2015

Citation: Beckstead v. Canada (Minister of Transport), 2015 TATCE 22 (Review)

Heard in: Montreal, Quebec, on October 1, 2015

REVIEW DETERMINATION AND REASONS

Held: The Minister has not proven on a balance of probabilities that the applicant, James Christopher Beckstead, failed to meet the qualifications or conditions necessary for issuance of a Canadian aviation document. Therefore, the matter is referred back to the Minister for reconsideration.

I. BACKGROUND

[1] The Minister of Transport (Minister) issued a Notice of Refusal to Issue or Amend a Canadian Aviation Document (Notice) to the applicant, Mr. James Christopher Beckstead, on December 24, 2014, pursuant to paragraph 6.71(1)(b) of the Aeronautics Act, R.S.C., 1985, c. A‑2 (Act). Qualifications or conditions necessary for issuance were not met or fulfilled.

[2] The Notice reads as follows:

During the flight test that occurred on December 19th, 2014, you demonstrated that you did not meet the required standard for a recurrent DH-8 Pilot Proficiency Check (PPC) in that you failed to correct the Pilot Flying (PF) when he did not call for the required passenger evacuation as per AOM and SOP. This was a deviation of procedures that was not recognized and not corrected.

II. STATUTES, REGULATIONS AND POLICIES

[3] The basis for the refusal to issue is established under paragraph 6.71(1)(b) of the Act, which states:

6.71 (1) The Minister may refuse to issue or amend a Canadian aviation document on the grounds that

[…]

(b) the applicant or any aircraft, aerodrome, airport or other facility in respect of which the application is made does not meet the qualifications or fulfil the conditions necessary for the issuance or amendment of the document; or

[4] The applicable regulations are the Canadian Aviation Regulations (SOR/96-433) (CARs) 705.106 (1), which state:

705.106 (1) Subject to subsection (3), no air operator shall permit a person to act and no person shall act as the pilot-in-command, second-in-command or cruise relief pilot of an aircraft unless the person

[…]

(c) has successfully completed a pilot proficiency check, the validity period of which has not expired, for that type of aircraft, in accordance with the Commercial Air Service Standards;

[5] CARs Standard 725 - Airline Operations - Aeroplanes, Part VII Commercial Air Services, states:

725.106 (2) Pilot Proficiency Check (refers to paragraph 705.106(1)(c) Canadian Aviation Regulations)

(a) The pilot proficiency check (PPC) shall be conducted in accordance with Schedule I, Schedule II or Schedule III of this Section.

[…]

(c) A pilot proficiency check shall be conducted in a manner that enables the pilot to demonstrate the knowledge and the skill respecting:

(i) the air operator's aeroplane, its systems and components;

(ii) proper control of airspeed, direction, altitude, attitude and configuration of the aeroplane, in accordance with normal, abnormal and emergency procedures and limitations set out in the aeroplane flight manual, aeroplane operating manual, (if applicable), the air operator's standard operating procedures, the check list, and any other information relating to the operation of the aeroplane type;

(iii) departure, enroute and arrival instrument procedures and other applicable procedures; and

(iv) adherence to approved procedures.

[…]

(g) A proficiency check of a pilot-in-command shall be completed in the seat normally occupied by the pilot-in-command and a check of a second-in-command shall be completed in the seat normally occupied by the second-in-command. The pilot proficiency check shall consist of a demonstration of both pilot flying (PF) duties and pilot not flying (PNF) duties.

[6] CARs Standard 725, Schedule I - Pilot Proficiency Check (PPC) - Synthetic Training Device, states:

(2) Flight Phase

[…]

(h) Abnormal and Emergency Procedures

(i) The crew shall demonstrate use of as many of the air operator's approved Standard Operating Procedures and abnormal and emergency procedures for as many of the situations as are necessary to confirm that the crew has an adequate knowledge and ability to perform these procedures.

[7] The Transport Canada Approved Check Pilot Manual (TP 6533) Ninth Edition:

Chapter 4: Conducting the Flight Check

4.1 Flight Check Philosophy

4.3 The Flight Check

4.3.2 Pilot Proficiency Checks Schedule

4.6 Assessment of Performance

[8] The Transport Canada Pilot Proficiency Check and Aircraft Type Rating Flight Test Guide (Aeroplanes), First Edition, TP 14727E, 11/2007 (Flight Test Guide).

III. PRELIMINARY MOTIONS

[9] No preliminary motions or statement of facts were presented in regard to this review.

IV. EVIDENCE

A. Minister

(1) Mary Pollock

[10] Mary Pollock is a civil aviation safety inspector and an occupational health and safety officer with Transport Canada. Ms. Pollock testified that she is familiar with the failed check ride conducted by Geoffrey Craig Brown on December 19, 2014 on Mr. Beckstead, as well as the subsequent Notice issued to the applicant. She confirmed that Transport Canada supported the decision by the ACP (Approved Check Pilot) in regard to the assessment and failure attributed, as per the email dated February 19, 2015, and entered into evidence as Exhibit M-1.

(2) Geoffrey Craig Brown

[11] Geoffrey Brown is an approved check pilot for Jazz Aviation LP. His role with the company includes flight and simulator training, and administering check rides. He has been a company check pilot for 10 years and has accumulated approximately 8,000 hours of flight time. He confirmed that on December 19, 2014, he conducted, in simulator, the failed check ride on Mr. Beckstead.

[12] He stated that part of his role as an approved check airman is to ensure that all twenty-plus elements of the pilot proficiency check (Exhibit M-7) are completed successfully, including ensuring that the candidate is adhering to the approved operational manuals at Air Canada Jazz. These manuals include the COM (Company Operations Manual) (Exhibit M-5) and the SOP (Standard Operations Manual). Along with these items, the check airman must also evaluate and analyze the crew's CRM (Crew Resource Management) skills. This CRM skill is evaluated in all of the individual twenty-plus items also being assessed. Mr. Brown explained that CRM training is provided by the company annually, and its purpose is to ensure crews interact properly and efficiently on board.

[13] Mr. Brown testified that some elements of CRM comprise situational awareness and communication between crew members. He confirmed that Mr. Beckstead had been properly trained in 2014 in regard to CRM.

[14] The check ride scenario itself is conducted using an approved Transport Canada script (Exhibit M-6). This script serves as a guideline, ensuring the ride is properly administered using scripted failures and scenarios. The pre-brief prior to the check was conducted normally, and both pilots were properly briefed as to the role and responsibility each has during the check. Pilot Flying (PF) and Pilot Not Flying (PNF) duties were explained. Mr. Brown explained that both candidates that morning were Captains. Each would be assessed from the left seat as PF, and from the right seat as PNF. The crew coordination concept for each role was discussed during the pre-brief. No questions were raised by Mr. Beckstead during this time, and Mr. Brown had no issues in regard to the candidate being ready and fit to take this check ride.

[15] Mr. Brown introduced the flight test report (Exhibit M-7). He explained that the first Captain was PF during the first part of the check ride, and Mr. Beckstead would be flying right seat and acting as PNF. The ride went well up until the last exercise. A rejected takeoff scenario, as per the script, was introduced in the simulator. The scenario as scripted, was to introduce after the rejected takeoff, a forward baggage smoke condition that would lead the Captain to call for a passenger evacuation. During this drill, the Captain, PF in the left seat, omitted to call out the “evacuate, evacuate, evacuate” on the PA as per the company AOM (Aircraft Operating Manual) (Exhibit M-4).

[16] Mr. Brown testified that Mr. Beckstead, as PNF, should have corrected this omission from the Captain, as per the COM. He did not, and was thus attributed a failing score of 1 on item 21 (PNF Duties) of the flight test report. He did state however, that he would have given the applicant a re-test for this item. This can be done, as per the privileges of an approved check pilot (Exhibit M-2, the Pilot Proficiency Check and Aircraft Type Rating Flight Test Guide) but only if the candidate does not subsequently score a 2 or lower during the remainder of the evaluation check. Since Mr. Beckstead did in fact score a 2 on an item during his check ride as Captain left seat, this option was no longer available.

[17] Mr. Brown went on to explain the grading scale of the Flight Test Guide, in particular the grade of 1. A score of 1 is assessed if the pilot performs an exercise and performs outside the prescribed limits and standards expected, and then proceeds to not be aware of the error or correct the error in a timely manner. In the case of the evacuation command omission by the first Captain, Mr. Brown would expect the First Officer, i.e. Mr. Beckstead, to correct the Captain if an error is made. Since it was not done, the score was warranted in his opinion.

[18] The witness re-iterated that the COM does specify that one of the duties of the First Officer is to inform the Captain immediately and at any time he believes the aircraft is being handled improperly or placed in jeopardy. Since both pilots are assessed as a crew and operating under CRM, the expectations were that the First Officer pick up on the error that was made during the evacuation drill.

Cross-examination of Mr. Brown

[19] Under cross-examination, Mr. Brown was asked about his pass/failure rate as an approved check airman. He could not recall. He was asked if the rate would be higher when the ride is being conducted by a Captain/Captain duo. Mr. Brown stated that this should be no factor, as both are trained in either seat of the aircraft and for PNF duties.

[20] Mr. Brown does confirm that a conversation did take place between himself and the applicant as to the rescheduling of the failed check ride. He recalls that Mr. Beckstead would not be available the next day since he had to return home to attend to personal matters.

Re-direct of Mr. Brown

[21] During re-examination, Mr. Brown stated that the applicant did discuss some issues as to his personal life back home but felt that at no time was there an issue as to the fitness of the applicant to take the check ride.

B. Applicant

(1) James Christopher Beckstead

[22] Mr. Beckstead testified that all the items required to be completed by him, in his role as PNF right seat, were done properly. He explained that during the evacuation drill, and upon completing his specific items, the Captain asked him if he was “ready”. He assumed that this meant that all the items related to the passenger evacuation by both pilots had been completed. At no time did it occur to him to ask the Captain if he had completed his specific items in this drill.

[23] In response to the AOM (page 13/ 3.2-6) and more specifically to the First Officer duties, he testified that his dedicated items in the drill were completed properly and that when the Captain asked if he was ready, all the items had been done.

Cross-examination of Mr. Beckstead

[24] Under cross-examination, Mr. Beckstead confirmed that he had received right seat training and had flown right seat several times during the past year in the role of PNF. He also confirmed that he completed CRM training on a yearly basis.

[25] When asked if he felt he was fit to take the check ride, Mr. Beckstead responded yes. He also stated that he did not lose situational awareness. He re-iterated that when the Captain asked him if he was ready, he had assumed all items by both pilots were completed in the drill. He also responded that there is no mention or requirement in the AOM to ask the PF if he had completed his items. He testified that he did not notice that the evacuation PA was not completed. He did agree that if this was noticed, it would be his duty to inform the other crew member.

[26] Lastly, Mr. Beckstead agreed that the emergency evacuation is a very serious event on board.

V. ARGUMENTS

A. Minister

[27] The Minister's representative argued that the Minister has the obligation to apply the regulatory requirements in place in relation to the Canadian Aviation Regulations. A 705 operator, as is the case with Air Canada Jazz, must adhere to these regulations. In particular, the Minister points to the requirement of each pilot to be successful on a pilot proficiency check. To be successful, the applicants must demonstrate knowledge and skills in respect to aircraft systems, component's control, and normal, abnormal and emergency procedures. They must respect airplane operating limitations, standard operating procedures and checklists.

[28] With regard to approved procedures within the company, the Minister's representative said that during a pilot proficiency check, the pilot must demonstrate proficiency in both PF and PNF duties. He also made a point of mentioning that in keeping with regulatory requirements found in the CARs for 705 operators, as per Schedule 1 of Standard 725, under abnormal and emergency procedures:

The crew shall demonstrate use of as many of the air operator's approved Standard Operating Procedures and abnormal and emergency procedures for as many of the situations as are necessary to confirm that the crew has an adequate knowledge and ability to perform these procedures.

[29] The Minister's representative argued that Mr. Brown's testimony was concise and clear. The check ride conducted was reasonable, fair, and respected the approved recurrent PPC script. A re-test was considered but this option was unfortunately unavailable once Mr. Beckstead was issued a score of 2 on his pilot proficiency check as PF.

[30] The Minister's representative re-iterated that CRM and crew concept are important parts of the check being evaluated and that the two pilots are assessed as a one-team crew. The Minister argued that this philosophy is clear in the company's manuals and all Transport Canada publications in effect for this operator. The Minister also argued that it is clear from the applicant's testimony that he struggled with his own memory items and that his focus was on completing his own checks without paying attention to the whole context of the emergency evacuation. As such, the Minister submits that situational awareness was lost.

[31] The Minister further submits that it is clear from testimony that the applicant considers the emergency evacuation as a very serious event. The Minister's representative stated that due to the seriousness, no checklists are available, and the crew must act quickly and execute the drill from memory. Each crew member has his/her responsibility, and part of this drill is for the Captain to call for the evacuation and then conduct a PA call of “evacuate, evacuate, evacuate”. Mr. Beckstead should have picked up on the fact that this PA was not made. The “are you ready” call from the Captain's testimony is not in any company operating manual or CRM philosophy.

B. Applicant

[32] The applicant argued that he was not occupying his normal seat when the check ride was conducted. His normal seat on the line is the left one. A clear distinction exists between the duties of a PF and PNF, and a Captain and First Officer. He stated that he had never completed an emergency evacuation in real life.

[33] As per the Jazz AOM, during an emergency evacuation, which is characterized as a memory drill, a First Officer can be a PF for a given flight leg but cannot command an evacuation. If the Captain is the PM (or PNF) for this leg, he always has the responsibility of commanding an evacuation. In fact he/she are the only crew members (unless incapacitated) who can command an evacuation. The Captain initiates the evacuation and the PF/PNF roles disappear.

VI.  ANALYSIS

[34] I find that the flight test was, in general, conducted fairly and in accordance with the Pilot Proficiency Check and Aircraft Type Rating Flight Test Guide (Flight Test Guide). Testimony from both parties was credible and sincere. Throughout the review, a number of matters were generally agreed on by both the Minister and the applicant:

  1. Mr. Beckstead received right seat pilot training.
  2. The applicant received CRM training on a yearly basis.
  3. The applicant was fit to take the check ride.
  4. The passenger evacuation drill is a time-critical drill.

[35] The evidence and the interpretation of the regulatory requirements offered by the Minister's representative is not compelling enough for me to determine that Mr. Beckstead failed his pilot proficiency check.

[36] Consequently, in this analysis, I will address the three main factors that the Minister raised in testimony and documentary evidence that the applicant did not meet the qualifications or conditions necessary for issuance of a PPC on the DH-8.

1) Failure to correct the PF during the passenger evacuation drill as per the AOM and COM

[37] I will discuss the AOM (Exhibit M-4) and COM (Exhibit M-5). Both parties have agreed to the urgency and importance of a passenger evacuation. Both documents refer to this drill, which is categorized as time-critical. The COM does offer some guidance regarding these types of checklists/drills. According to page 22, the emergency evacuation drill falls into this category in that each pilotmust have the knowledge and apply the procedure without reference to any publication or material. As such, no cross reference to a checklist or quick reference checklist is expected. This drill is in fact a memory drill and each pilot must perform his/her respective memory items concurrently but separately.

[38] If we reference the AOM (page 3.2-6/ or page 13 in the exhibit), the emergency evacuation is a memory item drill performed at the command of the Captain. Based on testimony by Mr. Brown, the Captain did call for an evacuation. From this point on, as per the AOM, each crew member is expected to accomplish their individual memory items. This drill, as opposed to other checklists that are performed normally by both crew members as a read-and-do or challenge-and-response checklist, clearly spells out two distinct separate tasks; one is for the Captain and the other for the First Officer.

[39] At this point the First Officer, Mr. Beckstead, has six distinct actions to perform by memory, as per the summary table on this same page of the AOM. The first involves ATC communication in which he must declare the emergency to ATC and include information such as the runway or aircraft location, number of passengers and advise if any dangerous goods are on board. With this communication completed, he then proceeds to fasten his seat belt, ensures emergency lights are on, and if required, opens the escape hatch, deploys the escape rope and exits the aircraft. The Captain's memory items after advising of evacuation comprise three items: shut down engines, PA to passengers “Evacuate, Evacuate, Evacuate”, and finally, ensure “Batteries & APU or EXT PWR – OFF”.

[40] It is clear that based on the time-critical element of this action, the crew is not expected to call for a subsequent checklist or quick reference card. Each crew member is expected to know and execute their respective memory item drills. From testimony, the only person that did carry out the drill properly was Mr. Beckstead.

[41] It is critical we understand the sequence of events once the Captain has called for an evacuation. The First Officer is expected to immediately proceed to contact ATC. Simultaneously, the Captain would now be initiating his actual drill items. In establishing the radio contact with ATC, the First Officer is at this point out of the loop and understandably so. He is in conversation with ATC and ensuring the information required is transmitted properly and clearly. Based on many check ride experiences both as a pilot and check airman, the cockpit is now divided in two.

[42] The sequence inevitably brings both crew members back into the loop but only at the very end of the drill. In all probability, it would be almost impossible for the First Officer to be on the radio with ATC and then complete his respective drill, while at the same time being expected to observe and correct the Captain's own memory items. The AOM or COM does not call for any further checklists or reference cards to ensure that each item was completed. Based on the time-sensitive nature of the evacuation, the crew must vacate the aircraft as soon as possible. Mr. Beckstead testified that he did not notice that the Captain had failed to call for an evacuation. He acknowledged that had he noticed that the Captain had missed an item during the drill, it would have been his duty to inform him. I find that the Minister has mis-interpreted that the AOM or COM required Mr. Beckstead to be responsible for correcting the Captain's memory item failure, while those same documents require him to be simultaneously executing his own memory items.

2) Cockpit Resource Management/crew concept

[43] I agree with the Minster that CRM is not only critical in all phases of flight, but also that it must be properly evaluated throughout the totality of the check ride. The Flight Test Guide clearly provides guidance in this matter. Testimony and evidence presented to the Tribunal indicate that CRM was observed as meeting standards throughout the duration of the first check ride (Exhibit M-7). No issues were detected or observed based on the overall scores on all the items assessed up until item 21.

[44] The Flight Test Guide (page 6) defines CRM as the effective use of all available resources; human resources, hardware and information. The definition further states that CRM is not a single task, but a set of competencies which must be evident in all exercises. Furthermore, page 6 and 7 specify that CRM evaluations are largely subjective and that examiners are required to exercise proper CRM competencies in conducting tests; a fail judgement cannot be based solely on CRM issues since CRM may be entirely subjective.Based on Mr. Beckstead's overall performance, I would find it difficult to see CRM being a factor during the evacuation procedure drill. I note that Mr. Brown could not recall if the Captain had asked the First Officer if he was “ready” at the end of the drill. This would seem, in all probability, to be something that could have been asked to ensure that the crew would then proceed to the final items on the check. This, along with the CRM displayed throughout the ride, would seem to counter the Minister's claim that CRM was an issue during this particular drill.

[45] Although the PPC is based on individual assessment of pilots, it is most important that the crew concept philosophy remain at the centre of the evaluation on a multi-crew aircraft, (as per the Flight Test Guide, page 3). Mr. Brown conducted the check ride using the approved PCW-2 script (Exhibit M-6). Mr. Brown testified that the passenger evacuation was the final item on the first check. Based on the script, the scenario has at the very minimum four emergencies that would involve abnormal checklists, CRM skills and crew coordination. Again, no issues or concerns were raised regarding crew concept during the three hours of the first session. Though both crew members are in jeopardy throughout the ride, the ACP must be able at times to evaluate each pilot separately in certain circumstances to ensure that they meet the required standards for issuance of a licence. I would normally expect this can be the case during the emergency evacuation drill, in which each crew member must perform individually during the memory item drill. Assessing Mr. Beckstead a failing grade for this particular circumstance is, in my opinion, wrong.

3) PNF duties – grading score of 1 and loss of situational awareness

[46] Mr. Brown attributed a score of 1 to Mr. Beckstead in regard to his PNF duties (page 39 of the Flight Test Guide refers), and in particular to the emergency evacuation drill. He supports this score in the comment section of the flight test report, indicating “did not correct PF when he did not call for a passenger evacuation as per Company AOM”. I have already discussed the AOM and COM references in regard to the non-guidance or requirement to correct the Captain, if recognized, during this individual drill, but I find that it is important we look at the grading score of 1 as it pertains to the specific actions performed by Mr. Beckstead. The Flight Test Guide (pages 11 to 13) refers to the six elements that are evaluated according to a four-point marking scale and provides the following guideline: A mark of 1 or 2 must be aligned to a safety issue, a qualification standard (performance criteria), or an approved technique or procedure.

[47] The individual tasks required to be accomplished by the applicant as First Officer were completed. The aircraft was depressurized and ATC was informed of the evacuation as per the company procedure. Aircraft safety was not a factor at this stage. The qualification standard associated with this drill, as a First Officer, was met using the approved emergency memory item drill. It is difficult to see how a score of 1 can be attributed to the applicant based on an error of omission by the Captain, when the First Officer would be out of the loop, on the radio with ATC and providing information related to the evacuation. I also note that there are no post-checklists to correct any error or deviations associated with this emergency action and that this drill is carried out simultaneously by each pilot. I agree that the PNF must always be able to correct the PF when conditions in-flight are warranted. This should be done in all normal, abnormal and emergency situations. However, in a situation of time-critical drill in which the crews are split into two clear separate taskings, asking one to complete his or her items and then have the responsibility to observe, listen and correct the other pilot would seem highly unrealistic. I am sure the First Officer (Mr. Beckstead) would have corrected the Captain during this drill but only if he had noticed, or may have noticed, an omission.

[48] I have no reason to believe that in all probability the Captain would have mentioned something to the First Officer at the end of the drill to see if he would be ready to evacuate the aircraft. A reasonable conclusion by the First Officer would be to assume at this stage that all the items had been completed. Scoring this as a failing grade of 1 would be, in my opinion, unwarranted.

[49] The Minister reproaches the applicant for having lost situational awareness (one of the six elements evaluated as specified in the Flight Test Guide, page 11) but at no time was situational awareness explained by the Minister. I note that on page 13 of the guide, under “Situational Awareness”, it states:

General

a. resides in the candidate's mind and can only be assessed by monitoring behaviour

Behaviour

b. actively monitors weather, aircraft systems, instruments, ATC communications

c. avoids tunnel vision and fixation

d. stays “ahead of the aircraft”, stays “with the aircraft”, gets “behind the aircraft”

Identification and correction of errors

a. Oops!, Slips and Lapses

b. are some errors going undetected or uncorrected?

[50] As mentioned before, correcting the PF duties seems unrealistic for this specific drill. In the aviation industry, and as defined by SKYbrary Eurocontrol, situational awareness is commonly known to be:

…having a mental picture of the existing inter-relationship of location, flight conditions, configuration and energy state of your aircraft as well as any other factors that could be about to affect its safety such as proximate terrain, obstructions, airspace reservations and weather systems.

[51] Based on the testimony from Mr. Brown, other than the Captain omitting to call for the evacuation on the PA, the crew was fully engaged in the need to evacuate this aircraft. As per the COM, page 4.2-21 (page 25 in the exhibit), the crew can apply the PACE (Probing, Alerting, Challenging and Emergency Warning) CRM tool in the case of a loss of situational awareness. This company procedure enables the crew to respectfully escalate and challenge the other crew member to ensure both have the same mental model.

[52] I note from testimony from both witnesses that the crew did properly reject the takeoff and then conclude that an evacuation was required. Both proceeded at this stage to initiate their respective memory items. PACE can be applied if a threat and error condition exists. I note that based on the evidence provided, this was not the case at any time during the check ride and not up to the time the evacuation was ordered by the Captain.

[53] Further, Transport Canada's Approved Check Pilot Manual (Exhibit M-3), section 4.3.2, states that a realistic Flight Check environment will result in an effective assessment. To have the First Officer now challenge the Captain on a missing item of the memory drill, during a critical time-sensitive event when he is unaware that this step was omitted, is unrealistic. Add to this that no checklist is required to verify that the items are completed, then the First Officer must, in all probability, assume the action items were completed.

VII. DETERMINATION

[54] The Minister has not proven on a balance of probabilities that the applicant, James Christopher Beckstead, failed to meet the qualifications or conditions necessary for issuance of a Canadian aviation document. Therefore, the matter is referred back to the Minister for reconsideration.

December 17, 2015

Franco Pietracupa

Member