TATC File No. H-3927-27
MoT File No. 5802- 298135
TRANSPORTATION APPEAL TRIBUNAL OF CANADA
Jeffrey David Karelsen, Applicant
- and -
Minister of Transport, Respondent
paragraph 6.71(1)(b) of the Aeronautics Act, R.S.C. 1985, c. A-2
Decision: February 27, 2014
Citation: Karelsen v. Canada (Minister of Transport), 2014 TATCE 9 (Review)
Heard in: Toronto, Ontario, on December 5, 2013
REVIEW DETERMINATION AND REASONS
Held: The Minister of Transport has not proven, on the balance of probabilities, that the applicant, Jeffrey David Karelsen, failed to meet the qualifications or conditions necessary for issuance of a pilot proficiency check. Therefore, the matter is referred back to the Minister for reconsideration.
TATC File No.: H-3927-27 (Jeffrey David Karelsen)
TATC File No.: H-3928-27 (Paula Cathy Nicolle)
 On October 29, 2012, the Minister of Transport (Minister) issued Notices of Refusal to Issue or Amend a Canadian Aviation Document to the applicants, Captain Jeffrey David Karelsen and First Officer (F/O) Paula Cathy Nicolle, with respect to failed pilot proficiency checks (PPCs) on October 28, 2012, pursuant to paragraph 6.71(1)(b) of the Aeronautics Act, R.S.C. 1985, c. A‑2 (Act).
 Captain Karelsen's and F/O Nicolle's requests for review were filed with the Transportation Appeal Tribunal of Canada (Tribunal) on November 21 and 22, 2012, respectively.
II. STATUTES AND POLICIES
 The basis for the refusal to issue is established under paragraph 6.71(1)(b) of the Act, which reads as follows:
6.71 (1) The Minister may refuse to issue or amend a Canadian aviation document on the grounds that
(b) the applicant or any aircraft, aerodrome, airport or other facility in respect of which the application is made does not meet the qualifications or fulfill the conditions necessary for the issuance or amendment of the document
 Excerpts from sections 3.1, 4.1, 4.3, 4.3.2, and 4.6 of the Approved Check Pilot Manual, 9th edition, TP 6533 (ACP Manual), read as follows:
3.1 AIM OF THE FLIGHT CHECK
Flight Checks conducted under and Part VII of the Canadian Aviation Regulations consist of Pilot Proficiency Checks (PPCs) and Line Checks.
The aim of a PPC is to:
(a) determine that the candidate/crew meets the skill requirements to fulfill their assigned responsibilities in a safe and competent manner for the PPC, Line Check or rating sought;
(c) to ensure acceptable levels of safety are maintained and, where possible, improved throughout the aviation industry, by requiring the application of sound airmanship and flight discipline.
4.1 FLIGHT CHECK PHILOSOPHY
Flight test principles intent is to focus on Threat and Error Management strategies and performance where it is recognized that from time to time, errors or deviations from standard practices will occur. While undesirable, it is a fact that flight crew or others associated with flight operations will make errors and that these errors if not recognized and managed properly could have serious consequences [italics added].
Today's Flight Check evaluators must recognize the potential safety threat for any given situation or commission of errors, and then determine the effectiveness of crew actions in managing the situation so as not to jeopardize safety.
4.3 THE FLIGHT CHECK
The ACP must create an environment conducive to a true demonstration of the pilot's ability [italics added].
4.3.2 Pilot Proficiency Checks
A realistic Flight Check environment will result in an effective assessment.
The PPC schedules define the PPC as a requirement for the flight crew to demonstrate their ability to safely operate a specific type of aircraft throughout the normal, abnormal, and emergency flight envelopes set out in the AFM, HFM, AOM, QRH, and SOPs [italics added].
4.6 ASSESSMENT OF PERFORMANCE
An ACP may also tolerate an excursion from specified limits in the performance criteria if the candidate recovers in a timely manner. However, an excursion from prescribed limits, with or without a timely recovery, which jeopardizes the safety of the aircraft is unacceptable. [italics added].
 Under the section titled Failure of a PPC, in the Transport Canada publication, Pilot Proficiency Check and Aircraft Type Rating Flight Test Guide (Aeroplane) (Flight Test Guide), TP 14727, a failed PPC is described as follows:
When an ACP assesses at least one sequence or item as “(1)”, the flight check will receive a General Assessment of “Failed”. A PPC that has five or more sequences or items assessed as “(2)” will also receive a General Assessment of “Failed” A PPC that has less than 5 sequences or items assessed as “(2)” and the remainder of sequences rated as “(3)” or “(4)” will receive a General Assessment of “Pass.”
III. PRE-HEARING RULINGS
 Captain Karelsen submitted three motions in writing to the Tribunal.
A. Motion of October 3, 2013
 The first motion requested disclosure from the Minister of the names of the health care providers for Captain Rodney Dahl, the approved check pilot (ACP) who conducted the check ride. The applicant argued that the names of the health care providers are important to the case and relevant to his perceived excessive failure rate of the ACP. In the ruling on the motion, dated October 23, 2013, the Tribunal was not convinced that the applicant had demonstrated this relevance, as no clear nexus between the health issues that existed at the time the ACP took sick leave, four months after the check ride, and the allegedly high failure rate, was demonstrated. The Tribunal also found that the names of Captain Dahl's health care providers would not be required for the applicant to have a full and fair hearing before the Tribunal. As such, the motion was dismissed.
B. Motion of November 5, 2013
 The second motion requested from the Minister the following data during the period of January 1, 2012 to February 28, 2013:
- The failure rate of PPCs and instrument ratings for Jazz Air as compared to the Canadian industry average for the stated period;
- Captain Dahl's failure rate and the amount of marks of “2” (Basic Standard) given per check ride as compared to all other Jazz Air check pilots, and compared to Canadian industry standards.
 In the ruling on the motion, dated November 18, 2013, the Tribunal found that the information sought by the applicant had no relevance to the case before the Tribunal. Captain Dahl was fully qualified and current in his role as ACP at the time of the check ride. As such, the motion was dismissed.
C. Motion of November 11, 2013
 In the third motion, the applicant requested the handwritten notes and comments taken by Captain Dahl during the check ride in question. The Minister agreed to submit the notes to the applicant, and no ruling was required by the Tribunal.
(1) Mary Pollock
 Mary Pollock is a civil aviation safety inspector and an occupational health and safety officer with Transport Canada. Ms. Pollock testified that she is familiar with the failed check ride conducted by Captain Dahl on October 28, 2012, as well as the Notice of Refusal letters issued to Captain Karelsen and F/O Nicolle following the failure on October 28, 2012 (Exhibits M‑1 and M‑2).
(2) Captain Rodney Dahl
 Captain Rodney Dahl has been a pilot since 1986, employed by Jazz since 1990, and an ACP since 2000. He explained that he was assigned to conduct the recurrent PPCs on Captain Karelsen and F/O Nicolle on October 28, 2012. A pre‑flight briefing was provided to both candidates following the company guidelines and the crew appeared relaxed. He testified that he used an approved check ride script for a recurrent PPC on a CRJ-200 (Exhibit M‑3), which provided him with the general guidelines in conducting the pre-flight briefing and the flight test scenario.
 Captain Dahl explained that the check ride was conducted in an approved Transport Canada simulator at the Canadian Aviation Electronics (CAE) training site in Toronto, Ontario. He verified the simulator logbook in order to ensure that the simulator had no deficiencies. He explained that the two pilots were flying as a crew for this check ride and that this would be a recurrent check ride for both of them. Captain Dahl explained his role during the check ride, including the mandatory items that needed to be assessed in the course of the PPC. He stated that he explained to them that crew coordination was an important element being observed, and that any mistake made on the part of the crew cannot be corrected by him.
 Captain Dahl was asked by the Minister to review Captain Karelsen's Flight Test Report (Exhibit M‑10), and in particular, the items that had been assessed a “2”. He testified that on Item 6, Take-off, an evaluation of “2” was attributed since Captain Karelsen had not performed a standing take-off as required by the aircraft operating manual (AOM) during low visibility conditions. He explained that the weather had been set at 600 feet (ft.) runway visual range (RVR) and that, although Captain Karelsen had briefed the F/O that he would be conducting a standing take-off, all take-offs were conducted as rolling take-offs. He testified that standing take-offs are required as per company standards in visibility below 1200 ft. RVR. He also explained that Captain Karelsen did not explicitly identify the take-off runway during the take‑offs he conducted during the check ride. He added that he did not feel that safety of flight was compromised.
 Captain Dahl was asked to briefly explain the four-point evaluation scale used by Transport Canada for PPCs. He explained that a score of “4” indicates a performance deemed Above Standard, a “3” means Standard, a “2” means Basic Standard, and a “1” means Below Standard and a failed item.
 Captain Dahl testified that on Item 2, Flight planning, the break release fuel amount entered into the flight management system (FMS) was contrary to AOM procedure, and caused a 300 pound (lb.) split between the FMS, and the engine-indicating and crew-alert system (EICAS) fuel indication. As such, the risk of inaccuracies in the programmed fuel total was poorly mitigated because the crew had not properly estimated their auxiliary power unit (APU) fuel burn. On Item 17, Go-around, Captain Dahl explained that the crew had to, as per global navigation satellite system (GNSS) navigation limitations specific to the AOM, monitor the underlying navigational aid (NAVAID) tracking system related to the missed approach facility. Again, he felt that the crew had poorly mitigated the risk.
 On Item 27, Passenger evacuation, Captain Dahl testified that Captain Karelsen did not follow the order of the drill, and this happened not once but twice. He clarified his assessment in testifying that the crew should conduct this drill in the sequence that the aircraft manufacturer has certified for the procedure. Since the sequence was not performed in the proper order, a “2” was attributed.
 Captain Dahl explained that after the break, the crew switched roles and Captain Karelsen, still captain seated in the left seat, took on the role of pilot not flying (PNF) for the second part of the evaluation. F/O Nicolle was now being evaluated as the pilot flying (PF). On Item 21, PNF Duties, Captain Karelsen was given a failing mark of “1”. Captain Dahl explained that a right engine oil pressure warning message had been programmed into the simulator. He stated that Captain Karelsen was excessively slow in diagnosing and actioning the correct warning checklist, resulting in an N2 shaft seizure, and severe engine damage.
 Captain Dahl testified that when the simulator is programmed to depict low oil pressure in an engine, the crew will hear a triple chime, see a master warning red light illuminate, on the glare shield, followed by an oral message. He stated that all of these triggers occurred. He further explained that F/O Nicolle, as PF, requested the emergency checklist.
 He testified that, at that point, Captain Karelsen had difficulty finding the applicable checklist and took a considerable amount of time to eventually find the correct checklist. In Captain Dahl's opinion, Captain Karelsen took between five and ten minutes to action the correct checklist, whereas the correct procedure when using the proper checklist from the start would have taken Captain Karelsen approximately two to three minutes. Captain Dahl explained that an engine operating at low oil pressure could seize and suffer severe engine damage, which is why the extra minutes were excessive.
 Captain Dahl explained that at approximately the five-minute mark of this malfunction event, a vibration was felt, a bang was heard and the N2 core seized rapidly, indicating a value of zero. He stated that these are all signs of an engine N2 seizure. He further testified that crew communication during this event was minimal and that Captain Karelsen was heard telling the PF, F/O Nicolle, to basically hang on to it, and that he would be shutting down the engine shortly. Captain Dahl was expecting the crew to proceed to an emergency drill, but when he heard the crew proceeding to an engine shutdown checklist instead, he decided to terminate the check ride. Captain Dahl testified that in severe engine damage drills, his expectations are for the crew to determine the appropriate drills and checklists within 60 to 90 seconds.
 Captain Dahl stated that the message annunciator left little doubt that this was a red master warning message and that the crew should have gone to the red emergency section of the Quick Reference Handbook (QRH) (Exhibit M‑5). He also elaborated that lower grade malfunctions are normally found in the amber (or yellow) abnormal section of the QRH. He further explained that some amber messages can be found in the red section of the QRH and that crews are properly trained regarding these anomalies. He stated that each respective chapter of the QRH also has an index page that would guide the crew as to which page to proceed to.
 Captain Dahl testified that in the QRH under the respective power plant index pages for both the emergency red section and the abnormal amber section a reference for a right engine oil pressure warning can be found. He explained that this reference in the index page for the abnormal power plant section was a mistake, as it was a misprint when this version of the QRH was being produced. He stated that this reference should not have been placed in the amber abnormal power plant section, but rather the red power plant section.
 Captain Dahl explained that Captain Karelsen, faced with a red master warning message and oral warning, should have proceeded to the red emergency power plant section of the QRH; instead he went to the abnormal amber power plant section for the fluctuating oil pressure checklist. Captain Dahl explained that Captain Karelsen was confused when he began reading this checklist as it did not pertain to the message and indications he was seeing, and stated that he should have self-corrected himself and proceeded to the red section, but instead wasted precious minutes. He stated that the fluctuating oil pressure checklist, if actioned properly, would have led to the correct checklist. Captain Karelsen did eventually get to the proper red emergency section of the power plant checklist. He stated that this took approximately five to ten minutes.
 When the crew did eventually proceed to the correct section of the QRH, the engine had seized. In his view, had the crew correctly consulted and executed the proper checklist, the engine seizure could have been avoided. The mark of “1” was attributed to both pilots due to the fact they did not deal with the malfunction in a timely matter.
 Captain Dahl reiterated that crew communication during the malfunction was minimal and that too much time was taken to properly assess the situation. He stated that had the crew secured the engine after it had seized, he would have still evaluated the item as a “1” because they had not achieved the objective of the exercise since they were not able to secure the engine with the right engine oil pressure warning message checklist, which would have led to other checklists that would have advised them how to safely shut down the engine. He stated that they were not able to accomplish that in a timely manner.
 Captain Dahl discussed the items marked as “2s” on F/O Nicolle's Flight Test Report (Exhibit M‑11). Item 2, Flight planning, Item 6, Take-off, and Item 21, PNF Duties, were marked as “2s” based on the same Basic Standard displayed by Captain Karelsen. On Item 24, right engine oil pressure, F/O Nicolle was the PF and, as such, her primary task was ensuring the safety of the aircraft.
 Regarding Item 24, Captain Dahl stated that F/O Nicolle did call for the emergency checklist when the master warning message appeared. From that point on, however, the communication between the PF and PNF was minimal. He stated that after the engine had seized and a bang had been heard, she should have been communicating with the PNF to reassess the situation. Captain Dahl expected F/O Nicolle to step up and start getting involved in a timely manner in order to correct Captain Karelsen's mistakes; as she did not, a mark of “1” was attributed.
 During cross‑examination, Captain Dahl was asked if he could recall a conversation in which he asked F/O Nicolle to subtract 300 pounds (lbs.) from the weight and balance, but he responded that he could not. He was also asked if the company had a definition for a rolling take‑off. Captain Dahl responded that he would need to verify the documentation, but could not recall in which manual it would be found. In regard to runway identification, Captain Dahl was asked if this requirement was to be done verbally. He responded that in training environments, crews are trained to verbalize the runway confirmation requirement. He stated that in all the observed take‑offs conducted by Captain Karelsen, including the runway repositioning ones, no verbal confirmation of the runway was heard.
 Captain Dahl was asked he if was aware of a printing error in the QRH version that was used during the flight test, specifically in regard to the abnormal power plant index page indicating a right engine oil pressure warning message (Exhibit M‑5). He responded that he was made aware of the printing error in the QRH when advised by Captain Karelsen in the weeks following the PPC. Captain Dahl was also asked if he knew the timeframe programmed into the simulator with respect to the engine seizure once the low oil pressure fault had been inserted and activated. He replied that he was unaware of that information.
 Captain Dahl testified that an indication of zero on the N2 display does not necessarily signify severe engine damage. He explained that an associated bang, as well as an increase in inter-turbine temperature (ITT), are other indications that an engine is experiencing severe damage. When asked if he could recall at what volume the simulator ambiance sound was set at, his response was that he could not recall, but that the crew did not voice any complaints regarding the sound level.
 Finally, Captain Dahl was asked if he observed Captain Karelsen originally going to the emergency power plant tab when the right engine oil pressure warning light came on. Captain Dahl responded that he did, at one point, stand and place himself between the two pilots after realizing that Captain Karelsen had some difficulty with locating the appropriate section in the QRH, and observed him in the abnormal section of the QRH. He could not recall if Captain Karelsen had initially proceeded to the red section of the QRH.
 Captain Dahl acknowledged that certain exceptions are found in the QRH in regard to colour symbology. He stated that as an example, the thrust reverser unlock message would appear on the EICAS screen as an amber message, but the appropriate checklist in the QRH is found in the red section. Captain Dahl explained that these exceptions are made aware to the crews in training so as to avoid confusion.
 Captain Dahl was asked why Captain Karelsen's performance in regard to Item 24 of F/O Nicolle's Flight test Report, Oil Pressure, was not discussed at the debriefing. He explained that the crew engaged him in regard to the failed item in the simulator once the check ride had been terminated. Although not preferable, he chose to discuss the failure there since the crew was upset. Once they were in the actual debriefing room, the crew was asked if they had any other issues, and none were mentioned.
 Captain Dahl also explained that no discussion during the debriefing took place in regard to the misprint in the QRH. He verified that the power plant emergency section had the correctright engine oil pressure warning tab and was satisfied that the crew would have been able to properly navigate this fault in the red emergency section. He does not recall being made aware of the error in the abnormal section of the QRH until sometime later, and cannot recall when it was noticed.
 Captain Dahl clarified that the ACP's position in the instructor operating system (IOS) seat during the check ride limits one's movements. He testified that he needed to stand up and position himself over Captain Karelsen's shoulder in order to confirm and verify which checklist he was using. He confirmed that this type of intervention should be kept to a minimum.
 On further cross‑examination, Captain Dahl was asked if he could recall telling F/O Nicolle to change the fuel in the FMS by 300 lbs., but Captain Dahl responded that he could not.
 During re-direct, Captain Dahl was asked what a crew should do when they observe a red light warning message being annunciated. He responded that the PNF is required to cancel the master warning light, and verbalize the message that is seen; and that the PF is tasked with calling for an emergency checklist, or if required, the appropriate drill in a given situation.
 Captain Dahl was asked by the Minister if he could recall Captain Karelsen mentioning an error in the QRH with regard to the right engine oil pressure warning message being in the abnormal section. He replied that he could not explicitly recall that, but does recall looking for that information in the QRH, but not seeing an obvious problem. He further explained that, after the debriefing, he investigated why Captain Karelsen ended up in the abnormal power plant, fluctuating oil pressure checklist, but he did not elaborate further in testimony.
(1) Captain Jeffrey David Karelsen
 Captain Jeffrey David Karelsen testified that for the initial right engine oil pressure warning, he proceeded to the red emergency section index of the QRH. He admitted that he did not initially see the reference tab to the right engine oil pressure warning message in the emergency index. He proceeded to the abnormal section index in the QRH as he knew that some red messages can be found there and vice versa. Once in the abnormal section for the right engine oil pressure warning message on page 1-16, he found himself in the fluctuating engine oil pressure checklist. Captain Karelsen admits that this was an error on his part, but that he was led to this page by way of the error in the QRH's abnormal power plant index page.
 Captain Karelsen explained that once the checklist error had been detected, he informed the PF that he would still quickly review this abnormal checklist as some abnormal messages are found in the red section and vice versa. He stated that this did eventually lead him to the correct checklist, which is the emergency power plant checklist at page 1-13. Captain Karelsen explained that this checklist was actioned correctly. The red emergency Checklist at page 1-13 has the crew proceed to single engine procedure – in-flight engine shutdown, in the abnormal section at page 1-1, if two of the three indications are displayed. Captain Karelsen testified that this was the case and proceeded to this section.
 Captain Karelsen began the abnormal single engine procedure – in-flight engine shutdown. Part of this checklist requires the crew to decide if the shutdown is due to suspected or intentional engine damage. He stated that Captain Dahl interrupted the check ride as he began to ask the PF this question. He acknowledged in testimony that the N2 had seized at that point. He further stated that the opportunity to action the severe damage drill, whether it was required in this situation, was denied the candidates because the ACP stopped the ride at precisely the point that the checklist would have prompted the crew to discuss the reason for the shutdown.
 Captain Karelsen testified that during this malfunction sequence, the crew established proper communication, were in the process of securing the engine; that situational awareness was not lost, and that flight safety was never compromised. He explained that an N2 instrument indication of zero alone is not a clear enough indication of an engine experiencing severe damage as noted in the airframe/engine maintenance training manual for the aircraft (Exhibit A‑2). He stated that multiple symptoms need to be established prior to proceeding with an engine severe damage drill. He testified that no loud bang was ever heard by the crew.
 Captain Karelsen explained that the error in the QRH abnormal power plant index played a large part in his ability to secure the engine in a timely fashion. Furthermore, when Jazz Air informed Transport Canada of this misprint, an internal memo was issued by the company on November 16, 2012, informing the pilots of the error in the QRH (Exhibit A‑1).
 Captain Karelsen testified that, as per the Company Operations Manual (Exhibit A‑1), runway identification procedures do not necessarily need to be performed verbally. As for the weight and balance issue, he recalled the ACP leaning forward and instructing F/O Nicolle to take 300 lbs. off for the taxi. Captain Karelsen does not dispute the assessment given in regard to Item 17 of his Flight Test Report, Go-around. As for Item 27, Passenger Evacuation, Captain Karelsen believes that the drill was well executed, but does concede that one or two items may have been out of sequence.
 In commenting on the conduct of the check ride, Captain Karelsen stated that he took exception to some inappropriate comments made by the ACP during and after the check ride. In particular, he testified that the ACP interrupted the crew during a flight spoiler fault checklist, positioned himself between the crew members in the simulator, and made negative comments regarding his ability to navigate the checklist.
 Under cross‑examination, Captain Karelsen was asked when he received a copy of his check ride report. He stated that he received it on November 2 or 3, 2012. He was also asked why he did not bring the error in the QRH to the attention of the ACP. Captain Karelsen responded that he only noticed the error in the QRH sometime after the check ride.
 Captain Karelsen was asked by the Minister what his actions were when he saw a red warning light, followed by three chimes, followed by an oral message. Captain Karelsen stated that the PF called for the emergency checklist, that he went to the red emergency index section in the QRH for power plant, but missed the right engine oil pressure warning message tab. He admitted that the section exists, and that he made an error and missed it. As he did not see the tab, he proceeded to the abnormal section of the QRH and saw the message in that section.
 Captain Karelsen stated that, once in the abnormal section, he informed the PF that this was not the correct checklist, but nevertheless read through it in the hope that it would lead him to the correct checklist. He estimated that just under three minutes elapsed from the time he was directed by the PF to proceed to the emergency checklist, to the time he was in the correct section, which is page 1-13 of the red emergency section of the QRH. Once there, it would have taken him another two minutes to action this checklist. He was asked if a loud vibration or bang was heard in the simulator during this event, but stated that he did not hear anything.
 Captain Karelsen explained that 30 to 45 seconds had elapsed, once he was in the correct red section, prior to the engine seizing and shutting down. He stated that no severe engine damage drill was conducted as he did not think that the engine had experienced severe damage, based solely on an N2 indication of zero. Captain Karelsen clarified as well that both he and the ACP were unaware of the error in the abnormal index page at the time that the check ride was terminated.
 When asked if he had verbalized the runway identification as required by the Company Operations Manual, Captain Karelsen responded that he verbally identified the runway on the first take-off, but could not recall if he verbalized the runway identification on any other subsequent take-offs, including the ones that had been repositioned by the ACP. Captain Karelsen did not admit to conducting any rolling take-offs during the check ride.
 On examination by F/O Nicolle, Captain Karelsen testified that he underwent additional training and successfully passed his PPC approximately one week after the failed attempt.
 With respect to the failed check ride, he did not recall the ACP referencing any notes during the debriefing. He was asked if during the debriefing, as outlined in the ACP Manual, any strengths were mentioned by the ACP regarding the check ride. He responded that none were mentioned.
 Captain Karelsen explained that at the initial master warning message, the PF, F/O Nicolle, instructed him to proceed to the emergency checklist. He also recalled the PF indicating to him that the fluctuating oil pressure checklist was the incorrect one, and that he explained that he was aware but would nevertheless read through it without actioning the items mentioned. He also mentioned that, at this point, the ACP had taken a position between them, which did not help the situation.
C. Captain Rodney Dahl, recalled by the Minister
 Captain Dahl testified that he does not recall any communication between the pilots in regard to them being in the wrong checklist. Captain Dahl evaluated and marked this item as a “1” because the situation got away from the pilots and they lost situational awareness. He also confirmed hearing a loud bang in the simulator. Captain Dahl was asked by the Minister if he could recall advising either of the pilots to remove 300 lbs. from the fuel load programming. He could not recall if he had said that, but also that he was not saying it did or did not happen.
 Captain Dahl explained that the crew was visibly upset after the check ride. He stated that discussions were held in the simulator after the check ride was terminated, but this was due to the crew engaging the ACP. Captain Dahl explained that he would not normally debrief in the simulator.
 The Minister argues that the preponderance of proof is clear that Captain Karelsen made a huge error by not proceeding to the applicable section in the emergency index tab of the QRH. He made an error and proceeded to the abnormal section of the QRH instead. As a result of this failure, and directly predicated on this initial mistake, which had nothing to do with the error in the index page for the abnormal section, the ACP evaluated the performance on this item as excessively slow, indicating that the aim of the task was not achieved. The master warning was clear: the crew should have immediately gone to the emergency section of the QRH. The error in the index for the abnormal section of the QRH was minor and a red herring used by Captain Karelsen. The malfunction, if dealt with properly, should have taken the crew no more than two minutes, and not the five or more minutes taken.
 With regard to F/O Nicolle, the PF at the time, the Minister argues that once the initial call for the emergency checklist had been made, no other communication or assistance was provided to the PNF. When she saw that the PNF was confused and unsure as to the correct checklist to use, she should have provided guidance and leadership, and redirected him to the emergency section of the QRH.
 The Applicants argue that the error in the QRH was not a minor one, but a major error that could have happened not only in training, but also on the line with other pilots. He argues that the error was corrected by the company within weeks of the event. He further submits that the opportunity to action the emergency drill during the engine malfunction was denied to the crew, since the ACP terminated the check ride too quickly.
C. Minister's Rebuttal
 The Minister submits that it would be normal for the company and Transport Canada to address the error in the QRH promptly. He also argues that the crew should never have been in the abnormal section of the QRH in the first place. The malfunction was clearly a red warning message, and the crew had to proceed directly to the red emergency section of the QRH. Their actioning of this master warning message was excessively slow, and based on the huge mistake of proceeding to the abnormal section, which Captain Karelsen admitted.
 I have carefully considered all of the evidence and have found that testimony from the Minister's witnesses and the applicants was generally credible. The applicants raised some issues in regard to the general environment during the check ride and the demeanour of the ACP. However, I have heard no evidence to support this claim. It is my belief that Captain Dahl set forth the appropriate conditions for the crew to be properly evaluated.
 I will now address the items raised during the review hearing that were evaluated as Basic Standard (2) and Below Standard (1).
A. Item 2: Flight Planning (FLP) (both Applicants)
 Item 2, Flight planning, was rated a Basic Standard mark of “2” for both F/O Nicolle and Captain Karelsen.
 Captain Karelsen testified that the fuel amount entered into the FMS was done with the knowledge of the ACP. This created a discrepancy of 300 lbs. in the FMS, versus the actual fuel indication on the EICAS screen. Captain Dahl's testimony on this subject is contradictory in that he initially claimed that the crew did not take into consideration the additional APU burn, and thus, the error on the fuel input; however, when then questioned by the Minister on that issue in re‑examination, he stated as follows:
Q. Do you recall saying to the crew, either F/O Nicolle or Captain Karelsen, “Remove 300 pounds from the tank”?
A. I do not recall that. I'm not saying it did or didn't happen. I just do not recall it…
 Based on this statement, I agree with Mr. Karelsen's testimony and rate this item a Standard mark of “3”.
B. Item 6: Take-off (TOF) (both Applicants)
 Item 6, Take-off, was rated a Basic Standard mark of “2” for both F/O Nicolle and Captain Karelsen.
 Captain Dahl testified that all take-offs conducted by Captain Karelsen were carried out as rolling take-offs and not as standing take-offs, as required by the AOM under minimum visibility conditions. The Script Summary (Exhibit M‑6) submitted by the Minister displays the ACP's hand‑written notes to this effect. Captain Karelsen argued that based on pilot technique, such as when the toe brakes are released, the take-offs were performed as standing, and not rolling, as alleged by the Minister.
 The Jazz Air CRJ AOM, Volume 2 (Exhibit M‑8) clearly defines the procedure for a rolling take-off as “set takeoff thrust without holding the brakes” [italics added]. The procedure for a standing takeoff is defined as “advance thrust to not less than 70% N1 thrust prior to brake release, then set takeoff thrust”. Although I can understand that there could be some possible ambiguity as to when the aircraft is repositioned in the simulator, as well as the pilot brake release technique, the difference between these two take-offs is easily detectable. I agree with the ACP in his observation that the take-offs were rolling instead of standing, as required under the operator specifications for minimum visibility.
 There were several discussions in regard to the runway identification required by the crew when in low visibility, which produced contradictory positions. The applicants raised the issue that, as per section 4.5 of the Company Operations Manual, the requirement to identify the departure runway can be done either visually or by cockpit instrumentation (Exhibit A‑1).
 However, since no testimony was heard by either party on the use of instrumentation, I deduce that visual confirmation was used. Having said that, however, I agree with the ACP that on a check ride, this must be done verbally so as to confirm that it has been completed. I agree with a Basic Standard mark of “2” for this exercise.
C. Item 17: Go-around (GOA) (Captain Karelsen) and Item 21: PNF Duties (F/O Nicolle)
 Both pilots were rated a Basic Standard mark of “2” for this exercise, in which Captain Karelsen acted as PF, and F/O Nicolle as PNF. Both applicants agree with the assessment.
D. Item 27: Passenger (Pax) Evacuation (Captain Karelsen)
 Captain Karelsen has admitted that “there very well may have been two items out of order”. As such, I agree that this exercise was properly graded a Basic Standard mark of “2”.
E. Item 21: PNF Duties (Captain Karelsen) and Item 24: Oil Pressure (F/O Nicolle)
 Both pilots were given a Below Standard mark of “1” for this exercise.
 Throughout the review hearing, there were several facts that the Minister and Captain Karelsen agreed to in regard to the sequence of events on the annunciation of the right engine oil pressure warning message.
- The PF called for the appropriate emergency checklist;
- The PNF proceeded at some point to the abnormal power plant index page and found the relevant message tab in this section;
- The PNF eventually proceeded back to the emergency section of the power plant QRH, and to the right engine oil pressure warning message tab.
 The Minister argued that the crew:
- Failed to diagnose, in a timely fashion, a right engine oil pressure warning message, which resulted in an N2 seizure and severe engine damage (Exhibits M‑1-A and M‑1‑B); and that
- The PNF (Captain Karelsen) did not complete the severe engine damage checklist.
 Both Captain Dahl and Captain Karelsen testified that the PF, F/O Nicolle, properly called for the emergency checklist on recognition of this fault. Her role as PF, as outlined in section 4.2.12 of the Company Operations Manual on page 4.2-20 (Exhibit M‑9) would be as follows: “[d]uring abnormal or emergency troubleshooting or at any time that cockpit activities are likely to involve more than one crew member, one pilot will be designated to control the aircraft and maintain ATC listening watch”. I have heard no contradictory testimony that this was not completed properly. Based on the indication presented to her on the EICAS display, her response in my view was correct.
 The PNF, Captain Karelsen, proceeded as directed to the QRH emergency section of the power plant index page. Captain Karelsen admitted that he did not see or simply missed the appropriate checklist tab in this section. This was an error on his part. Under “Threat and Error Management” in the ACP Manual (Exhibit M‑4), it is recognized that a pilot will err from time to time, but that this should be taken into consideration during a check ride. I find no reason to doubt Captain Karelsen's testimony that in missing the applicable tab, he proceeded to the abnormal section and found the inappropriately placed message tab in this section.
 Once in the abnormal section, the message associated to the fault lead him to page 1-16, fluctuating engine oil pressure. He admits that this was not the correct checklist but nevertheless quickly read through it as certain exceptions can be found in the QRH, in which emergency and abnormal messages are not in their respective colour sections. This checklist, nevertheless, redirected him back to the correct tab. His error in missing the applicable tab was admitted to, but the threat, in my opinion, was recognized by him as he did proceed back to the emergency checklist on page 1-13.
 As noted above, testimony was heard from both parties as to some of the exceptions found in the QRH as to colour coding. For example, a thrust reverser unlock message is displayed in the amber abnormal section, but the appropriate checklist is found in the red emergency section. This may have played a part in the PNF proceeding to the abnormal section when faced with a red message annunciation that was overlooked in the red section, and in fact, finding the applicable message in the abnormal power plant index page. Both the ACP and Captain Karelsen testified that, at the time, they were unaware that this was a printing error. As such, the PNF read through this chapter and wasted valuable time.
 The Minister contends that this printing error is minor in nature, but I believe it played a major role in terms of delaying Captain Karelsen in finding the correct emergency checklist. Furthermore, it is clear that Revision 2 of the QRH, dated February 1, 2012, which was used by the crew, is flawed with what I would term as a major printing error. The right engine oil pressure warning message should not have been printed in this section. It had a direct impact on the crew's performance during this malfunction event. The Minister has admitted that the printing error exists, and the company rectified the error within three weeks of the event.
 The Notices outline several grounds for non-issuance, including a failure on the part of both pilots to “diagnose in a timely fashion a R Eng Oil Pressure Warning”. This implies that the crew did eventually diagnose the failure, but not in a timely fashion.
 I do not agree with the Minister's position that this printing error played a minor role in the overall execution of this malfunction, specifically regarding the amount of time it took the crew to diagnose the fault. The amount of time spent in the abnormal checklist prior to returning to the correct section contributed directly to the amount of time it took the crew to properly secure the engine. I agree that taking five to ten minutes to diagnose the malfunction is excessive, but had this printing error not been there, the timeframe would certainly have been reduced. I cannot simply ignore this fact and it is evident from Exhibit A‑1 that an email from the company correcting this error was issued on Nov 16, 2012 – 19 days after the check ride.
 Additionally, I sympathize with the ACP's role in trying to determine what would constitute the correct amount of time to perform a QRH checklist. “Timely” is defined by Merriam Webster as “happening at the correct or most useful time; not happening too late”. It would be difficult to establish such a timeframe to complete an applicable checklist. In my view, based on the circumstances of a check ride, in that the crew would normally be more methodical in reading through checklists, the actions by the crew would have been appropriate and executed in a timely manner had the printing error not been present.
 The initial action by the crew was correct, but was derailed based in large part on this printing error. One can estimate that without this error, valuable time, estimated in testimony to be two to three minutes, would have been saved and the engine seizure could have been possibly avoided altogether. The ACP eventually stopped the check ride while the crew was securing the engine. It would seem from testimony by Captain Karelsen, and in part by the ACP, that the crew was in the process of securing the engine when this occurred. Captain Karelsen testified that no vibration or loud bang was heard. Nevertheless, based on testimony, the engine secure checklist was being performed prior to the check ride being terminated.
 The ACP testified that, “this procedure dealing with, getting to the oil pressure checklist and dealing just with the oil pressure checklist without going into any other checklist would take probably 30 seconds to 60 seconds to find the checklist and in the neighbourhood of a few minutes to action the checklist, two or three minutes”. However, no testimony or evidence was presented in regard to how this fault is programmed to occur in the simulator once it has been triggered. The time allocated by the simulated software program for the right engine oil pressure malfunction would be based on a set timeline, but it would seem likely that, had the abnormal power plant index not had the printing error, the PNF would have returned back to the initial checklist called for by the PF.
 The applicants have stated that from the time of the initial fault to the time that the check ride was terminated, approximately five minutes elapsed. The ACP testified that the timeframe was between five, but closer to 10 minutes. This range is wide and a more accurate estimate was not provided by the Minister. In reviewing the documentary evidence and the testimony, I would estimate the time to be five to six minutes. Based on the estimated time spent in the erred section of the checklist, it would seem, in all probability, that the crew would have been able to perform the applicable checklists in a timely fashion and possibly avoided the engine seizure.
 A mark of “1” was attributed to the crew on this exercise, and I will review the four-point marking scale guideline (Exhibit M‑7)
(1) Below Standard – “1”
 According to the Flight Test Guide, “Below Standard” is defined as “unacceptable deviations from the qualification standards occur, which may include excursions beyond prescribed limits that are not recognized or corrected in a timely matter”, and as “risk is unacceptably mitigated”.
 In my view, the crew recognized the malfunction properly as an emergency procedure checklist, managed the initial error of missing the red tab, were delayed when this tab was found in the abnormal section, and eventually corrected and proceeded to the red section of the checklist. Had this printing error not been in the abnormal section, I believe, in all probability, that the malfunction would have been performed in a timely fashion. The crew maintained its situational awareness, and maintained safety of flight and adequate crew resource management (CRM). Risk was being mitigated as the engine had been placed in idle, and was in the process of being secured prior to the check ride being terminated.
(2) Basic Standard – “2”
 “Basic Standard” is defined as “major deviations from the qualification standards occur, which may include momentary excursions beyond prescribed limits but these are recognized and corrected in a timely matter”, and as “risk is poorly mitigated”.
 A major deviation was admitted to, but it was adequately recognized and corrected. Risk was mitigated in ensuring that the first pilot action item on both checklists was performed, which is “affected thrust lever… confirm and idle”. Based on the testimony and evidence presented, it is the Tribunal's position that a mark of “2” better reflects the actions and performance by the crew in dealing with the engine oil pressure malfunction.
A. Captain Jeffrey David Karelsen
 The Minister of Transport has not proven, on the balance of probabilities, that the applicant, Jeffrey David Karelsen, failed to meet the qualifications or conditions necessary for issuance of a pilot proficiency check. Therefore, the matter is referred back to the Minister for reconsideration.
B. F/O Paula Cathy Nicolle
 The Minister of Transport has not proven, on the balance of probabilities, that the applicant, Paula Cathy Nicolle, failed to meet the qualifications or conditions necessary for issuance of a pilot proficiency check. Therefore, the matter is referred back to the Minister for reconsideration.
February 27, 2014
- Date modified: