Automatic Evaluation of Spatial Disorientation

The paper discusses the automated detection of spatial disorientation from recorded flight data. The first part of the paper briefly introduces middle ear function and its influence on spatial disorientation. The specially developed simulator and aerodrome manoeuvre flown by the pilot during the experiment is also mentioned. The second part of the paper analyses the recorded data from experimental flights and proposes an algorithm for the automatic evaluation of spatial disorientation. An example graphical interpretation of the recorded data is provided with a clear explanation of the important parameters. At the end of the paper, the most important conclusions of the work are presented.


Introduction
Flight illusions and the spatial disorientation that follows arise from many different causes -imperfections of human anatomy, particular weather or the time of the day.All pilots perceive the space around them primarily by sight.If for some reason sight is excluded as the main sensor for spatial orientation, then the other senses may start to misinterpret the surrounding space.One of the causes of illusions in flight is the misinterpretation of perceptions derived from the middle ear.The middle ear is the organ responsible for perception of changes in head position and its angular acceleration [1].If a pilot cannot properly evaluate all his or her sensations in cooperation with the onboard instruments, then sensations or instruments can be easily misinterpreted, leading to a flight illusion and spatial disorientation.
Flight illusions are fairly common for pilots who are flying according to IFR (instrument flight rules) under IMC (instrument meteorological conditions) [2].Without taking a corrective course of action, these illusions can deteriorate into a stage of spatial disorientation, which may have fatal consequences.There are many types of inflight illusions, and military pilots should be prepared for the effects of their influence.However, the experience of a flight illusion is a very subjective sensation that can be difficult to quantify.Another problem may be a certain reluctance of pilots to talk about this issue because such situations are infrequent and to some extent they are even a taboo subject [3].Based on a questionnaire survey among Czech military pilots, it was nonetheless found that most respondents would welcome more frequent education about in-flight illusions and spatial disorientation.
This would increase awareness of the phenomenon and make it easier to evaluate flights affected by illusions in future.Based on a deeper understanding of pilot behaviour when under an illusion, it would be possible to build a system that could protect pilots from the loss of spatial orientation during flight.A similar system is used, for example, in the automotive industry to protect the driver from micro-sleeps on the basis of eye tracking.

Simulator
To improve the understanding of the origin of spatial disorientation, a special simulator was developed to evaluate the effects of an illusion on airplane pilots.Its structure comprises a platform allowing a change of bank angle in the range of ±10° around the longitudinal axis.The simulator is based on a MS FSX, an Oculus RIFT DKII VR Headset and a simulator motion platform controlling application [4][5].
For the test flights, an approach pattern was created using one NDB (nondirectional beacon) and an ILS (instrument landing system) to provide take-off, flight in the vicinity of the airfield and landing.The weather during the flight corresponds to real conditions, i.e. ensuring the pilot has minimal visual contact with the ground for as long as possible.The mission for each pilot consists of three circuits according to the approach chart (Fig. 1), where during last two circuits the platform offset was introduced according defined parameters.Spatial disorientation during the experimental flight is caused by long-term exposure of the pilot to the platform offset.The illusion is created and pilot is affected by it immediately when the platform offset disappears.
The simulator allows the bank angle of the platform to be changed in order for the pilot to experience the leans during the ILS approach.According to the flight scheme and principles of the leans illusion, the illusion can only be easily generated on the left side.
Before the experiment, the pilots were familiarized with the various types of inflight illusion and the function of the middle ear apparatus.The pilots were also acquainted with the behaviour of the simulator during simulated flight, mainly with its change of bank angle around the longitudinal axis, which may seem unusual in the initial phase of the test due to the absence of other expected movements.The pilots were also informed about possible experience of spatial disorientation during simulated flight, but they were not informed about the exact time of the experimental activation of the platform bank offset.The next step was to explain the placement of the flight and navigation devices on the instrument panel and the functionality of all the instruments relevant to controlling the airplane under IMC and instrument flight rules.This step is especially important due to the possible variety of instruments used in different types of aircraft.Understanding their function is essential to successfully completing the planned task on the simulator.

Fig. 1 Approach chart/flight scheme
The pilots received basic instructions about the airplane speeds that should be maintained at various stages of the flight and HOTAS (hands on throttle and stick), which is essential to fly the simulator due to no visual contact with the controls.All pilots also completed an introductory flight in VMC (visual meteorological conditions) to familiarize themselves with the airplane.It was also necessary to introduce the pilot to the Oculus Rift DK2 VR Headset and customize it to the specific person.
One of the important features of the simulator is the pilot's ability to mark and signal any non-standard situation (such as discomfort, illusion in flight, spatial disorientation).This marking is done using a specially assigned button by which the pilot creates a mark in the recorded flight data.This tag or set of tags is later used to evaluate the dataset and develop the algorithm for post-processed automated data evaluation.

Detection algorithm
Based on data measured during experimental flights and pilot feedback, it was possible to create an algorithm to determine the possibility of an in-flight illusion or a spatial disorientation.The purpose of the algorithm is not only to evaluate the data measured during the experiment, but it is also used to classify pilots.
By combining all the limit values in the experimental flight records and also based on pilot feedback, a further algorithm was developed.This algorithm is based on the airplane heading values, deviations from the glide path track, airplane bank, airplane bank increment and the bank of the simulator platform.At this time, calculations are not performed in real time but once the simulated flight is complete.Post processing of the recorded data allows us to adjust the formula in order to find all relationships in the subsequent analysis of the obtained values.Evaluation of the possible presence of an in-flight illusion (or spatial disorientation) is simplified by the chosen flight execution procedure.Evaluation takes place on a previously fixed ILS course line, which clearly gives the idea of the ideal flight path.Measuring and recording of flight parameters is done with a frequency of 6 Hz and is currently sufficient for the experiment.
The limit values of the formula parameters were determined experimentally based on an analysis of the measured flight parameters, comparing them with the parameters of previous measurements and the reports of the pilots conducting the given flights.The main element entering an evaluation of the possible presence of an illusion is the dRoll value, equal to the airplane bank angle at a time interval of 1 second separated from measured data.Evaluating the identification of a leans illusion is based on a number of conditions which are determined by the predicted simulation effect on the pilot and an analysis of the performed flight.
The final algorithm (collection of conditions) is shown in the form of a flowchart in Fig. 2.

Fig. 2 Flowchart of algorithm
The flowchart shows a sequence of conditions that must be fulfilled to identify the possibility of an in-flight illusion.The algorithm currently works only with a final approach to the airport from a direction of 237°, and only calculates the possibility of the leans illusion on the left of the glide path direction; this is because of the pre-set conditions of the simulator platform movement when the offset angle is added during the simulated flight.Parameters such as Heading, CDI (final approach path deviation), AirplaneRoll and PlatformRoll are based on a description of the leans illusion and the presumed effect of the simulator.The dRoll formula represents the change in the airplane bank backwards in 1 second intervals.If the value of dRoll is greater than the experimentally obtained limit, the presence of SD is possible.
Based on the algorithm described above (Fig. 2), it is possible to deduce conclusions from the measured values, graphically evaluated as the ILL curve in Fig. 3.In addition, the individual curves of each item of data taken and used in the automatic evaluation of the flight are shown here and in Fig. 4.

Fig. 3 Graphical evaluation of the leans illusion
In the graphical evaluation of flight parameters, it is possible to see the results that detect a possible leans illusion during simulated flight (ILL curve).Also, it is possible to see the termination of the simulator platform offset (PLTF_Offset curve) and the small inclination of the airplane bank (Roll) to the right and left.Additionally, the glide path is intercepted and an on-board indication (CDI curve) displayed.This part of flight is relatively standard, although it was marked by one pilot as feeling nonstandard (possibly a flight illusion) immediately after the offset angle of the platform was decreased.
This feeling was probably due to the platform returning to its standard position, corresponding to the bank of the airplane (without any platform offset angle).In the 70 th second, the algorithm identified the possibility of an in-flight illusion, and as seen from both the CDI and the bank of the airplane (Roll curve), this is very likely.In the Fig. 4 Airplane heading and bank angle from both the CDI and the bank of the airplane (Roll curve), this is very likely.In the next set of ILL impulses we can see the simultaneous designation of the illusion marked by the pilot (IM curve).It is already possible to record some oscillation in the vicinity of value 60 on the CDI curve from 80 th to 150 th second.This value corresponds to the position between the second and third dots left of centre on the horizontal scale (Fig. 5) of the airplane ILS indicator.The vertical situation was recorded during the experiment, but the data have not been used for evaluation at this time.Information about the vertical position on the glide path is not so important for the evaluation because of the algorithm's primary concept.

Fig. 5 Visualization of the on-board ILS indicator
This oscillation is caused by an attempt to approach the descent route.However, once the value has been reached, this effort is again overcome by the pilot's sensation of an incorrect bank angle that was subconsciously balanced.From the development of the CDI line, however, it can be seen that this compensation proceeded with decreasing intensity, suggesting that the leans illusion was overcome or disappeared.
Small changes in airplane roll (up to 5°) seen from the 110 th second are performed as minor corrections to bring the airplane heading and approach closer to the glide path.However, the pilot was close to the glide path in the 160 th second before again veering away from it.This little deviation, almost at the end of the descent track, can be explained in two ways.First, it may be an inadequate pilot's reaction to his or her proximity to the airport, or it may be a continuing illusion in flight and the pilot (because of concentrating on the completed task) was unable to mark it with the illusion presence button.Further analysis of the recorded data (beyond the range in Fig. 3) showed that the deviation could be the result of poor steering interventions near the runway, caused by the angular character of the ILS landing equipment.
During real descent, pilots use relatively small changes of rudder deflection to achieve the required descent.This cannot be applied on an experimental simulator due to the simplified use of the airplane control system in the form of automatic rudder control.

Conclusion
The proposed algorithm retrospectively evaluates the possible influence of in-flight illusions on a pilot and his or her possible subsequent spatial disorientation.This evaluation has been performed based on conditions regarding the bank angle of the platform and the airplane, the heading and other recorded flight parameters.These parameters are described in detail in the text and the algorithm is shown in flowchart form.However, at this stage of the development, the algorithm only evaluates flight illusions created during a simple flight scheme.The flight is conducted according to the circuit diagram with ILS approach and setting the offset angle of the platform at a predefined time.This fact does not reduce the functionality of the algorithm in the assumed field of use.The results of the algorithm were confirmed by the positive identification of the illusion in flight by the tested pilots.