Development of image processing algorithms for taxiway line detection and tracking

Abstract

In the taxiing phase of flight, the pilots are usually busy executing pre-flight checklists and communicating with Air Traffic Control (ATC). This may lead to a loss of situational awareness and can result in a collision with another aircraft, vehicle or other objects present in the vicinity on the aerodrome. Furthermore, pilots might miss an exit or turn if they are not familiar with the airport or if the visibility is not good. This thesis describes a novel computer vision-based method to detect taxiway lines on an aerodrome and to estimate the deviation of a large passenger aircraft from the taxiway centerline. This method processes video footage captured from an onboard camera located on the vertical stabilizer of an A380 aircraft.

The proposed method can be applied as part of a larger system to increase the situation awareness of pilots during taxiing. It can also alert them if the aircraft deviates from the centerline. The method makes use of colour and edge information present in the camera images. The thesis proposes a Sliding Window (SW) method and clustering techniques to detect and process the taxiways markings.

In the first step, the input image is transformed to a top-down view by applying the Homographic Transform. Then, colour and edge detection techniques are applied to the top-down view and a binary image of pixels belonging to taxiway lines is generated. In the next step, the region of the image directly in front of the aircraft is processed to determine whether the aircraft is turning or moving in a straight line. If it is determined that the aircraft is in a turn manoeuver, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering technique is applied to the binary image obtained in the previous step. Otherwise, a SW method is used to detect the taxiway centerline ahead of the aircraft and any taxiway line intersections present in the image. In the final step, the aircraft’s lateral deviation from the taxiway centerline (the cross track error) is estimated using a template matching approach.

The entire algorithm was tested on simulated video sequences of taxi scenarios in different illumination and visibility conditions. The test results obtained during daytime (without fog) showed that the SW method achieved a detection rate of 80% and a false positive rate of 3%. On the other hand, the clustering technique achieved a detection rate of 76% and a false positive rate of 4%. The aircraft’s deviation from the taxiway centerline was estimated with a mean error of 0.8 pixels and a worst case error of ±3 pixels.