Enhanced flight vision system

The forward-facing camera used for the PlaneView EVS on a Gulfstream G450.

An enhanced flight vision system (EFVS, sometimes EVS) is an airborne system which provides an image of the scene and displays it to the pilot, in order to provide an image in which the scene and objects in it can be better detected. In other words, an EFVS is a system which provides the pilot with an image which is better than unaided human vision. An EFVS includes imaging sensors (one or many) such as a color camera, infrared camera or radar, and typically a display for the pilot, which can be a head-mounted display or head-up display. An EFVS may be combined with a synthetic vision system to create a combined vision system. ^[1]

An EFVS can be mounted on military or civilian aircraft, fixed wing (airplane) or rotary wing (helicopter). The image must be displayed to the pilot conformal to the scene, i.e. the pilot must see the artificially displayed elements in exact positions relative to the real world. Usually along with the enhanced image, the system will display visual cues such as a horizon bar and runway location.

Enhanced vision

EVS Camera under a Global 6000 windshield

Enhanced vision is a related to Synthetic vision system which incorporates information from aircraft based sensors (e.g., near-infrared cameras, millimeter wave radar) to provide vision in limited visibility environments.

Night vision systems have been available to pilots of military aircraft for many years. More recently business jets have added similar capabilities to aircraft to enhance pilot situational awareness in poor visibility due to weather or haze, and at night. The first civil certification of an enhanced vision system on an aircraft was pioneered by Gulfstream Aerospace using a Kollsman IR camera. Originally offered as an option on the Gulfstream V aircraft, it was made standard equipment in 2003 when the Gulfstream G550 was introduced and followed on the Gulfstream G450 and Gulfstream G650. As of 2009, Gulfstream has delivered over 500 aircraft with a certified EVS installed. Other aircraft OEMs followed, with EVS now available on some Bombardier and Dassault business jet products. Boeing has begun offering EVS on its line of Boeing business jets and is likely to include it as an option on the B787 and B737 MAX.

The Gulfstream EVS ^[2] and later EVS II systems use an IR camera mounted in the aircraft's nose to project a raster image on the head-up display (HUD). The IR image on the HUD is conformal to the outside scene, meaning that objects detected by the IR camera are the same size and aligned with objects outside the aircraft. Thus in poor visibility the pilot is able to view the IR camera image and is able to seamlessly and easily transition to the outside world as the aircraft gets closer.

The advantage of EVS is that safety in nearly all phases of flight are enhanced, especially during approach and landing in limited visibility. A pilot on a stabilized approach is able to recognize the runway environment (lights, runway markings, etc.) earlier in preparation for touchdown. Obstacles such as terrain, structures, and vehicles or other aircraft on the runway that might not otherwise be seen are clearly visible on the IR image.

The FAA grants some additional operating minimums to aircraft equipped with certified enhanced vision systems allowing Category I approaches to Category II minimums. Typically an operator is permitted to descend to lower altitudes closer to the runway surface (typically as low as 100 ft) in poor visibility in order to improve the chances of spotting the runway environment prior to landing. Aircraft not equipped with such systems would not be allowed to descend as low and often would be required to execute a missed approach and fly to a suitable alternate airport.

Other sensor types have been flown for research purposes, including active and passive millimeter wave radar. In 2009, DARPA provided funding to develop "Sandblaster", a millimeter wave radar based enhanced vision system installed on helicopters which enables the pilot to see and avoid obstacles in the landing area that may be obscured by smoke, sand, or dust.

The combination of dissimilar sensor types such as long wave IR, short wave IR, and millimeter wave radar can help ensure that real time video imagery of the outside scene can be provided to the pilot in all types of visibility conditions. For example, long wave IR sensor performance can be degraded in some types of large water droplet precipitation where millimeter wave radar would be less affected.

History

thermal image viewed through a head-up display

Night vision devices for military personnel have been operational since the time of World War II. Their use has been adopted also by military pilots, mainly in rotary-wing aircraft (helicopters). The use of such devices has been suggested for use by commercial pilots since the 1970s, but it was not until 1999 that the first commercial, FAA certified system, was airborne. Still, the pilot could not use the system to lower an aircraft below the required natural vision limit.

Gulfstream in 2001 became the first civilian aircraft manufacturer to develop and earn certification on its aircraft for EVS produced by Elbit's Kollsman. ^[3] The FAA permitted the use of the EVS to descend down to 100 feet above Touch-down zone, if no other restrictions apply. ^[4] It was not clear at the time whether an EFVS could be used for descending below that height. The situation was amended in 2004 with corrections to FAA FAR 91.175. ^[5] This marks the first time an EFVS gave a concrete commercial advantage over unaided vision.

Generation I EFVS

The first EVS's comprised a cooled mid-wave (MWIR) Forward looking infrared (FLIR) camera, and a HUD, certified for flight with the Gulfstream V aircraft. The camera has a cooled MWIR sensor

Airport LED transition and multispectral EFVS

EVSs are traditionally based on a Forward looking infrared camera which gives a thermal image of the world, and shows up heat released from airport approach lights. Most airports use incandescent Parabolic aluminized reflector lights, ^[6] though energy efficiency standards (such as the Energy Independence and Security Act of 2007 ) have caused some airports to switch to LED lighting, which has a lower thermal signature.

However, since 2007, airports are switching to the more energy efficient LED lighting, which has a lower thermal profile. The new EVS designs are multispectral, to capture both visual light from LED lights and the thermal image of previous EVS generations. Future EVS designs focus on all-weather vision, which can be accomplished by intelligently fusing images and data from cameras operating in visible light, infrared, and millimeter-wave.

Aircraft

An EFVS can be mounted on any type of craft. The typical platform is a small passenger plane, since it is more cost-effective to use an EFVS than an instrumental landing system, which is used in larger passenger airplanes.

NASA is developing a new supersonic airplane, the X-59 QueSST, to study technology related to better supersonic passenger planes. A key feature is an opaque nosecone, which the pilot cannot see through. NASA is considering using an EFVS to enable pilot vision on this plane. ^[7]

Technology

Sensors

The sensor unit of the EFVS can include a single imaging sensor, multiple cameras and also additional navigation-aiding sensors.

FLIR

Traditionally, the EVS sensor was a single forward looking infrared (FLIR) camera. FLIRs are of two major types: one is the high-end, cooled, MWIR band (3–5 um) camera, which has better temperature resolution and frame rate but is more expensive and bulky, and the other is uncooled microbolometers which operate in the LWIR band (8–14 um) of the light spectrum, are small and cheap but are less "sharp" with regards to temperature contrast.

The EVS sensor in a single FLIR EVS is usually the high-end cooled sensor. In multi-spectral applications the preferred sensor is usually uncooled since it has better atmospheric penetration in most cases (will see farther), while the fine image details will be provided by a complementary sensor.

VIS and NIR

Natural unaided vision in the visible portion of the light spectrum, along with the near-infrared, can be improved by using high end cameras. Such a camera can be a high dynamic range camera for day vision, a low-light CMOS camera (sometimes called scientific CMOS or sCMOS) and night vision goggles.

In day vision and bright light it may seem that there is no need to improve the natural vision, but there are certain cases in which it may be necessary. For example, in a strong haze situation where the whole scene is very bright and features are not distinguishable, a high dynamic range camera can filter the background and present a high-contrast image, and detect the runway approach lights further away than natural vision.

SWIR

A SWIR (short-wavelength infrared) camera is a relatively new technology. It can offer advantages for an EFVS, such as: better haze penetration than VIS, natural scene contrast similar to VIS unlike a MWIR or LWIR. SWIR cameras are available commercially, but there is no reported use of a SWIR camera in a commercial EFVS.

Millimeter wave camera

A passive millimeter wave (PMMW) camera is capable of producing a real time video image, with the advantage of seeing through clouds, fog and sand. Use of passive millimeter wave cameras are a promising technology for aircraft based Enhanced Flight Vision Systems ^{[8] [ circular reference ]} as well as ship navigation in low visibility, and industrial applications. The first commercially available passive millimeter wave camera for use in aircraft was created by Vū Systems ^[9] and launched at the National Business Aviation Association (NBAA) Conference in October 2019. ^[10]

Short range passive millimeter wave scanners are in use today for airport screening ^[11] and many scientific research programs. ^{[12] [13]}

Operation of a passive millimeter wave camera is based on measuring the difference or contrast in temperatures, but at millimeter wave frequencies, anywhere from 30 GHz to 300 GHz range. ^{[14] [ circular reference ]}

Imaging radar

An imaging radar has also been proposed by NASA in the 1990s. ^[15] It can offer the same scene resolution as a PMMW, but has different properties. It does not rely on natural radiation bu emits radio waves, which are reflected from the target and captured in the receiver. The image will be nearly the same under all conditions since it does not depend on the object temperature. An imaging radar requires very high resources for computation, since the image is formed by digital calculation and not by a lens. There have been flying prototypes, but it is not yet commercially available.

Lidar

A lidar is a laser system which scans the surrounding volume and provides 3D location of objects. From the data can be produced a synthetic image and also other critical flight data. The operational distance of a lidar depends on the output power. It is typically under 1 km distance, but is not limited in principle. Due to the relatively short distance it is considered more for helicopters than for airplanes. It can also aid in penetrating light to moderate atmospheric low-visibility conditions, such as fog and dust. Lidar is used in automotive applications (cars), and is being tested for helicopter landing applications.

Navigational sensors

A navigational sensor may aid in complementing the image. A synthetic image can be produced based on scene data in memory and location of the aircraft, and displayed top the pilot. In principle, a pilot could land based on this synthetic image, subject to its precision and fidelity.

• The most common navigation aid is a GPS . An enhanced GPS can provide the 3D location of the aircraft with accuracy of a 10 cm (4"). There are integrity issues that prevent it from being a full navigation solution. It can be blocked or tricked into reporting a false position, or lose the position, and not be able to report the problem in the first few seconds. These drawbacks prevent the GPS to be used as a stand-alone sensor in critical flight phases such as landing.
• Image registration is the comparison of the image acquired from an imaging sensor to a recorded image (usually from satellite) which has a known global position. The comparison allows to place the image, and therefore the camera (and with it the aircraft) in a precise global position and orientation, up to a precision which depends on the image resolution.
• An inertial navigation system (INS) or inertial measurement unit (IMU) is a device which measures acceleration, angular velocity, and sometimes the magnetic field, using a combination of accelerometers and gyroscopes, sometimes also magnetometers. The INS uses the information to determine position and orientation over time, by dead reckoning, i.e. only relative to a previously known position. Combined with a GPS or image registration, it can provide an accurate absolute position.
• A radar altimeter can provide the aircraft elevation above the terrain with high precision and fidelity. Altitude is information which can be combined with other data to provide a precise location.

Display

Not displayed, HUD, head-mounted display

The display to the pilot is a see-through display, which means it allows both seeing the scene directly with unaided vision and seeing a projected image. The display is one of two types:

1. Head-mounted display or helmet-mounted display. It includes glasses-like surfaces in front of the pilot's eyes and mounted on the head, and a projection system which projects an image on the glasses to be reflected or refracted to the pilot's eyes. Augmented-reality goggles are a notable example of such a display. Since it is moving with the pilot's head it must include tracking sensors to project the correct image according to the direction it is facing.
2. Head-up display is a system composed of a large reflecting plate (called combiner) positioned in front of the pilot, and a projection system. The system generates an image which is reflected from the combiner to the pilot.

A head-down display is an LCD screen installed below the window, hence the name "head-down". It is generally not used as an EFVS display, since the external scene cannot be seen when looking at it.

In addition to the improved sensors image, the image displayed to the pilot will include symbology, which is a collection of visual cues displayed to a pilot regarding altitude, azimuth, horizon orientation, flight path, fuel state, other aircraft etc., and in military avionics additional friend/foe symbols, targeting system cues, weapon sights etc.

The displayed EFVS imagery and symbology must be presented so that they are aligned with and scaled to the external view. The process of alignment is called harmonization. A head-up display must be harmonized with the imaging sensors. A head-mounted display moves constantly with the pilot's head, and must therefore be tracked continuously so that the image displayed conforms to the scene in real-time, see Helmet-mounted display. There is an additional issue of lag time between the image and head motion, which must be very small so as not to cause dizziness.

Functionality

Precision instrument approach/landing ^[16]

Category	Decision height
I	> 200 ft (60m)
II	30–60m (100–200 ft)
III A	< 100 ft (30m)
III B	< 50 ft (15m)
III C	no limit

The main purpose of an EVS is to permit takeoff, landing and taxiing in poor visibility conditions, where landing would not be safe otherwise. An EVS is certified for landing by the FAA only if it is combined with a HUD, in which case it is called an EFVS. ^[17]

The criterion for landing is known as decision height. ICAO defines Decision Height as "a specified altitude or height in the precision approach at which a missed approach must be initiated if the required visual reference to continue the approach has not been established." When a pilot is approaching the ground, they must see a visual reference to continue the approach. The visual references must be one of the following (see runway ):

1. The approach lighting system (if it exists).
2. Both the runway threshold and the touchdown zone, which are identifiable by their markings or lights.

If the pilot cannot see such a reference in the decision height, they must abort the landing, and then circle for a second approach or land elsewhere.

Above the decision height, the pilot uses mostly the aircraft displays. Below decision height, the pilot must look outside to identify visual references. In this stage the pilot alternates between looking at displays and looking out the window. This switching can be avoided if a see-through display is installed to display information to the pilot while also looking out.

Combined with synthetic vision

HUDs then EVS came to business jets in 2001 and the FAA published EVFS rules in 2016 to land in poor visibility through a HUD, precluding PFD use, with combined enhanced and synthetic vision system (CVS). Under current FAR 91.175 regulations, airplanes with HUDs can attain 100 ft (30 m) before switching to natural vision to land, permitting all-weather landing in airports without ILS Cat II/III approaches. ^[18] After beginning work in 2011, Dassault was first to certify its CVS with its Elbit HUD and camera, FalconEye, in October 2016 in the Falcon 2000 and 900, then in the 8X in early 2017. ^[18]

In July 2018, FAA certification of the Gulfstream G500 allowed the EFVS to provide the only visual cues for landing down to 1,000 ft (300 m) runway visual range, to touchdown and rollout, after 50 test approaches, and testing to lower visibilities could allow dropping the limit, with approvals for previous Gulfstreams to follow. ^[19] By October 2018, the Falcon 8X FalconEye was approved by the FAA and EASA for approaches down to 100 ft (30 m). ^[20] The Falcon 2000 and 900LX were approved in early 2019. ^[21] A dual HUD FalconEye will allow EVS-to-land in 2020, without using natural vision. ^[20] Rockwell Collins's conformal overlay of EVS and SVS is expected to enter service with the updated Global 5500/6500 around 2020. ^[18]

Bombardier Globals use a Rockwell Collins HUD and camera while Gulfstreams have a cooled Kollsman (Elbit) camera and a Rockwell Collins HUD. ^[18] Early cryogenically cooled, indium antimonide (InSb) cameras could detect 1.0–5.0-micron mid-IR for hot incandescent runway lights and some background radiation from its surface, blind to visible wavelengths for LED airport lights or long-wave IR for finer environment details: the Elbit FalconEye sees in the 0.4–1.1-micron visible light and near-IR band and the 8.0–12.5-micron long-wave-IR. ^[22]

Alternatives to EVS-assisted landing

Instrument landing system

An Instrument landing system, or ILS, relies on radio signals to allow operation in any weather. For an ILS landing to be allowed, the system must be installed on the ground, and a suitably equipped aircraft and appropriately qualified crew are required. Not all airports and runways are suitable for ILS installation, because of terrain conditions (hills in the way of the signal, non-straight landing slope).

GPS-assisted landing

While the GPS has a very high inherent precision, the reliability is not high enough for landing. GPS signals may be intentionally jammed, or lose integrity. In such cases, it may take the GPS receiver a few seconds to detect the malfunction, which is too long for critical flight stages. GPS can be used to lower the decision height below the unaided threshold, down to cat I decision height minima, but not lower.

See also

• External vision system
• Instrument approach
• Instrument landing system
• Synthetic vision system

References

1. RTCA DO-341, Sep. 2012
2. "Enhanced Vision System". Gulfstream.
3. Let's look at FAA's final rule on EFVS use published Dec 13, 2016 Archived February 14, 2018, at the Wayback Machine Professional Pilot website,
4. Special Conditions: Enhanced Vision System (EVS) for Gulfstream Model G-V Airplanes, FAA (June 2001)
5. General Operating and Flught Rules – Instrument Flight Rules Sec. 91.175.
6. Lighting Systems - Medium Approach Light System with Runway Alignment Indicator Lights (MALSR), FAA (August 2014)
7. "NASA's X-59A Quiet Supersonic Test Jet Will Have Zero Forward Visibility for Its Pilot". 23 August 2018.
8. Enhanced flight vision system
9. "Home". vusystems.com.
10. "Vū Systems' New Cube Will Change Instrument Approach Flying Forever". 6 November 2019.
11. "How Millimeter Wave Scanners Work". 28 November 2012.
12. https://www.millivision.com/technology.html
13. http://www.trexenterprises.com/Pages/Products%20and%20Services/Sensors/avionics.html
14. Extremely high frequency
15. The 94 GHz MMW imaging radar system, Proceedings of the Workshop on Augmented Visual Display (AVID) Research; p 47–60 (December 1993)
16. "Getting to grips with CAT II / CAT III operations" (PDF). Airbus. Oct 2001.
17. RTCA DO-315B Archived 2016-04-06 at the Wayback Machine (2012), "Minimum Aviation System Performance Standards (MASPS) for Enhanced Vision Systems, Synthetic Vision Systems, Combined Vision Systems and Enhanced Flight Vision Systems"
18. Matt Thurber (July 20, 2018). "Flying Dassault's FalconEye Combined Vision System". AIN online.
19. Matt Thurber (November 13, 2018). "Gulfstream First to Certify EFVS Landing System". AIN online.
20. Matt Thurber (October 9, 2018). "FAA, EASA OK Dassault 8X EFVS Down to 100 Feet". AIN online.
21. Matt Thurber (February 22, 2019). "Dassault Expands Certifications for FalconEye". AIN online.
22. Fred George (Aug 23, 2018). "Dassault FalconEye: A Head-Up Leap Forward In Situational Awareness". Business & Commercial Aviation.