Underwater computer vision

Last updated

Underwater computer vision is a subfield of computer vision. In recent years, with the development of underwater vehicles ( ROV, AUV, gliders), the need to be able to record and process huge amounts of information has become increasingly important. Applications range from inspection of underwater structures for the offshore industry to the identification and counting of fishes for biological research. However, no matter how big the impact of this technology can be to industry and research, it still is in a very early stage of development compared to traditional computer vision. One reason for this is that, the moment the camera goes into the water, a whole new set of challenges appear. On one hand, cameras have to be made waterproof, marine corrosion deteriorates materials quickly and access and modifications to experimental setups are costly, both in time and resources. On the other hand, the physical properties of the water make light behave differently, changing the appearance of a same object with variations of depth, organic material, currents, temperature etc.

Contents

Applications

Medium differences

Illumination

In air, light comes from the whole hemisphere on cloudy days, and is dominated by the sun. In water lighting comes from a finite cone above the scene. This phenomenon is called Snell's window.

Light attenuation

Unlike air, water attenuates light exponentially. This results in hazy images with very low contrast. The main reasons for light attenuation are light absorption (where energy is removed from the light) and light scattering, by which the direction of light is changed. Light scattering can further be divided into forward scattering, which results in an increased blurriness and backward scattering that limits the contrast and is responsible for the characteristic veil of underwater images. Both scattering and attenuation are heavily influenced by the amount of organic matter dissolved or suspended in the water.

Another problem with water is that light attenuation is a function of the wavelength. This means that different colours are attenuated faster or slower than others, leading to colour degradation. Red and orange light is the first to be attenuated, followed by yellows and greens. Blue is the least attenuated visual wavelength.

Challenges

In high level computer vision, human structures are frequently used as image features for image matching in different applications. However, the sea bottom lacks such features, making it hard to find correspondences in two images.

In order to be able to use a camera in the water, a watertight housing is required. However, refraction will happen at the water-glass and glass-air interface due to differences in density of the materials. This has the effect of introducing a non-linear image deformation.

The motion of the vehicle presents another special challenge. Underwater vehicles are constantly moving due to currents and other phenomena. This introduces another uncertainty to algorithms, where small motions may appear in all directions. This can be specially important for video tracking. In order to reduce this problem image stabilization algorithms may be applied.

Frequent methods

[ clarification needed ]

Image restoration

Image restoration [2] [3] aims to model the degradation process and then invert it, obtaining the new image after solving. It is generally a complex approach that requires plenty of parameters[ clarification needed ] that vary a lot between different water conditions.

Image enhancement

Image enhancement [4] only tries to provide a visually more appealing image without taking the physical image formation process into account. These methods are usually simpler and less computational intensive.

Color correction

Different algorithms exist that perform automatic color correction. [5] [6] The UCM (Unsupervised Color Correction Method), for example, does this in the following steps: It firstly reduces the color cast by equalizing the color values. Then it enhances contrast by stretching the red histogram towards the maximum and finally saturation and intensity components are optimized.

Underwater stereo vision

It is usually assumed that stereo cameras have been calibrated previously, geometrically and radiometrically. This leads to the assumption that corresponding pixels should have the same color. However this can not be guaranteed in an underwater scene, because of dispersion and backscatter as mentioned earlier. However, it is possible to digitally model this phenomenon and create a virtual image with those effects removed

Other application fields

In recent years imaging sonars [7] [8] have become more and more accessible and gained resolution, delivering better images. Sidescan sonars are used to produce complete maps of regions of the sea floor stitching together sequences of sonar images. However, imaging sonar images often lack proper contrast and are degraded by artefacts and distortions due to noise, attitude changes of the AUV/ROV carrying the sonar or non uniform beam patterns. Another common problem with sonar computer vision is the comparatively low frame rate of sonar images. [9]

Related Research Articles

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Hydrographic survey</span> Science of measurement and description of features which affect maritime activities

Hydrographic survey is the science of measurement and description of features which affect maritime navigation, marine construction, dredging, offshore wind farms, offshore oil exploration and drilling and related activities. Surveys may also be conducted to determine the route of subsea cables such as telecommunications cables, cables associated with wind farms, and HVDC power cables. Strong emphasis is placed on soundings, shorelines, tides, currents, seabed and submerged obstructions that relate to the previously mentioned activities. The term hydrography is used synonymously to describe maritime cartography, which in the final stages of the hydrographic process uses the raw data collected through hydrographic survey into information usable by the end user.

<span class="mw-page-title-main">Autonomous underwater vehicle</span> Unmanned underwater vehicle with autonomous guidance system

An autonomous underwater vehicle (AUV) is a robot that travels underwater without requiring continuous input from an operator. AUVs constitute part of a larger group of undersea systems known as unmanned underwater vehicles, a classification that includes non-autonomous remotely operated underwater vehicles (ROVs) – controlled and powered from the surface by an operator/pilot via an umbilical or using remote control. In military applications an AUV is more often referred to as an unmanned undersea vehicle (UUV). Underwater gliders are a subclass of AUVs.

<span class="mw-page-title-main">Monterey Bay Aquarium Research Institute</span> American oceanographic research institute

The Monterey Bay Aquarium Research Institute (MBARI) is a private, non-profit oceanographic research center in Moss Landing, California. MBARI was founded in 1987 by David Packard, and is primarily funded by the David and Lucile Packard Foundation. Christopher Scholin serves as the institute's president and chief executive officer, managing a work force of approximately 220 scientists, engineers, and operations and administrative staff.

<span class="mw-page-title-main">Unmanned underwater vehicle</span> Submersible vehicles that can operate underwater without a human occupant

Unmanned underwater vehicles (UUV), also known as uncrewed underwater vehicles and underwater drones, are submersible vehicles that can operate underwater without a human occupant. These vehicles may be divided into two categories: remotely operated underwater vehicles (ROUVs) and autonomous underwater vehicles (AUVs). ROUVs are remotely controlled by a human operator. AUVs are automated and operate independently of direct human input.

<span class="mw-page-title-main">Underwater vision</span> The ability to see objects underwater

Underwater vision is the ability to see objects underwater, and this is significantly affected by several factors. Underwater, objects are less visible because of lower levels of natural illumination caused by rapid attenuation of light with distance passed through the water. They are also blurred by scattering of light between the object and the viewer, also resulting in lower contrast. These effects vary with wavelength of the light, and color and turbidity of the water. The vertebrate eye is usually either optimised for underwater vision or air vision, as is the case in the human eye. The visual acuity of the air-optimised eye is severely adversely affected by the difference in refractive index between air and water when immersed in direct contact. Provision of an airspace between the cornea and the water can compensate, but has the side effect of scale and distance distortion. The diver learns to compensate for these distortions. Artificial illumination is effective to improve illumination at short range.

The term post-processing is used in the video/film business for quality-improvement image processing methods used in video playback devices, such as stand-alone DVD-Video players; video playing software; and transcoding software. It is also commonly used in real-time 3D rendering to add additional effects.

Image quality can refer to the level of accuracy with which different imaging systems capture, process, store, compress, transmit and display the signals that form an image. Another definition refers to image quality as "the weighted combination of all of the visually significant attributes of an image". The difference between the two definitions is that one focuses on the characteristics of signal processing in different imaging systems and the latter on the perceptual assessments that make an image pleasant for human viewers.

Intervention AUV or I-AUV is a type of autonomous underwater vehicle. Its characteristic feature is that it is capable of autonomous interventions on the subsea installations, a task usually carried out by remotely operated underwater vehicles (ROVs) or human divers.

Acoustic seabed classification is the partitioning of a seabed acoustic image into discrete physical entities or classes. This is a particularly active area of development in the field of seabed mapping, marine geophysics, underwater acoustics and benthic habitat mapping. Seabed classification is one route to characterizing the seabed and its habitats. Seabed characterization makes the link between the classified regions and the seabed physical, geological, chemical or biological properties. Acoustic seabed classification is possible using a wide range of acoustic imaging systems including multibeam echosounders, sidescan sonar, single-beam echosounders, interferometric systems and sub-bottom profilers. Seabed classification based on acoustic properties can be divided into two main categories; surficial seabed classification and sub-surface seabed classification. Sub-surface imaging technologies use lower frequency sound to provide higher penetration, whereas surficial imaging technologies provide higher resolution imagery by utilizing higher frequencies.

<span class="mw-page-title-main">Short baseline acoustic positioning system</span> Class of underwater acoustic positioning systems used to track underwater vehicles and divers

A short baseline (SBL) acoustic positioning system is one of three broad classes of underwater acoustic positioning systems that are used to track underwater vehicles and divers. The other two classes are ultra short baseline systems (USBL) and long baseline systems (LBL). Like USBL systems, SBL systems do not require any seafloor mounted transponders or equipment and are thus suitable for tracking underwater targets from boats or ships that are either anchored or under way. However, unlike USBL systems, which offer a fixed accuracy, SBL positioning accuracy improves with transducer spacing. Thus, where space permits, such as when operating from larger vessels or a dock, the SBL system can achieve a precision and position robustness that is similar to that of sea floor mounted LBL systems, making the system suitable for high-accuracy survey work. When operating from a smaller vessel where transducer spacing is limited, the SBL system will exhibit reduced precision.

Bistatic sonar is a sonar configuration in which transmitter and receiver are separated by a distance large enough to be comparable to the distance to the target. Most sonar systems are monostatic, in that the transmitter and receiver are located in the same place. A configuration with multiple receivers is called multistatic.

Explorer autonomous underwater vehicle (AUV) is a Chinese AUV developed in the People's Republic of China (PRC), first entering service in November 1994. It should not be confused with another two Anglo-American AUVs that share the same name: the American Autonomous Benthic Explorer AUV (ABE) built by Woods Hole Oceanographic Institution, and the British Columbia-based International Submarine Engineering built Canadian Explorer AUV, which is based on its earlier ARCS AUV. Many Chinese AUVs later developed, such as Wukong, WZODA, CR series, Exploration series, Micro Dragon series, Sea Whale series, Submerged Dragon series AUVs, are all based on experienced gained from Explorer AUV.

AUV - 150 is an unmanned underwater vehicle (UUV) being developed by Central Mechanical Engineering Research Institute (CMERI) scientists in Durgapur in the Indian state of West Bengal. The project is sponsored by the Ministry of Earth Sciences and has technical assistance from IIT-Kharagpur.

<span class="mw-page-title-main">Image editing</span> Processes of altering images

Image editing encompasses the processes of altering images, whether they are digital photographs, traditional photo-chemical photographs, or illustrations. Traditional analog image editing is known as photo retouching, using tools such as an airbrush to modify photographs or editing illustrations with any traditional art medium. Graphic software programs, which can be broadly grouped into vector graphics editors, raster graphics editors, and 3D modelers, are the primary tools with which a user may manipulate, enhance, and transform images. Many image editing programs are also used to render or create computer art from scratch. The term "image editing" usually refers only to the editing of 2D images, not 3D ones.

Underwater searches are procedures to find a known or suspected target object or objects in a specified search area under water. They may be carried out underwater by divers, manned submersibles, remotely operated underwater vehicles, or autonomous underwater vehicles, or from the surface by other agents, including surface vessels, aircraft and cadaver dogs.

The RV Denar 2 is a Turkish research and survey vessel owned by TOMA Maritime S.A. Istanbul, Turkey and operated 2E Maritime in Istanbul, Turkey.

<span class="mw-page-title-main">Ocean optics</span> The study of light interaction with water and submerged materials

Ocean optics is the study of how light interacts with water and the materials in water. Although research often focuses on the sea, the field broadly includes rivers, lakes, inland waters, coastal waters, and large ocean basins. How light acts in water is critical to how ecosystems function underwater. Knowledge of ocean optics is needed in aquatic remote sensing research in order to understand what information can be extracted from the color of the water as it appears from satellite sensors in space. The color of the water as seen by satellites is known as ocean color. While ocean color is a key theme of ocean optics, optics is a broader term that also includes the development of underwater sensors using optical methods to study much more than just color, including ocean chemistry, particle size, imaging of microscopic plants and animals, and more.

<span class="mw-page-title-main">Underwater survey</span> Inspection or measurement in or of an underwater environment

An underwater survey is a survey performed in an underwater environment or conducted remotely on an underwater object or region. Survey can have several meanings. The word originates in Medieval Latin with meanings of looking over and detailed study of a subject. One meaning is the accurate measurement of a geographical region, usually with the intention of plotting the positions of features as a scale map of the region. This meaning is often used in scientific contexts, and also in civil engineering and mineral extraction. Another meaning, often used in a civil, structural, or marine engineering context, is the inspection of a structure or vessel to compare actual condition with the specified nominal condition, usually with the purpose of reporting on the actual condition and compliance with, or deviations from, the nominal condition, for quality control, damage assessment, valuation, insurance, maintenance, and similar purposes. In other contexts it can mean inspection of a region to establish presence and distribution of specified content, such as living organisms, either to establish a baseline, or to compare with a baseline.

<span class="mw-page-title-main">Underwater exploration</span> Investigating or traveling around underwater for the purpose of discovery

Underwater exploration is the exploration of any underwater environment, either by direct observation by the explorer, or by remote observation and measurement under the direction of the investigators. Systematic, targeted exploration is the most effective method to increase understanding of the ocean and other underwater regions, so they can be effectively managed, conserved, regulated, and their resources discovered, accessed, and used. Less than 10% of the ocean has been mapped in any detail, less has been visually observed, and the total diversity of life and distribution of populations is similarly obscure.

References

  1. Horgan, Jonathan; Toal, Daniel (2009). "Computer Vision Applications In the Navigation of Unmanned Underwater Vehicles". Underwater Vehicles. doi:10.5772/6703. ISBN   978-953-7619-49-7. S2CID   2940888.
  2. Y. Schechner, Yoav; Karpel, Nir. "Clear Underwater vision". Proc. Computer Vision & Pattern Recognition. I: 536–543.
  3. Hou, Weilin; J.Gray, Deric; Weidemann, Alan D.; A.Arnone, Robert (2008). "Comparison and Validation of point spread models for imaging in natural waters". Optics Express. 16 (13): 9958–9965. Bibcode:2008OExpr..16.9958H. doi: 10.1364/OE.16.009958 . PMID   18575566.
  4. Schettini, Raimondo; Corchs, Silvia (2010). "Underwater Image Processing: State of the Art Image Enhancement Methods". EURASIP Journal on Advances in Signal Processing. 2010: 14. doi: 10.1155/2010/746052 .
  5. Akkaynak, Derya, and Tali Treibitz. "Sea-Thru: A Method for Removing Water From Underwater Images." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
  6. Iqbal, K.; Odetayo, M.; James, A.; Salam, R.A. "Enhancing the low quality images using Unsupervised Color Correction Methods" (PDF). Systems Man and Cybernetics.[ dead link ]
  7. Mignotte, M.; Collet, C. (2000). "Markov Random Field and Fuzzy Logic Modeling in Sonar Imagery". Computer Vision and Image Understanding. 79: 4–24. CiteSeerX   10.1.1.38.4225 . doi:10.1006/cviu.2000.0844.
  8. Cervenka, Pierre; de Moustier, Christian (1993). "Sidescan Sonar Image Processing Techniques". IEEE Journal of Oceanic Engineering. 18 (2): 108. Bibcode:1993IJOE...18..108C. doi:10.1109/48.219531.
  9. Trucco, E.; Petillot, Y.R.; Tena Ruiz, I. (2000). "Feature Tracking in Video and Sonar Subsea Sequences with Applications". Computer Vision and Image Understanding. 79: 92–122. doi:10.1006/cviu.2000.0846.