Underwater computer vision

Last updated

Underwater computer vision is a subfield of computer vision. In recent years, with the development of underwater vehicles ( ROV, AUV, gliders), the need to be able to record and process huge amounts of information has become increasingly important. Applications range from inspection of underwater structures for the offshore industry to the identification and counting of fishes for biological research. However, no matter how big the impact of this technology can be to industry and research, it still is in a very early stage of development compared to traditional computer vision. One reason for this is that, the moment the camera goes into the water, a whole new set of challenges appear. On one hand, cameras have to be made waterproof, marine corrosion deteriorates materials quickly and access and modifications to experimental setups are costly, both in time and resources. On the other hand, the physical properties of the water make light behave differently, changing the appearance of a same object with variations of depth, organic material, currents, temperature etc.

Contents

Applications

Medium differences

Illumination

In air, light comes from the whole hemisphere on cloudy days, and is dominated by the sun. In water direct lighting comes from a cone about 96° wide above the scene. This phenomenon is called Snell's window. [2]

Artificial lighting can be used where natural light levels are insufficient and where the light path is too long to produce acceptable colour, as the loss of colour is a function of the total distance through water from the source to the camera lens port. [3]

Light attenuation

Comparison of penetration of light of different wavelengths in the open ocean and coastal waters NOAA Deep Light diagram3.jpg
Comparison of penetration of light of different wavelengths in the open ocean and coastal waters

Unlike air,[ citation needed ] water attenuates light exponentially. This results in hazy images with very low contrast.[ clarification needed ] The main reasons for light attenuation are light absorption (where energy is removed from the light) and light scattering, by which the direction of light is changed. Light scattering can further be divided into forward scattering, which results in an increased blurriness and backward scattering that limits the contrast and is responsible for the characteristic veil of underwater images. Both scattering and attenuation are heavily influenced by the amount of organic matter dissolved or suspended in the water.[ citation needed ]

Light attenuation in water is also a function of the wavelength. This means that different colours are attenuated at different rates, leading to colour degradation.with depth and distance. Red and orange light are attenuated faster, followed by yellows and greens. Blue is the least attenuated visible wavelength. [4]

Artificial lighting

Challenges

In high level computer vision, human structures are frequently used as image features for image matching in different applications. However, the sea bottom lacks such features, making it hard to find correspondences in two images.

In order to be able to use a camera in the water, a watertight housing is required. However, refraction will happen at the water-glass and glass-air interface due to differences in density of the materials. This has the effect of introducing a non-linear image deformation.

The motion of the vehicle presents another special challenge. Underwater vehicles are constantly moving due to currents and other phenomena. This introduces another uncertainty to algorithms, where small motions may appear in all directions. This can be specially important for video tracking. In order to reduce this problem image stabilization algorithms may be applied.

Relevant technology

Image restoration

Image restoration< techniques are intended to model the degradation process and then invert it, obtaining the new image after solving. [5] [6] It is generally a complex approach that requires plenty of parameters[ clarification needed ] that vary a lot between different water conditions.

Image enhancement

Image enhancement [7] only tries to provide a visually more appealing image without taking the physical image formation process into account. These methods are usually simpler and less computational intensive.

Color correction

Various algorithms exist that perform automatic color correction. [8] [9] The UCM (Unsupervised Color Correction Method), for example, does this in the following steps: It firstly reduces the color cast by equalizing the color values. Then it enhances contrast by stretching the red histogram towards the maximum and finally saturation and intensity components are optimized.[ citation needed ]

Underwater stereo vision

It is usually assumed that stereo cameras have been calibrated previously, geometrically and radiometrically. This leads to the assumption that corresponding pixels should have the same color. However this can not be guaranteed in an underwater scene, because of dispersion and backscatter. However, it is possible to digitally model this phenomenon and create a virtual image with those effects removed

Other application fields

Imaging sonars [10] [11] have become more and more accessible and gained resolution, delivering better images. Sidescan sonars are used to produce complete maps of regions of the sea floor stitching together sequences of sonar images. However, sonar images often lack proper contrast[ clarification needed ] and are degraded by artefacts and distortions due to noise, attitude changes of the AUV/ROV carrying the sonar or non uniform beam patterns. Another common problem with sonar computer vision is the comparatively low frame rate of sonar images. [12]

Related Research Articles

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Side-scan sonar</span> Tool for seafloor mapping

Side-scan sonar is a category of sonar system that is used to efficiently create an image of large areas of the sea floor.

<span class="mw-page-title-main">Underwater archaeology</span> Archaeological techniques practiced at underwater sites

Underwater archaeology is archaeology practiced underwater. As with all other branches of archaeology, it evolved from its roots in pre-history and in the classical era to include sites from the historical and industrial eras.

<span class="mw-page-title-main">Hydrographic survey</span> Science of measurement and description of features which affect maritime activities

Hydrographic survey is the science of measurement and description of features which affect maritime navigation, marine construction, dredging, offshore wind farms, offshore oil exploration and drilling and related activities. Surveys may also be conducted to determine the route of subsea cables such as telecommunications cables, cables associated with wind farms, and HVDC power cables. Strong emphasis is placed on soundings, shorelines, tides, currents, seabed and submerged obstructions that relate to the previously mentioned activities. The term hydrography is used synonymously to describe maritime cartography, which in the final stages of the hydrographic process uses the raw data collected through hydrographic survey into information usable by the end user.

<span class="mw-page-title-main">Remotely operated underwater vehicle</span> A tethered underwater mobile device operated by a remote crew

A remotely operated underwater vehicle (ROUV) or remotely operated vehicle (ROV) is a free-swimming submersible craft used to perform underwater observation, inspection and physical tasks such as valve operations, hydraulic functions and other general tasks within the subsea oil and gas industry, military, scientific and other applications. ROVs can also carry tooling packages for undertaking specific tasks such as pull-in and connection of flexible flowlines and umbilicals, and component replacement.

<span class="mw-page-title-main">Autonomous underwater vehicle</span> Uncrewed underwater vehicle with autonomous guidance system

An autonomous underwater vehicle (AUV) is a robot that travels underwater without requiring continuous input from an operator. AUVs constitute part of a larger group of undersea systems known as unmanned underwater vehicles, a classification that includes non-autonomous remotely operated underwater vehicles (ROVs) – controlled and powered from the surface by an operator/pilot via an umbilical or using remote control. In military applications an AUV is more often referred to as an unmanned undersea vehicle (UUV). Underwater gliders are a subclass of AUVs.

<span class="mw-page-title-main">Synthetic-aperture sonar</span> Form of sonar using post-processing of sonar data

Synthetic-aperture sonar (SAS) is a form of sonar in which sophisticated post-processing of sonar data is used in ways closely analogous to synthetic-aperture radar.

<span class="mw-page-title-main">Monterey Bay Aquarium Research Institute</span> American oceanographic research institute

The Monterey Bay Aquarium Research Institute (MBARI) is a private, non-profit oceanographic research center in Moss Landing, California. MBARI was founded in 1987 by David Packard, and is primarily funded by the David and Lucile Packard Foundation. Christopher Scholin serves as the institute's president and chief executive officer, managing a work force of approximately 220 scientists, engineers, and operations and administrative staff.

<span class="mw-page-title-main">Unmanned underwater vehicle</span> Submersible vehicles that can operate underwater without a human occupant

Unmanned underwater vehicles (UUV), also known as uncrewed underwater vehicles and underwater drones, are submersible vehicles that can operate underwater without a human occupant. These vehicles may be divided into two categories: remotely operated underwater vehicles (ROUVs) and autonomous underwater vehicles (AUVs). ROUVs are remotely controlled by a human operator. AUVs are automated and operate independently of direct human input.

<span class="mw-page-title-main">Underwater vision</span> The ability to see objects underwater

Underwater vision is the ability to see objects underwater, and this is significantly affected by several factors. Underwater, objects are less visible because of lower levels of natural illumination caused by rapid attenuation of light with distance passed through the water. They are also blurred by scattering of light between the object and the viewer, also resulting in lower contrast. These effects vary with wavelength of the light, and color and turbidity of the water. The vertebrate eye is usually either optimised for underwater vision or air vision, as is the case in the human eye. The visual acuity of the air-optimised eye is severely adversely affected by the difference in refractive index between air and water when immersed in direct contact. Provision of an airspace between the cornea and the water can compensate, but has the side effect of scale and distance distortion. The diver learns to compensate for these distortions. Artificial illumination is effective to improve illumination at short range.

Intervention AUV or I-AUV is a type of autonomous underwater vehicle. Its characteristic feature is that it is capable of autonomous interventions on the subsea installations, a task usually carried out by remotely operated underwater vehicles (ROVs) or human divers.

Acoustic seabed classification is the partitioning of a seabed acoustic image into discrete physical entities or classes. This is a particularly active area of development in the field of seabed mapping, marine geophysics, underwater acoustics and benthic habitat mapping. Seabed classification is one route to characterizing the seabed and its habitats. Seabed characterization makes the link between the classified regions and the seabed physical, geological, chemical or biological properties. Acoustic seabed classification is possible using a wide range of acoustic imaging systems including multibeam echosounders, sidescan sonar, single-beam echosounders, interferometric systems and sub-bottom profilers. Seabed classification based on acoustic properties can be divided into two main categories; surficial seabed classification and sub-surface seabed classification. Sub-surface imaging technologies use lower frequency sound to provide higher penetration, whereas surficial imaging technologies provide higher resolution imagery by utilizing higher frequencies.

An underwater acoustic positioning system is a system for the tracking and navigation of underwater vehicles or divers by means of acoustic distance and/or direction measurements, and subsequent position triangulation. Underwater acoustic positioning systems are commonly used in a wide variety of underwater work, including oil and gas exploration, ocean sciences, salvage operations, marine archaeology, law enforcement and military activities.

Bistatic sonar is a sonar configuration in which transmitter and receiver are separated by a distance large enough to be comparable to the distance to the target. Most sonar systems are monostatic, in that the transmitter and receiver are located in the same place. A configuration with multiple receivers is called multistatic.

Explorer autonomous underwater vehicle (AUV) is a Chinese AUV developed in the People's Republic of China (PRC), first entering service in November 1994. It should not be confused with another two Anglo-American AUVs that share the same name: the American Autonomous Benthic Explorer AUV (ABE) built by Woods Hole Oceanographic Institution, and the British Columbia-based International Submarine Engineering built Canadian Explorer AUV, which is based on its earlier ARCS AUV. Many Chinese AUVs later developed, such as Wukong, WZODA, CR series, Exploration series, Micro Dragon series, Sea Whale series, Submerged Dragon series AUVs, are all based on experienced gained from Explorer AUV.

Underwater searches are procedures to find a known or suspected target object or objects in a specified search area under water. They may be carried out underwater by divers, manned submersibles, remotely operated underwater vehicles, or autonomous underwater vehicles, or from the surface by other agents, including surface vessels, aircraft and cadaver dogs.

The RV Denar 2 is a Turkish research and survey vessel owned by TOMA Maritime S.A. Istanbul, Turkey and operated 2E Maritime in Istanbul, Turkey.

<span class="mw-page-title-main">Ocean optics</span> The study of light interaction with water and submerged materials

Ocean optics is the study of how light interacts with water and the materials in water. Although research often focuses on the sea, the field broadly includes rivers, lakes, inland waters, coastal waters, and large ocean basins. How light acts in water is critical to how ecosystems function underwater. Knowledge of ocean optics is needed in aquatic remote sensing research in order to understand what information can be extracted from the color of the water as it appears from satellite sensors in space. The color of the water as seen by satellites is known as ocean color. While ocean color is a key theme of ocean optics, optics is a broader term that also includes the development of underwater sensors using optical methods to study much more than just color, including ocean chemistry, particle size, imaging of microscopic plants and animals, and more.

<span class="mw-page-title-main">Underwater survey</span> Inspection or measurement in or of an underwater environment

An underwater survey is a survey performed in an underwater environment or conducted remotely on an underwater object or region. Survey can have several meanings. The word originates in Medieval Latin with meanings of looking over and detailed study of a subject. One meaning is the accurate measurement of a geographical region, usually with the intention of plotting the positions of features as a scale map of the region. This meaning is often used in scientific contexts, and also in civil engineering and mineral extraction. Another meaning, often used in a civil, structural, or marine engineering context, is the inspection of a structure or vessel to compare actual condition with the specified nominal condition, usually with the purpose of reporting on the actual condition and compliance with, or deviations from, the nominal condition, for quality control, damage assessment, valuation, insurance, maintenance, and similar purposes. In other contexts it can mean inspection of a region to establish presence and distribution of specified content, such as living organisms, either to establish a baseline, or to compare with a baseline.

<span class="mw-page-title-main">Underwater exploration</span> Investigating or traveling around underwater for the purpose of discovery

Underwater exploration is the exploration of any underwater environment, either by direct observation by the explorer, or by remote observation and measurement under the direction of the investigators. Systematic, targeted exploration is the most effective method to increase understanding of the ocean and other underwater regions, so they can be effectively managed, conserved, regulated, and their resources discovered, accessed, and used. Less than 10% of the ocean has been mapped in any detail, less has been visually observed, and the total diversity of life and distribution of populations is similarly obscure.

References

  1. Horgan, Jonathan; Toal, Daniel (2009). "Computer Vision Applications In the Navigation of Unmanned Underwater Vehicles". Underwater Vehicles. doi:10.5772/6703. ISBN   978-953-7619-49-7. S2CID   2940888.
  2. Martin Edge and Ian Turner (1999). The Underwater Photographer. Focal Press. ISBN   0-240-51581-1. Archived from the original on 2023-10-22. Retrieved 2024-07-24.
  3. "Color underwater". Deep-six.com. Archived from the original on 23 July 2024. Retrieved 23 July 2024.
  4. Hegde, M. (30 September 2009). "The Blue, the Bluer, and the Bluest Ocean". NASA Goddard Earth Sciences Data and Information Services. Archived from the original on 12 July 2009. Retrieved 27 May 2011.
  5. Y. Schechner, Yoav; Karpel, Nir. "Clear Underwater vision". Proc. Computer Vision & Pattern Recognition. I: 536–543.
  6. Hou, Weilin; J.Gray, Deric; Weidemann, Alan D.; A.Arnone, Robert (2008). "Comparison and Validation of point spread models for imaging in natural waters". Optics Express. 16 (13): 9958–9965. Bibcode:2008OExpr..16.9958H. doi: 10.1364/OE.16.009958 . PMID   18575566.
  7. Schettini, Raimondo; Corchs, Silvia (2010). "Underwater Image Processing: State of the Art Image Enhancement Methods". EURASIP Journal on Advances in Signal Processing. 2010: 14. doi: 10.1155/2010/746052 .
  8. Akkaynak, Derya, and Tali Treibitz. "Sea-Thru: A Method for Removing Water From Underwater Images." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
  9. Iqbal, K.; Odetayo, M.; James, A.; Salam, R.A. "Enhancing the low quality images using Unsupervised Color Correction Methods" (PDF). Systems Man and Cybernetics.[ dead link ]
  10. Mignotte, M.; Collet, C. (2000). "Markov Random Field and Fuzzy Logic Modeling in Sonar Imagery". Computer Vision and Image Understanding. 79: 4–24. CiteSeerX   10.1.1.38.4225 . doi:10.1006/cviu.2000.0844.
  11. Cervenka, Pierre; de Moustier, Christian (1993). "Sidescan Sonar Image Processing Techniques". IEEE Journal of Oceanic Engineering. 18 (2): 108. Bibcode:1993IJOE...18..108C. doi:10.1109/48.219531.
  12. Trucco, E.; Petillot, Y.R.; Tena Ruiz, I. (2000). "Feature Tracking in Video and Sonar Subsea Sequences with Applications". Computer Vision and Image Understanding. 79: 92–122. doi:10.1006/cviu.2000.0846.