Smart camera

Last updated
Early smart camera (ca. 1985, in red) with an 8MHz Z80 compared to a modern device featuring Texas Instruments' C64 @1GHz MicroScanner VC smartcamera.jpg
Early smart camera (ca. 1985, in red) with an 8MHz Z80 compared to a modern device featuring Texas Instruments' C64 @1GHz

A smart camera (sensor) or intelligent camera (sensor) or (smart) vision sensor or intelligent vision sensor or smart optical sensor or intelligent optical sensor or smart visual sensor or intelligent visual sensor is a machine vision system which, in addition to image capture circuitry, is capable of extracting application-specific information from the captured images, along with generating event descriptions or making decisions that are used in an intelligent and automated system. [1] [2] A smart camera is a self-contained, standalone vision system with built-in image sensor in the housing of an industrial video camera. The vision system and the image sensor can be integrated into one single piece of hardware known as intelligent image sensor or smart image sensor. It contains all necessary communication interfaces, e.g. Ethernet, as well as industry-proof 24V I/O lines for connection to a PLC, actuators, relays or pneumatic valves, and can be either static or mobile. [3] It is not necessarily larger than an industrial or surveillance camera. A capability in machine vision generally means a degree of development such that these capabilities are ready for use on individual applications. This architecture has the advantage of a more compact volume compared to PC-based vision systems and often achieves lower cost, at the expense of a somewhat simpler (or omitted) user interface. Smart cameras are also referred to by the more general term smart sensors. [4]

Contents

History

The first publication of the term smart camera was in 1975 [5] as according to Belbachir et al. [6] In 1976, the General Electric's Electronic Systems Division indicated requirements of two industrial firms for smart cameras in a report for National Technical Information Service. [7] Authors affiliated in HRL Laboratories defined a smart camera as "a camera that could process its pictures before recording them" in 1976. [8] One of the first mentions of smart optical sensors appeared in a concept evaluation for satellites by NASA and General Electric Space Division from 1977. [9] They were suggested as a means for intelligent on-board editing and reduction of data.

Smart cameras have been marketed since the mid 80s. In the 21st century they have reached widespread use, since technology allowed their size to be reduced and their processing power reached several thousand MIPS (devices with 1 GHz processors and up to 8000MIPS are available as of end of 2006).

Artificial intelligence and photonics boost each other. [10] Photonics accelerates the process of data collection for AI and AI improves the spectrum of applications of photinics. In 2020, Sony has launched the first intelligent vision sensors with AI edge computing capabilies. [11] It is a further development of Exmor technology.

Components

A smart camera usually consists of several (but not necessarily all) of the following components:

Fields of application

Having a dedicated processor in each unit, smart cameras are especially suited for applications where several cameras must operate independently and often asynchronously, or when distributed vision is required (multiple inspection or surveillance points along a production line or within an assembly machine). In general smart cameras can be used for the same kind of applications where more complex vision systems are used, and can additionally be applied in some applications where volume, pricing or reliability constraints forbid use of bulkier devices and PC's.

Typical fields of application are:

Developers can purchase smart cameras and develop their own programs for special, custom made applications, or they can purchase ready made application software from the camera manufacturer or from third party sources. Custom programs can be developed by programming in various languages (typically C or C++) or by using more intuitive, albeit somewhat less flexible, visual development tools where existing functionalities (often called tool or blocks) can be connected in a list (a sequence or a bi-dimensional flowchart) that describes the desired flow of operations without any need to write program code. The main advantage of the visual approach versus programming is the shorter and somewhat easier development process, available also to non-programmers. Other development tools are available with relatively few but comparatively high level functionalities, which can be configured and deployed with very limited effort.

See also

Related Research Articles

<span class="mw-page-title-main">Computer vision</span> Computerized information extraction from images

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Intelligent transportation system</span> Advanced application

An intelligent transportation system (ITS) is an advanced application which aims to provide innovative services relating to different modes of transport and traffic management and enable users to be better informed and make safer, more coordinated, and 'smarter' use of transport networks.

<span class="mw-page-title-main">Machine vision</span> Technology and methods used to provide imaging-based automatic inspection and analysis

Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to many technologies, software and hardware products, integrated systems, actions, methods and expertise. Machine vision as a systems engineering discipline can be considered distinct from computer vision, a form of computer science. It attempts to integrate existing technologies in new ways and apply them to solve real world problems. The term is the prevalent one for these functions in industrial automation environments but is also used for these functions in other environment vehicle guidance.

<span class="mw-page-title-main">Forward-looking infrared</span> Type of thermographic camera

Forward-looking infrared (FLIR) cameras, typically used on military and civilian aircraft, use a thermographic camera that senses infrared radiation.

<span class="mw-page-title-main">Photonics</span> Technical applications of optics

Photonics is a branch of optics that involves the application of generation, detection, and manipulation of light in form of photons through emission, transmission, modulation, signal processing, switching, amplification, and sensing. Photonics is closely related to quantum electronics, where quantum electronics deals with the theoretical part of it while photonics deal with its engineering applications. Though covering all light's technical applications over the whole spectrum, most photonic applications are in the range of visible and near-infrared light. The term photonics developed as an outgrowth of the first practical semiconductor light emitters invented in the early 1960s and optical fibers developed in the 1970s.

<span class="mw-page-title-main">Thermographic camera</span> Imaging device using infrared radiation

A thermographic camera is a device that creates an image using infrared (IR) radiation, similar to a normal camera that forms an image using visible light. Instead of the 400–700 nanometre (nm) range of the visible light camera, infrared cameras are sensitive to wavelengths from about 1,000 nm to about 14,000 nm (14 μm). The practice of capturing and analyzing the data they provide is called thermography.

<span class="mw-page-title-main">Thermography</span> Use of thermograms to study heat distribution in structures or regions

Infrared thermography (IRT), thermal video and/or thermal imaging, is a process where a thermal camera captures and creates an image of an object by using infrared radiation emitted from the object in a process, which are examples of infrared imaging science. Thermographic cameras usually detect radiation in the long-infrared range of the electromagnetic spectrum and produce images of that radiation, called thermograms. Since infrared radiation is emitted by all objects with a temperature above absolute zero according to the black body radiation law, thermography makes it possible to see one's environment with or without visible illumination. The amount of radiation emitted by an object increases with temperature; therefore, thermography allows one to see variations in temperature. When viewed through a thermal imaging camera, warm objects stand out well against cooler backgrounds; humans and other warm-blooded animals become easily visible against the environment, day or night. As a result, thermography is particularly useful to the military and other users of surveillance cameras.

<span class="mw-page-title-main">Video camera</span> Camera used for electronic motion picture acquisition

A video camera is an optical instrument that captures videos. Video cameras were initially developed for the television industry but have since become widely used for a variety of other purposes.

Smartdust is a system of many tiny microelectromechanical systems (MEMS) such as sensors, robots, or other devices, that can detect, for example, light, temperature, vibration, magnetism, or chemicals. They are usually operated on a computer network wirelessly and are distributed over some area to perform tasks, usually sensing through radio-frequency identification. Without an antenna of much greater size the range of tiny smart dust communication devices is measured in a few millimeters and they may be vulnerable to electromagnetic disablement and destruction by microwave exposure.

Cognex Corporation is an American manufacturer of machine vision systems, software and sensors used in automated manufacturing to inspect and identify parts, detect defects, verify product assembly, and guide assembly robots. Cognex is headquartered in Natick, Massachusetts, USA and has offices in more than 20 countries.

<span class="mw-page-title-main">Hyperspectral imaging</span> Multi-wavelength imaging method

Hyperspectral imaging collects and processes information from across the electromagnetic spectrum. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifying materials, or detecting processes. There are three general types of spectral imagers. There are push broom scanners and the related whisk broom scanners, which read images over time, band sequential scanners, which acquire images of an area at different wavelengths, and snapshot hyperspectral imagers, which uses a staring array to generate an image in an instant.

The following are common definitions related to the machine vision field.

The following outline is provided as an overview of and topical guide to computer vision:

A visual sensor network or smart camera network or intelligent camera network is a network of spatially distributed smart camera devices capable of processing, exchanging data and fusing images of a scene from a variety of viewpoints into some form more useful than the individual images. A visual sensor network may be a type of wireless sensor network, and much of the theory and application of the latter applies to the former. The network generally consists of the cameras themselves, which have some local image processing, communication and storage capabilities, and possibly one or more central computers, where image data from multiple cameras is further processed and fused. Visual sensor networks also provide some high-level services to the user so that the large amount of data can be distilled into information of interest using specific queries.

An under-vehicle inspection (UVI) system generally consists of imaging systems mounted on a roadway and used at facility access points, particularly at secure facilities. An under-vehicle inspection system is used to detect threats—such as bombs—that are hidden underneath vehicles. Cameras capture images of the undercarriage of the vehicle for manual or automated visual inspection by security personnel or systems.

Visual privacy is the relationship between collection and dissemination of visual information, the expectation of privacy, and the legal issues surrounding them. These days digital cameras are ubiquitous. They are one of the most common sensors found in electronic devices, ranging from smartphones to tablets, and laptops to surveillance cams. However, privacy and trust implications surrounding it limit its ability to seamlessly blend into computing environment. In particular, large-scale camera networks have created increasing interest in understanding the advantages and disadvantages of such deployments. It is estimated that over 4 million CCTV cameras deployed in the UK. Due to increasing security concerns, camera networks have continued to proliferate across other countries such as the United States. While the impact of such systems continues to be evaluated, in parallel, tools for controlling how these camera networks are used and modifications to the images and video sent to end-users have been explored.

Optical sorting is the automated process of sorting solid products using cameras and/or lasers.

The Australian Research Centre for Aerospace Automation (ARCAA) was a research centre of the Queensland University of Technology. ARCAA conducted research into all aspects of aviation automation, with a particular research focus on autonomous technologies which support the more efficient and safer utilisation of airspace, and the development of autonomous aircraft and on-board sensor systems for a wide range of commercial applications.

An AI accelerator is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and machine vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2018, a typical AI integrated circuit chip contains billions of MOSFET transistors. A number of vendor-specific terms exist for devices in this category, and it is an emerging technology without a dominant design.

An event camera, also known as a neuromorphic camera, silicon retina or dynamic vision sensor, is an imaging sensor that responds to local changes in brightness. Event cameras do not capture images using a shutter as conventional (frame) cameras do. Instead, each pixel inside an event camera operates independently and asynchronously, reporting changes in brightness as they occur, and staying silent otherwise.

References

  1. Ahmed Nabil Belbachir, ed. (2009). Smart Cameras. Springer. ISBN   978-1-4419-0952-7.
  2. Alexander Hornberg (2006). Handbook of Machine Vision. Wiley-VCH. ISBN   3-527-40584-4.
  3. Rinner, Bernhard; Wolf, Wayne (2018-10-17). "An Introduction to Distributed Smart Cameras". Proceedings of the IEEE. 96 (10): 1565–1575. doi:10.1109/JPROC.2008.928742. S2CID   22637359 . Retrieved 2021-09-13.
  4. Birem, Merwan; Berry, François (2014). "DreamCam: A modular FPGA-based smart camera architecture". Journal of Systems Architecture. 60 (6): 519–527. doi:10.1016/j.sysarc.2014.01.006. ISSN   1383-7621. S2CID   2095590 . Retrieved 15 May 2021.
  5. SCHNEIDERMAN, R; R, SCHNEIDERMAN (1975). "SMART CAMERAS CLICKING WITH ELECTRONIC FUNCTIONS". Electronics. 48 (17): 74–81. Retrieved 27 June 2021.
  6. Belbachir, Ahmed Nabil, ed. (2010). Smart Cameras. doi:10.1007/978-1-4419-0953-4. ISBN   978-1-4419-0954-1 . Retrieved 27 June 2021.
  7. General Electric (1976). "SOLID STATE TELEVISION CAMERA (CID)" (PDF). NATIONAL TECHNICAL INFORMATION SERVICE. Retrieved 15 May 2021.
  8. Reif, P. G.; Jacobson, A. D.; Bleha, W. P.; Grinberg, J. (12 January 1977). Casasent, David P; Sawchuk, Alexander A (eds.). "HYBRID LIQUID CRYSTAL LIGHT VALVE -IMAGE TUBE DEVICES FOR OPTICAL DATA PROCESSING". Optical Information Processing: Real Time Devices & Novel Techniques. International Society for Optics and Photonics. 0083: 34–43. doi:10.1117/12.954897. S2CID   110882955 . Retrieved 15 May 2021.
  9. "POST LANDSAT D ADVANCED CONCEPT EVALUATION" (PDF). core.ac.uk/. NASA. 1977. Retrieved 15 May 2021.
  10. Goda, Keisuke; Jalali, Bahram; Lei, Cheng; Situ, Guohai; Westbrook, Paul (2020-07-01). "AI boosts photonics and vice versa". APL Photonics. 5 (7): 070401. Bibcode:2020APLP....5g0401G. doi: 10.1063/5.0017902 . S2CID   225783016.
  11. Lorbeer/pi, Klaus (14 May 2020). "Sony launcht weltweit erste Intelligent-Vision-Sensoren mit KI-Verarbeitung". computerwelt.at (in German). Archived from the original on 15 May 2021. Retrieved 15 May 2021.