Vision processing unit

Last updated

A vision processing unit (VPU) is (as of 2023) an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks. [1] [2]

Contents

Overview

Vision processing units are distinct from graphics processing units (which are specialised for video encoding and decoding) in their suitability for running machine vision algorithms such as CNN (convolutional neural networks), SIFT (scale-invariant feature transform) and similar.

They may include direct interfaces to take data from cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip dataflow between many parallel execution units with scratchpad memory, like a manycore DSP. But, like video processing units, they may have a focus on low precision fixed point arithmetic for image processing.

Contrast with GPUs

They are distinct from GPUs, which contain specialised hardware for rasterization and texture mapping (for 3D graphics), and whose memory architecture is optimised for manipulating bitmap images in off-chip memory (reading textures, and modifying frame buffers, with random access patterns). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.

Target markets are robotics, the internet of things (IoT), new classes of digital cameras for virtual reality and augmented reality, smart cameras, and integrating machine vision acceleration into smartphones and other mobile devices.

Examples

Broader category

Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of AI accelerators (to which VPUs may also belong), however as of 2016 there is no consensus on the name:

See also

Related Research Articles

<span class="mw-page-title-main">ATI Technologies</span> Canadian technology corporation

ATI Technologies Inc., commonly called ATI, was a Canadian semiconductor technology corporation based in Markham, Ontario, that specialized in the development of graphics processing units and chipsets. Founded in 1985, the company listed publicly in 1993 and was acquired by AMD in 2006. As a major fabrication-less or fabless semiconductor company, ATI conducted research and development in-house and outsourced the manufacturing and assembly of its products. With the decline and eventual bankruptcy of 3dfx in 2000, ATI and its chief rival Nvidia emerged as the two dominant players in the graphics processors industry, eventually forcing other manufacturers into niche roles.

<span class="mw-page-title-main">Coprocessor</span> Type of computer processor

A coprocessor is a computer processor used to supplement the functions of the primary processor. Operations performed by the coprocessor may be floating-point arithmetic, graphics, signal processing, string processing, cryptography or I/O interfacing with peripheral devices. By offloading processor-intensive tasks from the main processor, coprocessors can accelerate system performance. Coprocessors allow a line of computers to be customized, so that customers who do not need the extra performance do not need to pay for it.

<span class="mw-page-title-main">Graphics processing unit</span> Specialized electronic circuit; graphics accelerator

A graphics processing unit (GPU) is a specialized electronic circuit initially designed to accelerate computer graphics and image processing. After their initial design, GPUs were found to be useful for non-graphic calculations involving embarrassingly parallel problems due to their parallel structure. Other non-graphical uses include the training of neural networks and cryptocurrency mining.

<span class="mw-page-title-main">Hardware acceleration</span> Specialized computer hardware

Hardware acceleration is the use of computer hardware designed to perform specific functions more efficiently when compared to software running on a general-purpose central processing unit (CPU). Any transformation of data that can be calculated in software running on a generic CPU can also be calculated in custom-made hardware, or in some mix of both.

Scratchpad memory (SPM), also known as scratchpad, scratchpad RAM or local store in computer terminology, is an internal memory, usually high-speed, used for temporary storage of calculations, data, and other work in progress. In reference to a microprocessor, scratchpad refers to a special high-speed memory used to hold small items of data for rapid retrieval. It is similar to the usage and size of a scratchpad in life: a pad of paper for preliminary notes or sketches or writings, etc. When the scratchpad is a hidden portion of the main memory then it is sometimes referred to as bump storage.

VPU may refer to:

Manycore processors are special kinds of multi-core processors designed for a high degree of parallel processing, containing numerous simpler, independent processor cores. Manycore processors are used extensively in embedded computers and high-performance computing.

<span class="mw-page-title-main">Stick PC</span>

A stick PC or PC on a stick is a single-board computer in a small elongated casing resembling a stick, that can usually be plugged directly into an HDMI video port. A stick PC is a device which has independent CPUs or processing chips and which does not rely on another computer. It should not be confused with passive storage devices such as thumb drives.

A cognitive computer is a computer that hardwires artificial intelligence and machine learning algorithms into an integrated circuit that closely reproduces the behavior of the human brain. It generally adopts a neuromorphic engineering approach. Synonyms include neuromorphic chip and cognitive chip.

Arteris, Inc. is a multinational technology firm headquartered in Campbell, California. It develops the Network-on-Chip (NoC) on-chip interconnect IP and System-on-Chip (SoC) integration automation software used to create semiconductor designs for a variety of devices, particularly in automotive electronics, artificial intelligence/machine learning and consumer markets. The company specializes in the development and distribution of Network-on-Chip (NoC) interconnect Intellectual Property (IP) and SoC integration automation products used in the development of systems-on-chip.

Heterogeneous computing refers to systems that use more than one kind of processor or core. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar coprocessors, usually incorporating specialized processing capabilities to handle particular tasks.

Zeroth is a platform for brain-inspired computing from Qualcomm. It is based around a neural processing unit (NPU) AI accelerator chip and a software API to interact with the platform. It makes a form of machine learning known as deep learning available to mobile devices. It is used for image and sound processing, including speech recognition. The software operates locally rather than as a cloud application.

<span class="mw-page-title-main">Movidius</span> American computer processor chip design company

Movidius is a company based in San Mateo, California, that designs low-power processor chips for computer vision. The company was acquired by Intel in September 2016.

<span class="mw-page-title-main">Tensor Processing Unit</span> AI accelerator ASIC by Google

Tensor Processing Unit (TPU) is an AI accelerator application-specific integrated circuit (ASIC) developed by Google for neural network machine learning, using Google's own TensorFlow software. Google began using TPUs internally in 2015, and in 2018 made them available for third-party use, both as part of its cloud infrastructure and by offering a smaller version of the chip for sale.

In computing, a cache control instruction is a hint embedded in the instruction stream of a processor intended to improve the performance of hardware caches, using foreknowledge of the memory access pattern supplied by the programmer or compiler. They may reduce cache pollution, reduce bandwidth requirement, bypass latencies, by providing better control over the working set. Most cache control instructions do not affect the semantics of a program, although some can.

An AI accelerator, deep learning processor, or neural processing unit is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and machine vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2024, a typical AI integrated circuit chip contains tens of billions of MOSFET transistors.

<span class="mw-page-title-main">Remi El-Ouazzane</span>

Remi El-Ouazzane is a French businessman and embedded systems engineer who has led various initiatives in mobile computing, machine vision and embedded artificial intelligence. El-Ouazzane currently serves as STMicroelectronics (ST) President, Microcontrollers and Digital ICs Group and has held this position since January 1, 2022. He is a member of ST's executive committee.

Coherent Accelerator Processor Interface (CAPI), is a high-speed processor expansion bus standard for use in large data center computers, initially designed to be layered on top of PCI Express, for directly connecting central processing units (CPUs) to external accelerators like graphics processing units (GPUs), ASICs, FPGAs or fast storage. It offers low latency, high speed, direct memory access connectivity between devices of different instruction set architectures.

<span class="mw-page-title-main">AMD Instinct</span> Brand name by AMD; professional GPUs for high-performance-computing, machine learning

AMD Instinct is AMD's brand of professional GPUs. It replaced AMD's FirePro S brand in 2016. Compared to the Radeon brand of mainstream consumer/gamer products, the Instinct product line is intended to accelerate deep learning, artificial neural network, and high-performance computing/GPGPU applications.

Specialized computer hardware is often used to execute artificial intelligence (AI) programs faster, and with less energy, such as Lisp machines, neuromorphic engineering, event cameras, and physical neural networks. As of 2023, the market for AI hardware is dominated by GPUs.

References

  1. Seth Colaner; Matthew Humrick (January 3, 2016). "A third type of processor for AR/VR: Movidius' Myriad 2 VPU". Tom's Hardware.
  2. Prasid Banerje (March 28, 2016). "The rise of VPUs: Giving Eyes to Machines". Digit.in.
  3. "Intel® Movidius™ Vision Processing Units (VPUs)". Intel.
  4. Weckler, Adrian. "Dublin tech firm Movidius to power Google's new virtual reality headset". Independent.ie. Retrieved 15 March 2016.
  5. "DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius". www.movidius.com.
  6. Fred O'Connor (May 1, 2015). "Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed". PCWorld.
  7. Chen, Yu-Hsin; Krishna, Tushar; Emer, Joel & Sze, Vivienne (2016). "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks". IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers. pp. 262–263.
  8. "Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing". Qualcomm. October 10, 2013.
  9. "Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips". PCMAG.