The Pixel Visual Core (PVC) is a series of ARM-based system in package (SiP) image processors designed by Google. [1] The PVC is a fully programmable image, vision and AI multi-core domain-specific architecture (DSA) for mobile devices and in future for IoT. [2] It first appeared in the Google Pixel 2 and 2 XL which were introduced on October 19, 2017. It has also appeared in the Google Pixel 3 and 3 XL. Starting with the Pixel 4, this chip was replaced with the Pixel Neural Core.
Google previously used Qualcomm Snapdragon's CPU, GPU, IPU, and DSP to handle its image processing for their Google Nexus and Google Pixel devices. With the increasing importance of computational photography techniques, Google developed the Pixel Visual Core (PVC). Google claims the PVC uses less power than using CPU and GPU while still being fully programmable, unlike their tensor processing unit (TPU) application-specific integrated circuit (ASIC). Indeed, classical mobile devices equip an image signal processor (ISP) that is a fixed functionality image processing pipeline. In contrast to this, the PVC has a flexible programmable functionality, not limited only to image processing.
The PVC in the Google Pixel 2 and 2 XL is labeled SR3HX X726C502. [3]
The PVC in the Google Pixel 3 and 3 XL is labeled SR3HX X739F030. [4]
Thanks to the PVC, the Pixel 2 and Pixel 3 obtained a mobile DxOMark of 98 [5] and 101. [6] The latter one was the top-ranked single-lens mobile DxOMark score, tied with the iPhone XR. [7]
A typical image-processing program of the PVC is written in Halide. Currently, it supports just a subset of Halide programming language without floating point operations and with limited memory access patterns. [8] Halide is a domain-specific language that lets the user decouple the algorithm and the scheduling of its execution. In this way, the developer can write a program that is optimized for the target hardware architecture. [2]
The PVC has two types of instruction set architecture (ISA), a virtual and a physical one. First, a high-level language program is compiled into a virtual ISA (vISA), inspired by RISC-V ISA, [2] which abstracts completely from the target hardware generation. Then, the vISA program is compiled into the so-called physical ISA (pISA), that is a VLIW ISA. This compilation step takes into account the target hardware parameters (e.g. array of PEs size, STP size, etc...) and specify explicitly memory movements. The decoupling of vISA and pISA lets the first one to be cross-architecture and generation-independent, while pISA can be compiled offline or through JIT compilation. [8]
The Pixel Visual Core is designed to be a scalable multi-core energy-efficient architecture, ranging from even numbers between 2 and 16 core designs. [2] The core of a PVC is the image processing unit (IPU) a programmable unit tailored for image processing. The Pixel Visual Core architecture was also designed either to be its own chip, like the SR3HX, or as an IP block for System on a chip (SOC). [2]
The IPU core has a stencil processor (STP), a line buffer pool (LBP) and a NoC. The STP mainly provides a 2-D SIMD array of processing elements (PEs) able to perform stencil computations, a small neighborhood of pixels. Though it seems similar to systolic array and wavefront computations, the STP has an explicit software controlled data movement. Each PEs features 2x 16-bit arithmetic logic units (ALUs), 1x 16-bit Multiplier–accumulator unit (MAC), 10x 16-bit registers, and 10x 1-bit predicate registers. [2]
Considering that one of the most energy costly operation is DRAM access, each STP has temporary buffers to increase data locality, namely LBP. The used LBP is a 2-D FIFO that accommodates different sizes of reading and writing. The LBP uses single-producer multi-consumer behavioral model. Each LBP can have eight logical LB memories and one for DMA input-output operations. [8] Due to the real high complexity of the memory system, the PVC designers state the LBP controller as one of the most challenging components. [2] The NoC used is a ring network on chip used to communicate with only neighbor cores for energy savings and pipelined computational pattern preservation. [2]
The STP has a 2-D array of PEs: for example, a 16x16 array of full PEs and four lanes of simplified PEs called "halo". The STP has a scalar processor, called scalar lane (SCL), that adds control instructions with a small instruction memory. The last component of an STP is a load store unit called sheet generator (SHG), where the sheet is the PVC memory access unit. [2]
The SR3HX PVC features a 64-bit ARMv8a ARM Cortex-A53 CPU, 8x image processing unit (IPU) cores, 512 MB LPDDR4, MIPI, PCIe. The IPU cores each have 512 arithmetic logic units (ALUs) consisting of 256 processing elements (PEs) arranged as a 16 x 16 2-dimensional array. Those cores execute a custom VLIW ISA. There are two 16-bit ALUs per processing element and they can operate in three distinct ways: independent, joined, and fused. [9] The SR3HX PVC is manufactured as a SiP by TSMC using their 28HPM HKMG process. [1] It was designed over 4 years in partnership with Intel. (Codename: Monette Hill) [10] Google claims the SR3HX PVC is 7-16x more energy-efficient than the Snapdragon 835. [1] And that the SR3HX PVC can perform 3 trillion operations per second, HDR+ can run 5x faster and at less than one-tenth the energy than the Snapdragon 835. [11] It supports Halide for image processing and TensorFlow for machine learning. [11] The current chip runs at 426 MHz and the single IPU is able to perform more than 1 TeraOPS. [2] [8]
In digital imaging, a pixel, pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device. In most digital display devices, pixels are the smallest element that can be manipulated through software.
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobile phones, personal computers, workstations, and game consoles.
The R420 GPU, developed by ATI Technologies, was the company's basis for its 3rd-generation DirectX 9.0/OpenGL 2.0-capable graphics cards. Used first on the Radeon X800, the R420 was produced on a 0.13 micrometer low-K photolithography process and used GDDR-3 memory. The chip was designed for AGP graphics cards.
The R520 is a graphics processing unit (GPU) developed by ATI Technologies and produced by TSMC. It was the first GPU produced using a 90 nm photolithography process.
Marc Levoy is a computer graphics researcher and Professor Emeritus of Computer Science and Electrical Engineering at Stanford University, a vice president and Fellow at Adobe Inc., and a Distinguished Engineer at Google. He is noted for pioneering work in volume rendering, light fields, and computational photography.
PWRficient is a microprocessor series by P.A. Semi where the PA6T-1682M was the only one that became an actual product.
An image processor, also known as an image processing engine, image processing unit (IPU), or image signal processor (ISP), is a type of media processor or specialized digital signal processor (DSP) used for image processing, in digital cameras or other devices. Image processors often employ parallel computing even with SIMD or MIMD technologies to increase speed and efficiency. The digital image processing engine can perform a range of tasks. To increase the system integration on embedded devices, often it is a system on a chip with multi-core processor architecture.
Google Pixel is a brand of portable consumer electronic devices developed by Google that run either ChromeOS or the Android operating system. The main line of Pixel products consist of Android-powered smartphones, which have been produced since October 2016 as the replacement of the older Nexus, and of which the Pixel 7 is the current model. The Pixel brand also includes laptop and tablet computers, as well as several accessories, and was originally introduced in February 2013 with the Chromebook Pixel.
Halide is a computer programming language designed for writing digital image processing code that takes advantage of memory locality, vectorized computation and multi-core CPUs and GPUs. Halide is implemented as an internal domain-specific language (DSL) in C++.
Heterogeneous computing refers to systems that use more than one kind of processor or core. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar coprocessors, usually incorporating specialized processing capabilities to handle particular tasks.
An eSIM (embedded-SIM) is a form of SIM card that is embedded directly into a device. Instead of an integrated circuit located on a removable universal integrated circuit card (UICC), typically made of PVC, an eSIM consists of software installed onto an eUICC chip permanently attached to a device. If the eSIM is eUICC compatible, it can be re-programmed with new SIM information. Otherwise, the eSIM is programmed with its ICCID/IMSI and other information at the time it is manufactured, and cannot be changed.
A vision processing unit (VPU) is an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.
Tensor Processing Unit (TPU) is an AI accelerator application-specific integrated circuit (ASIC) developed by Google for neural network machine learning, using Google's own TensorFlow software. Google began using TPUs internally in 2015, and in 2018 made them available for third party use, both as part of its cloud infrastructure and by offering a smaller version of the chip for sale.
DXOMARK is a commercial website described as "an independent benchmark that scientifically assesses smartphones, lenses and cameras". Founded in 2008, DxOMark was originally owned by DxO Labs, a French engineering and consulting company, which is headquartered in Boulogne-Billancourt, Paris, France. DXOMARK Image Labs was separated from DxO Labs in September 2017, and was later re-branded to DXOMARK in 2019. DXOMARK is now a wholly independent privately-owned company.
The Pixel 2 and Pixel 2 XL are a pair of Android smartphones designed, developed, and marketed by Google as part of the Google Pixel product line. They collectively serve as the successors to the Pixel and Pixel XL. They were officially announced on October 4, 2017 at the Made by Google event and released in the United States on October 19. On October 9, 2018, they were succeeded by the Pixel 3 and Pixel 3 XL.
The Nokia 9 PureView is a Nokia-branded flagship smartphone developed by HMD Global. It was introduced at the 2019 Mobile World Congress (MWC) in Barcelona, Spain as a successor to the Nokia 8 Sirocco. It alludes to Nokia's previous camera-centric PureView devices, which ended with the Lumia 950 and 950 XL. Like most of HMD's Nokia smartphones, the 9 PureView is part of the Android One program.
The Pixel 3a and Pixel 3a XL are a pair of Android smartphones designed, developed, and marketed by Google as part of the Google Pixel product line. They collectively serve as mid-range variants of the Pixel 3 and Pixel 3 XL. They were officially announced on May 7, 2019 at Google I/O, seven months after the announcement of the original Pixel 3 lineup, and were released on the same day. On August 3, 2020, it was succeeded by the Pixel 4a.
The Pixel 4 and Pixel 4 XL are a pair of Android smartphones designed, developed, and marketed by Google as part of the Google Pixel product line. They collectively serve as the successors to the Pixel 3 and Pixel 3 XL. They were officially announced on October 15, 2019 at the Made by Google event and released in the United States on October 24, 2019. On September 30, 2020, they were succeeded by the Pixel 5.
The ARM Cortex-X1 is a central processing unit implementing the ARMv8.2-A 64-bit instruction set designed by ARM Holdings' Austin design centre as part of ARM's Cortex-X Custom (CXC) program.
Google Tensor is a series of ARM64-based system-on-chip (SoC) processors designed by Google for its Pixel devices. The first-generation chip debuted on the Pixel 6 smartphone series in 2021, and were succeeded by the second-generation chip on the Pixel 7 and Pixel 7 Pro smartphone series in 2022.