Intel RealSense

Last updated

Intel RealSense
Company type Product Line
Industry Artificial intelligence, Camera, Depth perception, 3D reconstruction
Founded2015;10 years ago (2015)
Headquarters Santa Clara, California
ProductsIntel RealSense
Parent Intel
Website www.intelrealsense.com
RealSense D435 Depth Camera
Intel Realsense depth camera D435.jpg
DeveloperIntel RealSense
Manufacturer Intel
Type Cameras, Technology

Intel RealSense Technology, formerly known as Intel Perceptual Computing, is a product range of depth and tracking technologies designed to give machines and devices depth perception capabilities. The technologies, owned by Intel are used in autonomous drones, robots, AR/VR, smart home devices amongst many others broad market products.

Contents

The RealSense products are made of Vision Processors, Depth and Tracking Modules, and Depth Cameras, supported by an open source, cross-platform SDK in an attempt to simplify supporting cameras for third party software developers, system integrators, ODMs and OEMs. [1]

History

Intel began producing hardware and software that utilized depth tracking, gestures, facial recognition, eye tracking, and other technologies under the branding Perceptual Computing in 2013. [2] [3] According to Intel, much of their research into the technologies is focused around "sensory inputs that make [computers] more human like". They initially hoped to begin including 3D cameras that could support their Perceptual Computing as opposed to traditional 2D cameras by late 2014. [4]

In 2013, Intel ran a competition among seven teams to create software highlighting the capabilities of its Perceptual Computing technology entitled "Intel Ultimate Coder Challenge: Going Perceptual". [3] [5]

In 2014, Intel rebranded their Perceptual Computing line of technology as Intel RealSense. [2] [6]

Intel RealSense Group supports multiple depth and tracking technologies including Coded Light Depth, Stereo Depth and Positional Tracking. [7]

To address the lack of applications built on the RealSense platform and to promote the platform among software developers, in 2014 Intel organized the "Intel RealSense App Challenge". The winners were awarded large sums of money. [8]

Product series

Intel RealSense D400 Product Family

As of January 2018, new Intel RealSense D400 Product Family was launched with the Intel RealSense Vision Processor D4, Intel RealSense Depth Module D400 Series, and 2 ready to use depth cameras: Intel RealSense Depth Cameras D435 and D415.

Intel RealSense Vision Processor D4 Series

The Intel RealSense Vision Processor D4 series are vision processors based on 28 nanometer (nm) process technology to compute real-time stereo depth data. They utilise a depth algorithm that enables more accurate and longer range depth perception than previously available. There are two products in this family: RealSense Vision processor D4 and RealSense Vision Processor D4m.[ citation needed ]

Other products

The Intel RealSense Depth Module D400 Series is designed for easy integration to bring 3D into devices and machines. Intel also released the D415 and D435 in 2018. Both cameras feature the RealSense Vision processor D4 and camera sensors. They are supported by the cross-platform and open source Intel RealSense SDK 2.0. The Intel D415 is designed for more precise measurements.[ citation needed ]

Intel RealSense Depth Camera D435
Food delivery robot with a RealSense camera mounted under the screen Serving robots "BellaBot" (52139754346).jpg
Food delivery robot with a RealSense camera mounted under the screen

The Intel RealSense Depth Camera D435 is for capturing stereo images as well as depth (by using a Structured-light 3D scanner) for a variety of applications that need perceive the world in 3D. [9] [10] [11] [12] [13] [14] [15] [16] [17]

Previous Generations

Previous generations of Intel RealSense depth cameras (F200, R200 and SR300) were implemented in multiple laptop and tablet computers by Asus, HP, Dell, Lenovo, and Acer. [18] Additionally, Razer and Creative offered consumer ready standalone webcams with the Intel RealSense camera built into the design.: [19] Razer Stargazer and the Creative BlasterX Senz3D. [20]

Intel RealSense 3D Camera (Front F200)

This is a stand-alone camera that can be attached to a desktop or laptop computer. [21] It is intended to be used for natural gesture-based interaction, face recognition, immersive, video conferencing and collaboration, gaming and learning and 3D scanning. [22] There was also version of this camera to be embedded into laptop computers. [18]

Intel RealSense Snapshot

Snapshot is a camera system intended to be built into tablet computers and possibly smartphones. Its intended uses include taking photographs and performing after the fact refocusing, distance measurements, and applying depth photo filters. [23] The refocus feature differs from a plenoptic camera in that RealSense Snapshot takes pictures with large depth of field so that initially the whole picture is in focus and then in software it selectively blurs parts of the image depending on their distance. The Dell Venue 8 and 10 7000 Series Android tablets are equipped with this camera. [24]

Intel RealSense 3D Camera (Rear R200)

Rear-mounted camera for Microsoft Surface or a similar tablet, like the HP Spectre X2. [25] This camera is intended for augmented reality applications, content creation, and object scanning. Its depth accuracy is on the order of millimeters and its range is up to 6.0 meters. The R200 is a stereo camera and is able to obtain accurate depth outdoors as well as indoors. [26]

Reception

In an early preview article in 2015, PC World's Mark Hachman concluded that RealSense is an enabling technology that will be largely defined by the software that will take advantage of its features. He noted that as of the time the article was written, the technology was new and there was no such software. [27]

Software using this technology

Camera 3D uses Intel RealSense (Serie D400) and Microsoft Kinect sensors to create holographic memories, 3D models and Facebook 3D photos

Product Technical Specifications

Specifications: Intel RealSense Depth Camera D415, D435 and D455 [28] [29] [30] [31]

D415D435D455
Use EnvironmentIndoor/Outdoor
Depth TechnologyActive infrared (IR) stereo
IR Projector & Left/Right Camera TypeStandardWide
Shutter TypeRollingGlobal
Image Sensor ModuleOV2740 (OV02740-H34A-Z) [32] OV9782 (OV09782-GA4A) [33]
Image Sensor TechnologyPureCel HDR [32] OmniPixel3-GS [33]
Image Sensor Size1⁄6 inch, 3855 μm × 2919 μm [32] 1⁄4 inch, 3896 μm × 2453 μm [33]
Image Sensor Pixel Size1.4 μm × 1.4 μm [32] 3 μm × 3 μm [33]
Vision Processor BoardRealSense Vision Processor D4
Depth Sensor ModuleRealSense Module D415RealSense Module D430 + RGB CameraRealSense Module D450
Depth Field of View for HDH:65°±2 V:40°±1 D:72°±2H:87°±3 V:58°±1 D:95°±3
Depth Field of View for VGAH:50°±2 V:40°±1 D:61°±2H:75°±3 V:62°±1 D:89°±3
Depth Resolution and FramerateUp to 1280px × 720px @ 90fps
Minimum Depth Distance at Maximum Resolution45 cm / 17.7"28 cm / 11.0"52 cm / 20.5"
Depth Accuracy<2% at 2.0m / 2.2yd<2% at 4.0m / 4.4yd
Ideal and Maximum Range0.5m to 3m / 0.6yd to 3.3yd0.3m to 3m / 0.3yd to 3.3yd0.6m to 6m / 0.3yd to 6.6yd
RGB Resolution, Framerate and Aspect Ratio1920px × 1080px @ 30fps (16:9)1280px × 800px @ 30fps (8:5)
RGB Field of ViewH:69.4° V:42.5° D:77.0°H:91.2° V:65.5° D:100.6°
RGB Lens Distortion≤1.5%
Device Dimensions99mm × 20mm × 23mm90mm × 25mm × 25mm124mm × 26mm × 29mm
ConnectorUSB Type-C 3.1 Gen 1
Mounting MechanismsOne 1/4-20UNC thread mounting point
Two M3 thread mounting pointsTwo M4 thread mounting points

Specifications: Intel RealSense Vision Processor D4 Series [34]
(Not available separately as these are just the bare PCB Vision Processor boards, only used as basis for the RealSense Depth Camera series)

RealSense Vision Processor D4RealSense Vision Processor D4M
Depth TechnologyStereoStereo
Form FactorASIC BGAASIC BGA
Package Size6.4mm x 6.4mm x 1mm4.7mm x 3.8mm x 0.55mm
Process Technology28 nm28 nm
Depth Max Throughput36.6 MP/sec (848×480@90fps)
Depth Stream Output ResolutionUp to 1280×720Up to 720×720
Depth Stream Output Frame RateUp to 90fps
RGB Sensor Max Resolution & Max Frame Rate1920×1080, Up to 60fps720×720, Up to 30 fps
IR Projector ControlsYesYes
Host InterfaceUSB 3.02x MIPI
Multi Camera SupportYes, up to 5Up to 30fps
I/O5x MIPI CSI-2, 5x I2C, 1x SPI, GPIO, Timer2x MIPI 1x I2C, 1x SPI, GPIO, Timer

Specifications: Intel Stereo DepthModule SKUs [35]
(Not available separately as these are just the bare PCB Depth Sensor Modules, only used as basis for the RealSense Depth Camera series)

D400D410D415D420D430
Depth technologyPassive IR StereoActive IR StereoActive IR StereoPassive IR StereoActive IR Stereo
Image Sensor technologyRolling ShutterRolling ShutterRolling ShutterGlobal ShutterGlobal Shutter
Depth FOV (HxV for HD 16:9)63.4degx 40.4deg63.4degx 40.4deg63.4degx 40.4deg85.2degx 58deg85.2degx 58deg
RGB Frame Rate and Resolution--Up to 60FPS--
Depth ResolutionUp to 1280x720Up to 1280x720Up to 1280x720Up to 1280x720Up to 1280x720
Depth Frame RateUp to 90fpsUp to 90fpsUp to 90fpsUp to 90fpsUp to 90fps
Range0.16-10m+0.16-10m+0.16-10m+0.11-10m+0.11-10m+

See also

Related Research Articles

<span class="mw-page-title-main">Gesture recognition</span> Topic in computer science and language technology

Gesture recognition is an area of research and development in computer science and language technology concerned with the recognition and interpretation of human gestures. A subdiscipline of computer vision, it employs mathematical algorithms to interpret gestures.

<span class="mw-page-title-main">Stereo camera</span> Camera with two or more lenses for capturing stereo views

A stereo camera is a type of camera with two or more lenses with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, and therefore gives it the ability to capture three-dimensional images, a process known as stereo photography. Stereo cameras may be used for making stereoviews and 3D pictures for movies, or for range imaging. The distance between the lenses in a typical stereo camera is about the distance between one's eyes and is about 6.35 cm, though a longer base line produces more extreme 3-dimensionality.

The stereo cameras approach is a method of distilling a noisy video signal into a coherent data set that a computer can begin to process into actionable symbolic objects, or abstractions. Stereo cameras is one of many approaches used in the broader fields of computer vision and machine vision.

<span class="mw-page-title-main">Microsoft PixelSense</span> Interactive surface computing platform by Microsoft

Microsoft PixelSense was an interactive surface computing platform that allowed one or more people to use and touch real-world objects, and share digital content at the same time. The PixelSense platform consists of software and hardware products that combine vision based multitouch PC hardware, 360-degree multiuser application design, and Windows software to create a natural user interface (NUI).

<span class="mw-page-title-main">Image processor</span> Specialized digital signal processor used for image processing

An image processor, also known as an image processing engine, image processing unit (IPU), or image signal processor (ISP), is a type of media processor or specialized digital signal processor (DSP) used for image processing, in digital cameras or other devices. Image processors often employ parallel computing even with SIMD or MIMD technologies to increase speed and efficiency. The digital image processing engine can perform a range of tasks. To increase the system integration on embedded devices, often it is a system on a chip with multi-core processor architecture.

<span class="mw-page-title-main">2D-plus-depth</span> Stereoscopic video coding format

2D-plus-Depth is a stereoscopic video coding format that is used for 3D displays, such as Philips WOWvx. Philips discontinued work on the WOWvx line in 2009, citing "current market developments". Currently, this Philips technology is used by SeeCubic company, led by former key 3D engineers and scientists of Philips. They offer autostereoscopic 3D displays which use the 2D-plus-Depth format for 3D video input.

<span class="mw-page-title-main">Time-of-flight camera</span> Range imaging camera system

A time-of-flight camera, also known as time-of-flight sensor, is a range imaging camera system for measuring distances between the camera and the subject for each point of the image based on time-of-flight, the round trip time of an artificial light signal, as provided by a laser or an LED. Laser-based time-of-flight cameras are part of a broader class of scannerless LIDAR, in which the entire scene is captured with each laser pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems. Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes allowed the production of components fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers.

Sony Depthsensing Solutions SA/NV, formerly known as SoftKinetic Systems, is a Belgian company originating from the merger of Optrima NV, founded by André Miodezky, Maarten Kuijk, Daniël Van Nieuwenhove, Ward Van der Tempel, Riemer Grootjans and Tomas Van den Hauwe and SoftKinetic SA founded by Eric Krzeslo, Thibaud Remacle, Gilles Pinault and Xavier Baele. Sony Depthsensing Solutions develops gesture recognition hardware and software for real-time range imaging (3D) cameras. SoftKinetic was founded in July 2007 providing gesture recognition solutions based on its technology to the interactive digital entertainment, consumer electronics, health & fitness, and serious game industries. SoftKinetic technology has been applied to interactive digital signage and advergaming, interactive television, and physical therapy.

<span class="mw-page-title-main">Kinect</span> Motion-sensing input device for the Xbox 360 and Xbox One

Kinect is a discontinued line of motion sensing input devices produced by Microsoft and first released in 2010. The devices generally contain RGB cameras, and infrared projectors and detectors that map depth through either structured light or time of flight calculations, which can in turn be used to perform real-time gesture recognition and body skeletal detection, among other capabilities. They also contain microphones that can be used for speech recognition and voice control.

Rockchip is a Chinese fabless semiconductor company based in Fuzhou, Fujian province. It has offices in Shanghai, Beijing, Shenzhen, Hangzhou and Hong Kong. It designs system on a chip (SoC) products, using the ARM architecture licensed from ARM Holdings for the majority of its projects.

<span class="mw-page-title-main">PrimeSense</span> Former Israeli company

PrimeSense was an Israeli 3D sensing company based in Tel Aviv. PrimeSense had offices in Israel, North America, Japan, Singapore, Korea, China and Taiwan. PrimeSense was bought by Apple Inc. for $360 million on November 24, 2013.

<span class="mw-page-title-main">IdeaCentre</span> Series of personal computers

The Lenovo IdeaCentre is a line of consumer-oriented desktop computers designed, developed and marketed by Lenovo. The first IdeaCentre desktop, the IdeaCentre K210, was announced by Lenovo on June 30, 2008 as a consumer-focussed alternative to the business-oriented ThinkCentre and ThinkStation brandnames acquired from IBM. While the IdeaCentre line consists entirely of desktops, they share a common design language with the IdeaPad line of laptops and hybrids. One such feature is Veriface facial recognition technology.

<span class="mw-page-title-main">Omek Interactive</span> Former technology company

Omek Interactive was a venture-backed technology company developing advanced motion-sensing software for human-computer interaction in automobiles, casinos and arcades, consumer electronics, video games, health care and digital signage. Omek was co-founded in 2006 by Janine Kutliroff and Gershom Kutliroff.

<span class="mw-page-title-main">Imageon</span> Defunct series of media coprocessors and chipsets

Imageon was a series of media coprocessors and mobile chipsets produced by ATI in 2002–2008, providing graphics acceleration and other multimedia features for handheld devices such as mobile phones and Personal Digital Assistants (PDAs). AMD later sold the Imageon mobile handheld graphics division to Qualcomm in 2009, where it was used exclusively inside their Snapdragon SoC processors under the Adreno brand name.

<span class="mw-page-title-main">Tango (platform)</span> Mobile computer vision platform for Android developed by Google

Tango was an augmented reality computing platform, developed and authored by the Advanced Technology and Projects (ATAP), a skunkworks division of Google. It used computer vision to enable mobile devices, such as smartphones and tablets, to detect their position relative to the world around them without using GPS or other external signals. This allowed application developers to create user experiences that include indoor navigation, 3D mapping, physical space measurement, environmental recognition, augmented reality, and windows into a virtual world.

Movidius Ltd. was a company based in San Mateo, California, that designed low-power processor chips for computer vision. The company was acquired by Intel in September 2016, who continues to sell the company's products under the Movidius line.

OpenXR is an open-source, royalty-free standard for access to virtual reality and augmented reality platforms and devices. It is developed by a working group managed by the Khronos Group consortium. OpenXR was announced by the Khronos Group on February 27, 2017, during GDC 2017. A provisional version of the standard was released on March 18, 2019, to enable developers and implementers to provide feedback on it. On July 29, 2019, OpenXR 1.0 was released to the public by Khronos Group at SIGGRAPH 2019 and on April 15, 2024, OpenXR 1.1 was released by Khronos.

The Sony Xperia XZ2 Premium is an Android smartphone manufactured and marketed by Sony. Part of the Xperia X series, the device was announced to the public on April 16, 2018, featuring a 4K HDR display and a MotionEye™ Dual Camera.

Gérard G. Medioni is a computer scientist, author, academic and inventor. He is a vice president and distinguished scientist at Amazon and serves as emeritus professor of Computer Science at the University of Southern California.

References

  1. ↑ "Intel RealSense". Intel RealSense. Retrieved April 14, 2018.
  2. 1 2 "Intel And Qualcomm Battle To Bring Computer Vision Into Phones And Drones". Forbes. August 10, 2023. Archived from the original on August 10, 2023. Retrieved August 10, 2023.
  3. 1 2 Owano, Nancy; Phys.org (January 14, 2013). "Intel's Perceptual Computing marks neo-desktop era". phys.org. Archived from the original on January 15, 2013. Retrieved August 10, 2023.
  4. ↑ "Inside Intel's perceptual computing lab (pictures)". CNET. November 29, 2013. Archived from the original on August 10, 2023. Retrieved August 10, 2023.
  5. ↑ "The Ultimate Coder Has Been Named - Final Results of Ultimate Coder Challenge: Going Perceptual". Intel® Developer Zone. July 31, 2013. Archived from the original on July 31, 2013. Retrieved August 10, 2023.
  6. ↑ tony2 (October 8, 2014). "Old Perceptual Computing SDK download". Intel Community Developer Software Forums. Archived from the original on August 10, 2023. Retrieved August 10, 2023.{{cite web}}: CS1 maint: numeric names: authors list (link)
  7. ↑ "Intel RealSense". Intel RealSense. Retrieved April 14, 2018.
  8. ↑ Healey, Nic (June 3, 2014). "Intel issues $1m challenge for devs to make RealSense apps". CNET. Archived from the original on October 6, 2022. Retrieved August 10, 2023.
  9. ↑ "$399 Intel Euclid Robotics Devkit Runs Ubuntu & ROS on Intel Atom x7-Z8700 Processor - CNX Software". May 22, 2017.
  10. ↑ "EUCLID1 Intel Euclid Developer Kit RF Exposure Info SA170103W003_CCS-G003_FCC SAR_Report_WLAN&BT Intel". FCC ID. Retrieved April 23, 2024.
  11. ↑ "Intel Announces Tools for RealSense Technology Development" . Retrieved April 23, 2024.
  12. ↑ Archived at Ghostarchive and the Mail incoming, trying out and tearing down the Intel Euclid @scanlime-in-progress. YouTube . Archived from the original on June 24, 2017. Retrieved December 11, 2021.{{cite AV media}}: CS1 maint: bot: original URL status unknown (link): Mail incoming, trying out and tearing down the Intel Euclid @scanlime-in-progress. YouTube .
  13. ↑ "Intel | Data Center Solutions, IoT, and PC Innovation".
  14. ↑ "Home". euclidcommunity.intel.com. Archived from the original on May 23, 2018. Retrieved May 23, 2018.
  15. ↑ "IntelEuclid - ROS Wiki".
  16. ↑ "Intel Euclid". GitHub .
  17. ↑ "Support for Intel Euclid Development Kit".
  18. 1 2 "Devices with Intel RealSense Technology". Intel.
  19. ↑ "At a Glance: Razer Stargazer | RZ20-01800". mysupport.razer.com. Retrieved October 18, 2022.
  20. ↑ "Razer and Creative want you to wave at your webcam". September 15, 2016.
  21. ↑ "Take Interaction to the Next Level – Intel RealSense Camera F200". Intel.
  22. ↑ "Developing for the Intel RealSense Camera (F200)". Intel.
  23. ↑ "Developing for the Intel RealSense Snapshot". Intel. Archived from the original on May 14, 2015.
  24. ↑ "Intel RealSense Snapshot". Intel. Archived from the original on May 3, 2015.
  25. ↑ "HP Spectre X2". HP.
  26. ↑ Keselman, Leonid; Woodfill, John Iselin; Grunnet-Jepsen, Anders; Bhowmik, Achintya (May 16, 2017). "Intel RealSense Stereoscopic Depth Cameras". arXiv: 1705.05548 [cs.CV].
  27. ↑ "Hands on: Without apps, Intel's RealSense camera is a puzzle". PC World. March 5, 2015. Retrieved March 25, 2015.
  28. ↑ "RealSense Depth Camera D415 Technical Specifications". Intel Store. Retrieved May 20, 2021.
  29. ↑ "RealSense Depth Camera D435 Technical Specifications". Intel Store. Retrieved May 20, 2021.
  30. ↑ "RealSense Depth Camera D455 Technical Specifications". Intel Store. Retrieved May 20, 2021.
  31. ↑ "Intel RealSense Product Family D400 Series Datasheet" (Datasheet). Intel RealSense. pp. 36, 37, 52. Retrieved May 20, 2021.
  32. 1 2 3 4 "OV2740 Color CMOS 1080p (1920x1080) HD PureCel Image Sensor". OmniVision. Retrieved May 20, 2021.
  33. 1 2 3 4 "OV9782 Color CMOS 1-Megapixel (1280x800) Image Sensor with OmniPixel3-GS Technology" . Retrieved May 20, 2021.
  34. ↑ "Intel RealSense Product Family D400 Series Datasheet" (Datasheet). Intel RealSense. Table 3-11. p. 36. Retrieved May 20, 2021.{{cite web}}: CS1 maint: location (link)
  35. ↑ "Intel RealSense Product Family D400 Series Datasheet" (Datasheet). Intel RealSense. Table 3-42. p. 55. Retrieved May 20, 2021.{{cite web}}: CS1 maint: location (link)
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.