Felix Heide

Last updated
Felix Heide
Alma mater University of British Columbia (PhD)
Stanford University (Postdoc)
Known for Computational imaging
AwardsAlain Fournier Ph.D. Dissertation Award (2016)
ACM SIGGRAPH Outstanding Doctoral Dissertation Award (2017)
AutoSens Young Engineer of the Year (2020)
ACM SIGGRAPH Significant New Researcher Award (2023)
Scientific career
Institutions Stanford University
Algolux
Princeton University
Thesis Structure-aware Computational Imaging  (2016)
Doctoral advisor Wolfgang Heidrich
Website www.cs.princeton.edu/~fheide/

Felix Heide is a German-born computer scientist known for his work in the fields of computational imaging, computer vision, computer graphics and deep learning. He is an assistant professor at Princeton University and was the head of the Computational Imaging Lab. He serves as Head of Artificial Intelligence at Torc Robotics. Heide co-founded Algolux, a startup in computer vision technology for self-driving vehicles, which later merged with Torc Robotics. [1]

Contents

Education

Heide completed his undergraduate and Master's degree in computer science at the University of Siegen, where he graduated summa cum laude. [2]

In 2016, he received his PhD from the University of British Columbia under the advisement of Professor Wolfgang Heidrich. His doctoral dissertation won the Alain Fournier PhD Dissertation Award for the best Canadian PhD dissertation in computer graphics and the ACM SIGGRAPH outstanding doctoral dissertation award for the best PhD dissertation in computer graphics and interactive techniques. Following his doctoral studies, he conducted postdoctoral research at Stanford University, with visiting research stints at institutions including the Massachusetts Institute of Technology (MIT) and King Abdullah University of Science and Technology (KAUST). [3]

Career

Heide is an assistant professor in the Computer Science Department at Princeton University, where he researches computational imaging, computer vision, and optics. His work explores end-to-end design of imaging, vision, and display systems, particularly those used in autonomous vehicle applications. [4]

In 2015, Heide co-founded Algolux. Under his leadership, Algolux grew to over 100+ employees, raised venture capital, and shipped computer vision software that has been used worldwide by automotive manufacturers. Algolux later became part of Torc Robotics. [5]

Heide has served as a technical advisory board member for Samsung Research and has held research and advisory roles with Nvidia, KAUST, and other institutions. [6]

Research

A central theme of Heide’s research is the integration of machine learning and optimization techniques with physical imaging systems. [7] He explores how algorithms can be co-designed with optics and hardware to improve the quality, robustness, and efficiency of imaging under challenging conditions. [8] This approach has resulted in novel methods for vision in adverse weather, low light, and occluded environments, which are critical for autonomous driving applications.

Computational imaging

Heide has pioneered methods that enhance image quality through innovative optics combined with computational processing. [9] His research in this area includes developing neural nano-optics for compact, high-performance imaging systems [10] , which hinge on differentiable camera models that allow for the joint optimization of optics and image processing algorithms. [11] [12] Neural nano-optics combine nanometer-scale metasurface optics with neural network-based image reconstruction to create high-quality imaging systems with wide field of view, chromatic aberration correction, and applications in fields like robotics and medicine. [13]

Heide’s pioneering work on transient imaging and non-line-of-sight (NLOS) imaging has focused on advancing methods for capturing and reconstructing images of objects that are not directly visible to the camera. [14]

End-to-end camera and vision systems

Heide's work in end-to-end design involves optimizing the entire imaging pipeline, from light capture to image reconstruction. This approach has applications in autonomous driving, where cameras need to perceive, act, and reconstruct complex environments. [15] [16] His research also explores novel display technologies, such as holography [17] , which leverage computational techniques for high-fidelity visual output. [18]

Machine learning for autonomous vehicle perception

In autonomous driving, perception is critical for safety and navigation. Heide's work has contributed significantly to developing perception systems that perform reliably in challenging conditions such as fog, rain, and low lighting. [19] These methods have been integrated into vision software used by companies like Waymo, Cruise, and Google. [20]

Awards and recognitions

Publications and patents

Heide has co-authored close to 50 publications and has received over 9,200 citations. [21] In 2020, six of his papers were accepted to the Conference on Computer Vision and Pattern Recognition – including three orals. One of them, “Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar” was picked up by numerous publications. [22]

Additionally, he has filed for 16 patents, and 7 of them have been granted as of yet. The last one, “Method and apparatus for joint image processing and perception” was granted in July 2020. [23]

Related Research Articles

Computer vision tasks include methods for acquiring, processing, analyzing, and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the form of decisions. "Understanding" in this context signifies the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Jürgen Schmidhuber</span> German computer scientist

Jürgen Schmidhuber is a German computer scientist noted for his work in the field of artificial intelligence, specifically artificial neural networks. He is a scientific director of the Dalle Molle Institute for Artificial Intelligence Research in Switzerland. He is also director of the Artificial Intelligence Initiative and professor of the Computer Science program in the Computer, Electrical, and Mathematical Sciences and Engineering (CEMSE) division at the King Abdullah University of Science and Technology (KAUST) in Saudi Arabia.

Neuromorphic computing is an approach to computing that is inspired by the structure and function of the human brain. A neuromorphic computer/chip is any device that uses physical artificial neurons to do computations. In recent times, the term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems. Recent advances have even discovered ways to mimic the human nervous system through liquid solutions of chemical systems.

A light field, or lightfield, is a vector function that describes the amount of light flowing in every direction through every point in a space. The space of all possible light rays is given by the five-dimensional plenoptic function, and the magnitude of each ray is given by its radiance. Michael Faraday was the first to propose that light should be interpreted as a field, much like the magnetic fields on which he had been working. The term light field was coined by Andrey Gershun in a classic 1936 paper on the radiometric properties of light in three-dimensional space.

The expression computational intelligence (CI) usually refers to the ability of a computer to learn a specific task from data or experimental observation. Even though it is commonly considered a synonym of soft computing, there is still no commonly accepted definition of computational intelligence.

<span class="mw-page-title-main">Daniela Rus</span> American computer scientist

Daniela L. Rus is a Romanian-American computer scientist. She serves as director of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), and the Andrew and Erna Viterbi Professor in the Department of Electrical Engineering and Computer Science (EECS) at the Massachusetts Institute of Technology. She is the author of the books Computing the Future, The Heart and the Chip: Our Bright Future with Robots, and The Mind's Mirror: Risk and Reward in the Age of AI.

Heung-Yeung "Harry" Shum is a Chinese computer scientist. He was a doctoral student of Raj Reddy. He was the Executive Vice President of Artificial Intelligence & Research at Microsoft. He is known for his research on computer vision and computer graphics, and for the development of the search engine Bing.

<span class="mw-page-title-main">University of Utah College of Engineering</span> John and Marcia Price College of Engineering in Utah, U.S.

The John and Marcia Price College of Engineering at the University of Utah is an academic college of the University of Utah in Salt Lake City, Utah. The college offers undergraduate and graduate degrees in engineering and computer science.

<span class="mw-page-title-main">Yann LeCun</span> French computer scientist (born 1960)

Yann André LeCun is a French-American computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice President, Chief AI Scientist at Meta.

<span class="mw-page-title-main">Joseph Sgro</span> American mathematician

Joseph A. Sgro is an American mathematician, neurologist / neurophysiologist, and an engineering technologist / entrepreneur in the field of frame grabbers, high-speed cameras, smart cameras, image processors, computer vision, and machine vision and learning technologies.

<span class="mw-page-title-main">Femto-photography</span> Technique for recording the propagation of ultrashort light pulses

Femto-photography is a technique for recording the propagation of ultrashort pulses of light through a scene at a very high speed (up to 1013 frames per second). A femto-photograph is equivalent to an optical impulse response of a scene and has also been denoted by terms such as a light-in-flight recording or transient image. Femto-photography of macroscopic objects was first demonstrated using a holographic process in the 1970s by Nils Abramsson at the Royal Institute of Technology (Sweden). A research team at the MIT Media Lab led by Ramesh Raskar, together with contributors from the Graphics and Imaging Lab at the Universidad de Zaragoza, Spain, more recently achieved a significant increase in image quality using a streak camera synchronized to a pulsed laser and modified to obtain 2D images instead of just a single scanline.

William Eric Leifur Grimson is a Canadian-born computer scientist and professor at the Massachusetts Institute of Technology, where he served as Chancellor from 2011 to 2014. An expert in computer vision, he headed MIT's Department of Electrical Engineering and Computer Science from 2005 to 2011 and currently serves as its Chancellor for Academic Advancement.

<span class="mw-page-title-main">John Tsotsos</span> Canadian Computer Scientist (born 1952)

John Tsotsos is a Canadian Computer Scientist whose research spans the fields of Computer Vision, Human Vision, Robotics, and Artificial Intelligence. He is best known for his work in visual attention, specifically for establishing the need for visual attention in both biological and computational systems through an argument based on the computational complexity of visual information processing and subsequently developing a computational framework for visual attention known as the Selective Tuning model. He is also acknowledged as a pioneer in the area of Active Vision, his students and he being first to propose strategies for active object recognition and object visual search by a robot. He has made many contributions to machine vision (particularly motion interpretation, colour processing, binocular vision, active robotic head design, shape analysis as well as to human vision, robotics and applied areas such as cardiology or dentistry and assistive robotics.

The Rice University Department of Electrical and Computer Engineering is one of nine academic departments at the George R. Brown School of Engineering at Rice University. Ashutosh Sabharwal is the Department Chair. Originally the Rice Department of Electrical Engineering, it was renamed in 1984 to Electrical and Computer Engineering.

<span class="mw-page-title-main">Michael F. Cohen</span> American computer scientist

Michael F. Cohen is an American computer scientist and researcher in computer graphics. He is currently a Senior Fellow at Meta in their Generative AI Group. He was a senior research scientist at Microsoft Research for 21 years until he joined Facebook in 2015. In 1998, he received the ACM SIGGRAPH CG Achievement Award for his work in developing radiosity methods for realistic image synthesis. He was elected a Fellow of the Association for Computing Machinery in 2007 for his "contributions to computer graphics and computer vision." In 2019, he received the ACM SIGGRAPH Steven A. Coons Award for Outstanding Creative Contributions to Computer Graphics for “his groundbreaking work in numerous areas of research—radiosity, motion simulation & editing, light field rendering, matting & compositing, and computational photography”.

<span class="mw-page-title-main">Gregory D. Hager</span> American computer scientist

Gregory D. Hager is the Mandell Bellmore Professor of Computer Science and founding director of the Johns Hopkins Malone Center for Engineering in Healthcare at Johns Hopkins University.

<span class="mw-page-title-main">Chan-Jin Chung</span> South Korean computer scientist (born 1959)

Chan-Jin Chung, commonly known as CJ Chung, is a full professor of computer science at Lawrence Technological University (LTU) in Michigan, USA. He founded an international autonomous robotics competition called Robofest in the 1999–2000 academic year as well as numerous educational programs for youth by integrating STEM, arts, autonomous robotics, and computer science. He also served as the founding USA National Organizer of World Robot Olympiad (WRO) in 2014 and 2015. He also started the WISER conference in 2014. He is working on developing a computer science curriculum for connected and autonomous vehicles (CAV) with a support from National Science Foundation . His research areas include evolutionary computation, cultural algorithms, intelligent systems & autonomous mobile robotics, software engineering, machine learning & deep learning, computer science education, and educational robotics.

Ralph Etienne-Cummings is an academic in the field of electrical engineering. He is a professor of Electrical & Computer Engineering at Johns Hopkins University.

Vladlen Koltun is an Israeli-American computer scientist and intelligent systems researcher. He currently serves as distinguished scientist at Apple Inc. His main areas of research are artificial intelligence, computer vision, machine learning, and pattern recognition. He also made a significant contribution to robotics and autonomous driving.

Wolfgang Heidrich is a German-Canadian computer scientist and Professor at the King Abdullah University of Science and Technology (KAUST), for which he served as the director of Visual Computing Center from 2014 to 2021. He was previously a professor at the University of British Columbia (UBC), where he was a Dolby Research Chair (2008-2013). His research has combined methods from computer graphics, optics, machine vision, imaging, inverse methods, and perception to develop new Computational Imaging and Display technologies. His more recent interest focuses on hardware-software co-design of the next generation of imaging systems, with applications such as high dynamic range (HDR) imaging, compact computational cameras, hyper-spectral cameras, wavefront sensors, to name just a few.

References

  1. ↑ "Felix Heide, Princeton University". Princeton University.
  2. ↑ "Computer Science Graduate Student Felix Heide Receives 2016 Alain Fournier Award". University of British Columbia.
  3. ↑ "Computer Science Graduate Student Felix Heide Receives 2017 ACM SIGGRAPH Outstanding Doctoral Dissertation Award". University of British Columbia.
  4. ↑ "Heide and Majumdar awarded 2023 Sloan Fellowships". Princeton University School of Engineering and Applied Science . 15 February 2023.
  5. ↑ "Felix Heide, Algolux". Algolux.
  6. ↑ "Researchers Create a Camera the Size of a Salt Grain Using Neural Nano-Optics". Nvidia . 9 December 2021.
  7. ↑ Tseng, Ethan; Kuo, Grace; Baek, Seung-Hwan; Matsuda, Nathan; Maimone, Andrew; Schiffers, Florian; Chakravarthula, Praneeth; Fu, Qiang; Heidrich, Wolfgang; Lanman, Douglas; Heide, Felix (22 April 2024). "Neural étendue expander for ultra-wide-angle high-fidelity holographic display". Nature (journal) . p. 2907. doi:10.1038/s41467-024-46915-3.
  8. ↑ Wilson, Andy (30 October 2013). "IMAGING SOFTWARE: Algorithm compensates for simple lens aberrations". Vision Systems Design.
  9. ↑ "Scientists develop a camera that's 'the size of a grain of salt'!". BBC Newsround . 1 December 2021.
  10. ↑ "Thin On-Sensor Nanophotonic Array Cameras – Princeton Computing Imaging Lab". Princeton University .
  11. ↑ "Neural Nano Optics". IEEE .
  12. ↑ "Non-line-of-sight Imaging with Partial Occluders and Surface Normals". gfx.cs.princeton.edu.
  13. ↑ "Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors". ar5iv.
  14. ↑ "Transient Convolutional Imaging". gfx.cs.princeton.edu.
  15. ↑ "CSDL - IEEE Computer Society". www.computer.org.
  16. ↑ "Researchers Made a Camera That's the Size of a Grain of Salt". Vice (magazine) . 7 December 2021.
  17. ↑ "Learned Hardware-in-the-loop Phase Retrieval for Holographic Near-Eye Displays – Princeton Computing Imaging Lab". Princeton University .
  18. ↑ Frishberg, Hannah (8 December 2021). "Microscopic camera created that's as tiny as a grain of salt". New York Post .
  19. ↑ "Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather – Princeton Computing Imaging Lab". Princeton University .
  20. ↑ "Neural Nano-Optics for High-quality Thin Lens Imaging – Princeton Computing Imaging Lab". Princeton University .
  21. ↑ "Felix Heide, Google Scholar". Google Scholar.
  22. ↑ "Princeton Researchers Use AI To Create Radar That Sees Around Corners". Clean Technica. 30 June 2020.
  23. ↑ "Patents by Inventor Felix Heide". Justia.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.