Non-photorealistic rendering

Last updated
A normal shader (left) and an NPR shader using cel-shading (right) Toon Shader.jpg
A normal shader (left) and an NPR shader using cel-shading (right)

Non-photorealistic rendering (NPR) is an area of computer graphics that focuses on enabling a wide variety of expressive styles for digital art, in contrast to traditional computer graphics, which focuses on photorealism. NPR is inspired by other artistic modes such as painting, drawing, technical illustration, and animated cartoons. NPR has appeared in movies and video games in the form of cel-shaded animation (also known as "toon" shading) as well as in scientific visualization, architectural illustration and experimental animation.[ citation needed ]

Contents

History and criticism of the term

The term non-photorealistic rendering is believed to have been coined by the SIGGRAPH 1990 papers committee, who held a session entitled "Non Photo Realistic Rendering". [1] [2]

The term has received some criticism:

The first conference on non-photorealistic animation and rendering[ when? ] included a discussion of possible alternative names. Among those suggested were "expressive graphics", "artistic rendering", "non-realistic graphics", "art-based rendering", and "psychographics". All of these terms have been used in various research papers on the topic, but the "non-photorealistic" term seems to have nonetheless taken hold.

The first technical meeting dedicated to NPR was the ACM-sponsored Symposium on Non-Photorealistic Rendering and Animation (NPAR) in 2000. NPAR is traditionally co-located with the Annecy Animated Film Festival, [3] running on even numbered years. From 2007 onward, NPAR began to also run on odd-numbered years, co-located with ACM SIGGRAPH. [4]

3D

An example of NPR used for technical illustrations Planar core assembly exploded.png
An example of NPR used for technical illustrations

Three-dimensional NPR is the style that is most commonly seen in video games and movies. The output from this technique is almost always a 3D model that has been modified from the original input model to portray a new artistic style. In many cases, the geometry of the model is identical to the original geometry, and only the material applied to the surface is modified. With increased availability of programmable GPU's, shaders have allowed NPR effects to be applied to the rasterised image that is to be displayed to the screen. [5] The majority of NPR techniques applied to 3D geometry are intended to make the scene appear two-dimensional.

NPR techniques for 3D images include cel shading and Gooch shading.

Many methods can be used to draw stylized outlines and strokes from 3D models, including occluding contours and Suggestive contours. [6]

For enhanced legibility, the most useful technical illustrations for technical communication are not necessarily photorealistic. Non-photorealistic renderings, such as exploded view diagrams, greatly assist in showing placement of parts in a complex system.

2D

A non-photorealistic rendering of an existing 2D (photographic) image Cradle Mountain Behind Dove Lake painted.jpg
A non-photorealistic rendering of an existing 2D (photographic) image
Original here Cradle Mountain Behind Dove Lake.jpg
Original here

The input to a two dimensional NPR system is typically an image or video. The output is a typically an artistic rendering of that input imagery (for example in a watercolor, painterly or sketched style) although some 2D NPR serves non-artistic purposes e.g. data visualization.

The artistic rendering of images and video (often referred to as image stylization [7] ) traditionally focused upon heuristic algorithms that seek to simulate the placement of brush strokes on a digital canvas. [8]

Arguably, the earliest example of 2D NPR is Paul Haeberli's 'Paint by Numbers' at SIGGRAPH 1990. This (and similar interactive techniques) provide the user with a canvas that they can "paint" on using the cursor — as the user paints, a stylized version of the image is revealed on the canvas. This is especially useful for people who want to simulate different sizes of brush strokes according to different areas of the image.

Subsequently, basic image processing operations using gradient operators [9] or statistical moments [10] were used to automate this process and minimize user interaction in the late nineties (although artistic control remains with the user via setting parameters of the algorithms). This automation enabled practical application of 2D NPR to video, for the first time in the living paintings of the movie What Dreams May Come (1998).

More sophisticated image abstractions techniques were developed in the early 2000s harnessing computer vision operators e.g. image salience, [11] or segmentation [12] operators to drive stroke placement. Around this time, machine learning began to influence image stylization algorithms notably image analogy [13] that could learn to mimic the style of an existing artwork.

The advent of deep learning has re-kindled activity in image stylization, notably with neural style transfer (NST) algorithms that can mimic a wide gamut of artistic styles from single visual examples. These algorithms underpin mobile apps capable of the same e.g. Prisma

In addition to the above stylization methods, a related class of techniques in 2D NPR address the simulation of artistic media. These methods include simulating the diffusion of ink through different kinds of paper, and also of pigments through water for simulation of watercolor.

Artistic rendering

Artistic rendering is the application of visual art styles to rendering. For photorealistic rendering styles, the emphasis is on accurate reproduction of light-and-shadow and the surface properties of the depicted objects, composition, or other more generic qualities. When the emphasis is on unique interpretive rendering styles, visual information is interpreted by the artist and displayed accordingly using the chosen art medium and level of abstraction in abstract art. In computer graphics, interpretive rendering styles are known as non-photorealistic rendering styles, but may be used to simplify technical illustrations. Rendering styles that combine photorealism with non-photorealism are known as hyperrealistic rendering styles.

Notable films and games

This section lists some seminal uses of NPR techniques in films, games and software. See cel-shaded animation for a list of uses of toon-shading in games and movies.

Short films
Technological Threat 1988Early use of toon shading together with Tex Avery-style cartoon characters
Gas Planet1992Pencil-sketching 3D rendering by Eric Darnell
Fishing2000Watercolor-style 3D rendering David Gainey
RoadHead
Snack and Drink
1998
1999
Short films created with Rotoshop by Bob Sabiston
Ryan 2004Nonlinear projection and other distortions of 3D geometry
The Girl Who Cried Flowers 2008Watercolor-style rendering by Auryn
Feature films
What Dreams May Come 1998Painterly rendering in the "painted world" sequence
Tarzan 1999First use of Disney's "Deep Canvas" system
Waking Life 2001First use of rotoshop in a feature film
A Scanner Darkly 2006"a 15-month animation process"
Video games and other software
Jet Set Radio 2000Early use of toon-shading in video games
SketchUp 2000Sketch-like modelling software with toon rendering
The Legend of Zelda: The Wind Waker 2002One of the most well-known cel-shaded games
Valkyria Chronicles 2008Uses a number of NPR techniques in the game, including a sketch-like shading method
XIII 2003A game made as "comic"-like as possible
Ōkami 2006A game whose visuals emulate the style of sumi-e (Japanese ink wash painting)
Guilty Gear Xrd 2014Fighting game using cel-shaded 3D characters with limited animation to imitate the look of 2D sprites
Vue Xstream 20153D environment creation software featuring an NPR renderer with various traditional art style emulating presets
Return of the Obra Dinn 2018A 3D game rendered in a unique monochrome, pointillist style
Manifold Garden 2019A 3D puzzle game using impossible geometry, notable for its novel edge-shading techniques. [14]

Related Research Articles

<span class="mw-page-title-main">Rendering (computer graphics)</span> Process of generating an image from a model

Rendering or image synthesis is the process of generating a photorealistic or non-photorealistic image from a 2D or 3D model by means of a computer program. The resulting image is referred to as the render. Multiple models can be defined in a scene file containing objects in a strictly defined language or data structure. The scene file contains geometry, viewpoint, texture, lighting, and shading information describing the virtual scene. The data contained in the scene file is then passed to a rendering program to be processed and output to a digital image or raster graphics image file. The term "rendering" is analogous to the concept of an artist's impression of a scene. The term "rendering" is also used to describe the process of calculating effects in a video editing program to produce the final video output.

Metropolis light transport (MLT) is a global illumination application of a variant of the Monte Carlo method called the Metropolis–Hastings algorithm to the rendering equation for generating images from detailed physical descriptions of three-dimensional scenes.

<span class="mw-page-title-main">Particle system</span> Technique in game physics, motion graphics and computer graphics

A particle system is a technique in game physics, motion graphics, and computer graphics that uses many minute sprites, 3D models, or other graphic objects to simulate certain kinds of "fuzzy" phenomena, which are otherwise very hard to reproduce with conventional rendering techniques – usually highly chaotic systems, natural phenomena, or processes caused by chemical reactions.

<span class="mw-page-title-main">Boids</span> Artificial life program

Boids is an artificial life program, developed by Craig Reynolds in 1986, which simulates the flocking behaviour of birds. His paper on this topic was published in 1987 in the proceedings of the ACM SIGGRAPH conference. The name "boid" corresponds to a shortened version of "bird-oid object", which refers to a bird-like object.

<span class="mw-page-title-main">Volume rendering</span> Representing a 3D-modeled object or dataset as a 2D projection

In scientific visualization and computer graphics, volume rendering is a set of techniques used to display a 2D projection of a 3D discretely sampled data set, typically a 3D scalar field.

<span class="mw-page-title-main">Catmull–Clark subdivision surface</span> Technique in 3D computer graphics

The Catmull–Clark algorithm is a technique used in 3D computer graphics to create curved surfaces by using subdivision surface modeling. It was devised by Edwin Catmull and Jim Clark in 1978 as a generalization of bi-cubic uniform B-spline surfaces to arbitrary topology.

<span class="mw-page-title-main">Rendering equation</span> Integral equation

In computer graphics, the rendering equation is an integral equation in which the equilibrium radiance leaving a point is given as the sum of emitted plus reflected radiance under a geometric optics approximation. It was simultaneously introduced into computer graphics by David Immel et al. and James Kajiya in 1986. The various realistic rendering techniques in computer graphics attempt to solve this equation.

<span class="mw-page-title-main">3D rendering</span> Process of converting 3D scenes into 2D images

3D rendering is the 3D computer graphics process of converting 3D models into 2D images on a computer. 3D renders may include photorealistic effects or non-photorealistic styles.

<span class="mw-page-title-main">Bidirectional scattering distribution function</span>

The definition of the BSDF is not well standardized. The term was probably introduced in 1980 by Bartell, Dereniak, and Wolfe. Most often it is used to name the general mathematical function which describes the way in which the light is scattered by a surface. However, in practice, this phenomenon is usually split into the reflected and transmitted components, which are then treated separately as BRDF and BTDF.

<span class="mw-page-title-main">Pat Hanrahan</span> American computer graphics researcher

Patrick M. Hanrahan is an American computer graphics researcher, the Canon USA Professor of Computer Science and Electrical Engineering in the Computer Graphics Laboratory at Stanford University. His research focuses on rendering algorithms, graphics processing units, as well as scientific illustration and visualization. He has received numerous awards, including the 2019 Turing Award.

<span class="mw-page-title-main">Fluid animation</span> Computer graphics techniques for generating realistic animations of fluids

Fluid animation refers to computer graphics techniques for generating realistic animations of fluids such as water and smoke. Fluid animations are typically focused on emulating the qualitative visual behavior of a fluid, with less emphasis placed on rigorously correct physical results, although they often still rely on approximate solutions to the Euler equations or Navier–Stokes equations that govern real fluid physics. Fluid animation can be performed with different levels of complexity, ranging from time-consuming, high-quality animations for films, or visual effects, to simple and fast animations for real-time animations like computer games.

<span class="mw-page-title-main">Computer graphics (computer science)</span> Sub-field of computer science

Computer graphics is a sub-field of computer science which studies methods for digitally synthesizing and manipulating visual content. Although the term often refers to the study of three-dimensional computer graphics, it also encompasses two-dimensional graphics and image processing.

<span class="mw-page-title-main">Andrew Witkin</span> American computer scientist (1952–2010)

Andrew Paul Witkin was an American computer scientist who made major contributions in computer vision and computer graphics.

<span class="mw-page-title-main">Line integral convolution</span> Method for visualizing vector fields

In scientific visualization, line integral convolution (LIC) is a method to visualize a vector field, such as fluid motion.

In computer graphics, free-form deformation (FFD) is a geometric technique used to model simple deformations of rigid objects. It is based on the idea of enclosing an object within a cube or another hull object, and transforming the object within the hull as the hull is deformed. Deformation of the hull is based on the concept of so-called hyper-patches, which are three-dimensional analogs of parametric curves such as Bézier curves, B-splines, or NURBs. The technique was first described by Thomas W. Sederberg and Scott R. Parry in 1986, and is based on an earlier technique by Alan Barr. It was extended by Coquillart to a technique described as extended free-form deformation, which refines the hull object by introducing additional geometry or by using different hull objects such as cylinders and prisms.

Maureen C. Stone is an American computer scientist, specializing in color modeling.

<span class="mw-page-title-main">Nadia Magnenat Thalmann</span> Computer scientist

Nadia Magnenat Thalmann is a computer graphics scientist and robotician and is the founder and head of MIRALab at the University of Geneva. She has chaired the Institute for Media Innovation at Nanyang Technological University (NTU), Singapore from 2009 to 2021.

Gradient domain image processing, also called Poisson image editing, is a type of digital image processing that operates on the differences between neighboring pixels, rather than on the pixel values directly. Mathematically, an image gradient represents the derivative of an image, so the goal of gradient domain processing is to construct a new image by integrating the gradient, which requires solving Poisson's equation.

<span class="mw-page-title-main">Gooch shading</span> Non-photorealistic rendering technique

Gooch shading is a non-photorealistic rendering technique for shading objects. It is also known as "cool to warm" shading, and is widely used in technical illustration.

Michael F. Cohen is an American computer scientist and researcher in computer graphics. He was a senior research scientist at Microsoft Research for 21 years until he joined Facebook Research in 2015. In 1998, he received the ACM SIGGRAPH CG Achievement Award for his work in developing radiosity methods for realistic image synthesis. He was elected a Fellow of the Association for Computing Machinery in 2007 for his "contributions to computer graphics and computer vision." In 2019, he received the ACM SIGGRAPH Steven A. Coons Award for Outstanding Creative Contributions to Computer Graphics for “his groundbreaking work in numerous areas of research—radiosity, motion simulation & editing, light field rendering, matting & compositing, and computational photography”.

References

  1. Baskett, Forest (1990). Proceedings of the 17th annual conference on Computer graphics and interactive techniques - SIGGRAPH '90. doi:10.1145/97879. ISBN   978-0-201-50933-5. S2CID   12199502.[ page needed ]
  2. Schofield, Simon (March 1994). Non-photorealistic rendering: a critical examination and proposed system (Thesis).
  3. "Site officiel du Festival et du Marché international du film d'animation d'Annecy". www.annecy.org. Retrieved 20 April 2018.
  4. "Archived copy". siggraph.org. Archived from the original on 21 December 1996. Retrieved 13 January 2022.{{cite web}}: CS1 maint: archived copy as title (link)
  5. Drew, Card (2002). "Non-Photorealistic Rendering with Pixel and Vertex Shaders". In Engel, Wolfgang F. (ed.). Direct3D Shaderx: Vertex & Pixel Shader Tips and Techniques. 87Wordware Game Developer's Library. Wordware Publishing. pp. 319–333. CiteSeerX   10.1.1.360.918 . ISBN   978-1-55622-041-8.
  6. Bénard, Pierre; Hertzmann, Aaron (2019). "Line Drawings from 3D Models: A Tutorial". Foundations and Trends in Computer Graphics and Vision. 11 (1–2): 1–159. arXiv: 1810.01175 . doi:10.1561/0600000075. S2CID   52912187.
  7. Rosin, Paul; Collomosse, John, eds. (2013). Image and Video-Based Artistic Stylisation. Computational Imaging and Vision. Vol. 42. doi:10.1007/978-1-4471-4519-6. ISBN   978-1-4471-4518-9. S2CID   40656135.[ page needed ]
  8. Kyprianidis, Jan Eric; Collomosse, John; Wang, Tinghuai; Isenberg, Tobias (May 2013). "State of the 'Art': A Taxonomy of Artistic Stylization Techniques for Images and Video" (PDF). IEEE Transactions on Visualization and Computer Graphics. 19 (5): 866–885. doi:10.1109/TVCG.2012.160. PMID   22802120. S2CID   2656810.
  9. Litwinowicz, Peter (1997). "Processing images and video for an impressionist effect". Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. pp. 407–414. doi:10.1145/258734.258893. ISBN   978-0-89791-896-1. S2CID   13139308.
  10. Shiraishi, Michio; Yamaguchi, Yasushi (2000). "An algorithm for automatic painterly rendering based on local source image approximation". Proceedings of the first international symposium on Non-photorealistic animation and rendering - NPAR '00. pp. 53–58. doi:10.1145/340916.340923. ISBN   978-1-58113-277-9. S2CID   16915734.
  11. Collomosse, J.P.; Hall, P.M. (2002). "Painterly rendering using image salience". Proceedings 20th Eurographics UK Conference. pp. 122–128. CiteSeerX   10.1.1.7.5383 . doi:10.1109/EGUK.2002.1011281. ISBN   978-0-7695-1518-2. S2CID   9610948.
  12. Gooch, Bruce; Coombe, Greg; Shirley, Peter (2002). "Artistic Vision". Proceedings of the second international symposium on Non-photorealistic animation and rendering - NPAR '02. p. 83. CiteSeerX   10.1.1.10.1761 . doi:10.1145/508530.508545. ISBN   978-1-58113-494-0. S2CID   1146198.
  13. Hertzmann, Aaron; Jacobs, Charles E.; Oliver, Nuria; Curless, Brian; Salesin, David H. (2001). "Image analogies". Proceedings of the 28th annual conference on Computer graphics and interactive techniques - SIGGRAPH '01. pp. 327–340. CiteSeerX   10.1.1.119.5127 . doi:10.1145/383259.383295. ISBN   978-1-58113-374-5. S2CID   2201072.
  14. Brussee, Arthur; Saraev, Andrew; Chyr, William (2020). "That's a wrap: Manifold Garden rendering retrospective". Special Interest Group on Computer Graphics and Interactive Techniques Conference Talks. pp. 1–2. doi:10.1145/3388767.3407385. ISBN   978-1-4503-7971-7. S2CID   221178781.

Further reading

Some key papers in the development of NPR are: