In this Bonus video on Material AOVs, I cover Cross Polarization photography, which is a technique that allows us to separate diffuse and specular components of everyday objects. I go into detail about the lighting concepts that allows this separation to occur, and how it’s used to gather reference and textures to recreate objects in 3D.
Electromagnetic Spectrum
Visible Light is a section of the Electromagnetic Spectrum
Light / Color is represented in 2D as a Sine Wave with a specific frequency
3D Light Wave Representation
The 2D representation looks a bit different in 3D space, since the light waves could be oriented in any and all directions along it’s forward axis
A light beam with randomly oriented Light Waves is referred to as an Unpolarized Light
Linear Polarization of Light
Linear Polarization isolates one specific angle of the light wavelength, only allowing a portion of the light waves that were oriented in the that direction, through the filter
Cross Polarization of Light
Cross Polarization uses 2 Polarizers that are perpendicular to each other, effectively eliminating the light wave passing through.
The first polarizer isolates the light wave to only one orientation
The second polarizer, if parallel to the first, continues to allow the polarized light through, but as it becomes more perpendicular, the light gets dimmer, and eventually blocked entirely
Polarization Upon Reflection
When unpolarized light hits a reflective surface (with a refractive index different than the surrounding medium, such as glass, snow, or water) the specular reflection is polarized or partially polarized to the angle perpendicular to the plane of incidence. (along the surface)
How polarized the Reflection depends on many factors; angle of incidence, material type, etc.
Brewster’s Angle
At a specific angle, the specular reflection is completely polarized to the angle perpendicular to the plane of incidence.
This angle is known as Brewster’s Angle.
Unpolarized Diffuse Component
Only the Specular Reflection has the effect of the Brewster’s Angle Polarization
The Diffuse Component is Unpolarized, because they are newly emitted photons from excited atoms
This phenomenon only happens when the light is reflected off dielectric materials such as water or glass.
When reflection occurs on a metallic surface, no Brewster Angle nor refracted light exist
Polarized Specular Reflections
Placing a Linear Polarizer filter in front of the observer will Cross Polarize some Specular Reflections if angled correctly. It blocks the polarized reflection light wave from shining through it
This is how Polarized Sunglasses are able to eliminate harsh glares and reflections from dielectric surfaces such as glass, water, snow, etc.
Cross Polarized Photography
If you polarize the light source, the Specular Reflection is also polarized (because it’s a mirror reflection of the light wave).
The Diffuse Component is unpolarized light because it is newly created lightwaves oriented randomly. Adding a second polarizer on the Camera, means we can block the Specular Component entirely depending on the angle of the Polarizers. When the 2 polarizers are parallel, we see Specular + Diffuse , and when they are perpendicular we will see only Diffuse.
The Parallel Polarized image gives use the Specular and Partial Diffuse (only Diffuse Component of that orientation)
The Cross Polarized image, negates the Specular, and only shows the other half of the Diffuse Component
To isolate the Specular Component, take Parallel Polarized image (Specular + Partial Diffuse) and minus the Cross Polarized image (Partial Diffuse). The Diffuse Components cancel out, and all that is left is the Specular Component
This Cross Polarization Photography allows CG Artists to collect photogrammetry data of everyday objects, and allows them to recreate these objects in 3D with accurate Diffuse and Specular Maps for Physically Based Rendering
What seems just like theoretical Diffuse/Specular Render Pass separation in CG is actually a lighting phenomenon that can be separated into Diffuse and Specular Components in the real world
Notice that Metallic Materials have no real Diffuse Color to them, They show up as completely black in the Cross Polarized result. Metals are entirely surface level Specular Reflections
Occasionally, the Diffuse Components of the Parallel Polarized and Cross Polarized Images are slightly different, (brighter or a shift in color for example)
In this case, when we minus the Cross Polarized result from the Parallel Polarized result, we are left with leftover color information or artifacts. The Specular Component can be desaturated to compensate for those color artifacts
Remember that in Dielectric Materials the Specular Component is the same color as the light source, but Metals can sometimes tint the Specular color depending on the type of Metal
Light Stage: Cross Polarization
The light stage used in films is capturing evenly lit, cross polarized textures of various facial expressions.
This helps separate Diffuse and Specular and aids in tracking features of the face
References:
Here are some great websites that go into more detail about polarizations:
In this post we are going to be focusing in on the Material AOVs Category.
Levels of Complexity
There are different levels of complexity to rebuilding Material AOVs into the beauty, and it all depends on how much flexibility and control you want with the cost of complexity and speed.
Simple
Diffuse
Specular
Emission
Other – Refraction / True Reflection
Intermediate
Diffuse
Direct Diffuse
Indirect Diffuse
Sub Surface Scattering
Specular
Direct Specular
Indirect Specular
Reflection
Coat
Sheen
Emission
Other – Refraction / True Reflection
Complex
Diffuse
Direct Diffuse
Indirect Diffuse
Sub Surface Scattering
Raw Diffuse
Albedo / Color / Texture
Specular
Direct Specular
Indirect Specular
Reflection
Coat
Sheen
Raw Specular
Albedo / Filter / Texture
Emission
Other – Refraction / True Reflection
Diffuse, Specular, and Emission are the Foundational Categories, and the complexities are subdivisions of the Diffuse and Specular Categories
So let’s first focus on the Simple category of Diffuse, Specular, and Emission and really break those down and understand them fully. This will make the future subdivisions easier, familiarise us with terms and concepts, and help us have a grounded foundation of knowledge for what we are adjusting when using these passes.
The full presentation from the video can be downloaded here in pdf format, for those who want to keep or study it offline:
Emission is any object, material, or texture that is actively emitting light into the scene
This includes any Lights, Super-heated metals, or Elemental FX like fire/ sparks / lightning / magic etc
Neon Lights, Screens, Monitors are all examples of real life Emission objects
Diffuse vs Specular
Specular – Surface Level Reflections
Diffuse – Light passes through surface and interacts with the material at a molecular level, Scattering and Absorption allow certain colors to re-exit and scatter into scene
Let’s talk about Specular first andSurface level Reflections
Specular
Law of Reflection
The angles of incidence is equal to the angle of reflection
Smooth Surface – Specular Reflections
Light Beam = a bundle of parallel light rays
Light Beam remains parallel on incidence and parallel on reflection
Planar Mirror and Virtual Image
An Image created by planar specular reflection that does not actually exist as a physical object is referred to as a Virtual Image.
The Virtual Image appears to be located “behind” the mirror
Virtual Image distance = Object to Mirror + Mirror to Observer.
Speculum is the Latin word for “mirror”, which is where “Specular” derives from
The people are witnessing a virtual image of themselves looking back, that is double the distance from them to the mirror. The light travels from them -> to the mirror, and then from the mirror -> back to their eye
Notice the reflected virtual image of the chess piece is in focus, even though the real piece (in the foreground) is out of focus. The camera lens is respecting the mirror’s virtual image distance, even though the mirror itself is out of focus.
Here you can see a ground plane mirror appearing to invert the tree in it’s virtual image
Rough Surface – Diffused Reflection
The uneven surface causes the Incidence Rays to hit at different angles
The outgoing reflection rays scatter in different directions
Here you see some examples of different CG materials along the Roughness / Glossiness spectrum
Wet Surface Reflections
When a surface is wet, the water fills the gaps and flattens the surface and causes more a specular reflection
Microscopic Surface Details
In these slides and examples we are discussing surfaces at a microscopic level. You might think a piece of paper looks smooth, but under a microscope it has quite a bit of roughness to it, which is what makes it so evenly lit and diffuse.
Metallic vs Dielectric Surfaces
The diffuse and specular terms describe two distinct effects going on. The Light interacts with materials differently depending on if the material is a metal, or a non-metal (Dielectric)
Dielectric – Absorbs and Scatters light
Metallic – Does not Absorb light. Only Reflects
Dielectric (Non-Metal)
Light penetrates the surface level and the molecules of the material absorb and scatter the light within
The light photons excite the atoms they hit below the surface. Some of the light is absorbed, and this energy is converted to heat. Then new light rays (photons) are emitted from the excited atoms. Those might excite nearby atoms or exit the surface as new photons. These new photons are same color as our material.
The Base Color Texture (Albedo Map) – determines the color of the diffusely scattered photons from excited atoms. It’s the color that is scattered back out and not absorbed by the material
Metallic
Does not Allow light to penetrate the surface and does not Absorb light. They only Reflect light on the surface
Metals can be thought of as positively charged ions suspended in a “sea of electrons” or “electron gas”. Attractions hold electrons near the ions, but not so tightly as to impede the electrons flow. This explains many of the properties of metals, like conductivity of heat and electricity
The incoming photon does not excite the atoms, but bounces directly off the electron gas
The Base Color (Albedo) is used to describe the color tint of the specular reflection
“Electron Gas” Model
Notice the Specular Reflections are tinted a certain color depending on the metal type:
On Dielectric Plastic balls, the material color changes, but notice the specular highlights are the same color, maintaining the color of the light or surrounding environment.
Comparison of a Metallic vs Dielectric Material in CG
Chrome Sphere and Diffuse Ball
Used as a reference to see what something 100% Smooth and Metal (Specular) and 100% Rough and Dielectric (Diffuse) looks like in the scene.
The diffuse component includes light that penetrates the surface and interacts with the materials molecules. This happens in different ways in the real world
Transmission
Light passing through the material / surface
Can be thought of as “transparency”
Refraction
when light changes angles as it goes through different materials or mediums
Absorption
When certain wavelength colors of light get absorbed by the material
Scattering
when light is dispersed in many directions when it comes into contact with small particles or structures in the material
Simplified Diffuse Calculation
When the distance that light travels beneath the surface is insignificant and negligible, the calculation can be simplified by the renderer and just calculated at the surface point where the light hits. It uses the Base Color Texture (Albedo) as the Diffuse Color that will scatter.
Sub Surface Scattering
When the distance the light travels beneath the surface of the material is significant, the interior scattering must be calculated. This is referred to as Sub Surface Scattering (SSS)
Physically Based Rendering Terminology
Albedo
Base Color Texture Map
On Dielectrics (non-metal) refers to color of material
On Metals, refers to the color tint of the specular reflection
Texture map is without highlights, shadows, or ambient occlusion
Metalness Map
What area is metallic or not. (will use Albedo Color differently). Usually Black or White
Roughness (Glossiness) Map
How blurry or how sharp the reflection will be
Real life objects often have a diffuse and a specular component
Diffuse describes the color of the billard balls, but the specular highlights are all the same color (reflecting the color of the light above the table)
Iridescence
There is also Iridescent materials that change specular color depending on viewing angle.
Iridescence is a kind of structural coloration due to wave interference of light in microstructures or thin films.
Nuke – Simple Material AOV setup
We can break our fruit bowl render into the 3 simple components, Diffuse, Specular, and Emission. They layers look like this:
You can download the nuke script shown in the Tutorial. I created the mini setups for the 3 different types of renderers, Arnold, RedShift, and Octane. Dividing the Beauty render up into their 3 Diffuse, Specular, Emission Components, and Recombining them.
The project files and the Renders are separate downloads, so if you have already downloaded 1.1 What and Why files or the Fruitbowl Renders, there are a couple ways to combine them to work.
Either add the .nk script to the previous package (in the folder above SourceImages, with the other .nk scripts)
Or simply drop the Render files into the SourceImages folder of the new 1.2 project folder
This will help the Read nodes auto-reconnect to the sourceImages for you.
Recap
Emission / Illumination materials emit light
Specular and Diffuse can be separated by Surface Level Reflections and below surface Material Interactions
Each individual light ray follows the Law of Reflection.
The smoother a surface is, the more mirror-like the specular reflection will be.
The roughness of a surface will cause the reflected rays to scatter, and reflection to be blurred.
Metallic materials do not allow light to enter the surface. They only reflect light
Dielectric materials allow light to enter the surface. Light rays are refracted, absorbed, scattered by the materials molecules. Certain color wavelengths re-exit the surface in random directions, which is what we perceive as the materials color
Albedo – Base Color Texture. On Dielectrics – color of material | On Metals – color tint of the specular reflection.
Sub Surface Scattering is when light below the surface travels a significant distance before re-exiting
Iridescent materials tint the color of the specular reflection depending on viewing angle.
References, Resources, Credits
Firstly, Thanks to Pexels for providing such a good resource for stock reference images
I did a hell of a lot of research on this topic before creating the video, I really encourage you to dig a little further and explore the topics more using these great resources:
The project files and the Renders are separate downloads, so if you have already downloaded 1.1 What and Why files or the Fruitbowl Renders, there are a couple ways to combine them to work.
Either add the .nk script to the previous package (in the folder above SourceImages, with the other .nk scripts)
Or simply drop the Render files into the SourceImages folder of the new 1.2 project folder
This will help the Read nodes auto-reconnect to the sourceImages for you.
Often there are a lot of renders passes to sort, and it’s useful to divide them into categories based on their functions. We can divide up all the render passes by how they are used.
There are 2 Overarching Types of CG Passes:
Beauty Rebuild Passes – Will recreate the Beauty Render
Data Passes – Helper passes
There are 4 Main Categories of CG Render Passes
Material AOVs
Light Groups
Utilities
IDs
Material AOVs
Used to adjust the Material Attributes (Shader) of objects in the scene
Examples:
Diffuse, Specular, Reflection, Sub-Surface Scattering, Refraction, Texture/Color, Emission, Raw Lighting, etc.
The passes in this category should add up to recreate the beauty render, as demonstrated in the previous video
From now on in the series, if I only say “AOVs”, I am referring to this category here. I will try my best to say Material AOVs, but I am so used to it being in my terminology and don’t find the AOV “all render pass” definition very useful.
Material AOVs are passes related to the shader or material from the 3D application. When we use these passes, we are wanting to manipulate the material or the shader of the object
Key, Rim, Fill, HDRI, Light-Emitting Objects, etc.
You can separate your lights however you like. Usually you see things like the 3 point lighting set up broken out into different lights. Along with HDRI and light emitting objects separated.
We are usually adjusting light attributes such as temperature and intensity
The ID category could probably live under the Utilities Category, but I do think the separation of these 2 categories is useful.
ID’s sole purpose is to pull out an alpha or matte channel, whereas Utilities can have many use cases beyond just that.
Many times a texture artist working on characters will make custom texture matte passes that can be rendered out as Texture RGB IDs to help isolate those important parts of the texture for adjustment in comp.
These also do not add up to the Beauty Render
Nuke Script: Breaking out Categories of the Renderers
Nuke script is a node graph representation of the slides table we looked at and I’ve broken out the passes in the categories for each of the 3 render engines.
In order for the LayerContactSheet node to display just the passes for each category, I am removing all layers from the other categories.
I’ve also broken out all of the Category’s Layers into shuffles when a text of the layer name into a contact sheet. The main difference would be that this contact sheet would be renderable, and the UI text on the layerContactSheet is not.
In the Beauty Rebuild Passes Section, underneath we have a Material AOV rebuild and a Light Group Rebuild, showing that these passes add up to equal the Beauty.
Please look through the different categories and different Render Engines to familiarise yourself.
Tips and Tricks for making contact sheets
Split Layers
Here are some links to some various Split out layers / shuffle layers python scripts found on nukepedia:
Place the FruitBowl renders files into the /SourceImages/ folder of the project files and nuke will reconnect the read nodes.
What is a CG multi-pass Render?
A CG Render with multiple extra layers or passes that are to be used to recreate the Beauty Render and to aid in further manipulation while Compositing.
Why do we need it?
Renders are Expensive, and Changes are often necessary. It can take too long to make tweaks and hit notes if you have to re-render the image.
Sometimes it’s faster to find the “look” you are going for in Comp, rather than waiting for the Render results.
Some effects are better achieved in Comp and need additional passes to help achieve the effect in Compositing.
Terms and Definitions
Here are some useful Terms and Definitions that I will be using in this series. They are commonly used in the industry, but sometimes they can be confusing or interchangeable, so I will try and define them for us to help while discussing CG Compositing
Render – The output image or final result of the export calculation from the CG software.
Renderer – The Render Engine or algorithm used to produce the render.
Render Passes – A general term for additional layers exported by the CG renderer meant to be used alongside the main render. These might come contained within a multi-pass EXR or be rendered as separate images.
SourceImages and Stamps
All of the read nodes and source images in the nuke scripts will be located at the top of each nuke script under a “Source Images” Backdrop
You will need to re-link the files in this area if you are following along
We will be using Adrian Pueyo’s “Stamps” add-on to nuke in order to populate our nuke script with the files in the source image folder.
LayerContactSheet is the easiest, fastest, and most convenient way to get a visual overview of all the passes contained in your render.
Turn on Show Layer Names to get UI labels of each pass name. This is only a GUI overlay, so you cannot render it out, it’s just for viewing purposes, but it’s great for identifying the pass names we are looking at
The Viewer
The Viewer shows an alphabetical dropdown list of channels of the stream where the viewer is plugged into.
Remember to set the viewer back to RBGA when you are done viewing that layer
You can use the PageUp PageDown hotkeys to cycle through layers in the Viewer
Along the bottom left of the viewer, it also lists all the channels separated by commas. It’s good to occasionally look at this part of the viewer to keep track of if you’ve lost your layers from the stream, or you are accidentally carrying layers that you do not need anymore in the stream.
Shuffle node
The Old Shuffle node will show a list of all layers in the stream which it is plugged into if you use the “in 1” dropdown
Good way to quickly check what layers are in your stream, but not as visual as layerContactSheet
ShuffleCycleLayers python script:
I wrote a tool called “ShuffleCycleLayers” which you can use hotkeys like Page Up, Page Down or + , – to cycle through the layers of the selected shuffle node, just like the viewer layer cycler. Maybe some people will find this handy if they don’t like to changed the viewer channel dropdown and would prefer to cycle through Shuffle node layers
Old shuffle only displays list of layers within the stream the input is plugged into
New shuffle displays list of every layer in the nuke script
If you’d like to exclusively use the old shuffle node instead of the new shuffle node, you can add this line of code to your menu.py in your User/.nuke/ folder
Split Layers is a python script that shuffles out all available layers from a selected node
This will make 1 shuffle per layer all connected to the source.
You can then just view and toggle between all the layers in the nodegraph
selecting all and hitting the hotkey alt + p will toggle on the postage stamp feature in all the shuffles, and if you visual thumbnails for all the passes. This can be useful for grouping and organising the passes.
Here are some links to some various Split out layers / shuffle layers python scripts found on nukepedia:
Channels are the individual pieces that make up a Layer, or Channel Set. The most common example is red, green, blue and alpha, channels that make up the rgba layer
A layer must contain at least 1 channel, but often has multiple channels.
Nuke prefers layers to have a maximum of 4 Channels per layer, any more and it has difficulty displaying them in the GUI interface
It becomes significantly more difficult to see the channels beyond 4 that are in 1 layer. Nuke’s interface is built around displaying 4 channels.
An individual channel in nuke is written as LayerName.ChannelName, to let you know what layer it belongs to
Depth.Z for example, in which Depth is the LayerName, and Z is the ChannelName
Whenever there is only 1 Channel, this displays in the viewer as the red channel, since it’s the first channel visible in rgba
There are also many cases where someone will just refer to it as “The Depth Channel”, where they are recalling referring to the Layer, but since it commonly has only 1 Channel, they are talking about the same thing.
Some nodes in nuke deal with layers and channel differently, or prefer to deal with one vs the other
A shuffle dropdown displays LayerNames for example whereas a Copy node displays Channels, and therefore the list is much bigger since it is displaying the individual pieces of the layer
Blur node “channels” dropdown actually lists layers, and then you can toggle the channels of that layer on/off
Basically any node with a mask input is dealing with channels since it only needs 1 channel to function
The first 4 channels of a layer are mapped to, and will display as Red, Green, Blue, and Alpha in the viewer, regardless the actual name of the layer. Any more than 4 channels in a layer and nuke has a hard time displaying them
A motion pass for example, is describing motion in XY directions. Left-Right and Up-Down. So only 2 channels are needed in the Layer and they display as Red and Green
A position pass, for example, is usually describing XYZ – 3D space coordinates, and sometimes the channels are actually named x, y, and z. So Position.x, Position.y, Position.z
Since X, Y, and Z are taking up the first 3 channels in this layer, they will display as red, green, blue
AOVs
AOVs stand for Arbitrary Output Variables
Arbitrary output variables (AOVs) allow data from a shader or renderer to be output during render calculations to provide additional options during compositing. This is usually data that is being calculated as part of the beauty pass, so comes with very little extra processing cost.
They can be considered ”checkpoints” or “steps” in the rendering process. The render engine splits up many calculations while making the final image (Beauty) and is exporting these smaller steps out to disk so we can combine them and manipulate them in Comp.
The important thing to take away is the renderer takes these “pieces, these AOVs, and combines them together to form the final Beauty render. We are essentially trying to recreate this process with our CG rebuild, while retaining control over the individual pieces.
One of the best things about AOVs is we get them “for free” since the renderer was going to calculate them anyway.
AOVs can sometimes be just a “catch all term” for all layers/passes you will render out
“What AOVs are you exporting” is a common question, and many 3D applications will use the term AOVs to define any render passes (even though some of them require extra work to get, like ID’s or custom passes)
Differences in the Render AOVs
All the renderers are essentially doing the same thing. They are crunching the numbers, using different algorithms, and coming up with the math needed to produce the final renders.
Since all the renders are basically doing the same steps / calculations, you just have to get used to what that renderer chooses to name these AOVs or lighting passes. All the passes will combine together and add up to the final Beauty output.
There are certain similarities or patterns between all the renderers.
Sometimes we’ll be looking at 1 renderer while explaining concepts, but they often translate over to the other renderers in some way. So keep an eye out for the patterns described and apply what is being taught to your renderer’s output.
Our renders have differences in amount of AOVs exported and differences in naming conventions for the AOVs
For a long time I wanted to release a CG compositing series. Many things stopped me in the past:
Time constraints
Access to good Render examples to work with
Not thinking I had too much to contribute to the subject matter
This series will be focused on answering the following question
How do I best rebuild my CG passes, for the most flexibility as a Compositor?
Download the FruitBowl Renders for the Series
My Friend and fellow artist, Chase Bickel, has kindly provided us with some high quality renders of a FruitBowl to download for free and play around with.
Download the FruitBowl renders now, or I will always post the links at the top of each video and blog post for you to download later:
You can place the FruitBowl renders files into the /SourceImages/ folder of the project files folder accompanying each video and nuke will reconnect the read nodes.
For Example:
These Renders are full of common passes you would find in production, including:
AOVs
Lightgroups
IDs
Utility
Gameplan
Start with the Basics –> Build our way to more advanced topics –> End with a proposed template for your CG Rebuild
I will go through different types of AOV passes you would typically find at a studio, what they are, how they are used, and how should think about them in relationship to one another. We will categorise and group different AOVs in order to define them better, and help us find the commonality and patterns between renderers.
This series aims to be useful no matter what renderer your CG comes from, as the principles are the same.
Topics Covered
Differences between Additive and Subtractive Workflows, and the pros and cons of both
Explaining the difference between Material AOVs and LightGroups and how to work with them together seamlessly
This includes an elegant solution to the infamous AOV – Lightgroup paradox
I will cover the importance of making Mattes and alphas, to help us isolate, and automate our CG manipulation. We will go over common utility passes and IDs and show how to do some cool things with them
Using Full CG Render
Will not cover how to integrate CG renders into a live-action plate
Will focus on the CG rebuild and various methods of manipulation to get the most out of your CG renders
Something for everyone
Juniors, Mids, Seniors, TDs, Comp Supervisors
There will be knowledge to be learned across all levels
Perhaps this will one day be a pre-requisite for a full CG Compositing into live-action plate course
This series will take some time to release all episodes, so please have patience
Compositing Mentor 
