---

Refraction & Reflection Passes (Exceptions)

In this video we aim to understanding the problem with refraction (transmission) and reflections (indirect specular) explore potential solutions. The problem with Indirect Specular (Mirror Reflections) and Transmission (or Refraction) passes is they reflect or refract the entire beauty of the environment, locking that information into 1 pass. There often seems there is not much we can do as compositors to separate those passes further.

---

Here we have a nightmare scenario from a AOV rebuild point of view: A glass jar full of balloons, that is also reflected in a mirror surface. Everything in the mirror Reflection shows up only in the Specular Indirect Pass, and everything seen through the glass jar shows up only in the Transmission (refraction) Pass.

We notice as well that objects that end up in the Transmission (Refraction) pass are missing from the Diffuse Pass.

Mirror Reflections, for example ground plane reflections for our subjects, are also limited to the Indirect Specular pass:

---

What is Transparency?

• Transparency is the ability to see – through an object or surface to what’s behind
• It’s as if the object or material is ignored or nonexistent and does not have to do with Light interacting with the material.
• The light passing through is not Distorted (Refract), nor does Scatter or change Color (which could be the case with Translucency or Transmission)

Transparency basically has only 1 setting: Amount – or “How much can i see through this”

---

What is Transmission?

• Transmission is the passing of light completely through a material
• Refractive, Transparent, and Translucent materials all transmit light, but Opaque materials do not. 
• If light is not transmitted, it may have been reflected (specular) or absorbed.

Transmission can sometimes cause the light to inherit a color tint as it passes through and interacts with the material.  Think of colored liquids or tinted glass.

---

What is Refraction?

• Refraction is the change in direction and speed of a light ray as it travels through or “Transmits” through different mediums, ie. from Air to Glass or Water or Plastic

The 2 more important characteristics of Refraction are:

1.) The Light passes through the material

2.) The Light changes direction

• The amount of distortion, “bending”, or change in direction of a light’s path while passing through the material, depends on factor’s like:
• Thickness of the material, Angle of View, and the material’s Index of Refraction

---

Refractions vs Transmission?

• Transmission is only referring to Light passing through an object
• Refraction is requiring the light to have changed direction, and to pass through
• The render pass is doing both things, so some Render Engines decided to call the pass Transmission, because it’s referring to light passing through the material
• Other renderers call the pass Refraction, referring to the Change of Direction, “bending” or distortion of the light
• Both terms in this case are referring to the same phenomena, just focusing on different aspects of the light’s behaviour
• Transmission might even be a more accurate label, because technically a material could have a Refraction index of 1.0, meaning no refraction/distortion is occurring, but the light is still Transmitting. 
• All Refractions require Transmission
• Not all Transmissions require Refraction

---

Why is Light Redirected during Refraction?

• Light travels through different mediums at different speeds, depending on the density and make up of the medium. 
• Examples of Mediums: Vacuum (space), Air, Glass, Plastic, Water, gases, etc.
• The change of light speed while passing from 1 medium into the next, causes the light to change direction when entering the 2nd medium.

---

Light Wave “Turning” or “Bending”

Light is a Wave:

One side of the wave hits the new medium and slows down first, turning/bending/redirecting the light wave towards a new direction.

---

Color Light Wave Frequencies

Remember that Different Frequencies of Light Spectrum show up as different colors

Different frequencies of light refract at slightly different angles, causing the colors to separate. This is what happens with Color Prisms.

---

Refraction / Reflection in Rainbows

A Combination of this Refraction Color Separation and Reflections within water droplets is what allows us to see Rainbows.

---

Index of Refraction

• Different materials have different densities and make ups and will cause light waves to move through at different speeds
• This is measured with an Index of Refraction, which measures how fast light moves through that medium, and therefore how much it refracts
• An Index of 1.0 is light’s speed in a Vacuum – or no change in direction
• Higher numbers mean light travels through the medium slower and light bends more

In CG, this Index of Refraction is an attribute setting on Materials that will make it more or less refractive

---

Refraction Re-Entering Original Medium

• When the Light goes from a fast medium, to slower medium, and back into the fast medium on the other side, it has another refraction turn
• This time, instead of one side of the light wavelength slowing first, one side speeds up first
• If the exit angle is the same as the entrance angle, it will reverse the lightwave back to the original direction, and is parallel to the orginal light direction, just offset

---

Refraction Angle

• The Angle that the light wave hits the surface also matters
• If the light hits the material exactly perpendicular to the surface normal then it will pass through and the light does not bend at all
• The more extreme the angle, the more refraction. This is why light appears most warped at the edges of curved surfaces.

This is exactly what causes lens distortion to be more extreme at the edges of frame vs the center of frame

---

Chromatic Aberration

Combining the more extreme distortion with the Color separation is why we get Chromatic Abberation more in the edges of frame as well.

---

Caustics

Light Refracting through complex shaped objects, changes direction, and concentrate towards certain areas more than others and create Caustics.

Complex shapes create complex caustics, and moving surfaces, like water, create dynamic and organic moving Caustic patterns.

---

What is Translucency?

• Transmissive materials have a Roughness or Glossiness setting that works in the same way as it does on Specular Highlights
• Increasing the Transmission Roughness causes the light rays traveling through to scatter / “diffuse” or blur together.  Think of Frosted Glass or Plastics.
• This effect of “Blurring” or Scattering the Transmitted light is called Translucency

---

Roughness Blurs Everything Together

Specular Roughness Setting:

Transmission Roughness Setting:

---

Recap

Transparency – You can see through to BG, as if the material or object is not visible or ignored

Transmission – Light allowed to pass through the surface / material

Refraction – Light changes direction as it passes through the material / surface

Translucency – Light passes through material and gets scattered / blurred 

---

Virtual Images / Worlds

When looking at fully reflective and refractive objects, we are seeing a distorted representation of our surroundings.

---

Concave/Convex Reflections

When looking at curved mirrors, it is very obvious that the object we are looking at, is a redirected and distorted view of our surrounding environment

---

Convex Reflections

• With Reflections, light bounces off the material and, depending on the surface shape, changes direction upon reflecting
• Convex shapes cause the light to Diverge – spread apart

---

Concave Reflections

• Concave shapes cause the light to Converge – come together

---

Concave / Convex Refractions

When looking at curved glass, or lenses, light that we are looking seeing through the glass, is a redirected and distorted view of our surrounding environment

---

Concave Refractions

• With Refractions, light passing through the material and, depending on the surface shape, changes direction upon refracting
• Concave shapes cause the refracted light to Diverge – spread apart 

---

Convex Refractions

Convex shapes cause the light to Converge – come together

---

Looking at them all next to each other, we can see Reflections and Refractions are both re-directing the light rays from another part of the scene. The biggest difference is Reflect = Light Bounces off, Refract = Light passes through.

---

There is No Spoon

---

There is No Glass Either…

---

Diffuse – Specular – Transmission (New Category)

Diffuse – All Light Interaction with Material / Object

Specular – All Surface Reflections (Bounces)

Transmission – All Pass Through Refractions

Here is an Example Scene with 1 sided Glass on the left, and 2 sided Glass on the right:

We can see the Direct Transmission shows the Light Source through only the 1 sided glass, but not the 2 sided glass

Almost all information in the 2 sided glass is stored in the Indirect Transmission:

Almost all objects that contain glass in 3D are supposed to be modelled with a thickness, meaning 2 or more sides. So more often than not, your Direct Transmission Pass will be empty and all information will go to the Indirect Transmission. This is also why very often it is not even split up and is just rendered combined as Overall Transmission.

---

Recap #2

• Transmission – Light passes through  
• Refraction – Light redirects.  
• The CG pass could be named either or but is often referring to the same phenomenon.
• Specular and Transmission are both similar in that they are capturing light redirecting and showing a virtual image of the distorted surroundings

• Emission is the light source
• Diffuse describes the object itself
• Specular Events captures light bouncing off the object’s surface
• Transmission Events capture light passing through an object. 
• These all get separated into their own categories.

• Both Specular and Transmission have: 
• A Direct pass that show the first reflection or first transmission of light
• An Indirect pass showing all subsequent bounces or pass throughs
• An Albedo Filter (mask)

• Transmissive surfaces like glass are often modelled with 2 sides
• Therefore the light usually passes through 2+ sides and ends up in the indirect pass, and the direct Transmission shows up empty
• Often rendered as just an overall combined Transmission pass, for convenience.

---

Incorporating Transmission (Refraction) Into AOV Template

Since most of the Refraction is in the Indirect, there is no need for space for splitting up and adjusting separate direct and indirect, like we do with the diffuse or spec. I recommend combining and keeping the Transmission Section Slim for Space Saving in the Template. I also recommend the layering to go: Diffuse, Transmission, Specular, Emission, Other. To me this was the clearest Layering.

I updated the Material AOV Rebuild Templates in the FruitBowl Renders for Arnold, RedShift and Octane incorporating the new Transmission / Refraction Section.

See the Downloads Section at the bottom for links to the whole nuke scripts for learning and template scripts updated per render engine, arnold, octane, redshift.

---

Handling Planar Mirror Reflections

One approach to rendering Planar Reflections with AOVs is flipping the Camera along the Mirror Plane

Flipping the Camera along the normal of the Mirror Plane will produce a Virtual camera for you to render the Mirrored Virtual Image from the right perspective

If your Object is sitting on top of the 3D origin ground plane, this can be as easy as making an Axis Node, Scaling the Y to -1 and plugging your camera Axis Input into this Axis Node.

This will view your scene from the perspective of your Mirror. In the above image, you can see after flipping the Camera in -Y, the Nuke rendered result is aligned with the rendered indirect Specular pass. We’ll need to do this method in the Render Application on Lighting side, or pass this camera back to the lighter in order to render the reflection with full AOVs.

Here is the re-rendered Mirror Camera Perspective of the Armored Mech, with full AOVs, matching the original reflection angle:

---

What about non-ground plane mirrors?

For all oriented mirror planes, the same concept applies, you want to flip the world from the pivot point and orientation of that card along it’s normal facing angle. This is easier to do in 3D applications, but can be done in nuke with a little Matrix Inversion.

I’ve made a tool called MirrorDimension to make this Camera Mirroring super easy. Just stick this node between the Mirror Card in nuke (must have it’s transformations and rotations) and the Camera node. The gizmo is acting as an Axis Node and is just flipping the world along the orientation of the Card input.

No Settings on the node, just the following instructions:

1.) Plug in the MirrorCard input to the Card or Axis node you would like to be the mirror.

– The scale of Card Does not matter as long as the orientation (translation/rotation) are correct.

– The Card’s +Z access is the front of mirror, point that towards subject / camera. This is the blue Z arrow in 3D viewer.

2.) Duplicate your Camera, and plug in the “axis” input of this new Camera to the output of this node.

3.) Your new Camera will be Mirrored according to the plane / card / axis.

4.) Render using this New Camera Setup to get the mirrored CG output.

Before MirrorDimension Node – Original Camera Position:

After Mirror Dimension Node Applied –

You would either do this in your 3D scene and render the AOVs or pass this camera to a Lighter to render from this mirror perspective.

---

Faking Reflections in Comp

If you suddenly need reflections but have no renders, you can use some of the above techniques to fake your reflections.

If you have your Geometry of the object, try projecting the rgba onto the geometry, and rendering it in nuke from the mirror dimension:

If you have no Geometry, but have a Position Pass. Try using a PositionToPoints node, plugged into your render and Position input plugged into your shuffled out Position pass (or select in the dropdown). You can render your rgb 3D point cloud of the object with the mirror camera and fake some reflections. It won’t be perfect, but perhaps in a pinch, it can save your ass and add more realism:

So the next question becomes, what can we do if it’s not a Planar Reflection? or if it’s multiple planar reflection, or surface is curved, or what about Refractions (Transmission) ?

---

Getting Help from Lighters

There is a serious limit to how much we can do in comp when encountering Indirect Specular or Refraction (Transmission) passes. Many times, if this is something that is a big feature of are shot and requires a lot of comp tweaks, we’ll need some help from our Lighting Department.

---

Julius Ihle – Head of Lighting and LookDev at Trixter

We talk to Julius Ihle – Head of Lighting and LookDev at Trixter for potential Lighting Solutions to these problems.

Julius is super knowledgeable, and introduces us to Light Path Expressions and Open Shading Language where lighters can help Build Additional AOVs and help us when the situation calls for it.

Julius is also an online educator and keeps a Lighting Blog discussing exactly these topics, check these tutorials out for more details:

Julius’ Blog:
https://julius-ihle.de/?page_id=346

---

Light Path Expressions

Julius’ Tutorial: LPE Quick Tip #1: Light Path Splitting for Transmission
https://julius-ihle.de/?p=2619

Here is an illustration of the drawing Julius used to explain how renderers are handling Reflection and Refraction Events

In a nutshell, the render engine keeps track of the light ray path and all the events that it undertakes on it’s journey from Camera back towards the Light

Lighters can create new AOVs with custom expressions telling the render engine exactly what parts and what events they want to see in the outputted pass.

Here is a link to the Light Path Expression community GitHub:
https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/wiki/OSL-Light-Path-Expressions

And here is the Arnold User Guide that Julius Mentions in the video to check out for more education:
https://help.autodesk.com/view/ARNOL/ENU/?guid=arnold_user_guide_ac_output_aovs_ac_expression_aovs_html

LPE’s are supported by many renderers so check if the one you are using supports them.

---

Open Shading Language

Julius’ Tutorial: Playing with OSL #5: Arnold Reflection Alpha + Utilities
https://julius-ihle.de/?p=2788

There are also Shaders that have been written that can Reflect various AOVs, such as Utility passes and Alpha channel so that reflections can be more useful for us in comp. Julius has written his own shader to do just that, download it from GitHub:

https://github.com/julsVFX/osl

---

Downloads:

If you haven’t downloaded the FruitBowl Renders already yet, you can do so now:

You can Choose to either download all 3 FruitBowls at once:
FruitBowl_All_Renders_Redshift_Arnold_Octane.zip (1.61 GB)

Or Each FruitBowl Render Individually for faster downloads:

FruitBowl_Redshift_Render.zip (569.1 MB)

FruitBowl_Arnold_Render.zip (562.8 MB)

FruitBowl_Octane_Render.zip (515.4 MB)

The project files and the Renders are separate downloads, so if you have already downloaded 1.1 What and Why files or the Fruitbowl Renders, there are a couple ways to combine them to work.

1. Either add the .nk script to the previous package (in the folder above SourceImages, with the other .nk scripts)
2. Or simply drop the Render files into the SourceImages folder of the new 1.2 project folder

---

Project Files for this Video:

Along with the fruitbowl renders above, here are the nuke script and project files from this video, so you can follow along:

All Nuke Project Files and template scripts:
CG_Compositing_Series_MaterialAOVs_RefractionsReflections_AllScripts.zip (88 KB)

Nuke scripts included in the above download, but can downloaded individually are:

CG_Compositing_Series_2_5_Material_AOVs_RefractionsReflections_DemoScript.nk

---

CG_Compositing_Series_2_5_Material_AOVs_ArmorMech_ReflectionsMirror_Demo.nk

---

CG_Compositing_Series_2_5_Material_AOVs_Updated_Transmission_Templates.nk

---

I have also updated these Individual AOV Rebuild Templates scripts for specific render engines to include a Transmission Section:

Realistic_AOV_Bebuild_Arnold_Template.nk

Realistic_AOV_Bebuild_Redshift_Template.nk

Realistic_AOV_Bebuild_Octane_Template.nk

Realistic_AOV_Bebuild_Blender_Template.nk

---

Glass Balloons (Houdini Solaris)

GlassBalloons_Renders.zip (2 EXRs – 101.4MB)

---

Armor Mech (Rendered in Blender):

Original model by Numata3D_98 on turbosquid:
https://www.turbosquid.com/3d-models/3d-attack-mecha-quadpod-1993489

4 EXR Renders and Geo (for nuke geo projection demo):

ArmorMech_RendersAndGeo.zip (179.4MB)

---

MirrorDimension

I am linking to the gizmo on the Nuke Survival Toolkit github, where you can download the raw file or copy/paste the RAW source code from your browser into nuke:

MirrorDimension gizmo

Or download the .nk file here:
MirrorDimension.nk

Or on Nukepedia:

https://www.nukepedia.com/gizmos/3d/mirrordimension

---

Blender JunkYard Scene:

Scene from https://www.blender.org/download/demo-files/

JunkShop_v01.exr (144.7MB )

---

Blender ClassRoom Scene:

Scene from https://www.blender.org/download/demo-files/

3 Render Files:

BlenderClassRoom_All_Renders.zip (213.6MB)

---

VRay Room Render:

Vray Room – Can be downloaded from this website, look for “download example scene” (36.6MB):

https://www.chaos.com/blog/how-to-use-cryptomatte-render-elements-in-v-ray-for-sketchup

---

Since I am using Stamps in the script, all renders can be swapped out at the top of the script where the “SourceImages” Backdrop is, and the rest of the script will get populated correctly.

---

Slide show PDF

Here is a PDF version of my slideshow in case you would like to save for future research or review:

---

References / Research

---

Light Path Expression Doc:
Github Wiki: OSL Light Path Expressions

Arnold Light Path Expression Help and Examples:
Arnold Help: Light Path Expression AOVs – Arnold User Guide

Julius Ihle Blog
Julius Ihle’s Github page : julsVFX/osl
Playing with OSL #5: Arnold Reflection Alpha + Utilities
LPE Quick Tip #1: Light Path Splitting for Transmission

---

Websites:

Refraction Wikipedia

Transparency_and_translucency – Wikipedia

https://notes.thatother.dev/physics/refraction

https://help.maxon.net/r3d/cinema/en-us/Content/html/Integrated+AOVs.html

https://www.scratchapixel.com/lessons/3d-basic-rendering/introduction-to-shading/reflection-refraction-fresnel.html

https://abnercabuang.wordpress.com/2017/11/19/reflection-refraction-transmission-and-absorption-of-light

https://study.com/learn/lesson/transmission-light-wave-examples.html

Basics of creating glass materials in Corona renderer and 3Ds Max

V-Ray Materials

3Delight Glass – Storage for referenced pages – 3DL Docs

https://macdesignstudio.wordpress.com/tag/reflection

Light Pipe Design: How TIR & Refraction Come into Play

Light and color. – ppt video online download

On BigCo Leaks: Transparency and disclosure

Can You See Through Me? | Lesson Plan

https://wbbsesolutions.guru/wbbse-solutions-for-class-10-physical-science-and-environment-chapter-5

FAQ/Combining 3D Passes – VFXPedia

Refraction – Definition, Refractive Index, Snell’s Law

The Physics Behind Rainbow Formation

Refraction | Sound Waves

Refraction Of Light – 2023

https://www.geocities.ws/rmackrell509/4thSpring.html

What are the uses of refraction in our daily life?

What is the Index of Refraction? Measurement, Definition & More –

Autodesk – arnold – Help

View topic – Help understanding Refraction, SSS and Transmission passes?

PPT – The Basics of Refraction PowerPoint Presentation, free download – ID:2558034

Molecular Expressions: Science, Optics, and You: Light and Color – Refraction of Light

https://slideplayer.com/slide/16831983

https://www.researchgate.net/figure/Distortions-of-the-light-field-generated-by-refractive-a-and-reflective-b-convex_fig1_308768656

https://global.canon/en/technology/s_labo/light/003/02.html

Delivering VR in Perfect Focus With Nanostructure Meta-lenses

https://osa.magnet.fsu.edu/teachersparents/articles/lensesgeometricaloptics.html

https://www.simply.science/images/content/physics/waves_optics/reflection/Concept_map/Convexconcave_mirrors.html

What is the difference between Translucency and Transparency?

https://www.linkedin.com/pulse/transparency-vs-translucency-whats-difference-between-archie-blake-3acne

Transparent vs Translucent

---

YouTube Links:

Light Absorption, Reflection, and Transmission

How is Light Absorbed, Reflected and Refracted

Why does light bend when it enters glass?

Refraction of Light

Reflection, Refraction and Absorption

Opacity Maps – PixPlant

Refractive index of water

How To Demonstrate Light Bending or Refraction

How Lenses Function (CanonOfficial)

Refraction Explained

Compositing/Render layers in Blender