I implemented transmission using the GGX microfacet model¹. The relevant command is transmission. Note that there is no specific ior command because the index of refraction is actually automatically determined from the specular value, according to Schlick’s approximation². Essentially, the value given in specular is the color seen when directly looking at the object, which allows us to solve for the index of refraction $\eta$. For example, glass has the index of refraction $\frac13\eta_\text{glass} = 1.5$. Using Schlick’s approximation, this roughly translates to a specular value $k_s = 0.04$.

In order to conserve energy, I have the following setup: first calculate the amount of specular reflection (e.g. the Fresnel term $F_r$), and then take $1 - F_r$ to be the remaining amount. This is then distributed among diffusion and transmission. Therefore diffuse and transmittance actually represent the percentage of the remaining energy that is diffused/transmitted. As long as diffuse + transmittance <= 1 and specular <= 1, energy does not increase. This setup is a little awkward, but it works well enough.

Also, I did not use Schlick’s approximation to compute $F_r$, despite using it to approximate $\eta$. When it came time to actually compute $F_r$, I instead went with the full formula described in the GGX paper¹. Originally I had not planned to do so, but there is a reason for this: Schlick’s approximation does not account for total internal reflection (TIR). It always interpolates over the full interval $\theta \in [0, \frac\pi2]$, rather than maxing out at an earlier $\theta$. However, how TIR actually works is that we should see a smooth fade between the refraction and reflection terms. At the TIR critical angle, there should not be a sharp visual effect caused by suddenly discarding invalid rays, because we should have $F_r = 1$ before that point.

Update: 6/12/2024

I realize this comes after the due date, so feel free to ignore the following section. I just thought the issue with Schlick’s approximation deserved a better explanation.

The easiest way to see how Schlick’s approximation does not handle TIR is that it does not distinguish between which index of refraction ($\eta_1$ or $\eta_2$) corresponds to the entered/exited medium. Schlick’s approximation is symmetric in that respect. However, the exact formula is not symmetric. Let $\eta_1$ be the entered medium and $\eta_2$ be the exited medium. Schlick’s approximation is only accurate when $\eta_1 > \eta_2$.

$F_\text{Schlick}$ vs. $F_\text{exact}$: Red is Schlick’s approximation, and black is the exact formula. $\eta_1$ and $\eta_2$ are chosen to represent glass and air. Technically, $\eta_\text{air} = 1.0003$, but I have chosen $\hat\eta_\text{air} = 1$ for my program.

Anyway, enough of the discussion on implementation. I think the following image best illustrates my implementation of transmission. I repurposed the GGX test scene from Homework 4.

GGX spheres and planes: The sphere on the left has specular 0.04 0.04 0.04, the plane on the middle-left has roughness 0.3, the plane on the middle-right is experiencing TIR (the black color is due to there being no sky), and the plane on the right has specular 0.00001 0.00001 0.00001 and is almost fully transparent. This scene was rendered with MIS at 2048spp.

Note that the planes in the above image are bending light, which doesn’t make much physical sense (refraction should be barely noticeable for thin mediums). The program checks $\mathrm{sign}(\omega_i \cdot n)$ to determine if light is entering/exiting a medium. If incoming light is aligned with the normal, it is treated as entering a medium. And if incoming light is not aligned with the normal, then it is treated as exiting a medium. So in order to be physically accurate, all objects must have front and back facing geometry. For demonstration purposes, these planes do not, which is why the refraction is noticeable. The plane on the middle-left is facing towards the camera (the camera’s perspective is from the “exterior”), while the plane on the middle-right is facing away from the camera (the camera’s perspective is from the “interior,” which is why we see TIR).

Depth of Field

Depth of field is quite simple to achieve. With path tracing, it is essentially a free feature. All that needs to be done is for the ray origins to be perturbed by some small amount while fixing points at a specific distance, called the “subject distance” (okay, it’s slightly more complicated than that, but it really is quite simple). The relevant commands here are aperture and subjectdistance. In photography, aperture is often described with an “f-number,” but the aperture command instead refers to relative aperture. Relative aperture is simply the reciprocal of the f-number (e.g. aperture 0.125 is equivalent to f/8).

GGX spheres and planes: Same as above, except with subjectdistance 13 and aperture 3. Note that the DOF effect occurs for the “interior” of the glass ball (e.g. everywhere but the edges). This is because the light coming through the ball is coming from far away, past the range of clarity.

Cornell box: Another simple demonstration of the DOF effect. subjectdistance 2.75, aperture 0.7. I think I may have set the aperture too high, because it was a little hard to get the sphere in focus. To be fair, it’s a little difficult to edit a scene through a text file.

Photon Mapping

Finally, I attempted to implement photon mapping³. However, I decided against fully committing to photon mapping: instead, I only created the caustics photon map. This is because I felt that global illumination was already done quite well by NEE/MIS. On the other hand, caustics look truly horrifying without thousands of samples.

In my implementation, using the photon map is a little cumbersome. Since photon maps can be reused, there is a separate command sampler photon and shader herophoton (it’s called herophoton because the actual shader is called hero). This makes the next render output a photon map (.photons file) instead of an image. To set the number of photons, we reuse the size <x> <y> command, where x * y photons are shot out (it doesn’t really matter that it’s a 2d size parameter; it’s just a relic from reusing commands). Afterwards, we have to switch back to sampler basic and shader hero to render the scene. By default it will ignore the photon map. To make it load/use the photon map, we need to use photonmap <radius> <count> with nonzero values. <radius> controls the maximum search radius for photons, and <count> controls the maximum number of photons to count.

The photons are stored in a $k$-d tree written from scratch, since it’s honestly easier to write one yourself than to take some open-source library. Most online C++ implementations seem to use a linked-node approach, which is really annoying to copy to device memory. My $k$-d tree was flat, which is also more performant. Here are some preliminary results showing the locations of photons in the scene:

Cornell box: The locations/counts of photons in the scene, with 1,000,000 photons. Keep in mind that there are far fewer than 1,000,000 photons here because the vast majority of them were not for caustics and were therefore discarded.

Finally, here is an actual render using the photon map:

Cornell box: Again, 1,000,000 photons with photonmap 0.1 256, nexteventestimation mis, spp 64. The caustics do look quite clean despite a low number of samples.

And here is the ground truth, rendered with brute force (MIS and 2000+ samples per pixel):

Cornell box: Ground truth. nexteventestimation mis, spp 2048.

The caustics do look comparatively clean, but they are evidently wrong. I haven’t quite figured out why yet. It’s a bit sad, but at least it wasn’t a complete failure. :/

My best guess is that I have calculated the power of the photons incorrectly, e.g. in BSDF evaluation. I say this because the shape of the caustics actually match up very closely between the photon-mapped image and the ground truth. My other reason for thinking my BSDF evaluation could be wrong is that when viewing the raw photon count (e.g. in the first image), we do see a higher concentration of photons at the center of the caustic. This indicates to me that my importance sampling is working correctly. I think the lack of a bright spot despite the much higher concentration of photons means that these photons are severely under-powered.

References

I have (almost) implemented microfacet transmission using the GGX distribution model¹ with BSDF importance sampling. The relevant commands are transmittance <r> <g> <b> and ior <x>. The transmission seems to be working correctly with roughness, and refraction kind of works, as illustrated by the following demo image (modified from the test GGX scene file for Homework 4).

GGX spheres and planes: The sphere on the left has ior 1 and roughness 0.25, and seems to be working okay (at least it passes the test by casual eye). The plane in the middle has ior 1.2 and roughness 1, but it’s weirdly bright. Finally, the plane on the right has ior 1 and roughness 0.001, and it seems fine.

Unforunately, “kind of works” is doing some heavy-lifting here.

GGX spheres and planes: Same as before, but all with ior 1.5. Evidently, not all is right. No clue what’s going on with the sphere on the left.

However, I would like to note that all of these effects were attempted without photon mapping. I initially wanted to experiment with extending previous techniques to transmission, just to see how feasible it would be. As such, the NEE is completely wrong, and even wholly transparent objects will cast shadows.

GGX spheres and planes: Additional plane on the right with specular 0 0 0, transmittance 1 1 1, roughness 0.001, and ior 1. Spooky shadow…

Turning off NEE and increasing the number of samples, we do get something “decent.” The spooky shadows are gone, and the purple translucent plane now casts a purple shadow. So everything seems mostly correct. Refraction still seems mostly broken, though.

GGX sphers and planes: Same as before, but with nee off and spp 2048.

ℹ️ Update: 5/31/2024

I got refraction working :) Again, without NEE.

GGX spheres and planes: Yay! The ball on the left is supposed to replicate glass, with ior 1.5. It accurately flips the image behind it. Note that the refraction here doesn’t make much sense. Planes are treated as faces of some larger object. So rays hitting the planes are treated as entering some object, but since there’s no back face, they just… stay refracted. To model a thin piece of glass, we should place backwards facing planes behind each of the planes. But then the refraction effect would be almost unnoticeable, so that is not done here for the sake of demonstration.

Photon Mapping

Turning off NEE and relying on brute force is pretty impractical, so I imagine this is where photon mapping comes into play. I’ve only just started reading into it, though, so I’m still not totally sure. Hence I have nothing to show related to photon mapping right now.

References

I implemented OptiX support, which massively reduced render times. Instead of using the provided OptiX 6.5 template, I set it up from scratch with a newer version (OptiX 8.0) and integrated it into my existing submission for CSE 167. However, the ray tracer still uses the CPU backend by default. To enable OptiX, the command backend optix must be included in the scene file. The default is backend cpu.

In my existing CPU implementation, I implemented a Bounding Volume Hierarchy along the lines of Pharr, Jakob, and Humphreys’s description in Physically Based Rendering 4e, alongside an efficient intersection algorithm for axis-aligned bounding volumes given by Ericson’s Real-Time Collision Detection. To disable the BVH, the command accelerator naive must be included in the scene file. The default is accelerator bvh.

A comparison of render times can be found below. The program was compiled with MSVC using /O2 optimization and /openmp paralellization, and it was run on my laptop’s Intel i7-12700H and NVIDIA RTX 3060.

Scene render time comparisons between CPU without BVH, CPU with BVH, and GPU with OptiX. Note that scenes 8–10 are too complex to render without acceleration in a reasonable amount of time, so the times are left as N/A. CSE 168 image-grader report.

Using the BVH and OptiX, I was able to render much more computationally taxing scenes (scenes 8–10), such as the one below of a reflective Stanford Dragon in a highly reflective box.

Scene 9, Stanford Dragon: shininess 70 specular .7 .7 .7, 3840x2160

Transmission

I added transmittance and ior (index of refraction) commands. The effect is decently realistic. For example, the following images demonstrate how a transparent sphere warps light at different refractive indices. Notably, a glass ball (ior 1.5) seems to turn the image upside down, which is physically accurate.

As an aside, I would like to note that while my submission in CSE 167 also included transmission, I have since improved upon it by handling total internal reflection. Interestingly, this significantly improved render times in certain cases. Normally, one ray becomes two rays: one reflected and one transmitted. But under total internal reflection, we only need the reflected ray. So one ray becomes one ray, which lessens the exponential explosion.

Scene 8, Refractive Sphere: ior 1.015, 3840x2880

Scene 8, Refractive Sphere: ior 1.5, 3840x2880

Just to experiment, I also rendered a few scenes of the Stanford dragon with varying refractive indices and transmittance.

Scene 10, Stanford Dragon: ior 1.015 transmittance .37 .74 .47, 3840x2160

Scene 10, Stanford Dragon: ior 1.5 transmittance .37 .74 .47, 3840x2160

Scene 10, Stanford Dragon: ior 1.5 transmittance .95 .95 .95, 3840x2160

It’s interesting how chaotic/noisy the final image is, as the refractive index is decently high and the model is quite complex/layered. There are also some dark spots near the tail caused by maxdepth being too low, but increasing maxdepth exponentially increases render times. The images above are already rendered with maxdepth 9.

Anti-aliasing

I added a sampler command for scene files with two available options: sampler basic and sampler rgss. With sampler rgss, the program uses Rotated Grid Supersampling (RGSS), which involves shooting 4 rays per pixel in a rotated square, which (loosely) creates a rotated grid. The grid is not perfect, but that could be viewed as a positive. Perfect grids can lead to visual artifacts when certain patterns align with the grid.

All of the images above have been rendered with sampler basic (no anti-aliasing), so below is a comparison between sampler basic and sampler rgss using scene 5. The top half of the renders are excluded because they are simply black.

Scene 5: sampler basic above, sampler rgss below, 640x480

Gamma Correction

I also added a colorspace command for scene files which allows input/output colors to be in either linear (default) or sRGB. The original assignment required not performing gamma correction. However, PNGs assume sRGB values, which leads to linear outputs looking harsh/wrong. But since the scenes used values based on those linear outputs, simply adding colorspace output srgb without tweaking the values looked quite odd, as shown below. Note that scenes 8–10 were configured with colorspace input srgb and colorspace output srgb.

Scene 6: colorspace input linear colorspace output linear, 3840x2880

Scene 6: colorspace input linear colorspace output srgb, 3840x2880

Scene 6: colorspace input srgb colorspace output srgb, 3840x2880

I implemented a Bounding Volume Hierarchy along the lines of Pharr, Jakob, and Humphreys’s description in Physically Based Rendering 4e, alongside an efficient intersection algorithm for axis-aligned bounding volumes given by Ericson’s Real-Time Collision Detection.

A comparison of render times can be found below. The program was compiled with MSVC using /O2 optimization and /openmp paralellization, and it was run on my laptop’s Intel i7-12700H.

Scene render times with BVH vs. without BVH. Notice how using the BVH was actually slower for very simple scenes, since tree traversal is slower than naive for-loops over few elements.

CSE 167 image-grader report. (The high accuracy of 0-10 hot pixels was achieved using precise intersection algorithms described by Wachter, Binder (2019) and Haines et al. (2019), both of which are found in Ray Tracing Gems)

Using the BVH, I was able to render the following scene of a reflective Stanford dragon in a highly reflective box, just for fun.

Stanford Dragon: shininess 70 specular .7 .7 .7, 3840x2160, 2m06s

Transmission

I added transmittance and ior (index of refraction) commands. The implementation is imperfect, as there are some effects missing (e.g. total internal reflection, intersections, shadows etc.). However, the effect is still decently realistic. For example, the following images demonstrate how a transparent sphere warps light at different refractive indices. Notably, a glass ball (ior 1.5) seems to turn the image upside down, which is physically accurate.

Sphere: ior 1.015, 1920x1440, 0m15s

Sphere: ior 1.5, 1920x1440, 0m15s

Just to experiment, I also rendered a few scenes of the Stanford dragon with varying refractive indices and transmittance. These scenes were pretty taxing, and I actually had to implement an approximate squareroot formula to significantly increase performance. Though std::sqrt out-performed my approximation in standalone tests (modern CPUs have a built-in squareroot unit), it drastically slowed down under high volume. I’m not entirely sure why, though.

Stanford Dragon: ior 1.015 transmittance .37 .74 .47, 3840x2160, 16m32s

Stanford Dragon: ior 1.5 transmittance .37 .74 .47, 3840x2160, 26m59s

Stanford Dragon: ior 1.5 transmittance .95 .95 .95, 3840x2160, 26m32s

It’s interesting how chaotic/noisy the final image is, as the refractive index is decently high and the model is quite complex/layered. There are also some dark spots near the tail caused by maxdepth being too low, but increasing maxdepth exponentially increases render times.

Gamma Correction

I also added a colorspace command for scene files which allows input/output colors to be in either linear (default) or sRGB. The original assignment required not performing gamma correction. However, PNGs assume sRGB values, which leads to linear outputs looking harsh/wrong. But since the scenes used values based on those linear outputs, simply adding colorspace output srgb without tweaking the values looked quite odd, as shown below. Note that all of the previously shown images were rendered with colorspace output srgb.

Scene 6: colorspace input linear colorspace output linear, 1920x1440, 0m01s

Scene 6: colorspace input linear colorspace output srgb, 1920x1440, 0m01s

Scene 6: colorspace input srgb colorspace output srgb, 1920x1440, 0m01s