Our group implemented participating media, hair BSDF and volumetric photon mapping to render this image of a feather hidden inside an amber. This image was selected as the runner-up in the rendering competition in the Rendering Algorithms course at Dartmouth. The theme for the rendering competition was "It's what's inside that counts".

If you want to learn about the technical details, the full project report is here.

My teammate, Yang Qi, was responsible for adding participating media, volumetric path tracer, microfacet BSDF and hair BSDF to our base renderer. I worked on implementing a progressive volumetric photon mapper and directional lights to achieve the caustic and light beam effects we wanted.

The photon mapper enabled us to render the caustic effect within reasonable time. Here is a comparison between path tracer with MIS (PT), photon mapper (PM), and stochastic progressive photon mapper (SPPM), rendering our caustics test scene. As you can see, photon mapper provides much faster convergence rate on caustics, and SPPM can reduce the bias on caustics caused by blurring, producing much defined edges on caustics.

The goal of this project is to implement the PIC-FLIP fluid simulation from the
paper Animating Sand as a Fluid by Yongning Zhu and Robert Bridson. This is a final project for the Computation Method for Physical Systems course at Dartmouth.

This is a group project with a group size of 2. I worked on coding marker particles, implementing PIC/FLIP algorithm, figuring out how to enforce boundary conditions and adapting the cell-centered functions to work with MAC grid.

Our group also tried comparing the results of using PIC, FLIP and combined PIC/FLIP. From the images below, we can see that the PIC method has a subdued wave, while the FLIP method shows many fluid particles breaking apart from the main fluid body. This demonstrates that PIC tends to dampen fluid due to the loss in velocity during interpolation. On the other hand, FLIP can preserve the lost energy from the PIC method, but is often noisy and can become unstable.

Our biggest challenge in the project was to figure out how to enforce boundary conditions on cell-centered grids in the projection step. We found going through Chapter 5 of Fluid Simulation for Computer Graphics book really helpful to us. It helped us understand the how the boundary conditions work mathematically. After a lot of trial and error we found that adapting the cell-centered velocity field to the face-centered velocity field by interpolation worked the best, since most of the equations makes more sense with face-centered grids.

If you want to learn more about how to simulate fluid using PIC/FLIP, check out our course report here.

I want to write down what our group has learned in our final project for the Computational Photography course. We wanted to explore how to capture a 360° environment map in real world and use that to perform image-based lighting in Maya. We tried two approaches, the mirror ball unwrapping approach and the panorama stitching approach. I will mainly talk about the panorama-based approach since that is what I worked on.

I mainly followed the slides and problem sets in MIT's 6.815 Digital Computation Photography course to implement the auto-stitching part. The auto-stitching consists of several stages to produce high quality corresponding feature point pairs between two images. It mainly consists of 4 stages: corner detection, descriptor creation, correspondence search and RANSAC.

In the first stage, we use the Harris Corner Detector to produce the feature points, which involves find structure tensors, calculating responses and find the local maximums of the responses. We also need a way to describe the neighborhood of a feature point, which is called “descriptors”, to help us measure the similarity of two feature points. We just used a simple patch descriptor, which is just a $k \times k$ patch around a feature point with some gaussian blur and values normalized. After getting the descriptors, we simply used L2 distance to determine how "close" any two descriptor are and kept the descriptor pairs whose distance is under a threshold. We also used the second-best test to filter out too ambiguous matches. Finally, even after the second-best test, usually our descriptor pairs will still contain some outliers. We used RANSAC to filter out these final outliners.

RANSAC is really powerful for eliminating outliners in descriptor matches, as demonstrated in the image below. The green lines shows the inlier matches while the red lines shows the outliner matches. The blue lines shows the matches selected by RANSAC to calculated the final rotation matrix $R$ between the two images.

After being able to stitch two images together with autostitching, we still need to figure out how to compose a full panorama by stitching N images. We can either stitch the images pair by pair locally, or use some global optimization approach. Due to the time constraint, we decided to use the local approach. We found that if you first stitch all pitch angles of a single yaw, then stich images of different yaw angles, you will tend to get much less ghosting in the output image.

And this is how we automatically generated the panorama image above given a series of individual images. Compared to the mirror ball unwrapping approach, the panorama-based approach can produce environment maps with much higher resolution. However, when using the panorama-based approach, the north and south pole area will cause a lot of problems. For example, if the sky is of a single color, the corner detector won't be able to find any corner and the whole pipeline will fail. To generate the final image combining a real-world photo with a virtual scene, we had to use some tricks (such as adding a hat to the mirror ball) to hide the holes in the north and south pole area.

We further compared these two methods in this table:

And this is basically all what we have learned in this project! I want to thank my teammates, Josephine Nguyen and Yang Qi, for their amazing work. Without them I wouldn't have the confidence to tackle such a broad project during COVID. I also had a lot of fun writing a job system to parallelize our C++ image processing code, but that will probably need its own post. Also, just for reference, Greg Zaal from HDRI Haven has a great blog post explaining what equipment he used to capture environment maps. I wish our media center also had a slide like his that can let the camera rotate around the entrance pupil...

Foresight is a VR camera system + mo-cap system aiming to expedite story and shot revision cycles in filmmaking. As a team, we tried visualizing different genres of scripts and built our own tools and pipelines along. If you want to check out our development team and the dev blogs, please visit our project website.

In this project, my main focus is the VR camera system and I worked on a wide variety of tasks ranging from developing tools within Unity to implementing specific camera effects. Here is a brief video about the features in the camera system.

My Contributions

• Recreated camera properties and moves such as adjustable focal lengths, tilting/panning and dolly in VR.
• Worked closely with the UI/UX designer to iterate on the camera panel UI and user interactions such as modifying track
• Wrote and debugged shaders to implement a gaussian-based depth of field effect with tweakable aperture based on Skylanders SWAP Force’s method
• Coded the serialization of camera motion and a tool to save/load camera motion

Special thanks to my teammates Angela, Arnav, Shera and Varun for their exceptional work in this project and valuable feedback to the camera system. Also I would like to appreciate my faculty advisors, Mo and Chris.

Scotty3D is a C++ 3D graphics toolkit including a mesh editor, a path tracer and an interactive animator. This is part of the coursework of the CMU 15-462/662 Computer Graphics course and the base code is available on GitHub.

The rendered image above used art assets by Rummy^[1] and Breanne Millette^[2].

PathTracer

• Coded key features for a Monte Carlo path tracer, including the BVH builder/traverser, the path tracing algorithm and importance-based environment map sampler
• Implemented extra techniques such as multi-jittered sampling
• Extended the Collada parser and the material class to support rendering textured surfaces

MeshEdit

• Finished key functions for a halfedge-based mesh editor
• Coded local operations such as edge collapse and global operations such as isotropic remeshing

Animation

• Finished key features of an interactive animator with skeleton kinematics
• Implemented "an integrator for the wave equation across the mesh" ^[1]

Deep is an immersive CAVE experience in which two guest will become the crew of a submarine and use their “vacuum” tools to collect lost treasures in danger underwater area. The guests will traverse through an undersea tunnel, dodge dangerous mines, collect valuables such as gold coins and pearls and reach the wreckage of a sunken pirate ship.

About this project

• Three-week BVW project with a team of 5.
• I was mainly the graphics programmer and the producer in our team.
• Tools used: Unity, Substance Designer, Photoshop

Key Activities

• Implemented a screen-space underwater shader featuring depth fog and simulated surface scattering of sun light.
• Created the cone tornado special effect shader for the vaccuum tools.
• Built a cone mesh generator to avoid creating different cone meshes from external tools.
• Tuned parameters for effects such as bloom and SSAO in the post processing stack to better convey a mysterious and adventurous feeling.

Screenshots

LabX aims to simulate chemistry and physics experiments in Mixed Reality for educational purposes. I implemented and optimized the whole real-time, screen-space fluid rendering pipeline of LabX and achieved an average frame time of 9.02 ms on GeForce GTX 1060 6GB when simulating 20K particles at 1080p.

Key Activities

• Rendered smooth particle-based fluid in real time with the screen-space filtering approach (20K particles, 1080p, 9.02ms average frame time on GeForce GTX 1060 6GB)
• Calculated and applied particle anisotropy efficiently using compute shaders to improve surface smoothness
• Implemented and compared two common filtering approaches, screen-space curvature flow and bilateral filtering
• Identified performance bottlenecks with tools such as Nsight, then reduced bilateral filter running time by up to 27.9% using techniques such as utilizing shared memory and employing loop unrolling

Pipeline Performance Metrics

The fluid rendering pipeline was tested in two different test scenes. The first "Single Dam Break" scene only tests the rendering of simple fluid, while the second "Fluid–rigid body Interaction" scene trials the performance of the pipeline during the interation between 3D fluid and rigid bodies.

Performance of the single dam break scene
1080p, 3.1K particles, using compute shader bilateral filtering with a diameter of 9 pixels

Performance of the fluid-rigid body interaction scene
1080p, 20.2K fluid particles, using adaptive radius bilateral filtering with a maximum diameter of 41 pixels

Buoyancy Lab provides a VR environment that can simulate and visualize hydrostatic forces. It is designed to make learning physics concepts more engaging to middle school students. The lab has 3 modes: the inspect mode, the build mode and the play mode.

Key Activities

• Implemented physics based hydrostatic force simulation with the formula $ \mathrm{d}\vec{F}=\rho gz\mathrm{d}S\vec{n} $. ^[1]
• Exploited parallelism in the algorithm and achieved up to 4x frame rate with a job system.

Project Bastion is a “shoot ’em up” game happening in 3D space: players build their own self-defense bastions and destroy others’ bastions and compete for resources. You need to maneuver through the dense barrage created by different kinds of enemy projectiles and try to locate and destroy the energy core somewhere inside the bastion to disable this devastating defense machine. Apart from challenging yourself in various pre-built levels, you can also unleash your creativity in the bastion builder mode. The bastion builder mode allows you to build your own self-defense bastions with 8 different kinds of bastion modules. You can fill the space with bullets with the Stormer module, or create a temporal protective shield using the Defender module. Build your invincible bastion and challenge your friends to destroy it!

Key Activities

• Designed the core gameplay mechanics and implemented behaviors for more than 5 kinds of enemy turrets
• Designed and coded the bastion builder system with serialization/deserialization and a flexible UI
• Utilized custom shader to achieve special effects such as holograms
• Reduced frame time by about 50% with techniques such as batching and instancing

Intentions

Many of the first wave of AR games, like the pet simulator AR Dragon and the puzzle game Conduct AR!, treats players as passive viewers of the virtual objects with limited agency to interact with the virtual world. However, one AR game called Pigeon Panic! is different because it uses player's movements in the real world directly as inputs to the amusing reactions of some scared virtual pigeons. This idea of a game world reacting to players' physical movements, together with my interests in traditional hardcore STGs like Strikers 1945, gave birth to the core mechanic in Project Bastion: players shoot at a bastion equipped with turrets while dodging incoming bullets.

After a short prototype stage, my work shifted to designing and implementing a variety of turrets. My first priority was to recreate the most important element of pleasure in traditional bullet hell games: the element of ilinx. The thrill of vertigo comes from the illusion that we become more vulnerable under the overwhelming bullet barrage. Thus the turret called "Stormer" was firstly created, which fires rapidly around you instead of towards you. Other kinds turrets was added to develop different rhythms. The red bullets fired by Crimson require instant reaction from the player, while the sphere energy shields generated by Defender encourage patience and push players to switch firing priorities. A flexible bastion builder system was designed and implemented later first to shorten our team's iteration time, as it often takes at least half an hour to install this game to an iOS device. Later we also decided to expose it to players so that a player can design a customized bastion and invite his friends to destroy it.

AR Graffiti is an eco-friendly graffiti app that empowers artists to create street art anywhere, anytime. Choose a wall to paint on, pick up the spray can of your color, shake your device and just start painting! Made with Unity + Apple ARKit.

Key Activities

• Implement a customized shader with easily congurable properties to simulate realworld spray paints: more area will be covered if the user sprays at a greater distance
• Built a scalable art board system capable of redoing & undoing with Render To Texture and achieved signicant frame rate gain

Hanabi is a mesh particle system demo that simulates petals and grass with modern OpenGL. Users can generate petals with a click of mouse, feeling the beauty of nature. The grass and the light would also naturally follow your mouse.

Source code has been uploaded to GitHub.

Key Activities

• Utilized the geometry shader to generate individual grass blades. Calculate and pass in the world position of the cursor to make the grass blades "breathe" with the cursor.
• Implemented basic light shading with Phong Illumination Model. Improved performance with the half-way vector optimization.
• Implemented basic physics simulation for the petals.
• Optimized performance with methods such as GPU Instancing.

In the game, two players cooperate as Hacker and AI, trying to survive by turning different enemies into allies and taking advantage of the environment with their connect and disconnect abilities in the cyber world.

Key Activities

• Implemented 3 distinct enemy AIs based on finite state machine
• Designed and implemented two complementary player skills: Connect and Disconnect
• Design room layouts for the level and coded environmental puzzles such as the laser tower

Light Chaser is a top-down shooting game where two alien creatures fire light waves to illuminate their surroundings and explore the scene. During their exploration process, they also need to locate and neutralize their enemies from the dark. Light is a kind of wave, so it can naturally bounce between walls. The light source you fired can also be retrieved for a faster reload. Utilize your light waves to expand your range of view and restrict your opponent's vision.

Key Activities

• Designed and implemented the rules around the "light wave", such as light bouncing and light retrieving
• Utilized and optimized sprite-based 2D lighting to deliver a dynamic sense of exploration and a thrill of the unknown with hundred of lights blended on screen

Download & Install Instructions

Currently, Light Chaser only support the Windows platform. Please visit Light Chaser's itcho.io page to download the latest version of the game: https://lykavin.itch.io/light-chaser.
After downloading the zip file and extracting it, connect two controllers to your PC and double-click LightChaser.exe to launch the game. Common controllers such as Dualshock 4, Xbox One Controllers are all supported. Visit this link to view all supported controllers: http://www.gallantgames.com/pages/incontrol-supported-controllers.

Interaction instructions

You can always click on the "help" button in the main menu to learn about all the basic controls for this game:

• Use the left joystick to move around.
• Use the right joystick to aim.
• Use the right trigger button (R2 on DualShock controllers, RT on Xbox controllers) to fire a light wave.
• While playing, press the "Option" button on DS4 / "Back" button on the Xbox 360 Controller / "Menu" button on the Xbox One Controller to return to the main menu.
• While playing, you can also press the "Esc" key on your keyboard to close the game directly.

Intentions

This game was initially created at Global Game Jam '17. The theme for that year's GGJ was "waves". Intrigued by real-time lighting at that time, I proposed the idea of player firing "light waves" at each other in a dark environment in our initial brainstorm. After some discussions, our team settled on this idea and started prototyping. After thinking deeper about the property of light waves, I introduced light bouncing to make illuminating the environment easier. The ability to retrieve the light source was also implemented to achieve an input pattern of "fire a light wave, follow the light path and retrieve the light source". These all add up to a sense of exploration with a natural rhythm in this game.

Later, I realized that the excitement from exploration will diminish as the player brightens every corner of the map. To solve this problem, we allowed a player's light wave to also darken his or her opponent's vision. This mechanic also helped us build a sense of thrill, as the player's range of view will be constantly transforming and the uncertainty about the opponent's position will be constant.