WIZARD WEI-CHE HSU
Hi! I'm Wizard - an interactive engineer (C++, Unity)
focusing on computer graphic, gameplay, and location-base entertainment.
Hi! I'm Wizard - an interactive engineer (C++, Unity)
focusing on computer graphic, gameplay, and location-base entertainment.
Challenge
It is an ETC project team working with the MuseumLab at the Children’s Museum of Pittsburgh to develop a mixed-reality installation inspired by the chain reaction machines found in the cartoons of Rube Goldberg. The experience will bring to life elements of the machines that would typically be impossible within an enclosed, public setting.
Design and Gameplay
We used the Augmented Reality Edge Network Architecture developed by the Conix Research Center to build the Rube Goldberg cartoon - a simple way to send snail mail. The multiple museum guests could simultaneously use their mobile devices to experience magic moments of surprise and humor that are core elements of Rube's original designs. For example, when the guest triggers the bird cage's open in the physical space, the virtual bird would fly across the room. We explore and deliver an interconnected series of AR interactions that contextually map onto and react to the museum space.
Notable Contributions
1. Designed the story and interaction of the Rube Goldberg machine cartoon
2. Deployed the hardware, furniture, wires of the installation in the physical museum space
3. Built networking between the mobile devices and Raspberry pi to trigger a series of animations at the specific time
4. Refined the mapping of virtual objects to the physical space and the animation duration to create the magic cartoon
Challenge
The project goal is to explore how VR interactive visualizations can facilitate music theory engagement for musically-interested university students. Our prototype goal is to foster musical curiosity by asking better musical questions. We define musical curiosity as a self-driven pursuit, reflection, or exploration of music. An example of a musical question is, “Why does this piece of music sound and feel this way?”.
Design and Gameplay
We created three prototypes exploring chord structures, chord progressions, and chord tonalities. Each prototype yielded a series of successes and flaws in relation to how well the experience a) conveyed the musical concept, b) leveraged the unique affordances of VR, and c) abstracted the musical concept. Our third prototype yielded a valuable opportunity to culminate all of our learnings into a singular experience, and it suffices as a capstone and testimony for all that we have learned throughout our journey navigating, charting, and discovering the crossroads between VR, music theory, and transformational design.
Notable Contributions
1. Designed the guest journey in the VR world
2. Implemented gameplay, interface, and physical mechanism in the experience
3. Created immersive panoramic scenes adding particle effect
4. Developed curve rendering tool with Cubic Bezier Curve for the accordion
4. Integrated visual (images, 3d models, shader, material) and audio (background music, minor and major chords, interface feedback sound) assets into Unity
Challenge
Tell a compelling, meaningful, interactive story where the multiple guests playing remotely and simultaneously to interact in order to bring the story to a fulfilling climax.
Story
Alice, a gloomy girl, disguised herself due to peer pressure. One day in the dream, Alice's inner truth begins to guide "herself" to destroy the mirror reflecting the false self. Alice starts the journey of finding the most important thing for her...Alice, a gloomy girl, disguised herself due to peer pressure. One day in the dream, Alice's inner truth begins to guide "herself" to destroy the mirror reflecting the false self. Alice starts the journey of finding the most important thing for her...
Design and Gameplay
Mirror is a two players game with networking. Player1 is a gloomy Alice who got lost in reality and can smash mirrors but cannot see mirrors that reflect her false self. Player2 is the inner true who can see the mirrors and is guiding Player1. Two players should collaborate to destroy all the mirrors to go through the story. Player1 can keep talking with Player2. On the other hand, Player2 only can write messages on the blackboard to send message to tell Player1 where the mirrors are. We designed to make players engage as the character - keep talking to herself to find her true self.
Notable Contributions
1. Design storyboard, cut scenes, and the gameplay for two players storytelling game
2. Implement networking with Photon and message writing
3. Set visual effect, lighting, and the transition of post-processing in each stage
4. The intense collaboration with artists to import the models for surrounding settings
Challenge
Create a game focusing on novelty, social, and accessibility. Make guests of all ages laugh, make them cry, make them think, make them go WOW.
Story
Welcome to Gnomes in a Robe, where guests will be playing as three gnome brothers working together to obtain a wizard license at the Bureau of Viable Wizards. The license exam is mainly designed for humans, so the gnomes can’t even reach the top of the desks! Work together with your brothers to somehow pass this exam and become a viable professional wizard!
Gameplay
Gnomes in a Robe is a three players cooperation game. Three players play as three parts respectively and should work together to use the spells to solve the puzzles:
Top Gnome: The one only can read tasks instruction: the dialogue with NPC, signs, and maps. Read them for the other two players and then teach the middle gnome how to cast the spell.
Middle Gnome: Cast spells by drawing the correct shape to solve puzzles.
Bottom Gnome: Control walking around and the direction of casting spells to go through obstacles.
Notable Contributions
1. Design the gameplay, game flow, and puzzle design with the other programmer and two artists
2. Implement physics mechanism: flying and collide detection, a boat floating and moving by the wind
3. Connect Unity with mobile controllers to set up different UI and data transitions for each part
4. Create visual effects with the particle system, shader, and VFX for the magic spells
5. Set the game scene, background music, UI, object animation, and scene transition
Challenge
Create a consistent, highly interactive, and engaging experience that lets naïve guests feel like they have a lot of freedom in their choices. The player only can control the guest in the game by face and voice. The game cannot use direct introductions to guide players; it should only use story, visual design, and sound design to let players know how to play and complete the game.
Design and Gameplay
The player is a vampire and then follows the monologue and the conversation of the human to attack them to get the power. The player would use the head direction detected by the Webcam to control the vampire's moving direction, stop, and run. When Webcam detected the player opening their mouth wide, it would trigger the "Attack" mode to drink human blood. After attacking several people, the player can speak the spell "Bat" to transform into the bats and fly, providing various gameplay and increasing the interest curve during the game.
Notable Contributions
1. Design the monologue indirect control and the time to play it in the game with the sound designer
2. Implement the voice recognition, visual effect, and physical mechanism for the bat transformation
3. Create the appealing start animation and the drink blood animation with Cinemachine and Post-Process
4. Create the particle system and simple Shader for setting the atmosphere of the game
Challenge
The game is fun to play with, preferably one that can be implemented simply.
Design and Gameplay
Two players control the cake ragdoll's left leg and the right leg respectively by mobile phones. They need to communicate with each other to go through the different Food Challenge to reach the goal as soon as possible. We set the ridiculous, funny, and adorable food theme to meet the "Fun-Focus" virtual world.
Notable Contributions
1. Design the gameplay, game flow, and level content with the other programmer and two artists
2. Implement ragdoll funny appearance, physical movement, and jump
3. Connect Unity with AirConsole and optimize data transition between them
4. Create the mobile phone user interface and design the experience with HTML/CSS/JS
5. Set the scene, skybox, food models with intense collaboration with artists.
6. Create start animation, smooth camera transition, and sound effects to enhance the appeal
Challenge
Create the interactive world to present theme - "The Guest Helps Character A Who is Afraid of Character B"
Design and Gameplay
We use visual models, sound effects, maze design, dark & light to create a world true to the "helping" theme in this game. The player uses a mouse click to trigger the Holding Hand and Moving Box to solve the maze and guide Character A (Blue Character) to escape from negative emotions.
Notable Contributions
1. Implement the physical mechanism of the pushing box, smooth guest moving, and CharacterB following
2. Implement the trick connection like pressing the button, then the slope show up with scripts and animations
3. Set the tone and atmosphere of each stage with light position, intensity, and post-process
4. Refine the model and camera moving speed to follow the pace of sound effect
Design and Gameplay
This game is made for round 0 of Building Virtual World class at Entertainment Technology Center. Round 0 aims at familiarizing students with Unity. I aimed to develop the sandbox game to allow players to explore Unity and use their creativity to create their world. I make two sandbox worlds; in the first one, players can control the time like pause, rewinding, and slow downtime. In the second space, players can manipulate gravity to experience different gameplay.
Notable Contributions
1. Individually completed a game in under 7 days, while attending class and doing other assignments
2. Combine 3D physical mechanism, gravity control, time control, and camera smooth moving, and my gameplay design knowledge to create a high level of interaction world that allow player to have the freedom to explore the sandbox
3. Polish visual effect by light rendering, particle system, and post-process in the Unity
Sampled triangle coverage by splitting each triangle into two triangles with a common horizontal base. In this algorithm, the navigation occurs along the lines contained in the triangles which increases the efficiency of the algorithm.
Furthermore, extended the rasterizer to anti-alias triangle edges via supersampling and averaging the sample values to smooth out the pixels shown in the images. In response to the user changing the screen sampling rate, the triangle edges become noticeably smoother with larger sample rates. The implementation is using a sample buffer vector to store each pixel's color components and then resample the supersampled results buffer (average their colors together) to obtain sample values for each pixel.
Implemented Simple Alpha Blending which is a method of displaying transparent elements in the SVG specification. This tasks focuses on pre-computed alpha that the formula for computation are based on the current value on the screen and the incoming value, creating a blend between the two using the alpha value(weight) specified.
Mapped a texture on a surface. Given location (x,y), the texel(texture coordinate) is obtained by normalizing the given coordinates first and then mapping in the texture dimensions. For each covered sample, the color of the image at the specified sample location would be computed using Bilinear Filtering of the input texture which would cause anti-aliasing.
Next, adding Trilinear Filtering to improve the anti-aliasing. My implementation first is generating mipmaps (Minification and Magnification) of desired resolution which are stored in a data structure. Based on the dimension of the texture, we computed the appropriate level at which to sample from the mip-hierarchy. As image elements shrink on screen, to avoid aliasing the rasterizer should sample from increasingly high (increasing pre-filtered) levels of the hierarchy. Finally, the values at any given pixel are interpolated by trilinear interpolation. That means we sampled more than one pixel at any given location and then average the value of all the locations to create smoother images,
Implemented three types of bevels: Vertex Bevel, Edge Bevel, and Face Bevel. The bevel action creates a new copy of the selected element that is inset and offset from the original element. Clicking and dragging on an element will perform a bevel; the horizontal motion of the cursor controls the amount by which the new element shrinks or expands relative to the original element, and the vertical motion of the cursor controls the amount by which the new element is offset from the original element.
Implemented a variety of commands used to alter the connectivity of the mesh (for instance, splitting or collapsing edges). These commands are applied by selecting a mesh element (in any mode) and pressing the appropriate key, as listed below. Local mesh editing operations include:
- Edge Flip: Rotates the edge in the anti-clockwise direction by one vertex.
- Edge Split: Split edge at its midpoint, and the new vertex is connected to the two opposite vertices.
- Erase Vertex: Replace a vertex together with all incident edges and faces with a single face.
- Erase Edge: Replace edge with the union of the faces containing it, producing a new face.
- Edge Collapse: Replace edge with a single vertex. This vertex is connected by edges to all vertices previously connected to either endpoint of e. Moreover, if either of the polygons containing e was a triangle, it will be replaced by an edge.
- Face Collapse: Replace face with a single vertex. All edges previously connected to vertices of face are now connected directly to the vertex.
All the operations consider the edge case of selecting the boundary vertex, edge, of face.
Implemented three schemes of geometry subdivision which upsample the low-resolution polygon mesh for display and simulation:
- Linear Subdivision: Each polygon in the selected mesh is split into quadrilaterals by inserting a vertex at the midpoint and connecting it to the midpoint of all edges. New vertices are placed at the average of old vertices and old vertices remain where they were. However, it will results in a mesh that is purely quad.
- Catmull Clark Subdivision: Just as with linear subdivision, each polygon is split into quadrilaterals, but this time the vertex positions are updated according to the Catmull-Clark subdivision rules, ultimately generating a nice rounded surface.
- Loop Subdivision: Need to operate Triangulation first and then each triangle is split into four by connecting the edge midpoints. Vertex positions are updated according to the Loop subdivision rules.
The mesh is resampled so that triangles all have roughly the same size and shape, and vertex valence is close to regular (i.e., about six edges incident on every vertex). The algorithm to be implemented is based on the paper Botsch and Kobbelt, “A Remeshing Approach to Multiresolution Modeling” (Section 4), and can be summarized in just a few simple steps:
1. If an edge is too long, split it.
2. If an edge is too short, collapse it.
3. If flipping an edge improves the degree of neighboring vertices, flip it.
4. Move vertices toward the average of their neighbors.
Repeating this simple process several times typically produces a mesh with fairly uniform triangle areas, angles, and vertex degrees. However, each of the steps deserves slightly more explanation.
The number of triangles in the mesh is reduced by a factor of about four, aiming to preserve the appearance of the original mesh as closely as possible. The simplification method simplifies a given triangle mesh by applying quadric error simplification, was originally developed by Michael Garland and Paul Heckbert, in their paper Surface Simplification Using Quadric Error Metrics.
Randomly generates uniformly distributed rays from the camera. Each ray can be expressed in the form: P = o + t * d, where o is the origin, t is a parameter to reach point P, and d is the direction. The program transforms the ray from a point in the image plane into the world space through a matrix multiplication. For each ray, detects the first intersection with a primitive object including triangles (using Möller-Trumbore algorithm) and spheres in the scene.
In order to accelerate rendering, implemented the bounding volume hierarchy (BVH) by spatially partitioning primitive objects into bounding boxes. Instead of ray tracing each individual primitive, this acceleration structure allows us to test for ray intersection against groups of primitives (axis-aligned planes).
The BVH algorithm is defined recursively, and takes in a vector of primitives and a max leaf size. When the node is a leaf, we simply return that bounding box and its list of primitives; otherwise, we split along the axis of the lowest Surface Area Heuristic, which is found by comparing x, y, and z axis values. Then, we partition the primitives into two vectors based on which axis the primitive centroids are on. With these left and right vectors, we can recursively call the construction algorithm, setting nodes accordingly.
Implemented accurate shadows according to light(hemisphere light, direction light, spot light, and point light) in the scene. The algorithm is checking whether a ray originating from the hit point, and traveling towards the light source (direction to light) hits any scene geometry before reaching the light.
Implemented Monte Carlo Rendering that considering materials BSDF and the complicated light paths, bouncing off many surfaces before eventually reaching the camera. Simulating this multi-bounce light is referred to as indirect illumination, and it is critical to producing realistic images, especially when specular surfaces are present.
For the BSDF, implemented Lambertian for diffuse reflections, Mirror for perfect specular reflection, and Glass for both reflecting light and transmit light. We use Snell’s Law and Schlick’s approximation to compute a transmittance BSDF in the Glass.
For the Ray Tracing, simulated multiple bounces using the Russian Roulette algorithm in order to avoid spending time evaluating function for samples that make a small contribution to the final results. The structure is:
(1) Randomly select a new ray direction
(2) Potentially terminate the path (using Russian Roulette)
(3) Recursively trace the ray to evaluate weighted reflectance contribution due to light from this direction.
Simulated the effects of de-focus blur and bokeh found in real cameras by modifying the generated rays from the camera according to member parameters - aperture and focal distance. Focal distance represents the distance between the camera aperture and the plane that is perfectly in focus. Setting aperture could randomly choose the ray origin from an square centered at the origin and facing the camera direction. Now it’s as if the same image was taken from slightly off origin. This simulates real cameras with non-pinhole apertures: the final photo is equivalent to averaging images taken by pinhole cameras placed at every point in the aperture.
Implemented an infinite environment light, a light that supplies incident radiance (really, the light intensity dPhi/dOmega) from all directions on the sphere. Rather than using a predefined collection of explicit lights, an environment light is a capture of the actual incoming light from some real-world scene; rendering using environment lighting can be quite striking.
Furthermore, implemented an importance sampling scheme for environment lights to sampling directions towards the directions for which incoming radiance is the greatest. In the real world, most of the energy provided by an environment light source is concentrated in the directions toward bright light sources. For environment lights with large variation in incoming light intensities, good importance sampling will significantly improve the quality of renderings.
Implemented the function allow users to make simple animations by translating, rotating, or scaling the mesh in the scene. I implemented Cubic Hermite Curve over the unit interval first, which evaluates a spline defined over the time interval given a pair of endpoints and tangents at endpoints. And then use this method to evaluates a general Catmull-Romspline at the specified time in a sequence of points (called “knots”).
Implemented these basic kinematics to allow user to define skeletons, set their positions at a collection of keyframes, and watch the skeleton smoothly interpolate the motion. A skeleton can be posed by selecting a joint and changing its pose i.e., rotating the joint.
In this project, I implemented a Capsule-Radius Linear Blend Skin method, which only moves vertices with a joint if they lie in the joint’s radius. This method could link the skeleton to the mesh in order to get the mesh to follow the movements of the skeleton
Implemented Inverse Kinematics, which will move the joints around in order to reach a target point. In this task, I implemented an iterative method called gradient descent in order to find the minimum of a function, more specifically, using a technique called Jacobian Transpose to update the angles along a single axis. And then extended it to 3 dimensions and support multi-target IK, which allows users to create a series of joints and get a particular joint to move to the desired final position users have selected.
Implemented a collection of non-interacting, physics-simulated, spherical particles that interact with the rest of the scene.I n this task, I use the simple forward Euler which not concerning with stability or energy conservation for our particle system. Forward Euler simply steps our position forward by our velocity, and then velocity by our acceleration.
Next, I used ray-tracing capabilities to find collisions along particles’ paths. If the ray intersects with the scene, computing when the particle would experience a collision. When finding a collision, assume all particles collide elastically - that is, the magnitude of their velocity should be the same before and after the collision, and its direction should be reflected about the normal of the collision surface. Repeated the ray-casting procedure(if the particle collided with the scene again?) in a loop until have used up the entire time-step.
Challenge: Design and implement the pipeline for creating realistic cave
Features: Use several layers of noise in volumes and points to add details of erosion and joints for cave walls
Tools: Houdini 19 / Unreal Engine 4.25
Challenge: Use fluid system, collider, light setting, and camera to render a delicious chocolate animation
Tools: Houdini 19
Challenge: Create various realistic visual effect with smoke pyro modules: candle fire, ground fire, wheel fire, and tornado
Tools: Houdini 19
Challenge: Create various magic effect and shader by Unity particle system and Visual Effect Graph for WebGl platform
Tools: Unity
Challenge: Set the light and post-process to create appealing, touching, and meaningful environment. Use the atmosphere of the environment to represent the character's mood change and allow the players engage in the story.
Tools: Unity
Challenge: Create various particle visual effect: Magical font, Colorful Lines
Tools: Houdini 19
Challenge: Use Scotty 3D which is a 3D modeling and rendering application implemented by myself to create 3D models and animation.
More details about implementing these tools are in the COMPUTER GRAPHICS.
Theme: Snoopy modeling, Glass and Mirror Material, and night environment light
Tools: Scotty 3D, C++
Introduction : Create intelligent flying hummingbirds that can navigate to flowers, dip their beaks in, and drink nectar by Reinforcement Learning.
Features : Craft a training environment, train neural networks, and set related machine learning parameters to stimulate the hummingbirds' behavior.
Reference: https://learn.unity.com/course/ml-agents-hummingbirds
Introduction : The game that player control the rolling ball to collect all crystals in the dark cave.
Features : Implement 3D physical mechanism and gameplay controller with Blueprint. Develop own levels by editing geometry and using lighting system to refine the atmosphere.
Reference: https://www.udemy.com/course/unrealblueprint/
Introduction : Reproduce the game that player has to avoid the bullet which is produced with the music tempo.
Features : Implement 2D physical mechanism and audio source control.
Reference: https://gamejolt.com/games/bullet-waltz/22372