WEEK 11 Unit Intro and Project Planning
This week, I began developing my project for the Advanced and Experimental 3D Computer Animation Techniques unit. The brief encourages us to create immersive experiences using animation, driven by storytelling and experimental design.
Initial Idea
My project explores the emotional contrast between adult life and childhood imagination. I plan to create a short story-driven animation combined with a 3D interactive memory space—where the user can experience fragments of childhood through stylized visuals and playful interaction.
Project Goals
- Use collage-inspired visuals to represent memory and emotion
- Contrast two emotional spaces: grey reality vs. colorful imagination
- Blend 2D storybook framing with interactive 3D scenes
- Leave users with a subtle but lasting emotional impact
Next Steps
- Finalize story structure and memory scenes
- Start collecting references on visual style and interaction design
- Begin building a simple prototype in Unreal Engine
WEEK 12 Plugins for Virtual Production — VCam
During this session, I explored the use of the VCam (Virtual Camera) plugin in Unreal Engine. It allowed me to control a digital camera in a 3D scene using a tablet, just like operating a real camera on set. I learned how to record camera movements in real time, including zooming, panning, and focusing.
This tool gave me a better understanding of cinematography in virtual production. It was exciting to see how we can block scenes and plan camera shots interactively in-engine. It also helped me realize how important virtual cameras are in previsualization and animation layout stages.
Project Progress – Character Design
I started designing the characters. I tried out a few different styles, and in the end, I went with a collage-paper look. It feels right for the story—not just visually, but emotionally too. In Your Diary, the main character’s memories, feelings, and daydreams are kind of messy and fragmented, like pieces of paper. Using a collage art style lets me show those layers of emotion visually, making the storytelling feel more vivid and deep.
I’m also planning to include the collage concept in the gameplay. Some of the mechanics and interactions will feel like assembling or peeling back layers, so players won’t just see a collage—they’ll actually take part in building it. I hope this makes the experience more engaging and helps players connect emotionally with the story.
WEEK 13 Exploring nDisplay and LED Wall Setup
In this class, I learned how Unreal Engine’s nDisplay system works for large-scale multi-screen or LED wall displays. We explored how to sync multiple viewports and render them in real time across different screens. I also learned how camera tracking works in this setup, which is essential for in-camera VFX.
This session showed me how virtual production blends digital and physical elements during live-action shoots. It made me appreciate the complexity behind setting up LED stages and how real-time rendering can transform traditional filmmaking. I now understand the pipeline and tools used in professional virtual production studios.
During the session, Serra also encouraged us to start thinking about how we want to present our final projects at the graduation exhibition. Although the final outcomes will be in video format, she reminded us that the exhibition experience is more than just playing a video on a screen. We were asked to consider how to create a more engaging and immersive experience for the audience—whether through physical setups, interactive elements, behind-the-scenes materials, or creative display design. This made me realize the importance of curating not just the animation itself, but also how it’s received and experienced in a public setting.
Project Progress – Character Frame by Frame animation
I’ve been thinking about how to create a visual style in Unreal Engine that blends both 2D and 3D elements. Since my project is closely connected to a picture book story, I didn’t want it to look like a typical fully 3D game. I wanted to keep a handmade, personal touch in the visuals.
While doing some research online, I found this super helpful tutorial explaining how to use the PaperZD plugin. It allows me to bring 2D sprites and animations into UE more easily, and I’m definitely planning to use it as the project moves forward.
I started working on the frame-by-frame animations for the character. It’s definitely a time-consuming process, but also kind of fun—especially since I haven’t done much drawing in a while. Getting back into it felt refreshing.
There’s something really satisfying about seeing the character come to life, one frame at a time. Even small movements, like blinking or tilting the head, take a lot of care to get right. But that’s also what makes it so rewarding—each frame adds a bit more personality and emotion.
I’m keeping the collage-paper texture consistent in the animations too, so the whole visual style stays unified. It’s a lot of work, but I think it’ll be worth it in the end.
Week14: Project Feedback
The Visitor by Antje Damm
A Grand Day by Samantha Cotterill
For the art direction, I chose a deliberately naive and childlike style. I found that this kind of slightly clumsy, simple aesthetic can actually create stronger emotional connections. It’s more relatable and accessible, which makes it easier for players of different ages and backgrounds to engage with the story on a deeper level.
This kind of visual approach also matches well with both the collage technique and the stylized 3D cartoon look I’m aiming for. Collage art focuses on bold shapes and clear colors, and 3D cartoon styles often exaggerate features in a fun, playful way. A more polished or realistic style would clash with that, so sticking to something a bit more naive helps tie everything together visually.
In terms of color, I went with bright, saturated tones to give the game a cheerful, picture-book feeling. These colors make characters and environments stand out more, and they help emphasize the childlike theme of the story. I want players to feel like they’re stepping into a memory—or even into a child’s imagination.
Bright colors are often associated with warmth, joy, and comfort, which is exactly the atmosphere I want to create. I hope that by combining this playful visual style with the emotional depth of the story, players will feel both visually delighted and emotionally connected as they explore the world of Your Diary.
Level 3 Scene
Scene 3 is the only level in the game that required extra 3D modeling, so I built the environment from scratch in Maya. After that, I used Substance Painter to create the textures, keeping the same collage style I used in the rest of the project.
Reference
Level 3 assets modeling and texturing
Even though it was more work, it was fun to bring this unique space to life. I made sure the textures still looked like pieces of paper cut and layered together, so the scene would feel consistent with the overall art direction. Having this one custom-built environment helped add a bit of surprise and variation to the game while still fitting in with the rest of the visual world.
Week 15: Mocap Breakout room and Resolume
This week’s session gave us hands-on experience with motion capture in the breakout room. We began by learning about the setup process, including putting on the mocap suit, attaching reflective markers to specific body joints, and calibrating the system for accurate tracking. It was important to make sure all markers were placed precisely, as any misalignment could affect the capture data.
Before the actual recording, we performed a T-pose to help the system recognize the default stance, and then did some basic test movements to check if all the markers were being tracked correctly. Once everything was set up, we took turns performing different actions—such as walking, turning, or gesturing—which were recorded by the mocap system and streamed into Unreal Engine.
I was impressed by how natural the recorded motion looked once it was applied to a 3D character. It really helped me understand the importance of clean data capture and how to direct performance for animation. This experience also gave me more confidence in integrating mocap data into future animation workflows.
Project Progress – environment building
WEEK 16 Projection Mapping — Mad Mapper
This week, we had the opportunity to learn MadMapper, a powerful tool for projection mapping. It allows us to take video or real-time visuals and project them onto physical surfaces in precise and creative ways. Serra guided us through the basic workflow—connecting a video output, creating projection surfaces, and mapping visuals accurately onto 3D shapes.
For our experiment, we chose a unique projection surface made from three plates stuck together. This irregular shape made the mapping process more interesting, as we had to carefully adjust the visuals to fit the curves and spacing of each plate. It taught us how to use masks and quads to align content correctly, and how different surface shapes can completely change the viewer’s perception of the visual material.
This session inspired me to think beyond flat screens when it comes to presenting animated work. Projection mapping offers an exciting way to extend animation into physical space, and I’m now considering how I might incorporate mapped elements into my final exhibition setup.
For interactive mini-games, I combined animation blueprints with scene elements. For example, I placed transparent cubes as colliders along the parents’ path. When a collision occurs, their speed is set to zero, triggering a turn-back animation via an event in their animation blueprint. After the animation plays, the cube is destroyed with a slight delay, allowing the characters to continue moving. Though the logic is simple, it effectively achieves the intended result.
In my game, I used the open-source plugin Paper ZD in Unreal Engine to create a visual style that combines 2D and 3D. It’s ideal for developing pixel-style or traditional 2D games. I hand-drew frame-by-frame animations for the character’s idle, walk, crouch, and turn-back actions. Each frame was exported as a PNG, then imported into UE to create Sprites and Flipbooks. I adjusted playback speed and looping, and integrated them into the character and animation blueprints.
Many thanks to the developers of the Paper ZD plugin for making such a powerful and accessible tool available to the community.
WEEK 17 Creating a 360° Video with Off World Live Plugin
This week, we explored how to produce 360° immersive videos using the Off World Live plugin in Unreal Engine. The process began with setting up the correct render settings to support equirectangular output, which is necessary for spherical video playback. We then integrated the plugin into our scene and configured the Off World Live camera rig to capture a full 360-degree view.
Using the Sequencer, we animated the environment and characters to create a short narrative loop. It was interesting to see how framing and composition work differently when the viewer has complete control over their perspective in the final video. After capturing the sequence, we rendered the footage and exported it for post-processing.
To finalize the video, we used Adobe Media Encoder to make minor adjustments and ensure the format was compatible with platforms that support 360° playback, such as YouTube or VR headsets.
WEEK 18 Project Progress and Feedback
The story follows a young woman navigating the monotony of adult life. One rainy evening, while eating a microwaved dinner after work, she questions the purpose of her years of effort. At that moment, her childhood diary begins to glow, pulling her into her past.
Players are then transported into her memories through a mini-game, where she relives moments like catching bugs with her grandfather, drawing in a pillow fort, and writing to her future self. These sequences were designed to emphasize warmth, curiosity, and freedom through stylized visuals and light interactivity.
Upon returning to reality, a younger version of herself steps out of the diary and begins to accompany her. This symbolic figure becomes a quiet yet powerful presence—bringing back the joy of splashing in puddles, the comfort of small celebrations, and the courage to face challenges with a lighter heart. In moments of pressure or creative block, the younger self reminds her to stay playful, curious, and unafraid. Through this companionship, she gradually rediscovers a more balanced, joyful way of living—where imagination and responsibility can coexist.
As Walt Disney said: “Growing old is mandatory, but growing up is optional.”
WEEK 19 Resolume Arena, VJing
Today’s session introduced us to the world of live visual performance through Resolume Arena. We started by exploring the basics of the software interface, learning how to import and layer video clips, apply real-time effects, and sync visuals to audio. It was exciting to see how visuals could be manipulated live, responding dynamically to rhythm and mood.
We also got an introduction to VJing, which is like being a DJ, but for visuals. Mixing video clips live, responding to music or mood, and creating visuals on the spot felt totally different from traditional animation work—and in a good way. It’s much more spontaneous and expressive.
A key part of the session was using a MIDI controller to interact with Resolume. By assigning buttons and knobs to video layers and effects, we were able to trigger changes instantly with physical input. It felt much more intuitive than using only a mouse and keyboard, and it made the experience feel like a true live performance.
Overall, it was a relaxed, hands-on session that sparked a lot of creative ideas. Whether for performance, installation, or just experimenting with visuals, Resolume feels like a great tool to have in the mix.
Project Progress – Gameplay Induction
Last week, Serra suggested that I should include more explanation to help players and viewers better understand how the game works—what the goals are and how each level is meant to be played.
So I came up with an idea that fits the theme of the project: before each level starts, the player will see a short diary entry written from the perspective of the child character. These entries are simple and heartfelt, like something a kid might actually write.
Each diary entry sets the emotional tone for the level, while also giving subtle hints about what the player will be doing or what the objective is. I also included a brief gameplay explanation alongside the diary, so players will know what kind of interaction to expect in that stage.
This way, players can connect more deeply with the story, while also clearly understanding how to play. It creates a smoother and more emotionally meaningful experience from the moment the level begins.
WEEK 20 Final Outcome
As the term comes to an end, I’ve been looking back at everything I’ve worked on over the past few months. From the first rough ideas to building a playable experience in Unreal Engine, this project has been a journey—not just technically, but emotionally too.
One of the biggest things I’ve learned is how important it is to let a project evolve naturally. At first, I had a vague idea of blending 2D and 3D styles, but it wasn’t until I started designing the characters and experimenting with collage that the tone of the project really started to take shape. The more I leaned into the childlike, handcrafted feeling, the more everything else started to fall into place—visually, narratively, and mechanically.
I’ve also realized how much I enjoy creating emotional experiences through simple interactions. Even though the gameplay isn’t complex, I wanted each level to feel personal and meaningful. Writing diary entries for each stage helped me frame the story in a way that feels intimate, and I hope it helps players connect to the character’s inner world as they play.
Of course, not everything went smoothly. There were times I felt stuck—especially during frame-by-frame animation or when things didn’t look quite how I imagined in-engine. But those challenges pushed me to improve, to ask for feedback, and to be okay with taking a step back and adjusting things. I’ve become more confident in both my creative decisions and technical skills.
Looking forward, I’m excited to keep developing this project. There’s still more to do, but I now have a clearer direction and a stronger understanding of how to balance visuals, story, and gameplay. Most importantly, I’ve grown to trust my instincts more and give space for ideas to grow—even if they start small and messy, like scraps in a collage.