By using 3D scanning to digitize household items and scenes, a digital replica of the home was created based on the Unity engine. Each scene and item, marked by significant traces of life, includes descriptions of family members’ memories associated with them. Through an immersive interactive installation, a surreal familial memory space was created, allowing viewers to interact with the author’s family memories by touching the projected images. This multi-sensory presentation offers the audience an immersive experience of the author’s family memories. The project aims to back up family memories to the cloud, providing a new possibility for preserving family memories in the future.

In this project, the SLAM LiDAR A2M8 was used as a sensor, and the collected data was converted into screen coordinate data using InteractiveEngine software. This data was then transmitted to Unity via the TUIO protocol. Due to issues such as poor lighting and narrow passageways in environments like my grandmother’s house, the point cloud scanning was conducted using Luma.AI’s NeRF-based scanning method. After exporting the point cloud files, the data was cleaned using the Point Cloud Visualizer plugin in Blender. In Unity, the VFX Graph was used to visualize the point cloud and create particle effects. Colliders were placed and hidden in each scene, and C# scripts was written to convert screen space points into three-dimensional points using raycasting. Touch effects were then generated at these points, influencing the point cloud.

This is a VJ system developed for an audio-visual performance at Sazareba #005. All effects are implemented in UE5 and rendered in real-time to produce interactive visuals synchronized with the audio. Specifically, TouchDesigner is used to receive and analyze audio signals in real-time, then transmit the audio data to UE5 via the OSC protocol. In UE5, the system is primarily divided into a camera controller and a VFX controller, both of which have their parameters controlled by a MIDI controller via the MIDI protocol.

Visual：RustyHead（TAN YANGGU）
Music：i.s.d（Yuki Ishida）

In the future, with the advancement of technology, I believe each of us will have a digital avatar. This avatar, reflecting our characteristics, will be constructed from data collected from our physical bodies. Building on this idea, I used AI to generate new images that resemble, but are not identical to, my physical appearance by using photos of parts of my body. I then used these newly generated images as a texture, filling the entire screen via video, transforming the screen into an extension of my digital avatar and exploring the broader possibilities of future digital embodiments.

Screenings:
2024.08 – Nakano ZERO Media Arts Program 2024
2025.02 – 13th International Science Film Festival Dome Festa
2025.02 – Immersive Dark Night Vol.3

Music: stabilizer / V.A.
Special Thanks: Oishi Hiroaki; Yamasaki Renki

Create a grid of square planes, then animate them through scaling and rotation. By applying materials to add color, the result can be used as a UV coordinate map.

After duplicating the UVs, apply downward motion using a Transform operation, then convert the result into a polar coordinate-based pattern. The AI-generated skin texture image is then processed and mapped onto the animated UVs using a Remap node, resulting in a dynamic visual effect.

I collected the hair I lost during home isolation and mandatory quarantine in hotels since the pandemic began. To me, the fallen hair embodies negative emotions such as anxiety and irritation. I plan to burn the collected hair and use a smoke density sensor to gather data, which I will then convert into audio and visual representations, resembling the echoes of the immense disaster of COVID-19. Through this project, I aim to remind everyone to reflect on what we should remember after a disaster, how to face painful memories, and how to gather the strength to move forward and face the inevitable tomorrow.

Test of audio-reactive particles in UE5 using TouchOSC.
Music：Lucky Charm (Original Mix) by Alix Perez

Some scenes constructed using Unreal Engine. The assets include some from MegaScan and some that I created myself.

Reconstructing the cyclical evolution of blood, this work attempts to interpret the connections and cycles of bloodlines through CG imagery.

It’s bedtime, and you’re lying in bed, planning to scroll through your phone a bit before sleep. Little do you know, your mind is being constantly stimulated by the fragmented information on your screen…

In this visualized mental world, the player’s excitement level is represented by an energy bar that depletes over time. The player needs to continuously use a shield to break the incoming blocks symbolizing points of excitement. If unsuccessful, these points will stimulate the core of your mind, increasing your excitement level. The goal is to reduce the excitement level to zero within a limited time. Success means you fall asleep and wake up refreshed the next day; failure means you stay up late and wake up the next day with dark circles, feeling groggy.