Shadow “Play”

The project aims to explore the use of tangible interaction to promote awareness of traditional shadow culture. Shadow “Play” is an interactive art installation that brings the traditional art - Chinese Shadow Play, closer to the public. People can choose their preferred scenes and shadow puppetries, and then interact with the scenes by manipulating actual shadow puppetries. The visual effects of the different objects within the scene are triggered depending on how people manipulate the shadow to move around on the curtain.

Project Concept & Intended Experience

"Shadow Play" utilizes a projector to mimic the lighting effects seen in traditional shadow plays, projecting scene frames directly to recreate the traditional storytelling experience. To begin, participants are treated to a brief video introduction outlining their roles and offering tips on manipulating shadow puppets. At the end of the video, instructions for the next steps are provided.

The experience offers four distinct scenes corresponding to the seasons—spring, summer, autumn, and winter—each featuring a selection of four different shadow puppets for participants to choose from. They can use the same puppet to explore multiple scenes or switch at any time. As participants manipulate the shadow puppets on the curtain and interact with scene objects, they receive visual and auditory feedback tailored to the specific season. This feedback aims to enhance the storytelling experience, echoing the ambiance of traditional shadow plays.

In traditional shadow plays, recurring characters often take on different roles in various storylines. Similarly, by selecting different scenes and puppets, participants effectively choose the "stories" they will witness. As participants engage with the shadow puppets on the curtain, they can immerse themselves in the seasonal scenes through the accompanying visual and auditory feedback, enhancing their overall experience.

Against the backdrop of technological advancements and people's growing interest in novelty, many traditional cultural works are fading away. To preserve these cultural treasures, scholars have proposed creating new interactive experiences to awaken people's cultural memories and enhance their interest in the culture. It is emphasized that a combination of performances and other forms is necessary to better enable users to comprehend the cultural background of shadow puppetry.

Therefore, our team is committed to achieving the following objectives:
1. Engage a wider audience in this art form through immersive interactive environments, fostering their appreciation for it.
2. Allow the general public to explore traditional artistic expressions through immersive environments and a series of heuristic teaching, bridging the gap between ordinary individuals and the demands of sophisticated performances, thereby cultivating their interest in Chinese shadow puppetry.
3. Build a bridge between modern technology and the ICH. By combining modern techniques with traditional shadow puppetry interactions, users can gain a deeper understanding of the essence of this ICH.
4. Encourage users to protect this culture by increasing their engagement and understanding of it through promotion.

Design Process

During the technical development of Shadow “Play”, the process unfolded in three distinct stages, each contributing to the final delivery. The first stage, the Conceptual Stage, involved laying the groundwork for the project's interaction medium within traditional arts. Here, tangential learning was explored as a theoretical framework, while research on previous work helped identify gaps between theory and practice. It was during this phase that the concept of using hand shadows to trigger traditional shadow puppet characters emerged. Implementing a machine-learning method to capture users' hand shadow movements, we aimed to create a seamless interaction flow for Shadow “Play”.

Transitioning to the Prototype Stage, feedback from users and technical limitations prompted significant changes to the initial concept. Hand shadows proved challenging for users to connect with traditional arts, leading to a shift towards restoring the authenticity of shadow play performance. This involved utilizing actual shadow puppetries for manipulation and interaction, supported by Hall Effect sensors to detect magnetic puppetry movements.

In the Final Exhibit Stage, additional refinements were made to enhance the intended experience. While retaining the use of Hall Effect sensors for detection, we addressed feedback regarding the interaction journey's integrity by introducing a background introduction video at the onset of the interaction. Furthermore, offering a broader selection of scenes empowered participants with greater flexibility in their interactions, facilitated by the integration of Lilypad color recognition sensors.

Through these iterative stages, Shadow “Play” successfully merged traditional Chinese Shadow Play elements with modern technology, culminating in an immersive and accessible experience for participants.

My Contribution

My job in our group is visual and animation designer, mainly responsible for the character design, and production of the physical shadow for the whole project, as well as the design of the promotional materials. I was also involved in the conception of the interactive concepts throughout the project. I think my contribution played a very important role in the whole project process. Since our project is an immersive experience, the visual design had to take into account the combination of aesthetics and user experience. Based on the feedback from the exhibition it was possible to find interactive forms that were aesthetically pleasing to the public and looked good and interesting, that could attract users and keep them interested in the shadow culture.

My contribution had a positive impact on the conceptualization of the final project. Through my visual and animation design, I was able to create unique and compelling scenes for the entire project, creating an atmosphere and emotion that was in line with the shadow culture. My attention to detail ensured that the scene design blended with the experience of user interaction, enabling the audience to better understand and appreciate the cultural connotations of shadow art.

In the design and production of physical shadow characters, I am committed to creating character images with personality and charm. By researching and understanding the characteristics of the shadow culture, I focus on the selection of materials and the refinement of craftsmanship to ensure the texture and expressiveness of the physical shadow to bring a more immersive experience to the audience.

❮ ❯

Project Outcomes

For the final outcome of the project, despite limited technological implementation, we were not able to present our concept perfectly. However, I am still quite satisfied with the results. When the audience began interacting with our physical shadow puppets and backdrop, their novelty reactions and positive encouragement made us realize that our final achievement could spark the interest of young people in shadow puppetry. They can learn about the cultural significance of shadow puppetry through our interactive device.

At the same time, the project's results have also validated that combining immersive experiences with heuristic teaching can to some extent enhance users' awareness of intangible cultural heritage and increase their consciousness of its preservation. However, there is still room for improvement in our project. Compared to other existing works in the field, our work is still lacking in terms of immersive experiences. By incorporating some guiding and engaging elements into the overall immersive experience, users would have more freedom to interact with our installation. For example:

1. A compelling storyline enhances user engagement and emotional resonance. Design an engaging storyline that allows users to experience plot developments and twists in an interactive way.
2. Interesting interactive elements that offer diverse and innovative ways of engagement enable users to actively participate and influence the story's progression.
3. Additionally, utilizing tactile feedback, vibrations, and other physical response mechanisms can enhance the interactive experience between users and the installation.