Interactive Media · 2023–2024
Resonance
共·鸣
An Interactive Musical-Ritual Environment
An interactive musical-ritual installation that translates Interaction Ritual Chain theory into spatial, audiovisual, and kinetic feedback, evaluated across three public exhibitions.
01 / Project Overview
Project overview
Resonance is an interactive musical-ritual installation that explores how co-presence and collective affect can be reconstructed in digitally mediated environments. A central frame-drum apparatus, four surrounding screens, robotic mallets, seat and strike sensors, and live camera mapping create a low-latency audiovisual call-and-response. Visitors move from spectators to co-performers as rhythm, image, light, and mechanical movement converge around shared attention.
《共·鸣》是一件探索共同在场与群体情感的音乐仪式交互装置。中心架鼓、四面屏幕、机械鼓槌、入座与敲击传感以及实时摄像映射，共同构成即时的视听与动作反馈，使观众围绕共同的节奏与注意中心，从旁观者转变为共同表演者。
02 / Project significance
Rebuilding ritual presence in digital space在数字空间中重建仪式性的共同在场
Resonance begins with a central question: when interaction is no longer organized around bodily co-presence, how can ritual recover affective weight and symbolic intensity in the “weightless” digital sphere?
《共·鸣》关注：当互动不再依赖身体共处时，如何在数字空间中重新建立仪式的情感力量与象征意义？
Rather than treating ritual as a visual theme, the project translates its social mechanisms into repeatable spatial and interaction principles: establish a legible center and boundary; use low-latency rhythmic feedback to support synchrony; anchor memory through a small set of strong symbols and replayable traces; and make participants mutually visible so they can move from spectators to co-performers. Developed through practice and examined across multiple public settings, these principles position the work as both an installation and a testable design method.
项目将仪式的社会机制转化为可重复的设计原则：建立明确的中心与边界，以低延迟节奏反馈促进同步，通过符号与回放痕迹留存共同记忆，并让参与者彼此可见，逐步从旁观者转为共同表演者。
03 / Theory translation
From Collins’s Interaction Ritual Chains to an immersive design framework从互动仪式链理论到沉浸式设计框架
Randall Collins’s Interaction Ritual Chain theory describes four ingredients of a successful ritual: bodily co-presence, a boundary to outsiders, mutual focus of attention, and a shared mood. These conditions can generate solidarity, emotional energy, shared symbols, and common standards. In digitally mediated settings, however, bodily co-presence and sensorimotor coupling are weakened or redistributed, making collective attention and shared feeling harder to sustain.
柯林斯的互动仪式链理论提出身体共在、对外边界、共同注意与共享情绪四个条件。在数字媒介环境中，身体共处与感知—动作联系受到削弱或重新分配，共同注意与情感联结也更难维持。
Resonance treats the smallest perceivable cycle—participant action, system feedback, shared trace, and role shift—as a designable micro-ritual. The theory is translated into concrete spatial, interface, and interaction decisions, then connected to observable HCI indicators and methods of evaluation. The result is a framework that can be applied and tested through practice rather than remaining at the level of metaphor.
项目把“参与者行动—系统反馈—共享痕迹—角色转变”视为可设计的微型仪式，将理论落实为空间、界面与交互决策，并与可观察的体验指标和评价方法相连接。
04 / Spatial architecture and apparatus
A musical gathering translated into an annular field将传统合乐关系转化为环形体验空间
The spatial concept begins with Hele Tu (合乐图), traditionally attributed to Zhou Wenju of the Five Dynasties–Southern Tang period. The handscroll gathers musicians around a shared field of performance, making rhythm, reciprocal attention, and ensemble relations visible. Resonance does not reproduce the painting literally; it translates this cultural structure into an inward-facing arrangement organized around a common center.
空间灵感来自传为五代南唐周文矩所作的《合乐图》。项目提炼画中合奏者围绕共同表演场域形成的节奏、注意力与协作关系，转化为面向共同中心的环形空间组织。
Within a darkened 4 × 2.5 × 3 metre room, a two-tier frame-drum apparatus is encircled by four digital screens. The upper mallets are system-driven and establish the pulse; the lower mallets respond to audience actions, creating a call-and-response between machine and participant. The apparatus also takes the Shanghai Museum’s Coiled-Dragon Frame Drum as a morphological reference, recasting the ritual instrument as a contemporary interactive interface. Live camera mapping places participants inside the surrounding imagery, reinforcing mutual visibility and the transition from spectator to co-performer.
在约4 × 2.5 × 3米的暗室内，双层架鼓装置由四面数字屏幕围合。上层鼓槌由系统驱动建立节拍，下层回应观众行为；造型参照上海博物馆藏盘龙架鼓，并通过实时摄像映射将观众置入影像，强化共同在场与参与感。
05 / Media integration and technical implementation
One control system coordinates image, rhythm, sensing, and movement统一协调影像、节奏、传感与机械运动
TouchDesigner functions as the central orchestration layer, integrating Unreal Engine 5 visual output with Arduino-based sensing and mechanical actuation. Capacitive touch sensors register strikes, photoresistors detect occupied seats, and a camera captures live images for screen mapping. Servo-driven mallets, multichannel audio, and UE5 sequence changes return immediate audiovisual and kinetic feedback.
TouchDesigner作为中央控制层，连接UE5影像、Arduino传感与机械执行。电容触摸传感器记录敲击，光敏电阻感知入座，摄像头采集实时画面；舵机鼓槌、多通道声音与影像切换共同提供即时反馈。
The system coordinates waiting, entry, resonance, and closure within a single control graph. Keeping this loop legible and low-latency allows each action to be perceived not as an isolated trigger, but as part of a shared rhythmic structure.
同一控制流程协调等待、进入、共鸣与结束四种状态，使每次操作都能被理解为共同节奏的一部分，而非孤立的触发事件。
06 / Interaction flow and ritual structure
Four acts move the visitor from observer to co-performer四幕体验：从旁观者到共同表演者
The experience unfolds as a continuous arc from perception to participation, collective intensification, and release. Each act coordinates spatial cues, sensing logic, tempo, and feedback so that the visitor’s changing role remains perceptible.
体验由感知、参与、群体共鸣走向结束。四个阶段协调空间提示、传感逻辑、节奏与反馈，使观众能够感知自身角色的变化。
-
Act I
Preludial attraction序幕吸引
The apparatus leads with a system-driven visual and rhythmic motif, establishing a clear pulse and focal center before the visitor enters.
装置先以自动运行的影像和节奏吸引注意，在观众进入前建立明确的节拍与视觉中心。
Act I · Preludial attraction第一幕 · 序幕吸引 -
Act II
Audience seating and virtual co-presence观众入座与虚拟共在
Seat sensing changes the tempo while live camera mapping places the visitor within the surrounding imagery. Crossing the spatial boundary turns a passerby into a participant.
入座感应改变节奏，实时摄像将观众映射到周围屏幕，使跨越空间边界的旁观者转变为参与者。
Act II · Audience seating and virtual co-presence第二幕 · 观众入座与虚拟共在 -
Act III
Collective resonance群体共鸣
Drum strikes intensify music, image, projection, and mallet movement through low-latency call-and-response. Human and machine now share the performance loop.
敲鼓通过低延迟反馈带动声音、影像、投影与鼓槌运动，人与机器共同进入表演循环。
Act III · Collective resonance第三幕 · 群体共鸣 -
Act IV
Ritual closure仪式结束
The tempo decrescendos, symbolic traces replay, and the participant leaves the interaction field. The system then returns to its waiting state for the next gathering.
节奏逐渐减弱，共同的象征痕迹再次呈现。参与者离开后，系统回到等待状态，迎接下一轮相聚。
Act IV · Ritual closure第四幕 · 仪式结束
Public encounter
The work becomes a situation shared by its visitors由观众共同构成的体验现场
Research output
Rhythm, Symbol, and Presence
Translating Interaction Ritual Chains for Digital-Space Installations
节奏、符号与在场：面向数字空间装置的互动仪式链转译。
Yiyi Qian, Zhen Yu
Proceedings of the 2025 International Conference on Human-Engaged Computing · ACM · pp. 1–11
Project record
Credits and references
Exhibitions
- China Academy of Art Graduation Exhibition, Hangzhou — June 2024
Awards / Selections
- Silver Award, China Academy of Art Graduation Exhibition — 2024
- Best Emerging Talent Award, HONOR Talents Global Design Awards — 2024
Publications
- Qian, Yiyi, and Zhen Yu. Rhythm, Symbol, and Presence: Translating Interaction Ritual Chains for Digital-Space Installations. Proceedings of ICHEC 2025, ACM, pp. 1–11. DOI: 10.1145/3786995.3787041
The team
- Creators
- Yiyi Qian, Ziheng Yu
- Music Production
- Dan Lu, Yuzhou Chen