At Keio University's Faculty of Environment and Information Studies, a course on data-driven art using VIE's earphone-type brainwave monitor has been launched. The lectures spanned four days, with VIE members also participating as guest speakers. On the final day of the course, students presented unique artworks created in groups using the earphone-type brainwave monitor, unveiling a new world of art using brainwave technology. In this report, we will share the highlights of the presentation event on the last day.
Recreating the enigmatic presence of brainwaves through dance, music, lighting, and visuals.
The first performance, themed 'Unknown,' expressed the attachment to the mysterious brainwaves through improvisational dance. The captivating collaboration between beautiful dance synchronized with real-time generated music, lighting, and visuals from the audience's brainwaves, accompanied by unpredictable and mysterious sounds, was truly impressive. The soundtrack incorporated everyday sounds seamlessly into electronic music, including the flowing of water, the sound of wind, and even family snoring, infusing humor and a sense of affinity with 'VIE Tunes' (https://vie.style/pages/vie-tunes), which incorporates natural sounds.
The instructors' comments included praise such as 'Feeling the vitality through the discomfort in the sound, with a fantastic contrast to the beauty of the dance' and 'It would be interesting if a large number of spectators wore brainwave monitors, creating a choir of brainwaves.' The moments when dance, music, lighting, and visuals seamlessly came together created a sense of awe and excitement, captivating the hearts of the audience. The mystique of witnessing the integration in real-time added to the allure of the performance.
Outputting words based on the patterns of brainwaves
The second performance involved selecting and connecting words from a pool of 300 registered words using brainwaves to generate output in the form of sentences. The mechanism behind this creation incorporated setting thresholds in brainwaves and randomly altering the rule for picking up characters based on whether the values exceeded those thresholds. The intriguing aspect of this piece lay in the meticulous attention the author, Mr. Ebina, had given to his feelings towards words. He expressed, 'I believe that words, like emotions, are ambiguous, and labeling them turns them into something concrete. I don't particularly like the characteristics of words. That's why I wanted to create a piece that respects ambiguity, leaving ambiguous things as they are.' Through the obscured visibility and complexity of the text, the work aimed to represent the intricacies of the mind.
In the feedback, there were comments such as 'It would be interesting if, as information accumulates, one could discern the individual's unique language' and 'The association of initially complex brainwaves with language made it more accessible, which was commendable.' The creative approach of countering the 'ambiguous' with 'clear language' was highly innovative. The anticipation for future updates was high, considering elements like reading aloud the completed sentences and the potential for creating content in Japanese.
Sharing the reverberations within the mind with the audience
The third piece focused on the reverberations within the mind experienced after live performances or concerts. This live performance aimed to share those reverberations with the audience. The artist manipulated in real-time the sounds of footsteps and piano using their own brainwaves, creating a space where the audience could concentrate solely on the sound. The unreal space, where the reverberations that typically occur only within oneself were released externally as sound, possessed a peculiar and delightful sensation, captivating the audience.
In the feedback, there were comments such as 'I truly felt what it's like after listening to music' and 'I realized that reverberation is not only a physical phenomenon but also has experiential significance.' The innovative direction of focusing solely on the sound received praise, with many expressing anticipation for the effects that could arise when combined with visuals. The shared reverberation of an individual, collaborating with the reverberation resonating within oneself, introduced a new way of enjoying music—each person feeling the sound of a new reverberation within their own mind.
Expressing the life of plants through sound
The fourth piece involved outputting the bioelectric potential flowing within plants as sound, creating a uniquely original work that allows one to feel the vitality of plants and draws the audience closer to the plant world. The presenter, Mr. Shinmi, regularly conducts performances using a 'Synthesizer Co-performing with Plants.' However, he faced a dilemma, questioning whether the sounds produced by plants were ultimately being modified by human preferences, potentially fabricating the intention of the plants. To address this, he expressed the concept of approaching plants by immersing himself in a fixed state and creating music based on the brainwaves generated while listening to the sounds emitted by the plants."
In the feedback, comments such as 'Choosing the dilemma you had been facing as the theme was a good decision' and 'I would like to see performances where the performer experiences what they have done to the plants, such as being watered' were noted. Mr. Shinmi expressed satisfaction, stating, 'There was a moment when sounds I didn't desire were produced, and it felt like I got a little closer to the plants, which was enjoyable.' However, he also conveyed a sense of ongoing dilemma, saying, 'The sound is still based on the mapping I did, and the dilemma hasn't disappeared yet,' demonstrating his enthusiasm for future updates.
Expressing the inner beauty of a human through heartbeat and brainwaves
The final piece contrasted the sounds of an artificial valve with the heartbeat, aiming to depict the beauty within the human body through a performance involving audio, visuals, and lighting. The artificial valve emitted a ticking sound reminiscent of a clock. The sensation of hearing this sound live left a lasting impression, prompting four individuals to come together with the idea of turning it into art. The first part of the performance portrayed the contrast between the heartbeat and the sound of the artificial valve, while the latter part featured enigmatic sounds visualizing brainwaves resonating throughout the venue.
In the feedback, there were comments such as 'They successfully translated the sound into visuals' and 'Even without explanations, it was clear what they wanted to create.' The heartbeat, with its intense yet somehow calming characteristics, conveyed the inner rhythm that humans unconsciously experience. The audio-visual presentation was impressive, featuring images of the heart and brain created with 3D computer graphics that unfolded unpredictably, evoking a deeply emotional response.
Experiencing the reality of 'If you want to go far, go together'
After the conclusion of all the presentations, the evaluations from the instructors were tallied, and an award ceremony took place. The VIE Award was granted to the piece that demonstrated the developmental potential of brainwaves and language, specifically the work generating sentences based on brainwaves. The Roland Award focused on the theme of reverberation in sound, the Technical Support Award was given to the piece creating music from plant sounds, and the Daito Manabe Special Award recognized the improvisational dance performance synchronized with brainwaves. The Daito Manabe Grand Prize, selected through a vote from the entire audience, was awarded to the final piece that depicted the contrast between the sounds of an artificial valve and the heartbeat.
During the award ceremony, there were reflections such as 'The strength of each individual was leveraged because this team came together to create excellent works,' 'Experiencing "If you want to go fast, go alone; if you want to go far, go together",' and 'Being able to implement and output within a short period gave me confidence.' The proud expressions on the students' faces, who gained confidence through challenging themselves and reflecting on the course, were truly impressive. VIE felt a deep connection to their growth and expressed a strengthened commitment to further convey the allure of neurotechnology to society.
