Thinking Beyond the Human: Design Approaches for Robots and AI in Opera

Authors

  • Evelyn Ficarra Sussex University, Brighton, UK Author
  • Tim Hopkins University of Sussex, Brighton, UK Author
  • Elizabeth Jochum Aalborg University, Aalborg, Denmark Author
  • Chris Kiefer University of Sussex, Brighton, UK Author

DOI:

https://doi.org/10.37130/hxypf659

Keywords:

robots, opera, dance, AI, puppetry, musical robots

Abstract

Opera has historically been a site for developing new and emerging technologies. More recently, opera has proven a rich environment for investigating future scenarios involving human-robot interaction and exploring the potential of robot performers. The explosion of enthusiasm for AI and the ubiquity of machine learning tools have created numerous possibilities for generating creative content in music, poetry, images, and choreography. This article outlines efforts to explore the creative possibilities of robots and AI through a dedicated workshop with researchers from diverse fields. Our goal is to begin thinking beyond the human to investigate what robot performers might look and behave like when designed from a relational rather than a representational approach. We provide a brief history of robots on stage, including the authors’ prior work developing original performances featuring robots and human performers. We describe our iterative design approach and low- fi prototypes, summarizing key insights from the workshop. Finally, we analyze the workshop outcomes and consider the trade-offs of working with existing robots and the possibilities for working with open, configurable systems.

Author Biographies

  • Evelyn Ficarra, Sussex University, Brighton, UK

    Evelyn Ficarra is Professor of Interdisciplinary Composition (Music) and Co- Director of the Centre for Research in Opera and Music (CROMT) at the University of Sussex. Composer, sound artist and educator; Evelyn’s work finds expression across a range of forms including music theatre, multimedia, installation, dance, film and the concert hall. Primary research interests include: investigations into sound objects in different contexts (theatre, installation); electroacoustic and multi-disciplinary improvisation, and robot opera. Evelyn started the robot opera research project at CROMT in 2017, exploring issues of embodiment, performance and vocality through the lens of radical technical innovation in opera.

  • Tim Hopkins, University of Sussex, Brighton, UK

    Tim Hopkins is Senior Research Fellow, Music, University of Sussex, and has a background in opera, music theatre and theatre direction, scenic design, moving image, performing, in the UK, Europe, and US. He joined Sussex in 2007 as AHRC Fellow in the Creative and Performing Arts. He is interested in opera and music theatre as a technological adventure and societal forum, researching new forms in performance, experimentation and collaboration, exploring how the art-form responds to technologies and their meanings. His own work often links historic and current practice, new spaces technology has created and the evolving artistic toolbox, to explore impacts of technological transformation itself—cultural, emotional, political— and imagine possible futures for opera.

  • Elizabeth Jochum, Aalborg University, Aalborg, Denmark

    Elizabeth Jochum is an Associate Professor and leads the RELATE Research Laboratory for Art and Technology at Aalborg University (Denmark). Her research uses the visual and performing arts as catalysts for re-thinking how we design and implement robots and other assistive technologies. From industrial robots to exoskeletons, Dr. Jochum’s work involves transdisciplinary collaboration in human- robot interaction, health, and engineering to develop creative, value-sensitive, and human-centered approaches to ensure the technologies we build address the real needs of people. She coordinates the European ABRA project on Artificial Biology, Robotics and Art, and the ImprovAIze project on machine learning, dance improvisation, and wearable technologies. She was named one of the top 50 Women in Robotics by Robohub in 2021.

  • Chris Kiefer, University of Sussex, Brighton, UK

    Chris Kiefer is a Senior Lecturer in Music Technology, Experimental Music Technologies Lab (EMUTE) at University of Sussex. Chris is a computer-musician and musical instrument designer, specialising in musician-computer interaction, physical computing, and machine learning. He performs with custom-made instruments including malleable foam interfaces, touch screen software, interactive sculptures, and a modified self-resonating cello. His work concentrates on machine learning and complex systems in new instrument design. He co-runs the AHRC Feedback Musicianship Network.

References

Bell, J. (2001) Puppets, Masks and Performing Objects. Cambridge: MIT Press.

Bemman, B., and Meredith, D. (2018) “Generating new musical works in the style of Milton Babbitt,” in Computer Music Journal, 42(1), pp. 60-79.

Burnham, J. (1968) “Systems Esthetics,” in Artforum, September 1968, pp. 30-35.

Francis, P. (2011) Puppetry: A Reader in Theatre Practice. Bloomsbury.

Henson, K. (editor) (2016) Technology and the Diva: Sopranos, Opera, and Media from Romanticism to the Digital Age. Cambridge: Cambridge University Press (Cambridge Studies in Opera).

Hoffman, G. and Ju, W. (2014) “Designing Robots with Movement in Mind,” in Journal of Human-Robot Interaction, 3(1), pp. 89-122.

Jochum, E. (2013) “Deus Ex Machina: Towards an Aesthetics of Autonomous and Semi-Autonomous Machines,” Doctoral Dissertation, University of Colorado. https://scholar.colorado.edu/concern/graduate_thesis_or_dissertations/qf85nb374

Jochum, E., Millar, P., and Nuñez, D. (2017) “Sequence and Chance: Design and Control methods for entertainment robots,” in Robotics and Autonomous Systems, 87, pp. 372-380.

Jochum, E., and Derks, J. (2019) “Tonight We Improvise! Real-time tracking for human-robot improvisational dance,” in MOCO’19: Proceedings of the 6th International Conference on Movement and Computing, pp. 1-11.

Jochum, E., and Murphey, E. (2014) “Programming Play: Puppets, Robots, and Engineering,” in Posner, D., Orenstein, C. & Bell, J. (editors), Routledge Companion to Puppetry and Material Performance. Routledge, pp. 308-321.

Kang, M. (2011) Sublime Dreams of Living Machines. Cambridge: Harvard University Press.

Li, C., Hansen, A.K., Chrysostomou, D., Bøgh, S., and Madsen, O. (2022) “Bringing a Natural Language-enabled Virtual Assistant to Industrial Mobile Robots for Learning, Training and Assistance of Manufacturing Tasks,” in 2022 IEEE/SICE International Symposium on System Integration (SII), pp. 238-243.

Nikolaidis, R., and Weinberg, G. (2010) “Playing with the masters: A model for improvisatory musical interaction between robots and humans,” in 19th International Symposium in Robot and Human Interactive Communication, Viareggio, Italy, 2010, pp. 712-717.

Poulten, C. (2014) “From Puppet to Robot: Technology and the Human in Japanse Theatre,” in Posner, D., Orenstein, C., and Bell, J. (editors), Routledge Companion to Puppetry and Material Performance. Routledge, pp. 308-321.

Reilly, K. (2019) “Robots and Anthropomorphism in Science Fiction Theatre: from Rebellion to Domesticity and Back Again,” in Wyants, N. (editor) Media Archaeology and Intermedial Performance Deep Time of the Theatre. London: Palgrave Macmillan, pp. 193-210

Schaeffer, S. (1999) “Enlightened Automata,” in Clark, W., Golinski, J., Schaffer, S. (editors), The Sciences in Enlightened Europe. Chicago: Chicago University Press. pp. 126-165.

Sokei, L.C. (2015) The Sound of Culture: Diaspora and Black Technopoetics. Wesleyan University Press.

Sone, Y. (2017) Japanese Robot Culture: Performance, Imagination, and Modernity. New York: Palgrave Macmillan.

Tatar, K., Cotton, K., and Bisig, D. (2023) “Sound Design Strategies for Latent Audio Space Explorations Using Deep Learning Architectures,” in Proceedings of Sound and Music Computing. arXiv:2305.15571

Topey, Peter (2009). “Disembodied Performance,” Master’s Thesis, MIT.

Tremblay, P.A., Roma, G., and Green, O. (2021). “Enabling Programmatic Data Mining as Musicking: The Fluid Corpus Manipulation Toolkit,” in Computer Music Journal, 45(2), pp. 9-23.

Koteen, D., and Stark Smith, N., (2023) Contact Quarterly, [Online]. Available at: https://contactquarterly.com/contact-improvisation/about/ (Accessed: 3 August 2023)

Park, E. (2011) The Ring Transformed [Online]. Available at: https://www.metopera.org/user-information/nightly-met-opera-streams/articles/the-ring-transformed/ (Accessed: 3 August 2023).

Tatar, K. (2022) Coding the Latent. [Online]. Available at: https://kivanctatar.com/Coding-the-Latent (Accessed: 11 August 2023).

The Uncanny Valley (2019) Directed by Stefan Kaegi [Gessnerallee, Zurich].

O, One (2017) Music and libretto by Evelyn Ficarra. Directed by Evelyn Ficarra [Centre for Research in Opera and Music Theatre, Sussex].

The Opposite of Familiarity (2017) Music by Ed Hughes, libretto by Eleanor Knight. Directed by Evelyn Ficarra [Centre for Research in Opera and Music Theatre, Sussex].

Robot Opera – What’s Next? (2019) Music and libretto by Loré Lixenberg, Anton Lukoszevieze and Sam Bilbow. Directed by Evelyn Ficarra [Centre for Research in Opera and Music Theatre, Sussex].

Robo_Po /// Robo_Op (2021) Music and libretto by Evelyn Ficarra. Directed by Tim Hopkins. [Centre for Research in Opera and Music Theatre, Sussex].

Downloads

Published

2023-06-03

Issue

Vol. 26 No. 1 (2023): Matei Vişniec’s Dramaturgy in the Context of Interdisciplinary Artistic Research / Beyond the Human: Theatre, Robots and Social Realities

Section

Articles

How to Cite

Ficarra, E., Hopkins, T., Jochum, E., & Kiefer, C. (2023). Thinking Beyond the Human: Design Approaches for Robots and AI in Opera. CONCEPT, 26(1), 129-151. https://doi.org/10.37130/hxypf659