Dot to dot : storytelling without words, through gesture and abstraction : What can robotics learn from puppetry's language of gesture? : insights from a theatrical experiment

Eszter Angéla Kocsis

doi:10.5817/TY2026-1-7

Dot to dot : storytelling without words, through gesture and abstraction : What can robotics learn from puppetry's language of gesture? : insights from a theatrical experiment

Roč.29,č.1(2026)
Puppetry with(out) Classics

Eszter Angéla Kocsis

https://doi.org/10.5817/TY2026-1-7

Trvalý odkaz (handle):
http://hdl.handle.net/11222.digilib/digilib.84151

PDF (angličtina)

Abstrakt

Dot to Dot., a 45-minute experimental puppet performance premiered in 2023 at the Budapest Puppet Theatre, demonstrates puppeteers conveying emotion and narrative through movement and gesture alone. Using abstract dramaturgy, non-linear scenes, non-realistic puppets, and rolling robot balls as simple social robots, it engages audiences in active meaning- making. Linked to the author's PhD research on puppetry-robotics intersections, particularly nonverbal channels in human–robot interaction (HRI), the study employed drama workshops with children, questionnaires, and an online emotion-recognition experiment. Findings show puppetry's non-verbal system – movement, body language, visual symbolism – as culturally responsive yet universally rooted, offering inspiration for contemporary puppetry and socially adept robots, plus a proposed gesture-based puppetry lexicon for technological interaction.

Klíčová slova:
puppetry; robotics; nonverbal; gestures; abstraction; robot theatre; object animation; theatre anthropology

Stránky:
113–134

Reference

ARTAUD, Antonin. 1985a. A Kegyetlen Színház (Első kiáltvány) [The Theatre of Cruelty (First Manifesto)]. In Antonin Artaud. A könyörtelen színház [The Theatre of Cruelty]. Gondolat Kiadó, 1985: 147–160. | DOI 10.5040/9781350928091

ARTAUD, Antonin. 1985b. Színház és kegyetlenség [Theatre and Cruelty]. In Antonin Artaud. A könyörtelen színház [The Theatre of Cruelty]. Gondolat Kiadó, 1985: 143–146. | DOI 10.5040/9781350928091

BALÁZS, Géza. 2015a. Gesztuskommunikáció [Gesture Communication]. In Géza Balázs. Antroposzemiotika [Anthroposemiotics]. Budapest: Magyar Szemiotikai Társaság, 2015: 55–64.

BALÁZS, Géza. 2015b. Jelkulturális, jelpedagógiai megjegyzések [Sign-Cultural and Sign- Pedagogical Remarks]. In Géza Balázs. Antroposzemiotika [Anthroposemiotics]. Budapest: Magyar Szemiotikai Társaság, 2015: 65–84.

BARBA, Eugenio and Nicola SAVARESE (eds.). 2021. A színész titkos művészete— Színházantropológiai szótár [The Actor's Secret Art – Dictionary of Theatre Anthropology]. Budapest: L'Harmattan Kiadó, 2021.

BLAIR, Preston. 1994. Cartoon Animation. Laguna Hills: Walter Foster Publishing, 1994.

BOYLE, Michael Shane, Matt CORNISH and Brandon WOOLF (eds.). 2019. Postdramatic Theatre and Form. London: Methuen Drama, Bloomsbury Publishing Plc, 2019.

BUJOREANU-HUŢANU, Raluca. 2019. The Language of Puppetry – Code for Remembering and Theatricalization [online]. Theatrical Colloquia 9 (2019): 2: 40–53. [accessed on 4. 3. 2026]. Available online at https://www.researchgate.net/publication/345356458_The_Language_of_Puppetry_Code_for_Remembering_and_Theatricalization. | DOI 10.2478/tco-2019-0015

EGRI, Lajos. 2022. A drámaírás művészete [The Art of Dramatic Writing]. Budapest: GABO Kiadó, 2022.

ELLINGER, Edina. 2017. Szubjektív objektum. A funkcionális tárgyjáték vizsgálata és A csomótündér doktori műalkotás alkotói munkafázisai [Subjective Object. Examination of Functional Object Play and the Creative Phases of the Doctoral Artwork The Knot Fairy] [online]. Unpublished doctoral thesis. Színház- és Filmművészeti Egyetem, 2017. [accessed on 13. 8. 2024] Available online at https://archiv.szfe.hu/wp-content/uploads/2018/01/Ellinger-Edina-DLAdolgozat.pdf.

FATLAND, Eivind and Xiang LI. 2004. The Puppet UI Tools for nonverbal communication in virtual environments [online]. [accessed on 9. 11. 2025]. Available online at http://www.thoughtbird.com/portfolio/puppetui/puppetui_paper.pdf.

GHANI, Dahlan Abdul. 2012. Relationship Between The Art of Wayang Kulit and Disney's Twelve Principles of Animation. Revista De Cercetare Si Interventie Sociala 37 (2012): 162–179. [accessed on 4. 3. 2026]. Available online at https://www.researchgate.net/publication/290573324_Relationship_Between_The_Art_of_Wayang_Kulit_and_Disney's_Twelve_Principles_of_Animation.

GOMEZ CUBERO et al. 2021. The Robot is Present: Creative Approaches for Artistic Expression With Robots. Frontiers in Robotics and AI 8 (2021). DOI: https://doi.org/10.3389/frobt.2021.662249. | DOI 10.3389/frobt.2021.662249

JOCHUM, Elizabeth et al. 2016. Using Theatre to Study Interaction with Care Robots. International Journal of Social Robotics 8 (2016): 4: 457–470. DOI: https://doi.org/10.1007/s12369-016-0370-y. | DOI 10.1007/s12369-016-0370-y

KAPITÁNY, Ágnes and Gábor KAPITÁNY. 2021. A szimbolizáció: Hogyan cselekszünk szimbólumokkal? [Symbolization: How Do We Act with Symbols?]. Budapest: Ventus Commerce, 2021.

KOCSIS, Eszter A. 2021. Animatronika – bábművészet és technológia [Animatronics –Puppetry and Technology] [online]. Theatron 15 (2021): 3. Available online at https://theatron.hu/theatron_cikkek/animatronika-babmuveszet-es-technologia/. | DOI 10.55502/the.2021.3.144

KOCSIS, Eszter A. 2024. Robotokkal kapcsolatos attitűdök vizsgálata a bábművészet segítségével [Examining Attitudes Toward Robots through Puppetry]. Educatio 33 (2024): 1: 85–92. DOI: https://doi.org/10.1556/2063.33.2024.1.10. | DOI 10.1556/2063.33.2024.1.10

KORSÓS, Noémi. 2021. A felfedezés folyamata: A színházi devising megközelítési kísérletei [The Process of Discovery: Attempts to Approach Theatrical Devising]. Theatron 15 (2021): 1: 160–165. DOI: https://doi.org/10.55502/the.2021.1.160. | DOI 10.55502/the.2021.1.160

LEHMANN, Hans-Thies. 2007. Postdramatic Theatre and the Tradition of Tragedy. Lecture presented at the Kortárs Drámafesztivál Budapest, Országos Színháztörténeti Múzeum és Intézet, 25. 11. 2007.

LEHMANN, Hans-Thies. 2009. Posztdramatikus színház [Postdramatic Theatre]. Budapest: Balassi Kiadó, 2009.

MAHDI, Hamza et al. 2022. A survey on the design and evolution of social robots – Past, present and future. Robotics and Autonomous Systems 156 (2022): 104193. DOI: https://doi.org/10.1016/j.robot.2022.104193. | DOI 10.1016/j.robot.2022.104193

MATHEWSON, Kory and Piotr MIROWSKI. 2018. Improbotics: Exploring the Imitation Game Using Machine Intelligence in Improvised Theatre. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 14 (2018): 1: 59–66. DOI: https://doi.org/10.1609/aiide.v14i1.13030. | DOI 10.1609/aiide.v14i1.13030

MCCROSKEY, James C. and Thomas A. MCCAIN. 1974. The measurement of interpersonal attraction. Speech Monographs 41 (1974): 3: 261–266. DOI: https://doi.org/10.1080/03637757409375845. | DOI 10.1080/03637757409375845

MOHOLY-NAGY, László. 1978. Színház, cirkusz, varieté [Theatre, Circus, Variety]. In Oskar Schlemmer et al. A Bauhaus színháza [The Bauhaus Theatre]. Budapest: Corvina Kiadó, 1978: 45–56.

PAIVA, Ana et al. 2003. SenToy: A tangible interface to control the emotions of a synthetic character. Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (2003): 1088–1089. DOI: https://doi.org/10.1145/860575.860809. | DOI 10.1145/860575.860809

POULTON, Cody. 2014. From Puppet to Robot. In John Bell, Claudia Orenstein and David N. Posner (eds.). The Routledge Companion to Puppetry and Material Performance. London: Routledge, 2014: 280–293. DOI: https://doi.org/10.4324/9781315850115.ch25. | DOI 10.4324/9781315850115.ch25

ROSENTHAL-VON DER PÜTTEN, Astrid M. et al. 2014. The Uncanny in the Wild. Analysis of Unscripted Human–Android Interaction in the Field. International Journal of Social Robotics 6 (2014): 1: 67–83. DOI: https://doi.org/10.1007/s12369-013-0198-7. | DOI 10.1007/s12369-013-0198-7

SAMANI, Hooman et al. 2024. Creative Robotics Theatre: Designing Creative Interactions with Tangible and Embodied Interfaces. Designing Interactive Systems Conference (2024): 389–391. DOI: https://doi.org/10.1145/3656156.3658387. | DOI 10.1145/3656156.3658387

SCHLEMMER, Oskar. 1978. Ember és műfigura [Human and Art Figure]. In Oskar Schlemmer et al. A Bauhaus színháza [The Bauhaus Theatre]. Budapest: Corvina Kiadó, 1978: 7–21.

SOTO-MORETTINI, Dario. 2017. Reverse engineering the human: Artificial intelligence and acting theory. Connection Science 29 (2017): 1: 64–76. DOI: https://doi.org/10.1080/09540091.2016.1271398. | DOI 10.1080/09540091.2016.1271398

STUBER, Andrea. 2024. Evolúció [Evolution] [online]. spirituszonline.hu (20. 1. 2024). [accessed on 13. 8. 2024]. Available online at https://www.spirituszonline.hu/kritika/evolucio/.

TRIFAN, Bianca. 2023. Devised Theatre Meets Telematic Theatre. In Marina Hanganu (ed.). Beyond the Human: Theatre, Robots and Social Realities. Craiova: Editura Universitaria, 2023: 327–360.

TUNG, Fang-Wu. 2011. Influence of Gender and Age on the Attitudes of Children towards Humanoid Robots. In J. A. Jacko (ed.). Human-Computer Interaction. Users and Applications 6764 (2011): 637–646. Springer Berlin Heidelberg. DOI: https://doi.org/10.1007/978-3-642-21619-0_76. | DOI 10.1007/978-3-642-21619-0_76

VINES, John et al. 2013. Configuring participation: On how we involve people in design. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2013): 429–438. DOI: https://doi.org/10.1145/2470654.2470716. | DOI 10.1145/2470654.2470716

VOIGT, Vilmos. 1977. Bevezetés a szemiotikába [Introduction to Semiotics]. Budapest: Gondolat Kiadó, 1977.

Metriky