Tactile Emoticon

In a contemporary globalized world, social communication is increasingly digitally mediated. The frequent exchange of emotional states and social feedback (e.g. ‘Likes’, ‘Emoticons’) is a significant aspect of communication, usually via visual technologies. But what about touch? An absence of touch in digital communication is important as social, affective touch is a unique aspect of social communication, the formation of social bonds, and of self-other boundaries. Touch is also the preferred, and the only reliable, non-verbal channel of communication for prosocial emotions (love, admiration), and social intentions such as social support (Kirsch et al., 2017; Sauter, 2017).

Tactile Emoticon brought IN-TOUCH into a new collaboration with UCL colleagues from neuroscience and computer science to explore how the established digital communicative practices of sending emoticons might be extended to touch to enhance social communication and positive social feedback. We asked:

  1. What features and specific types of affective touch experience can be translated into socially meaningful digital communicative experiences?
  2. What multi-sensorial cues best serve as ‘emoticons’ for such digital affective touch and its associative communications?
  3. Which technology best implements digitally-mediated affective touch for socially meaningful communicative experiences?
  4. How can qualitative social science methods, quantitative neuroscience methods, design-based computer science and affective computing methods support each other to explore affective touch in the context of digital communication technologies?

Our research methods include literature review, rapid prototyping, observation, user interaction study, and psychophysical assessments of affective touch. We undertook a literature review to identify the features of touch (giving, reception and interpretation) and multisensorial cues that have the potential to communicate distinct emotions in intimate and public digital environments (e.g. social media exchanges). A half-day workshop was facilitated with researchers from across the three collaborating labs to explore: practices of using emoticons; textural and tactile associations with emotional states; the production and interpretation of tactile emoticons using a range of materials. The insights gathered from this initial phase were used to explore how to computationally characterize types of social touch to automatically extract affective cues from touch behaviour and its possible affective meaning (social support, approval, intimacy). These informed the ideation and development of a prototype ‘tactile emoticon’ using capacitive textile touch sensors, vibrotactile, pressure, and heat feedback. We combined our different disciplinary lenses to explore people’s responses and uses of the prototype device to examine the features that would enhance the social use of this touch device, with attention to reciprocity, privacy, agency, and controllability. We then re-calibrated and iterated the device as a research probe.

Collaborators

Dr. Aikaterini Fotopoulou Reader in Affective Neuroscience, Research Department of Clinical, Health and Educational Psychology, Division of Psychology and Language Sciences, Brain Sciences, SLMS, UCL

Professor Nadia Berthouze Professor in Affective Computing and Interaction, UCL Interaction Centre, connected to both the Division of Psychology and Language Sciences and Computer Science, Brain Sciences, SLMS, UCL

Frederik Brudy Doctoral Candidate at UCL Interaction Centre.

This case study is supported by additional funding from the UCL Social Science Plus scheme, to purchase equipment and employ a computer scientist to build a prototype.