What does touch sound like?

At IN-TOUCH we like to think about touch in different ways and recently we asked ourselves the question ‘What does touch sound like?’ When you think about touch, you probably think of tactile sensations, the feeling of your body coming into contact with something. Yet much of our sensory experience comes about through a combination of information from our different senses, known as cross-modal interaction. Imagine the noises you hear when you rub or scratch a rough surface. What do they tell you about the thing you are touching? Studies have shown that we understand a lot from this kind of sonic information. For example the sounds we hear when eating crisps can change our opinion of how crisp, or stale and soggy they are. Zampini & Spence (2004) fed their participants the same crisps and asked them to use scales to rate how crisp and fresh they perceived them to be, when they listened to different sounds as they bit into them. They showed that when hearing louder or higher frequency sounds, perceptions of crispness (and freshness) increased. Consider that next time you bite into something you think is crunchy or soft.

Sound has also been shown to influence our perception of the way textiles feel  (Orzechowski, 2016 & Petreca et al. in publication). In an interface presenting interactive digital textile swatches, when sound was added to the swatches they were always rated as being more crisp. This is perhaps because this tactile quality is more often associated with fabrics making noise than some other qualities like smoothness and flexibility.

Different gestures to manipulate the fabric swatches were shown to be more effective at demonstrating the tactile qualities of the swatch if the gesture used would cause the fabric to make more noise. Implying that sound was helpful in understanding how a fabric felt, but also informed the gestures used to touch it when seeking this information.

Another example that shows how sounds can influence more than just our perceptions of a texture is a study by Bresin et al. (2010). They explored the tacit associations between sound and texture from an embodied perspective, investigating the effect of different types of ground texture sounds on the way participants walked. In relation to the surface sound, participants walked faster when hearing a surface sound spectrum with a higher central point (the example given in the study being crisp snow) than a lower central point (muddy ground). Harder texture sounds led to more aggressive walking styles. Thus sound can influence our behaviour and the way we touch and come into contact with our environment

So how might a hug sound? And how might we (unconsciously) respond to that sound? How could sound communicate a sense of touch or contact? And how can we become more aware of sound in relation to touch?

IN-TOUCH have been experimenting with some quick prototypes using the Bare Conductive Touch Board (https://www.bareconductive.com) to help us explore these questions.

The shirt in the images and video below was an experiment by Douglas Atkinson in playing a piano scale of body contact, with rising and falling tones wrapping around the body. The area of contact with the wearer’s body was highlighted by the tone of sound produced. It was interesting to see if ‘gamifying’ inter-personal touch could encourage people to overcome social taboos and shyness to ‘play’ the shirt?

There was a noticeable sense of reluctance to touch another person to play a tune (most people were more comfortable touching their own body), though we tried out some common inter-personal touches like hugs to see how they sounded. There was also a sense of surprise at the sounds produced. Whether this was surprise at touch being associated with a sound, or whether it was the incongruity of the sound compared to what was unconsciously expected was unclear and could be an interesting avenue for our future research.

Another interesting aspect of the shirt is way the visual experience of its design so strongly guides interaction. Many people’s first instinct is to stroke and trace the copper tape which forms the circuit, despite the sound being triggered by contact with the printed touch pads. This is an important consideration for the design of touch sensing objects used in research as they may well bias people’s touch habits.

References
Bresin, R., de Witt, A., Papetti, S., Civolani, M., & Fontana, F. (2010). Expressive sonification of footstep sounds. In Proceedings of ISon 2010 : 3rd Interactive Sonification Workshop. Stockholm, Sweden. pp. 51–54.

Orzechowski, P.M. (2016) Pinching sweaters on your phone – iShoogle : multi-gesture touchscreen fabric simulator using natural on-fabric gestures to communicate textile qualities. Thesis (Ph.D.), School of Mathematical and Computer Sciences, Heriot-Watt University.

Zampini, M. & Spence, C. (2004). The Role Of Auditory Cues In Modulating The Perceived Crispness And Staleness Of Potato Chips. Journal of Sensory Studies, 19, pp.347–363.