Blog

Sensory

Touch awareness: its role for IN-TOUCH

Recent visits and discussions with various technologists, including robotics engineers and computer scientists, have made us think about the question of how the development of touch-related digital technologies might bring about new awareness of ‘touch’: the conscious sensation of touching, and the specificity of those particular sensations. Some robotics engineers develop robots to work in situations that reduce the risk of harm or danger to humans, for example in bomb disposal or high radiation contexts (like the recent ‘little sunfish’ that can enter the Fukushima reactor). Where these robots have to undertake delicate tasks of manipulation or construction, there is a need to train them to ‘touch’ or engage with the machinery in an appropriate way – both manipulatively and with sensitivity to pressure, especially where the ‘touch’ involves delicate mechanisms.

One way of training these robots is through getting them to mirror human action of similar manipulations. In such training situations, we assume the human trainer has to consciously think and be aware of the specific touch quality being conveyed to the robot, as well as the parts of the hand (e.g. fingertips or palm) or arm that are critical to ensuring effective operation. This process, then, makes salient properties of touch that we are typically unconscious of, and is interesting to the In-Touch project in developing our understanding of touch in a digital context. More than this, though, the ‘awareness’ process must be instrumental in supporting the ongoing development of the ‘touch’ capabilities of the robots, specifically in terms of knowing what aspects of the mechanical operations are most essential, and which sensations, like pressure, including the degree of these different sensations – are most critical. While some engineers still work with human anatomy as a template for robotic form and action, many robot designs actively move away from mimicry, for instance by managing without the superfluous ‘little finger’ which usually serves a stabilising function. This raises questions of optimising robotic touch in digital contexts as similar or different from human touch and its associated, often taken-for-granted sensations, skills and knowledges.

Similarly, the Kissenger we are in the process of trying out brings to consciousness awareness of the ‘touch’ sensation on your lips or cheek in the context of conveying a ‘kiss’. The Kissenger uses motors under a soft ‘skin’ that create a tactile sensation when placed on your lips or cheek, and while this does not feel exactly the same as a kiss, it can enable us to think about what the qualities of a kiss are that are needed to convey a message. For In-Touch this will be valuable in developing a language of touch, and in informing about the role of digital touch in making salient certain sensations, what these sensations are, and the role they play in communication. It also raises the question of the role of intentionality in both sending and receiving such communication – if you know the intention is for your loved one or friend to send you a reassuring kiss, then the sensation you feel from the Kissenger can be mapped to your own experience of being kissed.

kissengerdevicewithimage

This brings to the fore the centrality of touch awareness and the meanings of touch with regard to human-to-human and human-to-machine communication. Within medical contexts, we have moreover encountered research and development around the notion of communication between body and environment through digitally mediated interfaces. Here, prosthetic devices might bridge sensory gaps for patients who have lost some touch awareness and need to be made aware of their surroundings, for instance regarding unwanted pressure on foot ulcers. In this context, it is the very notion of touch awareness itself that comes into focus for the design of digital touch communication.

 

Sensory

Visual touch or sensing with the eyes

The notion that it is possible to ‘see with the hands’, as Descartes once put it in Dioptrique (1637, see Paterson 2016), chimes with the popular imagination of the sense of touch as somehow enhanced in people with impaired vision. It is linked to centuries of philosophical debate and scientific research of how blind people ‘visualise’, what their sense of spatiality might be, and how touch may be part of providing and processing wider environmental information. These are in parts deeply political questions: labelling people as blind and trying to allow them to see in other ways is considered by many as simultaneously constructing them as ‘lacking’ fundamental sensory functions (Paterson 2016), as having a defect. While this deficit model is problematised as indicative of societies in which emphasis is put on vision as a dominant sense, it has certainly fuelled a lasting interest in cross-modal perception and, in the last forty years or so, the development of sensory substitution technologies.

Paul Bach-y-Rita was a pioneer of neuroplasticity, or the idea that the brain continually adapts and reorganises itself through new neurological connections in the course of a lifetime. He was perhaps best known for his development of Tactile-Visual Sensory Substitution (TVSS) systems, including the below chair that translates visual into tactile images and conveys these through vibration to the subject’s back (Fig. 1).

TVSS_tcnl.bme.wisc.edu_170518

Fig. 1 Early TVSS system https://tcnl.bme.wisc.edu/projects/completed/tvss

More recent developments of this idea include tongue-machine electro-tactile interfaces, such as the commercially available BrainPort (Fig. 2). As its manufacturer points out, it doesn’t enable vision per se but is classified as an ‘oral electronic vision aid’, working like a ‘refreshable Braille display from which you learn to interpret the bubble-like patterns on your tongue as representative of objects in [your] surroundings’ (http://www.wicab.com/faqs).

Screen Shot 2017-05-18 at 14.15.20

Fig. 2 BrainPort sensory substitution system http://www.wicab.com

While the hands and finger tips are still commonly regarded as the main sensory mediators of touch, both in traditional Braille and with regard to emerging tactile displays (Xu et al. 2011), BrainPort and similar technologies demonstrate an interesting shift away from the hand to also incorporate other parts of the human body. In a related vein, Ingold (2004) and Macpherson (2009) have pointed to the ways in which blind people (and others) sense the world through their feet.

The above technologies and approaches to touch tap into a dialectic of both sensory isolation, that is, the notion of distinct modalities of touch, hearing or vision, and a merging of sensory categories. On the one hand, one sense or kind of sensory information is seen to replace the other. On the other hand, neuroscientific research points to ‘the extraordinarily rich interconnectedness and interactions of the sensory areas of the brain’, making it difficult to say ‘that anything is purely visual or purely auditory, or purely anything’ (Sacks 2011, 238)

To sensory anthropologists and particularly those engaging in phenomenological anthropology, the merging of senses is nothing new, partly because the Western 5-sensorium (of taste, touch, smell, vision and hearing) has been revealed as a cultural construct. Accordingly, sensory categories are not universal givens but are socially and culturally roduced and experienced. A multimodal social semiotic perspective recognises the social and cultural contingency of the senses. It attends to the ways in which people take up and use the sensory categories and the resources that are available to them to develop communicational modes that meet their social meaning functions (Jewitt, in press). Bearing in mind their cultural distinction, and social use, if touch apparently offers a route to visual imagery and imagination, what – if anything – can vision tell us about touch?

Within film studies, Laura Marks has advanced the concept of ‘haptic cinema’ or ‘haptic visuality’. The latter is described as ‘a sense of physical touching or being touched engendered by an organization of the film image in which its material presence is foregrounded and which evokes close engagement with the surface detail and texture’ (Kuhn and Westwell 2015). As Marks explains:

Haptic looking tends to rest on the surface of its object rather than plunge into depth, tends not to distinguish form so much as discern texture. It is a labile, plastic sort of look, more inclined to move than focus. The video works I propose to call haptic invite a look that moves on the surface plane of the screen for some time before the viewer realizes what it is she is beholding. Haptic video […] invites [a] caressing look. (Marks 1998: 338)

This notion of embodied viewing is also seen to test ‘viewer’s own sense of separation between self and image’ (ibid). Marks draws inspiration from the phenomenology of film and the sensation of ‘synaesthesia’ as advanced, amongst others, by Vivien Sobchack. ‘The lived-body’, Sobchack writes, ‘does not have [separate] senses. It is, rather, sensible’. (Sobchack 1991, 77)

Dyer and Pink have further argued that it is film’s ‘mimetic’ (Marks 2000) properties that render it ‘capable of drawing us into sensory participation with its world’ (Dyer and Pink 2015). As such, ‘the experience of film does not simply involve us looking at something that is external to us, but it is through the affordances of film that, in relation to the other constituents of our environments/worlds that viewing becomes meaningful’ (ibid). The visual, in this sense, is more than a text that can be viewed and interpreted, it is an environment that can be known and experienced:

Should the drawing or painting be understood as a final image to be inspected and interpreted, as is conventional in studies of visual culture, or should we rather think of it as a node in a matrix of trails to be followed by observant eyes? Are drawings or paintings of things in the world, or are they like things in the world, in the sense that we have to find our way through and among them, inhabiting them as we do the world itself? (Ingold 2010, 16)

While anthropological film has long sought to create empathetic entry points for viewers across cultural boundaries, this notion of inhabiting a visual medium goes beyond empathy to include a corporeal dimension akin to that described by filmmaker David MacDougall. Accordingly, visual media allow us to ‘construct knowledge not by “description” […] but by a form of “acquaintance”’ (MacDougall 2006, 220). Vision in itself has of course been conceptualised as a situated practice that can be trained to become ‘skilled’ (Grasseni 2010).

As we are developing our methodology for exploring the situated meanings and affordances of digital touch, the above considerations are at the forefront of our mind, specifically as regards the possibilities of employing arts-based and video-based methods in our fieldwork. Arts-based methods (e.g. cultural design probes, design provocations, future scenarios, and prototyping) will be employed to elicit a response (e.g. tactile sensations) from participants, engaging their imagination and encouraging a fuller physical response in ways that more formal interview and questionnaire techniques cannot achieve.

Videos will be useful to create audio-visual records of people’s interactions with touch technologies, which we can then revisit and explore as a team. Beyond this, however, we are interested in making use of video as part of a sensory-ethnographically inspired set of methods, including video tours or re-enactments (Pink and Leder Mackley 2014). Designed, framed and analysed through a sensory-ethnographic lens, these can provide additional routes towards empathic-embodied understandings of touch as they are performed and demonstrated in relation to the objects, technologies and environments in which touch occurs.

 

References

Dyer, Adrian G, and Sarah Pink. 2015. ‘Movement, Attention and Movies: The Possibilities and Limitations of Eye Tracking?’ Refractory: A Journal of Entertainment Media 25 (February). http://refractory.unimelb.edu.au/2015/02/06/dyer-pink/.

Grasseni, Cristina. 2010. Skilled Visions: Between Apprenticeship and Standards. New York; Oxford: Berghahn Books.

Ingold, Tim. 2004. ‘Culture on the Ground: The World Perceived Through the Feet’. Journal of Material Culture 9 (3): 315–40. doi:10.1177/1359183504046896.

———. 2010. ‘Ways of Mind-Walking: Reading, Writing, Painting’. Visual Studies 25 (1): 15–23. doi:10.1080/14725861003606712.

Jewitt, Carey. 2017 (in press). ‘Towards a Multimodal Social Semiotic Agenda for Touch’. In Advancing Multimodal and Critical Discourse Studies, edited by Sumin Zhao, Emilia Djonov, Anders Björkvall, and Morten Boeriis. London: Routledge.

Kuhn, Annette, and Guy Westwell. 2015. Dictionary of Film Studies – Oxford Reference. Oxford. http://www.oxfordreference.com/view/10.1093/acref/9780199587261.001.0001/acref-9780199587261.

MacDougall, David. 2006. The Corporeal Image: Film, Ethnography, and the Senses. Princeton, N.J.: Princeton University Press.

Macpherson, Hannah. 2009. ‘Articulating Blind Touch: Thinking through the Feet’. The Senses and Society 4 (2): 179–93. doi:10.2752/174589309X425120.

Paterson, Mark. 2016. Seeing with the Hands: Blindness, Vision and Touch After Descartes. 1 edition. Edinburgh: Edinburgh University Press.

Pink, Sarah, and Kerstin Leder Mackley. 2014. ‘Re-Enactment Methodologies for Everyday Life Research: Art Therapy Insights for Video Ethnography’. Visual Studies 29 (2): 146–54. doi:10.1080/1472586X.2014.887266.

Sacks, Oliver W. 2011. The Mind’s Eye. London: Picador.

Sobchack, Vivian. 1991. The Address of the Eye: A Phenomenology of Film Experience. Princeton, NJ: Princeton University Press.

Xu, Cheng, Ali Israr, Ivan Poupyrev, Olivier Bau, and Chris Harrison. 2011. ‘Tactile Display for the Visually Impaired Using TeslaTouch’. In CHI’11 Extended Abstracts on Human Factors in Computing Systems, 317–322. ACM. http://dl.acm.org/citation.cfm?id=1979705.

 

Sensory

Machine touch?

IN-TOUCH visited the London Science Museum’s new Robots exhibition this week. The exhibition sets out to understand what it means to be human by exploring the ‘very human obsession to recreate ourselves’.

The quest to build ever more complex robots has transformed our understanding of the human body, and today robots are becoming increasingly human, learning from mistakes and expressing emotions.

img_0847img_0846The exhibition tells the cultural, historical and technological story of humanoid robots through over 100 robots from the 16th-century mechanical characters to robots from science fiction and contemporary cutting edge research labs. The exhibition opens with a robotic baby that slowly opens its eyes and reaches out. It was made for use in films apparently and is very life like. IN-TOUCH love babies and I found it disturbing not to feel anything toward the little squirming robot. Then it slowly opened it eyes – but it was creepy rather than appealing. We started at this uncanny valley. As the robots became ‘less humanoid’ my emotions grew, which I found interesting. The robots became more human to me as they became less humanoid.

Unsurprisingly IN-TOUCH was drawn to the robots potential for touching, handling and manipulating. The exhibition narrates the wide range and complex movements people can make with their hands, more so than any other living creature, and how this makes humans unique. It also visualizes the difficulty that robots have in handling objects as well as some of the ways in which touch and grip are designed.

A range of Shadow Robot Company hands were exhibited  capable of gripping a drill, threading a needle, and able to move thumb and fingers to mimic nearly every move of a human hand.img_0850

One exhibit shows Cyskin tactile sensors are flexible touch sensorsimg_0857 that can cover the curves of a robot’s body (like skin). It collects tactile data that enables a robot to sense which part of its ‘body’ is being touched, and with how much force. The aim is that this will enable robots to interactive with humans in more intuitive and safer ways.

The BioTac Sensor is an exhibit that shows the potential of robots to mimic the physical properties and sensory capabilities of the human fingertip – and to ‘discover the future of machine touch’. BioTac is capable of sensing: force, vibration, and temperature which is ‘identical to human touch capabilities’. img_0859

The potential of robot-assisted communication is a key theme of the exhibition, notably in the form of robots that receive, interpret and respond to facial expressions.

Zeno is a child sized robot that detects facial expression, and then mirrors it: a therapeutic potential that is being used by researchers to explore and assist children on the autism spectrum to understand and communicate emotions. img_0870

Telenoid is a ‘huggable’ ‘telecommunication avatar’ that stands in for the person on the other end of a telephone conversation. I have to say I am not sure I could hug a Telenoid. It is designed to ‘bring people on long-distance phone calls closer together’. The caller’s voice and movement are captured by a microphone and camera and these are physically communicated to the receiver.img_0874

The exhibition posses a range of questions underpinned by human fear, anxiety, desire, and playfulness. Asking for example ‘could technology with a face and body one day be so satisfyingly real that we drift even further apart? Could robot-mediated contact replace face-to face interaction entirely? On the basis of this exhibition I would say there is a long way to go before that happens. Although according to recent piece by Trudy Barber The potential for virtual reality falling in love could already be a deeper experience than ‘in real life.

After the exhibition we went onto the Barbican to watch Robot a performance by the Blanca Li Dance Company. The performance engages with humanity’s evolving dependence on technology and the increasingly blurred boundaries between human and machine.  In the performance the dancers and robots mimic each other’s characteristics, with the aim, which is not always fully achieved, of them becoming almost interchangeable. What screen-shot-2017-03-02-at-16-29-18I found interesting was the emotions that the Nao Robots evoked in me, and others in the audience, as it fell and managed to right itself after falling flat on its face. It is very toddler like and I felt so protective of the little creatures. But could I love a robotic baby or toddler?

screen-shot-2017-03-02-at-16-29-32

Sensory

Book review: Losing Touch

Losing Touch – a Man without his body. Jonathan Cole, 2016, Oxford University Press

screen-shot-2017-01-13-at-15-28-13

This book is about understanding the experience of living with the loss of touch – cutaneous touch and movement/position sense (proprioception).

‘Touch Ian, stroke him, or put a heavy weight in his hand and he cannot feel it. For all the world he has no sense of light touch; nothing in daily living is felt on the skin unless it is hot, cold, itchy, or painful. ‘(p.111)

 It draws on the lived experiences of Ian Waterman who lost these senses suddenly at the age of 19, bringing together Ian’s reflections and those of the author’s scientific experiments with Ian, alongside medical explanations and understandings.

Ian’s initial terrifying experience of loss of touch from below the neck is described in the first chapter of the book. A key feature of which was the varied medical reaction to his unexplainable condition, with no clear diagnosis or treatment. It was a decade later that Ian’s condition was first named as ‘acute sensory neuronpathy’. During his hospitalization through intensive physiotherapy over a year and a half, and becoming aware of the physics of movement, Ian eventually learnt to walk unaided and after a similar length of time was able to live independently. However this is not a book about ‘recovery’: Ian does not recover his sense of touch rather he finds ways to live without it.

Twelve years after Ian’s loss of touch he met Jonathan Cole who was starting out on his medical career. The starting point of their journey over more than 40 years was a series of neurophysiological tests to stimulate and record Ian’s nerves to better understand his condition and the extent of sensory nerve function. The experiments conducted with and on Ian conducted by Jonathan and a variety of other scientists around the world are described in some detail, as is Ian’s experiences and commentary on them. The book is a telling narrative, notably on the limitations and expectations of scientific understanding, and the relationship between lab and the everyday experiences.

img_0794

The book offers the reader a first-person approach to the experience of living without touch. It connects neuroscience with the work of social scientists, phenomenologists, and gesture studies as well as the arts – Ian’s experiences have been ‘translated’ into theatre and dance:

 ‘A fusion of narrative and neuroscience, of the personal and the physiological, to understand touch and proprioception and its loss’ (p. 14)

 Chapter 5 reports Ian’s encounter with the theatre director Peter Brook. It discusses how Brook’s dramaturgical approach to Ian’s movement and pose or his bodily ‘performance’ moved beyond the technical aspects of his movement that were of interest to science to explore how he looked when he moved. This approach served to break down Ian’s embodied performance as a route to understanding his ‘self-image’ and his internal feelings or way of being. This experience also seems to have given Ian a vocabulary for talking about his own learned movements as being choreographed and sequenced.

The book’s interdisciplinary stance to touch speaks to the concerns of the IN-TOUCH project. In addition to the book’s exploration of Ian’s loss of touch and sensation were through scientific experimentation and the arts, chapter 6 discusses the potential of robotics for providing insight on Ian’s experience. During a trip to NASA Ian and others engaged with a mock up of a Space Station that housed a large robot. The use of the robot led a colleague to explain ‘ how he could see the robot arm and move it, but had no sense of touch, and how curious that was’. But as Cole notes, the colleague could ‘move and see the robot while also feeling his own arms and hands’ a rather different experience than Ian’s, nonetheless the ultimate shared experience was one of becoming embodied and entwined in the robot.

img_0795

Some of the question’s raised in the book connect with those raised by the ways in which digitally mediated touch might reshape communication, including:

  • How might the digital reconfigure the limits to touch and movement with no feedback?
  • Will the digital re-mediate the boundary between touch and the embodied self?
  • How does temperature become heightened as a resource and newly embedded in experience when proprioception is absent? (In the book, Ian used temperature as a way of understanding the spatial environment and to achieve actions that usually rely on proprioception).
  • How does touch interact with the visual? How does augmented and virtual reality shift those interactions? what might that mean for communication and meaning making? (In the book, Ian talks of having a visual map ‘in his head’ which enabled him to move his hand accurately in different positions and that being able to visualize a movement enabled the movement)
  • What does it mean to touch an object digitally (e.g. moving an object with a robotic arm) but have no sense of what it feels like? Is that touch?

This book while focused on the loss of touch provides useful ground on which to think about how the digital may re-mediate touch through the design of sensory digital communication devices.

 

Sensory

What is the role of e-textiles and smart textiles for touch based communication?

Smart textiles or e-textiles are ordinary materials with a range of electronics, conductive materials and sensors woven into the fabric, and that respond to movement and/or touch. Smart textiles are materials that have ‘sensing and actuating properties’. This means they can both ‘sense’ and ‘react’ to external factors in the environment, e.g. mechanical, chemical, thermal, magnetic; they can release medication or regulate body temperature; change colour, light up and display other forms of visualisation (Syduzzaman et al, 2015). According to Syduzzaman et al there are three generations of smart textiles: (i) provides an additional feature in a ‘passive’ mode e.g. anti-microbial material; (ii) has both sensors and actuators, and automatically adapt to changing environments e.g. thermo regulated materials; and (iii) ultra smart textiles sense, react and adapt, they can sense, interact and communicate.

History of smart textiles: Electricity being combined with clothing is not particularly new. It was used in jewellery and gowns at the end of 19th century (Gere and Rudoe, 2010: Jewellery in the Age of Queen Victoria: A Mirror to the World); in astronaut space suits that could inflate, light up, heat or cool down in the ‘60s; for electroluminescent party dresses (Diana Dew); and a fully animated T-shirt in the ‘80s (Harry Wainwright. Since then developments in automatic embedding of fiber optics into fabrics, use of GSR sensors linked to Bluetooth, infra red digital displays and the development of sensor based conductive yarn, conductive rubber and conductive ink, and shape memory materials are all contributing to new ways of thinking about and potential uses of smart textiles (for more detail take a look at Syduzzaman et al., 2015)

Contexts of use, monitoring: Key applications found in the literature are health, safety, military defence, sports (tracking and monitoring systems) and fashion; and more recently super smart textiles are being further developed, driven by needs within space travel, sports etc. According to Martin et al. (2003) wearable devices have really taken off in the context of health and fitness, with tracking devices monitoring physiological changes; behaviour; eating, activity, sleeping habits, context-awareness and location sensing. However, smart clothing/ textiles that are currently available are limited to extreme sports, fitness and entertainment – due to durability limitations (Gorkey, 2016). This perhaps highlights a distinction between ‘wearable’ technologies and ‘smart textiles’, where sensors or conductive materials are woven into the fabric itself. Nevertheless, Clothing + is integrating sensors into clothing – basically to monitor and understand physiological data, such as heart rate, breathing, calorie burn; fatigue levels etc. recognising such changes in your body can alert you to stress levels, for example, and enable you to take steps to alleviate this (e.g Poupyrev, 2015). Another form of ‘monitoring’ can be seen in the fashion industry. For example Intel Edison’s Spider dress is a smart textile dress that responds to mood: it has moving arms on the shoulders that reach out when the wearer feels uncomfortable, sensing this through embedded sensors.

Screen Shot 2016-12-14 at 16.08.31.png

Robotic Spider dress, powered by Intel Smart Wearable Technology Fashion January 6th 2015

A key question here is whether these kinds of applications are ‘touch’? Are monitoring forms of activity generated through the clothing a form of touch, since they touch the body of the wearer? Are they communicating through touch, since they are sensing different bodily changes and adapting or responding accordingly? Is clothing that enables self-expression and a form of communicating your emotions a form of ‘touch’?

Contexts of use, art/ music: smart textiles are increasingly being use by artists to enhance engagement with audio or music, e.g. Soundbrenner, worn on the wrist or upper arm, helps you feel the beat and rhythm of music. Others are using these textiles to create new art forms, for example, Maggie Orth uses smart textiles for ‘painting’ through large cloth displays, using conductive yarn, thermochromic ink, drive electronics and software, focusing on programmable colour change textiles and touch sensitive textiles with light. These applications offer new material forms for artists and designers, and offer opportunities for new forms of touch-based interaction with art pieces.

Context of use, disability: e-textiles have often been used for their visual properties, but are less understood in terms of their tactile qualities (Giles and Van der Linden, 2014). Indeed the monitoring applications focus less on the tactile qualities of the fabric and more on the sensing capabilities, and displays or communication of sensor output, for example linked to visual forms of feedback through LEDs. Much of this research is not centrally related to touch – yet for the visually impaired the touch sensory system is critical. In the current landscape of smooth touchscreen technology then, smart textiles seem to have much to offer for the visually impaired. Emilie Giles, Janet van der Linden & Sarah Wiseman have done some interesting work – Flatland – looking at performance through touch using haptic technologies – looking at the ‘science of communicating by touch’.

screen-shot-2016-12-14-at-16-10-37

Ghostly-figure-walking-away Flatlands

“In our adaptation of the 19th century novella the audience is invited to explore this world using a haptic cube device, which leads them around the completely dark space, designed this way to give visually impaired and sighted audiences a shared experience. The project investigated how technologies centered around touch and bodily perception can form a new sensory means for audiences to engage with dramatic installations”. http://www.emiliegiles.co.uk/Flatland

This form of performance enables visually impaired and sighted audiences to have a shared experience through tactile forms of interaction.

Communication/ connecting through smart textiles

In parts of the fashion industry the focus – rather than monitoring – has been on connecting people through textiles e.g. the ‘hugshirt’ (CuteCircuit) or the ‘alert shirt’ (from We:eX and Foxtel). The hugshirt, developed by Rosella and Genz in 2002, won an award and was launched in 2006, then updated in 2014. It was designed – not to replace hugs – but to enable people to send hugs to one another when they were away at work or not co-present. The ‘alert shirt’ enables football ‘observers’ to feel what the players are feeling e.g. heart rate changes, bump from a collision between players through haptic feedback on the t-shirt. These two examples offer different ways of connecting – one is a more intimate expression of emotion for someone else, the other a shared experience aiming to enhance engagement – in this case through some kind of immersion in the football game. Both offer explicit ways of enhancing touch interaction, yet also raise issues about the implications for human-human touch contact.

screen-shot-2016-12-14-at-16-13-10

Right: The hugshirt, Left: The Alert shirt

Sharon Baurley describes the heat sensation of the hug shirt as ‘symbolising’ not ‘simulating’ the warmth of a touch/hug. This seems like a crucial distinction to make, especially for those that become concerned about technology ‘replacing’ the human-human, which is considered central to human communication and well being. The symbolic experience is thus different and distinguishable from the ‘real’ experience. This is good in the sense that new experiences are possible, experiences that symbolise something important that may not currently be possible on an everyday basis, or that symbolise someone else’s experience in a way that enables enhanced empathy or engagement e.g. the alert shirt. The symbolic becomes more questionable if it develops to increasingly replace and ultimately threaten commonplace human-human touch. What is important is that we develop a better understanding of what ‘touch’ and ‘digital touch are and the role they each play in human-human communication

According to Rosella what is important with the ‘hugshirt’ is that both the sender and the receiver are jointly creating the human-human communication via the technology. The use and control of the experience is jointly negotiated and becomes part of the emotional engagement between those two people. However this is a specific and constrained scenario – what would it be like or what would happen if all our clothing were smart textiles and the increase in touch communication extends beyond intimate or close family/friend relationships? Would the ‘touch’ sensation then take on a different meaning? Would you be able to distinguish ‘touch’ from different people or different sources? How difficult would it be to maintain control of both sending and receiving touch communications?

What other issues are raised?

Anne Cranny-Francis chapter on Smart Textiles raised a number of issues around touch, and led me to think more about some key questions that smart textiles give rise to in the context of InTouch.

She raises the issue of the ‘makers lack of awareness of the significance of human touch’. To me this raises the question about the difference – for both toucher and touched – between human-human contact, and ‘touch’ contact mediated through technology. Such questions often raise tensions about technology, where on the one hand escalated concerns emerge that human touch will be ‘replaced’ by technology, yet on the other hand the same technologies can be seen by others as serving a valuable purpose in enabling ‘touch’ forms of communication, when such communication cannot otherwise take place e.g. hugging /or touch from a distance when you are away from a loved one. In this case, the technology is extending not replacing touch communication.

The notion of ‘being replaced’ links to the question of whether these textiles can replicate human touch e.g. the right pressure, and highlighting that the lack of other sensory information would significantly change a ‘real’ touch experience, since it wouldn’t accompanied by, for example, the smell, sound of the other person. So we need to ask ourselves whether e-textiles or smart textiles need to ‘replicate’ and where they might be useful without this level of ‘replication’; or indeed, if technology develops such that they can be replicable to this degree, then what issues does this raise for touch communication? Does this mean that human proximity will lessen? One aspect of this idea of replication made me think more about the ‘function’ of touch, and whether this is critical in thinking about contexts of use. So, if the touch action is purely ‘functional’ ie you touch something to make something happen, pressing a switch, then this would not interfere with human-human touch communication – although it might change the individual interpretation of the interaction with the object depending on the feedback of the tactile interaction

Finally, I was struck by Anne’s experience with Fauxy, and the adaptation process she went through, from feeling surprised and responsible for the movement in the clothing beyond her control, to having a feeling of ‘being able to see behind’ her. I think this example demonstrates how an initial wariness of new technology and ‘automated’ behaviour can be reconfigured with a new and interesting awareness.

In thinking specifically about smart textiles, the thoughts here link into broader questions raised about what we mean by communication – with whom? With what? Two way or one way. It raises questions about touch, and what it is to touch or be touched through textiles, through technology, who is touching, who is being touched? What is the function of touch for oneself, for others? Are smart textiles about monitoring, communicating or feeling of fabric?

Loren Klein “to touch is to experience, but to feel is to live”

Readings

Cranny-Francis, A (2013) Technology and Touch: The Biopolitics of Emerging Technologies. Palgrave Macmillan.

Giles, E. and Van der Linden, J. (2014) Using e-Textile objects for touch based interaction for visual impairment. In ASGA Atelier of Smart Garments and Accessories, 14 September 2014, Seattle

Gokey, M. (2016) Why smart clothes, not watches are the future of wearables. Digital Trends, January 13th 2016

Syduzzaman, Patwary SU, Farhana K, Ahmed S. (2015) Smart Textiles and Nano-Technology: A General Overview. Journal of Science Engineering, 5, 181

http://www.digitaltrends.com/wearables/smart-clothing-is-the-future-of-wearables/

Sensory

A renaissance of touch in the museum?

Touch was a significant feature of visiting museums in the past. In the late seventeenth and eighteenth centuries visitors to the Ashmolean (Oxford, UK) and the British Museum (London, UK) were allowed to handle, rub, shake and even taste objects on display. Restrictions on how objects could be handled and by whom emerged over time and by the mid nineteenth century only curators and conservators were able to enjoy physical interaction with the objects within the museum.

The demise of touch has been linked by some with modernity and its turn to observation, objectivity, the visual display of capitalism, and the equation of touch with ‘primitive cultures’ (notably Howes and Classen, 2014). Howes, notes (2008) that touch was removed via people learning how to be ‘a museum visitor’ through taking up various prestigious bodily techniques that displayed ‘the requisite degree of cultural competence’ – ‘to stand at the correct distance from an art work, walking at a pace that is neither too fast nor too slow, and knowing what to “feel” (without touching, of course). This created the art work as an object to be contemplated which ‘resulted in the interposition of more and more distance between the average museum visitor and the work of art or exotic artifact, whether there were physical barriers (a glass case, a rope) or not.’

screen-shot-2016-11-28-at-18-52-14

Touch and class: Changes in the social class of museum visitors have also been linked to the demise of touch: ‘the upper classes always had license to touch and their touch was deemed rational and non-damaging. On the few occasions when the lower classes could touch museum collections it was considered unruly and dirty’ (Candlin, 2008: 9). Some argue that it is the move of museums from being restricted and elite to public and open that touch was prohibited and the reign of the visual spectacle of the museum began. For some, the shaping of the museum as a site of pure spectatorship can be linked to the development of visual technologies (the microscope, telescope) and the modern era. However others argue that touch opens up ‘imaginative, speculative and emotional ways of knowing material objects’ that are not in opposition to rationality but form part of the rational project. Thus, they dispute that there has been a progressive trajectory of ‘transition from multisensory, non-rational and premodern experience to visual, rational and modern knowledge’.

Visitors’ deep-rooted desire to touch objects: The work that museum or gallery needs to dedicate to protect artefacts from the touch of visitors is testimony to visitors’ deep-rooted desire to touch objects to acquire information about them:

Watch the crowds in any museum and, despite the prohibitive ropes and signs, somebody will be touching something…to investigate an object’s surface, to verify what they have seen, or in an attempt to make a connection with the past. (Candlin, 2008: 89)

This may account for the ever increasing role of the museum gift shop (related to our previous blog post on touch and retail: ‘Touch it: Want it’) touch and handle replica jewelry, textiles, and sculptures

A renaissance for touch in the museum: Over the past decade or so, however, the ‘sensory turn’ has led some contemporary museums to rethink their restriction of the senses and how they might revive sensory opportunities for visitors to engage with objects. The move to touch in the museum and gallery has been led by Blind and visually impaired activists and advocates. This has led to the establishment of a number of tactile museums around the world, the development of touchable collections (e.g. plaster cast rooms), as well as touch tours, and other events dedicated to the handling of objects (albeit objects of limited significance or value). There has been particular interest in how the role of touch could be exploited and expanded in the museum in the wake of studies that have shown the potential social, cognitive and therapeutic value of handling objects (Chatterjee, 2008, 2013).  Below we set out some of they key issues for touch in the museum below before briefly discussing the potential of the digital to speak to and these.

Touch as access: To some extent the shift to sensory experiences in the museum have become increasingly entwined with the politics of social inclusion and widening access to the museum. This shift can also be seen in schools, and other learning environments and institutions. However, as some have pointed out, touch alone does not equal ‘access’ to collections if it is not also accompanied by intellectual context; and while touch tours are usually restricted to blind or visually impaired visitors, it is not only those visitors whose understanding is enhanced by touch.

screen-shot-2016-11-28-at-18-53-46

Ways of knowing through experiencing the properties of objects – The use of touch is nonetheless a means of learning is fundamental to contemporary notions of exploratory learning and investigation as well as common practice. It is common practice to both visually inspect objects under study but this often needs to be accompanied by the handling of an object, turning it, feeling its texture and weight, temperature and other physical properties.

Evoking tactile memories – Museums can use objects to stimulate memories among visitors, supporting them to reconnect with a past experience, and how this can help to reconstruct lived narratives and identities. This can help visitors to access the benefits of reliving moments triggered by handling of objects. Touch is a part of the phenomena of artists designing exhibits to be touched and physically interacted with in the museum or gallery –Patrick Dougherty’s natural environments, John Bock’s strange buildings, Felix Gonzallez-Torres installations. They are explicitly designed to evoke touch-based memories, and responses in which visitors’ physical interaction is a part of the production of the piece.

Creating personal tactile dialogues with museum collections – touching objects in the museum can help to generate a sensation of intimacy with the original creators of the objects on display. In turn this can help visitors to empathise and re-enact lives of others. Historians and anthropologists have pointed to some of the challenges of telling historical narratives through the senses – for instance, the need to understand that the senses themselves are both a cultural and a natural phenomenon – and what felt soft, or new a century ago may not feel the same today (Levent and Mc Rainey, 2014).

Increased engagement, aesthetic tactile pleasure is also a part of touching in the museum.

Making objects meaningful through touch– it is not always clear what an object is, what its meaning and use is. Sometimes and object needs to be returned to its original communities, shared and placed into context, its material and techniques of production studied. For example, Gadoua (2008) sets out a study conducted at the McCord Museum in which groups of Inuit elders were invited to share their knowledge and memories about objects of everyday life collected in the Canadian Arctic during the first half of the twentieth century. The elders’ interpretations of ancient hunting equipment, sewing paraphernalia, household utensils, and personal ornaments informed the archaeological analyses of analogous artifacts. This necessitated the handling of the objects to generate various meanings for the objects: how they were made, and uses in the past, as well as their contemporary value for the preservation and promotion of Inuit traditions. Collaborations such as this can also support the handling of objects to regain traditional craft skills and cultural practices.

Digital touch and the museum – As we develop IN-TOUCH we will explore how the digital can be used to explore and exploit each of the key themes above. For example:

Tangible and embodied prototypes are being developed by the Touching the Untouchable project to support virtual handling of archaeological artefacts in meaningful ways by placing them in contexts of use and provide uses with a sensorial and embodied cultural heritage experiences. Such multisensory interactions present significant opportunities for interactive exhibits to expand our access to cultural history and its artifacts.

screen-shot-2016-11-28-at-18-48-36

Reverse electrovibration (virtual touch) can be used to facilitate electronic transmission of the human tactile sense, allowing end users to perceive the textures and contours of remote objects. For example, REVEL, imparts a low-level signal, creating an oscillating, weak electric field around the user’s skin. Signal variations correspond to texture variations in the distant object. The signals are generated in such a way that the resulting sensation in the fingers mimics the sensation of sliding the fingers over the object.

 

We will ask what, if anything, the digital can add to ensuring a more subtle understanding of the relationship between touch and meaning as socially and culturally constructed, and historically situated in the museum.

 

Readings

Candlin, F. (2008) Don’t touch! Hands off! Art, blindness and the conservation of exptertise in Pye, E. (ed) The power of touch: handling objects in museum and heritage contexts. Walnut Creek: California, USA.

Candlin, F. (2010) Art, museums and touch. Manchester University Press.

Chatterjee, H. J. and Noble, G. (2013) Museums, Health and Wellbeing. Ashgate Publishing Ltd. Farnham, UK

Chatterjee, H. J. (2008). Touch in Museums: Policy and Practice in Object Handling. Berg Publications.

Gadoua, M. (2008) Making Sense through Touch: Handling Collections with Inuit Elders at the McCord Museum. The senses and society. 3 (3): 323-341

Howes, D. (2008) Introduction to Sensory Museology, The Senses and Society, 9:3, 259-267, DOI: 10.2752/174589314X14023847039917

Levent, N and Pascual-Leone, A. (2014) The Multisensory Museum. Rowman and Littlefield: Plymouth UK

Pye, E. (ed) (2008) The power of touch: handling objects in museum and heritage contexts. Walnut Creek: California, USA.

debates, why touch matters

Touch it – Want it? Using Touch to Sell

Touch is key to retail and consumerism. The role of touch in the success of the mid-nineteenth century phenomenon of the department store is well documented by Classen in The Deepest Sense (2012), her seminal cultural history of touch. She writes of how the department store brought the goods previously out of sight and touch behind the counter into the customers’ view and tactile reach. Allowing customers to touch goods served to increase their confidence in the quality of products on sale and to create a subtle ‘bond of attachment’ or sense of ownership between customer and product. This bond is considered by marketing psychologists to drive impulsive, ‘must have’ buying behavior. It is this idea, rather than security, that underlies the movement of products from display cases into a customer’s hands in many contemporary stores – from iphones in Apple Stores to high-end jewellery. In short, products are increasingly designed and marketed to deliver a range of sensory experiences and emotions: touching a product tends to lead to more unplanned purchases.

Touch is also known to create symbolic connections of ‘warmth’ between people – in retail environments between customers and sellers. Interpersonal touch, such as a handshake or on the shoulder, can convey trustworthiness and lead people to feel safer and subsequently spend and consume more.

A recent ‘taxonomy of touch’ in consumer behavior identifies four types of touch (Peck, 2010). Three are subsumed within the category of ‘instrumental’ consumer behavior i.e. touch as a means to an end: goal directed, problem solving, and pre-purchase behavior. These include touch to purchase; touch to obtain non-haptic product information (e.g. visual inspection), touch to obtain haptic product information (e.g. specific material properties – weight, texture etc.). The fourth is ‘Hedonic touch’ in which touch is an end in itself (e.g. sensory exploration and pleasure). Being allowed to touch a garment or object in a shop influenced and increased the purchase of an object (Childers and Peck, 2010). Peck also developed a ‘Need for Touch’ scale that is defined as a preference for obtaining information through touch and hedonistic pleasurable touch.

Witnessing the incidental touch of other people in a retail environment can also shape consumer touch behavior (Morales, 2010) due to ‘contagion effect’. That is, the belief that once a person touches an object something of that person is transferred to the object. Research has shown that where the contagion effect is negative – that is it devalues an object this is related to fear of contamination or more directly to the consumer’s response to either the person who has touched the object (who is usually unknown to them). Positive contagion effect is rare, and related to the source of the touch being liked or aspirational. The specifics of a product, the preferences of an individual, and situational factors (the particularities of an environment, e.g. signs encouraging touch, the taking of objects out of cabinets to be touched, or a lack of touch) all combine to shape people’s motivations to touch.

How to seduce people into touching an object is at the heart of marketing. Strategies include offering consumers ‘tangible comparisons of materials’, novel textures and surfaces, modeling the enjoyment of touching an object.

Written descriptions and visual depictions can compensate for a lack of touch in some contexts and for some products. While people have different relationships to touch, for certain types of product qualities ‘there is no substitute for actual touch’, according to Peck and Childers.

Technology, notably online shopping and virtual reality, has changed the way retailers and brands operate over the past decade. However although more people are happy to purchase products online without the opportunity to feel them, a recent survey shows that the majority (75%) of people still want a physical experience when purchasing mobile phones, clothes, and furniture among other products. It is interesting to me that more than a century on from the first department store, it remains important for customers to touch and feel products in a physical store across a number of retail sectors. Although virtual try-on environments can give basic visual access to the experience of an object, touch currently remains out of reach. Current online environments provide a poor sense of materials, their weight, drape, how they might feel to touch, and do not convey the complex sensory experience of materials. iShoogle is an example of an innovative multi-touch interactive video representation of the tactile qualities of materials that aims to understand the felt properties of textiles and how people inherently touch and handle them to enhance the communication of tactile qualities of textiles. For the time being however, where there is no opportunity to touch (e.g. online or catalog) marketers and retailers continue to work to seduce and persuade consumers to buy. They use strategies to compensate for the lack of touch: the use of images and videos, showing others holding and touching object, writing that supports the imagining of owning and describing feeling, for example. These strategies are used alongside a good returns policy and an increased click and collect service: a combination enables shoppers a seamless experience that connects online (visual and audio) with physical (touch, olfactory) consumer experiences.

The potential to touch remains a strong aspect of marketing and purchasing behaviour: as consumers and researchers we need to be aware of and understand how it is being used to sell products.

References

Atkinson, Douglas and Orzechowski, Pawel and Petreca, Bruna and Bianchi-Berthouze, Nadia and Watkins, Penelope and Baurley, Sharon and Padilla, Stefano and Chantler, Mike (2013) Tactile Perceptions of Digital Textiles: A Design Research Methodology. In: CHI ’13 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp. 1669-1678. ISBN 9781450318990

Constance Classen (2012) The Deepest Sense. Berg: Oxford.

Joann Peck (2010) Does touch matter? Chapter 2 in Aradhna Krishna (ed) Sensory marketing: research on the sensuality of products, Routledge: London.

Andrea Morales (2010) Understanding the role of incidental touch in consumer behavior. Chapter 4 in Aradhna Krishna (ed) Sensory marketing: research on the sensuality of products, Routledge: London.

Terry Childers and Joann Peck (2010) Informational and affective influences in consumer behavior. Chapter 5 in Aradhna Krishna (ed) Sensory marketing: research on the sensuality of products, Routledge: London.

Verdict and British Land (2016) The True Value of Stores http://www.britishland.com/news-and-views/press-releases/2016/18-07-2016a