Sensory

‘Touch Tools’ at IEEE World Haptics 2021

By Kerstin Leder Mackley (@kayle10) with InTouch (@IN_TOUCH_UCL)

[Touch Tools Workshop Website]

In July 2021, InTouch brought together an international panel of speakers from academia and industry for an interactive workshop at the IEEE World Haptics Conference 2021. ‘Touch Tools: Bringing Social-Sensorial Tools to Digital Touch Design’ explored the kinds of conceptual and design tools needed to make the complex space of haptics a diverse and ‘touch-sensitive’ one, as well as a space that accounts for different social contexts and sensorial experiences. Its aim was to both explore connections between existing tools and identify needs and opportunities for the creation of new ones.

This thinking piece provides a summary of workshop contributions, along with some of the themes and creative tensions that emerged from the session.  

[Touch Tools speakers and organisers]

Part I: Setting the Scene

As with many events in the 2021 conference calendar, Touch Tools took place online. Three short provocations kicked off the workshop.

The Need for Touchy Tools

First, Carey Jewitt from InTouch presented highlights from the collaborative Manifesto of Social Touch in Crisis. A manifesto, Jewitt posited, offers a chance to question the status quo, and to prompt new ideas. It often emerges from moments of uncertainty or crisis. Touch can be seen as in crisis, not least in the face of COVID-19 which has arguably unsettled and shifted touch etiquette. Even pre-pandemic, touch had reached crisis points, with people seen to either touch too much, not enough, or inappropriately/illegally (e.g. #metoo).   

The manifesto’s call for action invited haptics designers and developers to engage more with the social and sensorial character of touch and, in the process, to ‘democratise touch’ (Manifesto Statement #3), ‘remake, don’t only replicate’ (Statement #7) and ‘develop open touchy tools’ (Statement #9). Everyone, the manifesto argues, should have a say in the future of digital touch. Haptics design should account for different voices, for diversity in backgrounds, bodies and sensorial experience. It should not be limited by analogue reflections but engage with the unfamiliar, break free from mimicry and foster alternative touches. Tools should be collaborative, shared, varied, enabling makers to communicate across design spaces, to expand touch in the realm of the digital.

The Design Space for Design Tools

Following on from Jewitt’s broad vision, Karon MacLean from the Sensory Perception & Interaction Research Group (SPIN) at the University of British Columbia honed in on the environment of haptics design, putting forward three crucial considerations for creating haptic sensations. First, MacLean reminded us of the different stages of design, each necessitating different tools for browsing, sketching, refining and sharing.

[MacLean: The design space for creating and sharing touch experiences]

These come with additional challenges in the haptics space where sketching and sharing, for instance, is not straightforward. Second, MacLean invited hapticians to position themselves as storytellers, and see touch sensations as actors in a theatrical production. Our job, she argued, is to set the stage – to explore and develop a narrative and setting, and consider other sensory modalities (sound, vision…) as additional actors… And to improvise! Like Jewitt, MacLean was keen to throw away preconceptions of how touch should work (e.g. vibration as mobile device notification) and to consider how touch could be used in new ways in concert with other modalities. What are the roles for haptic elements in supporting an application narrative? Third, again paralleling Jewitt’s call for action, MacLean opened up the question of who designs, and who for (e.g. specialists, end users). The process, she argued, can and should be shaped by a range of people (artists, engineers, UX experts…), all involved in different components (technology, haptic content, end-user experience…) and at different stages of design (early concept, refining, field testing…). Accordingly, touch tools should be accessible to support a broad range of ‘designers’ with different skills in different tasks.  

Creating Game Experiences with Multichannel Haptics

Dave Birnbaum’s inspiring talk concluded the first part of the Touch Tools workshop. Approaching touch tools from an industry perspective in his role as senior technology strategist at Immersion, Birnbaum focused in on three specific ‘tools’ at the hands of hapticians: the physical model of touch; a model of touch as meaning; and the landscape of tactile technology. Like MacLean, Birnbaum highlighted the wider environment – what he called the broader eco system – of haptics, allowing hapticians to orchestrate and synchronise touch sensations in relation to wider sensory experiences and interfaces. Because touch is immensely broad, he argued, it makes sense for designers to break it down into different sensory channels and explore each in its own right. Again chiming with previous talks, Birnbaum called for hapticians to look at what haptics can do, and what meanings touch can convey, bend and play with in the digital-material space. To do this, it helps to understand the landscape of tactile technology; the rich palette of tactile, kinesthetic, thermal and contactless interfaces that often begin life in research labs but are increasingly becoming commercially available. Birnbaum concluded with a portrait of the haptician as generalist – and of haptics teams as necessarily multidisciplinary – as they connect different levels of the haptics stack from design guidelines of what touch is capable of via software to hardware. (See Dave’s slides for a list of handy creation and prototyping tools.)

Part II: Introducing Touch Tools

In the second part of Touch Tools, Hasti Seifi, Bruna Petreca and Kerstin Leder Mackley introduced a set of tools and toolkits, each following their talks with hands-on activities for workshop attendees on the online collaborative space Miro.

Browsing Haptic Content

Presenting a vibrotactile design tool, VibViz, and an interconnected archive of force-feedback devices, Haptipedia, Seifi demonstrated the possibilities and significance of browsing haptic content via digital platforms. Browsing, a step often skipped by novice hapticians, is not only important for generating new ideas, she argued, but also to actively build on and modify existing designs, and move towards shared understandings of how to talk about and standardise haptics. Deciding whose (or what kind of) ‘language’ to use when working across different cultural contexts remains a key challenge.

Designing with (Digital) Textiles

It is a challenge Bruna Petreca from the Royal College of Art picked up on in her talk on sCrIPT, a methods card toolkit for designers engaged in digital fashion. sCrIPT (system for Collection, (re)Interrogation, Projection and Transformation) supports designers to articulate and explore the ‘feel’ of fabrics in the digital realm where their materiality is missing. Petreca’s work draws heavily on the tacit and embodied processes designers employ when handling and exploring textiles in the real world. It’s these details that not only allow for enhanced sensitivity towards sensorial nuances but also help translate these into digital contexts. The toolkit goes beyond the written or spoken word, with the methods cards encouraging users to sketch or act out activities, to engage with their bodies and imagination.

Designing Digital Touch

sCrIPT shares this embodied and activity-orientated approach with the Designing Digital Touch (DDT) Toolkit, presented by Kerstin Leder Mackley on behalf of InTouch and collaborators from the School of Design and Creative Arts at Loughborough University. Loosely based on the Double Diamond design framework (but also allowing to break free from this), the online toolkit supports designers to put touch at the forefront of their thinking and making across the design process. Three card types – filters, activity cards and wild cards – provide different entry points into the design space, prompting designers (and researchers) to unpack social, sensorial, spatial, temporal and material aspects of touch interaction in co-located and remote settings.

Part III: Toolkit Interactions on Miro

The toolkit introductions were followed by structured activities, allowing workshop attendees to explore toolkit cards and digital platforms collaboratively on Miro. The aim was for attendees to relate tools to their own devices or concepts, and explore the haptic sensations at the centre of these. Albeit all too short, the interactions brought to the fore and added to many of the challenges and opportunities identified by workshops speakers. They also inspired reflection on the creative spaces and tensions opened up by a combination of presented tools:

  • The need to attend to sensorial-material nuance and diversity, versus the drive away from direct replication of ‘real-life’ experience or feel;
  • The tension between using haptics for what it can do and mean, versus allowing for new and unexpected sensations/meanings;
  • The need for some form of shared language around haptics and sensorial meaning (to enable degrees of standardisation, for example), versus the incompatibility with a notion of detailed ‘linguistic’ information in touch;
  • The tension between focusing in on touch, versus allowing it to play a tactile part in multimodal constructions of meaning.

[Toolkit interactions on Miro]

Of course, the workshop reminded us – and MacLean, in particular, pointed out – that designers are dealing with a shifting haptics landscape. Whether it’s technological or conceptual advances, or situating these within new applied context, every development brings with it the potential for new meanings and communication.

Future avenues

There was a sense that the touch tools presented during the workshop already play an important role in unlocking this potential, highlighting the significance of making these more widely (or perhaps more centrally) available. Moreover, the workshop shed light on areas of sensorial experience in need of further investigation, such as the sense of touch (and its relation to other senses) in ageing, and other cultural and corporeal differences that affect sensation and social interaction.

Whether it’s through building broader socio-cultural contexts into haptics design tools or through generalist hapticians and interdisciplinary teams straddling technological complexity and social-sensorial meaning, generating creative dialogue across and beyond the different ‘layers’ of the haptics stack seems key.

We thank the World Haptics organisers and all attendees for making these exchanges possible. Special thanks to all speakers, also for their role in shaping and publicising the workshop in the lead-up. For a recording of the workshop, click here. Presentation slides and links to toolkits can be found on our workshop website.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s