Patricia Cornelio, William Frier, Lili Golmohammadi, Carey Jewitt, Sara Price.
These guidelines culminate from a collaboration between research (UCL) and industry (Ultraleap) to develop the Manifesto for Digital Social Touch in Crisis (Jewitt et al., 2021) – a proactive call to engage newly with the social and political aspects of haptics – into a set of ethical and socially informed design guidelines for groups working across haptics. The guidelines set out 10 high-level social considerations for haptic experience design, together with practical pointers for their implementation. Following Schneider et al., (2017) we define haptic experience design as the ‘design (planning, development, and evaluation) of user experiences deliberately connecting interactive technology to one or more perceived senses of touch, possibly as part of a multi-sensory experience’ (p.5). Some of the guideline considerations may be more pertinent than others depending on your area of work, the intention of your design, or the stage of your design process.
1. Anticipate unintended consequences: play out worst case scenarios of use.
We might set out to improve a haptic experience or use case; yet oriented to possibilities and hopes, this process may neglect to consider everyday unexpected negative consequences. Design is improved by infusing responsible innovation throughout the process. When planning and ideating with haptics, it is useful to consider the social context of the touch experience and create and interrogate worst-case use scenarios. To do this, ask ‘what if’ questions and consider how the design of the haptic experience could then be improved.
- Example 1: a designer might seek to integrate haptics into a communication technology to enable partners to comfort or express affection for one another during a video. How might this technology be misused in this scenario? How could such misuse, such as negative touch (e.g. prick/poke instead of caress) or physical abuse, be avoided? For example, ask, “what might happen if someone outside the relationship hacked into the experience of touch exchanges and the partners did not realise or realised later?”.
- Example 2: in cases of apps linked to electrostimulation (stimulating the sense of touch using electricity), a hacker could bypass the safety level of current and start harming the user. One may think “simply pull out the wearable” but in some environments, are we sure the user can simply “pull out the wearable”?
2. Seek out diverse voices for haptic experience design: disrupt the boundaries of accepted norms.
Seek out diverse voices for the design and imagination of future haptic experiences. Foster a design community that includes broadinterdisciplinary conversations and collaborations.Engage with people with differently abled, aged, and gendered bodies, and distinct sensory standpoints. Reflect on how tactile practices and norms differ between cultures, or economic circumstance.
- Example 1: consider haptic experiences that require two adult sized hands to be tracked; consider how they could be navigated by (a) the small hand(s) of a young child; (b) by someone with one hand; or (c) by an older person with reduced dexterous motor manipulation abilities.
- Example 2: Rüller and colleagues (2022) worked on ameliorating the consequences of illiteracy (written and digital) in collaboration with people in remote Morrocco by producing design fictions of haptics-based interactions. This included those inspired by, and through, traditional tattoos, and allowed participants to explore ‘the bounds and confines of their literacy’ and to reconceptualise how they might engage both with haptic technology and existing screen and text based interfaces.
3. Evaluate expectations of haptic technologies: establish when expectations do and do not need to be set.
Users’ expectations of haptics can be addressed in different ways. On the one hand, high user expectations that are set through overpromising can result in haptic technology feeling underwhelming. Touch is a broad set of senses, yet current technology typically stimulates a limited range of sensory aspects (e.g., force feedback, vibration, or thermal qualities), and offers an incomplete sensory touch experience. To mitigate potential disappointment, set expectations of haptic experiences within the bounds of technical capacity. On the other hand, designers may at times prefer not to set any expectations of haptics, which can be differently beneficial when managed well.
- Example 1: don’t promise that a user will be able to push a door if the technology employed can’t provide a sense of resistance in the hand movement.
- Example 2: don’t promote your haptic technology for e-commerce/virtual try-on clothing applications if the technology employed doesn’t sufficiently allow the user to discriminate between two different fabrics. If the technology does not sufficiently enable this, the user is likely to be disappointed and frustrated, rather than promising the ability to feel a fabric remotely, introduction to the haptic components may be couched around the ability to feel the ‘essence’ of a texture.
- Example 3: a natural flow in haptic content (i.e., when introduced gradually) can bring surprise and delight to a user as they become accustomed to an experience and play around with its novelty and intensity levels (e.g., the Haptex Playground demo (2017) which enables a user to feel grains of wheat, sense the rain, and a little fox that jumps onto the user’s hand. However, this also raises tensions where a user might be frustrated if some elements are haptically enabled, while others are not (e.g. can feel the wheat field, but not the bark of the tree at the edge of the field, can feel the fox paws but not its fur).
4. Bring attention to sensory and aesthetic considerations: engage with how emotional experience is evoked.
When defining your haptic experience, think further than the immediate human-to-human or human-to-object aspects. It is helpful to consider the haptic experience beyond a means to an end and explore its aesthetics in the design process. Think beyond functionality, and design haptic experiences to be playful, or create a sense of wonder, awe, and/or harmony.
- Example 1: Awareness of the sensory and aesthetic profiles of users is critical. Consider drawing on the notion of mood boards (e.g., from graphic design), where a “palette” of haptics, or ‘haptics mood board’ is used to evoke certain emotions or sensory perceptions.
- Example 2: a serious and formal experience might be implied through heavy and firm haptic feedback combined with slow sound, while a more light and joyful experience or communication might be implied through dynamic and rapidly changing haptic and visual stimuli. This is complex as the material, social and cultural histories, practices, and contexts that people bring to their use of technologies, including personal social relationships, will influence how haptic technologies, and our experiences of them are shaped.
5. Consider purposes for haptics beyond replication: generate touch in new forms.
Haptic technology does not always need to digitally replicate ‘real-world’ tactile experience. Consider ways of leveraging combinations of metaphor, imagination, and/or sensory equivalence to create new haptic experiences that complement in-person touch experiences, while attending to the purpose of the haptic technology.
- Example 1: paired bracelets (e.g., FeelHey, Bond Touch) which give a small squeeze to the wrist can be symbolic of a hug and more offer a new tactile communication to people rather than trying to recreate a hug (e.g., through a robotic upper body or arms).
- Example 2: the ‘alert shirt’ (We:eX and Foxtel, 2014) aims to enhance spectator experience of sport through enabling a wearer to receive real time haptic vibrations linked to live actions/interactions in the game, such as, touchdowns, interceptions.
6. Combine touch dimensions: consider the place of touch as part of wider multisensory experience.
Touch (whether digitally delivered or not) is rarely an experience on its own but is embedded in our wider multisensorial experience and understanding. Think about what touch dimensions your haptic technology can produce. Combine these to provide a richer perceptual space or consider how you might combine the output of different haptic technologies to create an improved harmonious and meaningful haptic experience. Keep in mind that enhancing the perception space can diminish the success of a design: consider how to balance the positive returns of mixing touch and other dimensions with increasing the complexity of the system.
- Example 1: pressure and temperature together can offer higher levels of comfort, and a greater sense of presence and physicality in affective touch communication. This could be further enhanced through other sensory experiences, for example through the use of scent.
- Example 2: furthermore, the speed of activation and changes in intensity can influence interpretation: slight differences in the amount of heat and pressure can change a tactile message from signifying affection to anger or changes in the tempo of a vibration offer more varied meanings than a consistent tempo, while combining tempo changes with varied pressure may convey levels of urgency or importance. Social context might be enhanced through hearing the voice of the person sending the tactile message or through reading an accompanying text that they have written.
- Example 3: in terms of multiple senses, we need to consider how touch sensation integrates with sight and sound. A key aspect of achieving this successfully is congruence. A simple example is lip-sync in a video: often image and audio are recorded separately, but when put together, if there is a delay between the two, and this breaks the illusion. Similarly with haptics: if I hit a bell, hear a bell, but feel the haptic “bang” too soon or too late, the immersion will break. Or, if the haptics is on time but feels more like stroking a coarse rather than smooth (silk) fabric, the illusion will break. This takes us back to the notion of anticipation – haptics need to be use with parsimony: only where it adds value rather than a requirement to be everywhere in the experience.
7. Devise an onboarding process for new haptic experiences: scaffold touch attunement.
People often need support to navigate new haptic experiences, which can sometimes be unintuitive. When devising an onboarding process that involves haptics, stagger the learning process and introduce users to one feature at a time before they use these in combination.
- Example 1: consider a videogame enhanced with haptic feedback. Introduce haptic effects as you introduce interactions and game mechanics. For instance, the popular game ‘Portal’ introduces game mechanics (e.g. red and blue portal creation) as the player progress in the game. Corresponding haptic effects, if they were to be added, could be gradually introduced at a similar pace.
- Example 2: haptic feedback can also be used to enhance the environment (e.g., car driving through mud, wind blowing). In this context, haptic effects also need to be introduced one a time. For instance, make sure the user experienced wind whilst walking, before experiencing wind and road surface whilst driving.
- Example 3: imagine a haptic slider. Ensure the user can determine whether the cursor has been selected or disengaged. Design the haptic feedback to convey whether the slider value is increasing or decreasing and when the maximum or minimum value is reached. Consider ways to inform the user how to learn these different haptic meanings.
8. Protect the privacy of touch and help the user identify touch sources: touch is intimate and personal.
Give users ways to configure and consent to the receiving, recording, sharing, and replaying of personal haptic interactions, and ensure the user can identify who or what is touching them. Without this, people may lose agency over their haptic interaction, which can in turn lead to loss of trust and enjoyment.
- Example 1: a personal touch sent to a partner can be re-experienced as many times as they like. Sometimes this might be comforting and welcome. Yet there is also a risk of violation of the sender’s personal touch; the recorded ‘touch’ could be shared by the recipient with others, or still experienced even if the relationship terminates. This would be problematic in, for example, a haptic lip device designed to reproduce a kiss for long distance partners (similar to Kissinger)
- Example 2: imagine a parent can record a hug sequence on a mobile app to send to their young child. They could deliver this recorded ‘hug data’ to their childminder’s phone so this could be transmitted to the child; who is doing the ‘hugging’ and how can the child tell?
- Example 3: a VR game mechanism relies on the player moving through levels full of enemies with the help of an avatar. Upon receiving haptic feedback, consider whether the player can distinguish between an enemy shooting at them or an avatar trying to communicate with them. Reflect on how the user can discern whether an enemy is a real person or a non-player character.
9. Make opting in and out of haptics quick and easy: tactile sensations may overwhelm.
People have distinct and dynamic sensory experiences of the world. Enable people to decide when, how, and where technology touches. Design frictionless options that allow them to do this quickly and intuitively and avoid intrusive or prolonged technological ‘assistance’.
- Example 1: a neurodivergent individual may have an even greater need to control stimuli than neurotypical individuals. Sometimes they may feel overwhelmed and need to disengage from haptics immediately – design an ‘off’ option that is easy and quick to activate.
- Example 2: a haptics device used to communicate with others could include varied profiles akin to audio options available on phones (e.g., personalising the ring tone, choosing between audio/vibration/silent modes, adjusting the volume differently for call/notifications/alarms). This could also temper the potential to overwhelm.
10. Build in options for personalisation: offer tailored touch experiences.
Attention to people’s varied touch preferences is important. Users may want to disable certain haptic features to reflect their personal touch sensitivities or preferences. Consider whether they can complete their intended task without haptic cues or substitute these for a more desirable haptic effect. Providing a degree of customisation enables a user to align haptics with ways that fulfil cultural, contextual, and personal preferences. Sometimes we want haptics to be anywhere anytime, and other times not; sometimes we want complexity, sometimes simplicity; sometimes we may want whole-body based haptics, at times others may not. Such dimensions offer trade-offs in design. Customisation which aligns with user control, location preferences, temporal choices etc. needs to balance rendering touch as over-burdensome and unspontaneous, with haptics being restricted, simplistic, empty, or meaningless.
- Example 1: a surgeon operating via a robotic platform with haptic feedback may want to disable specific haptic functionalities. Consider how they can achieve this personalisation, without negatively impacting the wider practices of the operation.
- Example 2: personalisation trade-offs in design can be illustrated for example, through VR ‘open-world wizard’ video games (where players explore a vast, non-linear world as a magical character); these may offer a wide variety of textural feedback – from realistic food on the table, to large spiders in caves, to gooey texture in the cauldrons: some players may have an adverse reaction to some of these haptic effects, and a designer may want to consider how their design might enable the player to progress in a game without experiencing textures they find undesirable.
Acknowledgements:
We would like to thank the following ‘critical friends’ for their thorough, insightful and valuable comments on our pre-final version of the guidelines: Oliver Schneider (University of Waterloo); Jürgen Steimle (Saarland University); Hasti Selfi (Arizona State University); Eric Vezzoli (Razor Inc.); Karon MacLean (University of British Columbia).
