Author bio:
Brendon Bussy is a special needs teacher, artist, maker and retired mandolin player. At the Dominican School for Deaf Children in Wittebome, Cape Town he teaches design, visual arts and performance. His interests include language (he is proficient in South African Sign Language) and multi-modal teaching approaches such as role playing. http://brendonbussy.wordpress.com/
Sight and Touch
A few years back I came across the BrainPort, a commercially available assistive sight device that works via the tongue. When the neuroscientist Bach-y-Rita first conceived this device in 1967, the tools he had at his disposal were large and cumbersome, repurposed lab equipment and a studio TV camera. The user was required to sit in a chair with a large input device, a vibrating grid of points, strapped to her back. This would translate what the camera was seeing and the user would be able to sense what the camera was seeing. So what was this, a translation of sight? Via a camera which sees? And sensed via touch?
The curious manner in which the BrainPort works further messes with our neat categories of senses. A small motion sensing camera is mounted on a pair of glasses rims worn by a visually impaired user. This sends a signal to a lozenge like tab which sits on the user’s tongue. Then this is what happens: The tongue is stimulated via a matrix of electrodes creating a pixelated ‘sensation’ which, after a surprisingly short period of practice allows the user to construct a mental image of whatever the camera is pointing at. So a camera communicates via tongue and the result is a, perception of sight?
An astonishing process. And a surprisingly difficult one to describe, in that the usual sensory descriptors such as sight, taste and touch, all active here, seem to be operating so far out of their expected functions that they are hard to describe. An attempt could be made by describing this as a logical process: visual input -> processing -> sensation -> perception of sight. However, intuitively, the description is hard to accept. Understanding the process requires a suspension of the normal links between sensor and sense, the common sense links between sight and the eyes, between taste and the tongue. And the practice of touch.
Bach-y-Rita’s impetus for the creation of the device came via his research into brain plasticity. Up until the mid 1900’s it was thought that the brain was relatively inflexible, however it has since been shown that a great deal of adaption as well as physical change is possible, change which is often hard to explain. In short, our brains may rapidly remap after injury, so much so that limbs rendered immobile through stroke will quickly lose their mapping allocation, resulting in a much lower chance of recovery. So today therapies that have been shown to be promising include strapping up and immobilizing healthy arms to force the brain to ‘re-acknowledge’ stroke impaired arms and reassign brain function to those arms. An example of this can be illustrated by Bach-y-Rita’s patient Cheryl Schiltz, who having almost completely lost her sense of spatial orientation, her vestibular sense, was able to use the Brainport as a virtual ‘spirit level’ and replace her ability to balance. The remarkable outcome was that after a time, she was eventually able to function without the BrainPort.
Current thinking sees the brain mapped according to task rather than sense. A person needs to perceive an object and conventional ‘eye’ sight might be used or another sense or combinations of senses might be used to perceive that object. Sight is no longer linked intrinsically to sight perception. In the same manner touch can be assigned to fulfill the role of sight, taste, and even balance, our sensors serving our perception, a means to an end.
So what do we do with this plasticity, flexibility of function and ‘swapping’ of senses? How does it inform our idea categories of perception? Touch as sight, sight divorced from the functional apparatus of the eyes.
Hearing and Touch
Differences in physical hearing reveals interesting perspectives on our assumptions about hearing. Deafness is not a form of anti-sound or non-sound. Instead it is another way of reflecting on sound. I have come to understand that not having use, or full use of, the hearing receptors that the hearing use (i.e the ears), does not mean that people who are deaf are immune to the sensation of sound, rather there are other ways in which sound is experienced, and most importantly, that there are ways in which the work of the ears can be substituted, and the objective of hearing fulfilled. This understanding has forced me to start interrogating the ‘job’ that our ears do.
A disturbance in air manipulating the outer ear’s ‘drum’. The resultant manipulation of a series of levers (which can also act as shock absorbers if the movement is too great). The agitation of fluid and movement of the small hair like ‘switches’ in the inner ear causing a stream of electrical pulses to the brain and this becoming the raw data we process and file in the category ‘sound’. Hearing is a very physical process which at most times is perceived in a subconscious manner. It is only when things don’t work as expected that we sense the very physical nature of hearing. When someone screams in your ear, when a puff of cold air is directed straight into the ear canal, or if we develop an unfortunate condition like Hyperacusis, an extreme sensitivity to particular sounds which can cause physical pain.
Despite the intangible nature of sound, we’re interacting directly with a physical substance, air. And air, the medium surrounding us, surrounds others. Moving through this mostly invisible medium allows us to communicate with each other at a distance, our movements agitating this medium, our ears, when working well, finely tuned to perceive the sound: a hand rubbing a head, a tired sigh, a dog running in its sleep. All of these actions which we also call sound, the result of oscillations through air engaging intimately with the sensitive apparatus of our inner ears.
But sound is even more than this. Our ears are good at capturing higher frequencies, but when our physical environment vibrates at lower frequencies we perceive in other ways. As a truck rumbles past in the distance, we sense this ‘sound’ primarily through the surface of our bodies, and through our skeletal structure. A sensing mechanism that works seamlessly in conjunction with our ears, our brain smoothing the ‘crossover’ between a sensation we would normally call ‘touch’ and one we’d normally call ‘hearing’.
The deaf musician Evelyn Glennie explains this so well: ‘Hearing is basically a specialized form of touch…The sense of hearing is not the only sense that can do this, touch can do this too…[but] For some reason we tend to make a distinction between hearing a sound and feeling a vibration, in reality they are the same thing. It is interesting to note that in the Italian language this distinction does not exist. The verb ‘sentire’ means to hear and the same verb in the reflexive form ‘sentirsi’ means to feel. Deafness does not mean that you can’t hear, only that there is something wrong with the ears. Even someone who is totally deaf can still hear/feel sounds.’
At times sitting on a plane I will try and focus on non-ear derived sensation, the vibration of my seat as the engines warm up. Essentially trying to ignore my ears, which also means that I have to ignore the way in which I am processing sound. And I’ve also experimented with devices that change my hearing, designing listening devices such as my Ear Shells which I use to manipulate the frequency response of my ear canals by effectively creating an extension of the canal and manipulating it with my finger-tips. Hearing can be experienced in so many ways.
From my previous life as a musician, I remember wincing from the physical squeal of feedback, the sensuous thrum deep in my core as I sensed low frequency and, after a long mixing session, the sense of my ears being over stimulated, a feeling of physical numbness and disorientation in the world. Hearing is not only communication, it is the way in which our bodies interpret subtle, and sometimes not so subtle, physical interactions in the collective medium in which we exist.
References and reading
Bussy, Brendon. Ear Shells at Sounding Out, blog post, 30 June 2012, accessed 30 April 2018, https://brendonbussy.wordpress.com/2012/06/30/ear-shells-at-sounding-out/
Bussy, Brendon. A New Adventure – Hearing with Touch, blog post, 30 June 2012, accessed 30 April 2018,
https://brendonbussy.wordpress.com/2012/06/30/a-new-adventure-hearing-with-touch/
Doidge, Norman, M.D. (2007), The Brain that Changes Itself, Viking
Glennie, Evelyn. Hearing Essay, 1 January 2015, web page, accessed 27 April 2018, https://www.evelyn.co.uk/hearing-essay/Rutkin, Aviva Hope. (2012), Champagne for the Blind, Paul Bach-y-Rita, Neuroscience’s Forgotten Genius, accessed 30 April 2018
https://cmsw.mit.edu/paul-bach-y-rita-neurosciences-forgotten-genius/
Twilley, Nicola. (15May 2017) Seeing with your tongue, The New Yorker, https://www.newyorker.com/magazine/2017/05/15/seeing-with-your-tongue
IN-TOUCH: DIGITAL TOUCH COMMUNICATION 