Hacking a hearing aid to hear data

Science writer Frank Swain collaborated with artist and engineer Daniel Jones to reimagine both his hearing aids and the new environment of data we move in

We’re slowly being educated in machines talking to us, in the tones of our smartphones, but in our world awash with data, machines are talking to each other, communicating with each other through wi-fi, internet exchanges and routers. It is this space that science writer Frank Swain hacked into with his project ‘Phantom Terrains’. 

Swain has been going deaf since his 20s and wanted to take control of the sound given by his hearing aids, re-thinking his hearing aids not just as a means to improve hearing but as a creative possibility to augment reality. Working with artist and software engineer Daniel Jones they hacked an iPhone – the wi-fi sensor within the phone picks up signals in environment around him and streams it as audio to his Bluetooth connected hearing aids. ‘Daniel was a sound artist who was recommended to me as someone who would do the kinds of things I wanted to do,’ explains Swain. ‘We received funding from Nesta [UK innovation charity] to carry out this project with help from Starkey Hearing who supplied the hearing aids, and Daniel’s job was really to do everything to make it work.’ 

Low Energy

As you can only imagine, the unfiltered noise from the data environment could be quite intense, so Swain worked with Jones to choose sound that would be comfortable to listen to for long periods of time, ‘that you wouldn’t notice but would be aware of and give a sense of your environment.’ 

So what particularly excited Nesta about the project? ‘I think they were interested in taking something that was typically viewed as a prosthetic and re-imagining it as an augment’, says Swain. ‘If I am going to have to have something in my ears my whole life then what can we build in that space? They were also interested by the emergent technology – the backbone of this project is a new kind of Bluetooth, voiceover, audio-over, low energy Bluetooth, the same kind of Bluetooth used with ‘Fitbit’ and apps like that. Very, very, low power, which allows you to beam sounds at small distance at almost no energetic cost.’

Swain believes that this low energy technology will drive an increasing growth of voice-activated systems (such as Apple’s Siri and OK Google), which is another reason to explore this area. ‘If we are going to be wearing earables to connect us to network devices what can you build in that space?’ 

Invisible Infrastructures

Swain pays tribute to the experimental work of people such as Timo Arnall, former Creative Director of the highly innovative but now defunct London-based technology company BERG (an acronym British Experimental Rocket Group taken from the 1950s BBC TV series ‘The Quatermass Experiment’). Arnall created a variety of projects exploring and visualizing the new ‘immaterials’ of our technological world. ‘The idea of bringing these invisible infrastructures into light is continuing work done by people like Timo Arnall. It’s not necessary to map wi-fi, this project, this system, can really translate any form of continuous data into sound. That could be stock market movements, that could be information about your environment, crime rate…walk somewhere there was a mugging or a road traffic accident happened you could make it play a sound, wi-fi is just one example of a variety of data that we could have translated into sound.’

Swain’s educational background is as a biologist and is trained in natural resource management, at Bangor University and Cranfield University. He is one of a new generation of young science graduates creating and pugging into networks of interesting people to experiment and play with scientific ideas. For ‘Phantom Terrains’ illustrator Stefanie Povasec created a visualization for visually mapping the audio experience of walking through the BBC centre. 

Collaborative Network

‘I’ve always been attracted to the fringe elements of knowledge, and the creative space you can explore once you detach yourself a little bit form the rigors of science, science fiction being a classic example of that,’ reflects Swain. ‘But I’m also lucky to be in contact with a lot of wonderful people such as Honor Harger who used to be at the Lighthouse in Brighton, a digital hub, she was the one who put me in touch with Daniel and encouraged me to explore the project.’ Swain feeds off, ‘the availability of people like that, both scientists and beyond, who you can bounce these ideas off and get encouragement from.’

Most of all Swain is part of a generation that doesn’t need to ask permission to try things out, to use scientific knowledge to generate creative experiments. ‘I think it’s just the manifestation of the creative urge, you have a question, you say I wonder if it would be possible to hack my hearing aids to hear something additional and then translate some form of data into sound. Then it is a case of making it happen. I don’t think there is anything magic about that. You set yourself a goal, get the resources and the necessary knowledge and you produce what you want to produce.’