How does the Cthulhu Shield work?
Find Examples and more explanation for the Cthulhu Shield on our github!
The Cthulhu Shield creates short biphasic voltage pulses on the electrodes that are in contact with the surface of the tongue. The electric field created by these pulses and the movement of ions in the electric field cause a change in the potential across the membrane of a somatosensory nerve fiber(s) in the tongue, opening voltage gated ion channels, causing the fiber to propagate an action potential to the brain where it is processed and perceived as a touch sensation. This form of neuromodulation is often referred to as electrotactile stimulation. Most of the sensations the Cthulhu is capable of producing feel a little like pop-rocks or bubbles from carbonated beverage.
Patterns and arrangements of these touch, or tactile, sensations can be used to convey information to a person in the form of sounds, images, or other forms of presentation. Initially, a person learns to associate a given pattern of sensation with some sort of meaning (think of tactile languages like Braille), research shows that over time mechanisms of neuroplasticity can route the information to other parts of the brain where images or sounds are usually processed. The net effect, one can argue, is truly seeing or hearing with the tongue. Of course none of this limited to sight and sound, you can feel radio waves, or stock prices on your tongue just as easily as a painting or concerto.
Why the tongue?
The tongue is incredibly sensitive. There are lots of nerves in your tongue, and it turns out that because of the presence of an electrolyte (saliva) and the type of tissue in the tongue, it’s incredibly easy to stimulate with low voltage electrical pulses. Additionally, there’s a lot of brainpower dedicated to both controlling the movement of the tongue, and processing touch signals coming from the tongue. This combination of features makes it ideal for sensory substitution, as well as interfacing with computers and machines.
An RC network attached to the Arduino I/O pins is used to generate the biphasic waveform used to depolarize nerve fibers. The network is based on designs developed by researchers who founded modern sensory substitution over 20 years ago but has been adapted to work more easily on Arduino based devices. It actually takes up very little space on the shield. This is the business end of the device that produces the electrotactile signal and stimulates the tongue.
The rest of the hardware is used to display which electrodes are being activated, to help the user more quickly make sense of the sensations they are feeling.
During electrotactile stimulation, we can quickly measure the voltage potential on certain individual electrodes. The potential on the electrode is different if your tongue is in contact with the electrode or not, enabling us to sense the position of your tongue on the array, and determine things like tongue swipes, presses, and gestures. We are excited to see the apps and software users develop to use their smartphones, computers, and other devices with their tongues.
As important as the hardware, the open source software library and examples we developed to let users quickly create exciting and innovative uses for tongue based sensory substitution.
The heart of the library is a simple pulse-train generating function that allows you to generate patterns of nerve-depolarizing pulses that are perceived as touch sensations. This pulse generating function is based directly off of techniques developed by the pioneers of this field, but has been slightly tweaked for some of the early research we performed. The function lets the user input different pulse-train/waveform parameters to create different sensations that have been described by researchers as ‘tactile colors’. These ‘tactile colors’ can be used represent actual colors if you are displaying visual information, or anything else, like sound frequency, temperature, etc.
We have provided example source code for Arduino, processing, and android, that will let you use your Cthulhu with your android device camera and microphone, GPS navigation app, or external ultrasonic sound detector and thermal camera without needing to write a single line of code.
What can I make?
Using the Cthulhu Shield, you can create sensory substitution and sensory augmentation devices.
Sensory Substitution: Sensory substitution is the name that has been given to tools and techniques that take sensory information you would normally receive on one sensory organ, and presents that information in another form to another sensory organ. Your brain then figures out what to do with this information.
You can use your phone’s onboard camera to send color, movement, and pixel intensity information to create low resolution images you can feel on your tongue. You can use your phone’s onboard microphone to send audio information to your tongue to use the Cthulhu as a tactile hearing aid, or just to feel your favorite song. You can also attach other external sensors to your Arduino, like a small pressure sensor to use the Cthulhu to feel the force you are exerting on an object if you have nerve damage in your hands or are wearing thick gloves. Many people have used tongue stimulation to improve their sense of balance as well by using pressure sensors or accelerometers.
Sensory Augmentation: Ever wanted thermal vision or extra-human hearing ability? Want to sense the electric fields of living creatures like a shark, or give yourself magnetoreception like a migratory bird? All of these things and more can be presented electrotactiley on the tongue. Your imagination is the only limit to the ways you can expand your sensory experience.
In addition to the senses we witness in nature, there are many exciting opportunities to use the Cthulhu to sense data directly from the internet or virtual world. Upvotes, likes, and stock prices can all be felt on your tongue with the Cthulhu and simple computer programs.
Tongue Controlled Devices: Our favorite applications of the Cthulhu tongue control are for assistive devices. The Cthulhu can be used very effectively to control your computer cursor and type with your tongue, but it can also be used to control wheelchairs, or even cars and other vehicles.
For those without impaired mobility, the Cthulhu can be used as an extra controller for gaming (to move or perform certain actions quicker), or to control your smartphone and send texts or emails silently with your tongue.
There are certainly many uses of the Cthulhu Shield we haven’t thought of, and are excited to see what users come up with.