Eyemusic Device Converts Images to Music

There are devices that are equipped with sensory substitution run by transforming one type of sensory input into another. Some of the examples are CASBLiP and Eye 21, through which blind people assign a sound to an image which eventually helps them see. A group of researchers at Hebrew University of Jerusalem have developed a device called EyeMusic. It works by employing sweet musical tones to express visual information, which helps the visually impaired see with the help of music. This non-invasive SSD converts images into a combination of musical notes, or “soundscapes.”

It consists of a pair of glasses incorporated by an attached video camera and transforms pictures into a combination of musical notes. The device also consists of a pair of headphones and conveys sound into the user’s scalp, instead of conveying directly into their ears.

The image is scanned by the camera on continuous basis from left to right, and is then translated into “soundscapes”. The higher the pixel is perpendicularly located in the scanned image, the higher a musical note will be used to symbolize it in the soundscape. However, the farther it goes within the scanned area, the later it will be transmitted within the soundscape.

“We demonstrated in this study that the EyeMusic, which employs pleasant musical scales to convey visual information, can be used after a short training period (in some cases, less than half an hour) to guide movements, similar to movements guided visually,” explained lead investigator Drs. Shelly Levy-Tzedek, an ELSC researcher at the Faculty of Medicine, Hebrew University, Jerusalem, and Prof. Amir Amedi. “The level of accuracy reached in our study indicates that performing daily tasks with an SSD is feasible, and indicates a potential for rehabilitative use.”

The intensity of each pixel represents how loud its sound is. Different instruments are used to represent different colors. Blue uses trumpet, reggae for red, green is described by reed, violin for yellow, and white being heard as vocals. Silence is used to represent black color.

An experiment was conducted on a group of 18 blind-folded individuals who found EyeMusic very useful and quickly learned how it actually works. It is expected that this device could be used by the blind in everyday activities such as reaching for a coffee cup or even playing video games.