Musically Deaf

Assistive Technology for Deaf Musicality Using Pure Data

Abstract

Research Question/Problem

Technological mediation in music education for deaf learners faces a critical challenge in virtual environments: the absence of tactile perception of sound vibrations, a central resource in face-to-face music lessons. For the deaf community — which understands deafness as a cultural and linguistic difference (Strobel, 2008) — screen-based mediation eliminates the possibility of physically feeling sound vibrations, a fundamental tactile resource for perceiving beat and rhythm, since music is a multisensory experience (Duarte, 2017). The challenge lies in the need for Assistive Technologies (AT) capable of translating sound information into the visual channel accurately and in real time, allowing students to identify the beat and rhythmic nuances remotely.

Objectives

This paper describes the development and application of the Assistive Technology tool “Musicalmente Surdo,” based on open-source software, aimed at promoting accessibility and remote rhythmic training for deaf learners. The main objective is to investigate how interactive visual interfaces can support deaf participants in maintaining the beat and engaging in collective musical practices in virtual environments.

Method

The visual programming language Pure Data was used to create a computational model for real-time audio processing. The system captures audio generated in virtual environments through the [fiddle~] object, which performs pitch and amplitude analysis using the FFT algorithm (Kreidler, 2013). The project was structured into two subpatches: (1) Filters, which segments the audio spectrum into four frequency channels (150 Hz to 200 Hz; 350 Hz to 800 Hz; 800 Hz to 1500 Hz; above 1900 Hz); and (2) Connect, which encapsulates amplitude and frequency data into OSC packets transmitted via UDP to the Processing environment, where the interactive graphical interface is generated

Results and/or Main contributions

Assistive Technology (AT) converted sound signals into dynamic circles on the screen, mapping four frequency regions to specific colors (red for low frequencies, green for mid frequencies, blue for upper-mid frequencies, and yellow for high frequencies), while amplitude controlled the size of the shapes. The application allowed deaf students to visualize their sound events in real time, as well as those of other participants, overcoming network latency and the absence of tactile stimuli. This ensured autonomy to adapt to the collective musical pulse, promoting active participation in rhythmic creation, as discussed by Louro (2012) regarding the importance of pulse in music-making. The visualization aligns with the observations of Prometi (2020) and Strobel (2013) that deaf people perceive music primarily through visual and bodily perception

Conclusions and/or Implications

The use of visual programming environments such as Pure Data proves effective for the development of Assistive Technologies (AT) that promote social inclusion and the musical and cognitive development of deaf people, meeting the validation criteria proposed by Bersch (2017). The proposal demonstrates that remote musical accessibility is a necessary and viable paradigm for the democratization of education, enabling Deaf Culture to participate autonomously in spaces of collective creation regardless of geographical context, and reaffirming the musical capacities of deaf people (Benassi, 2014).

Keywords

Deaf Musicality. Assistive Technology. Music Accessibility.