April 30, 2025

Maggie Liu

11th Grade

Fountain Valley High School

“Music is the medicine of the soul.” - Plato.

Music follows us everywhere we go– from speakers to headphones, from long study sessions to quiet moments during meditation. But what if music could not only soothe the soul, but also help decipher the complexities of the human heart? Through the combination of art and science, computers and music work hand-in-hand to turn data into melodies, a process known as sonification, which can be interpreted to detect life-threatening heart conditions. The rhythmic pulse of music, powered by AI-driven technology, is transforming the medical world as we know it, opening the doors to new opportunities and groundbreaking research.

Key to this process is the electrocardiogram (ECG), a test that records the heart’s activity through electrodes attached to the skin. Traditionally, ECG results are displayed visually through spikes and dips, requiring specialized tools and expertise to interpret. However, with the help of AI-driven technology, these visual patterns can be transformed into music, allowing us to hear anomalies in heart function. Sonification plays a central role in this transformation by using frequency modulation (FM) or other audio mapping techniques to convert ECG signals into audible sounds. A Filestacks Lab demonstrates what ECG Sonification looks and sounds like in action, showing that it is possible to audibly detect irregularities such as atrial fibrillation, a common heart rhythm disorder. These piano-like notes make the data more accessible and easier to interpret, even for individuals with basic medical knowledge.

In electrocardiography and sonification, AI and computers play a significant role by transforming visual data into music and monitoring heart activity. For patients who find healthcare inaccessible or struggle with advanced technology, such as the visually impaired or elderly, the roles of computers and AI are crucial to modify heart signals through ECG into music and vice versa. As of 2024, a study by Krasteva et al. explored the use of AI to process data from 8 out of the 12 total ECG leads, which act as cameras, providing perspectives of the heart’s electrical activity. With the help of frequency modulation (FM), ECG data is converted into low-frequency bands between 300 and 2,700 Hz, making it perfectly compatible with standard phone microphones. The purpose of these phone microphones is to record and playback ECG signals that have been transformed into audio. For individuals without easy access to healthcare, they can record audio signals, which can then be converted into music. The music can then be analyzed by a highly accurate AI model referred to as a 1D convolutional neural network (CNN). In this scenario, 1D-CNNs work well because of their ability to detect the patterns in 1D sequential data, such as sound waves found in audio data. Before the use of AI, a group of sound and music experts would listen to these files, pointing out any heart issues they detected. However, studies have shown that the 1D-CNNs have a 97% accuracy in detecting heart diseases such as arrhythmias, outperforming human listeners in both accuracy and efficiency.

While AI-based sonification stands out for its creative approach to accessibility, it is just one part of a broader movement in cardiovascular AI. Through deep-learning algorithms, AI-ECG models have been created to screen heart diseases such as cardiac amyloidosis and aortic stenosis. Not only have AI models been developed to diagnose diseases, but have also been developed to predict future health events regarding these diseases. AI has even been utilized to filter out unnecessary ECG signals caused by electrical interferences or body movements. 

So, what does the future hold? As computing power continues to advance, real-time ECG monitoring will be made possible. AI will be able to analyze heart data from wearable watches or implants and recognize when serious events might occur. The potential for AI in the cardiovascular field is essentially limitless; its implementations in ECG and transforming it into music are just the tip of the iceberg. 

Although music and medicine are two very distinct topics, with AI-driven technology acting as a bridge between the two, they come together in perfect harmony.