I am developing a real-time audio-to-visual tool that graphically represents piano sound to help students better understand the effects of the sustain pedal.
The problem as it stands
Helping students develop musical judgment, particularly in their use of the sustain pedal, remains one of the most elusive challenges in piano education.
Much of piano pedagogy for beginners focuses on concrete musical elements like pitch and rhythm, which students can relatively straightforwardly identify as “correct” or “incorrect” in practice. However, the technique of expert pianists demonstrates not just correct notes and rhythms, but more importantly an ability to obtain a greater variety of sounds and tone qualities from the instrument.
Dynamics and articulation are two domains in which students can explore a greater variety of sound. Staccato, legato, marcato, and sforzando involve different motions of the fingers, hands, and arms, and create distinct effects in the piano’s sound. Both can be difficult to master because they involve qualitative contrasts rather than quantitative certainties like pitch and rhythm.
One of the most difficult elements of tone for students to grasp is the damper pedal. It removes the dampers from the piano’s strings and allows them to resonate freely. This lets the pianist add varying degrees of resonance and depth to the instrument’s tone, and enhance dynamics and articulations by lifting and replacing the dampers.
To help students clear this hurdle faster, I am exploring a visual approach. Music is defined by the auditory sense, but I hypothesize that many piano students can benefit from a simple visualization of the audio emitted from the instrument.
The visual concept
The project takes microphone input and translates it into a screen with two major elements: colored circles and a colorable background.
Colored circles
- Each time a key or group of keys is pressed, a randomly colored circle appears.
- The circle’s size is proportional to the volume of the keys: a louder chord creates a larger circle.
- The circle shrinks over time with the remaining acoustic energy. As the energy decreases, the circles fade away.
A colorable background
- The background begins white, while the circles contribute their colors to it.
- Larger circles add more color, while the background continuously moves back toward white.
- The effect of the pedal appears through the background’s murkiness: the more blurred the music becomes, the darker and more opaque the background grows.
If a user plays sharply without pedal, the acoustic energy falls quickly. The circles fade rapidly and the background remains largely white. With continuously sustained pedal, the acoustic energy stays high, the circles remain large for longer, and the background grows progressively darker.
The eventual test is straightforward: compare how well students use the pedal when practicing with the visualizer against students practicing without it.