By the end of ESAP robotics, the piano player had moved from a set of actuator experiments into a much more complete system: servos mounted over keys, electronics drilled down more securely, timing logic rewritten around musical events, and a codebase tuned for speed. The final stretch was less about proving that a servo could press a key and more about making the whole machine respond fast enough and consistently enough to play real pieces.
A large part of the improvement came from code optimization. Earlier versions worked, but they left too much timing slack in places where the robot had to react quickly. The final code focused on reducing blocking behavior and making note handling more direct, so the controller could keep scheduling new actions while other notes were still being played.
One major change was the note structure. We moved toward a more MIDI-like model with three essential pieces of information: pitch, note on, and duration. Instead of treating a song as a long chain of delays, the scheduler could look at note events, decide when a pitch should turn on, and separately decide how long that note should remain active. This made the representation cleaner and closer to how music is usually encoded.
The challenge was keeping that cleaner structure as efficient as the previous one. We used non-blocking schedulers so the microcontroller did not have to wait idly during a note. While one servo was holding or releasing a note, the scheduler could continue checking the rest of the song timeline and preparing other servo commands. That let the new MIDI-style structure stay readable without giving up the timing efficiency needed for faster passages.
Understanding Lift Time
The most important speed improvement came from understanding lift times: the time it takes for a servo head to rise back up far enough that it can come down again for the next note. This mattered most for repeated notes. If the same key had to be played several times quickly, the servo could not simply stay down for the full written duration. It had to release early enough to reset before the next strike.
Our first solution used static lift times. The code assumed a worst case where the same note might repeat multiple times at a fast frequency, so it would call the servo head back up early enough to be ready for the next repeated note. This protected repeated-note sections, but it created a new problem: in very fast sections, notes were sometimes released before the servo head had fully touched the key. In those cases, the optimization made the note faster on paper but weaker or missing in sound.
We changed the logic so that strict lift-time enforcement only happened when notes were actually repeated multiple times in a row. That made the behavior more musical. A single short note no longer had to pay the full worst-case reset cost unless the upcoming pattern required it. The scheduler became more context-aware: repeated-note runs were handled aggressively, while other fast passages were allowed to use more of their available note duration.
We also adjusted the percentage of each note that the robot tried to play. In standard piano playing, notes are often held for around 80% of their expected duration. Holding much more can make the passage sound too connected, or legato; holding much less can make it too sharp, or staccato. We originally used the same idea here. However, for very short durations, 80% was sometimes not enough physical time for the servo to press the key clearly.
To fix that, we allowed short notes to use their full duration, and in some cases slightly exceed it, so the note could actually be heard. This mattered in the Bach pieces and in some of the faster notes in "Moondance." The tradeoff was that the robot sometimes had to bend the ideal written duration slightly, but the result was better because the listener could hear the intended notes.
Final Build
On the build side, we made the prototype more rigid. The electronics and servo mounts were drilled down solidly instead of relying on looser placement, and we used weights to prevent the MDF base from flexing. That was important because even small changes in height or angle could change whether a servo actually reached the key. The software timing only mattered if the mechanical frame kept the servo heads in the right positions.
The full repository for the project, including images, code, and videos, is available at namsysid/ESAP_Piano.git.