A real-time hand gesture-based chord player using MediaPipe Hands for finger detection and SoundDevice for audio playback. Play major chords by showing different numbers of fingers, and control volume by specific finger counts combined with vertical hand movement.
- Detects fingers using your webcam via MediaPipe Hands.
- Plays major chords (C, D, E, F, G, A, B, C-high) mapped to 1-8 fingers.
- Shows chord name and volume level directly on the video feed.
- Volume control gestures:
- Show 9 fingers and slowly move hand down to decrease volume.
- Show 10 fingers and slowly move hand up to increase volume.
- Real-time feedback with minimal lag (async audio playback).
- Visual overlays highlighting detected hands and finger counts.
- Python 3.7+
- OpenCV
- MediaPipe
- NumPy
- SoundDevice
Install dependencies with pip:
pip install mediapipe opencv-python numpy sounddeviceRun the main script:
python hand_chord_player.py
Press s to start chord detection and playback.
Press q to quit the application.
Show fingers in front of the camera to play chords:
1 finger = C Major
2 fingers = D Major
3 fingers = E Major
4 fingers = F Major
5 fingers = G Major
6 fingers = A Major
7 fingers = B Major
8 fingers = C Major (Next Octave)
Use volume control gestures:
Show 9 fingers and move hand down → volume down
Show 10 fingers and move hand up → volume up
- Captures webcam frames.
- Uses MediaPipe Hands to detect hands and finger landmarks.
- Counts extended fingers considering left/right hand orientation.
- Maps finger count to chord frequencies.
- Plays corresponding chord asynchronously.
- Tracks vertical hand position for volume control gestures.
- Displays chord and volume info overlayed on the webcam feed.
- For best results, use a well-lit environment.
- Volume control sensitivity can be adjusted via parameters in the code.
- Audio playback is done asynchronously for smooth real-time performance.
- The system supports two hands and counts total fingers extended across both.