Your webcam sees 21 points on each hand, and your fingers become the instrument. 5 play modes — air harp · air keys · chord conductor · theremin · drum pads · 15 synthesised instruments · 12 scales in any key · 10-finger polyphony · record video + WAV of your performance · fullscreen mode. Free, no signup — and the camera feed never leaves your browser.
An instrument you play in mid-air, in a browser tab
Every musical instrument is a compromise between what your body can do and what physics will let you get away with. A violin needs a decade before it sounds like anything; a piano needs to be in the room. This one needs a webcam. The camera watches your hands, a landmark model returns twenty-one points per hand about thirty times a second, and those points are read as musical intent: which fingers are out, where they are, how fast they are moving, whether the palm is tilted or pinched. Then a fifteen-voice synthesiser does the rest.
The result is closer to a theremin than a keyboard — you play in space rather than against a surface — but unlike a theremin it can be locked to a scale, which removes the single thing that makes gesture instruments frustrating. If C Minor Pentatonic is selected, C Minor Pentatonic is all the air contains. You can wave your arms around like a lunatic and it will still be in key.
Five surfaces, not one gimmick
Most webcam music demos implement a single idea. This one gives you five, because different music wants different physical gestures. Air Harp turns the frame into a set of strings you sweep through — plucky, percussive, immediately satisfying. Air Keys puts a keyboard strip under your hands for held notes and chords, which is what pads and organs actually want. Chord Conductor splits the work between hands the way a real accompanist does: left hand chooses the harmony, right hand voices and swells it. Theremin is the pure continuous-pitch experience with optional scale snapping. Drum Pads gives you nine synthesised drums you punch in the air.
Fifteen instruments, none of them samples
Nothing here is a recording. Every note is built live with the Web Audio API using whichever synthesis method actually suits the sound. Pads, leads, strings and brass are subtractive — stacks of detuned sawtooths pushed through a resonant filter with its own envelope. The electric piano and vibraphone are two-operator FM, which is how those sounds were made in the 1980s in the first place. The struck piano and music box are additive with deliberately inharmonic partials, because struck metal and stretched strings do not vibrate in tidy whole-number ratios. The nylon guitar and concert harp use Karplus-Strong: a burst of noise recirculating in a delay line one wavelength long, losing a little energy on each pass — a genuine physical model of a vibrating string, rendered per note and cached. The flute is filtered noise. The organ is a drawbar stack.
- Sustained: Warm Pad, Bowed Strings, Church Organ, Choir Aah, Flute, Brass Section
- Plucked and struck: Concert Harp, Nylon Guitar, Struck Piano, Music Box, Vibraphone, Electric Piano
- Synth: Bright Lead, Retro Square, Sub Bass
Recording that is actually usable
Pressing record captures two things at once. The first is a WebM video of the canvas — camera view, hand skeleton, glowing strings, note ripples and the heads-up display exactly as you see them — with the synth output mixed into the audio track. That is the file you post. The second is an uncompressed 16-bit stereo WAV written straight from the synth bus, with no Opus compression in the way. That is the file you drop into a DAW when the take turns out better than expected. Both stay in memory until you download them; there is no upload and no server.
Where the video goes: nowhere
This matters enough to say plainly. The camera stream is handed to a canvas and a model running inside your browser tab. It is not sent to a server, because there is no server — the whole instrument is one HTML file. The only recording that exists is one you start deliberately, and it lives in your machine's memory until you either download it or close the tab. The page never asks for your microphone at all, which is also why the recorded audio is completely clean of room noise.
Getting good results
Tracking quality is almost entirely a lighting problem. The model needs to see the outline of individual fingers, so put a window or lamp in front of you rather than behind, avoid a background that matches your skin tone, and keep whole hands inside the frame — the moment fingertips leave the edge, the model starts guessing. If small jitter is triggering notes you did not mean, raise Smoothing; if gentle movements are being ignored, raise Sensitivity. And use wired headphones: Bluetooth adds 100 to 200 milliseconds of latency that no browser can remove.
FAQ — hand tracking instrument
37 questions on how it works, what it can do and what happens to your camera feed.
Getting started
Do I need to install anything?
No. It is a normal web page. The hand tracking model is fetched once when you press Start camera and is then cached by your browser, so a second visit starts almost instantly. There is nothing to install, no account and no extension.
Which mode should a complete beginner start with?
Air Harp with a pentatonic scale. Pentatonic scales have no half-step clashes, so every string sounds good against every other string and there is no way to play a wrong note. Sweep an open hand slowly across the frame and you are already making music.
Why does nothing happen when I press Start camera?
Either the browser blocked the camera or another application is holding it. Check the camera icon in the address bar and allow the site, then close any video-call app that may still own the device. On macOS also check System Settings → Privacy & Security → Camera and make sure your browser is enabled. Reload the page after changing a permission.
Why is there no sound even though the hands are tracking?
Browsers do not allow audio to start until you interact with the page, so the audio engine is created on your first click. If the strings are lighting up but you hear nothing, click anywhere on the page, check that the master volume is up and that the tab is not muted, and make sure the right output device is selected in your system settings.
Does it work on a phone or tablet?
Yes, using the front camera, and landscape orientation with the phone propped up gives by far the best result — you need both hands free. Tracking runs slower on older phones, so switch to one-hand modes like Air Harp or Theremin, and reduce the string count if it feels sluggish.
Which browsers are supported?
Chrome and Edge give the best performance because they can run the model on the GPU. Firefox and Safari 16 and later both work. The page needs a secure context, which any https site or localhost provides. Very old browsers without WebGL or the MediaRecorder API will load the page but cannot track or record.
Playing & gestures
How does playing music with my hands actually work?
Your webcam frame is passed to a hand landmark model that returns 21 three-dimensional points per hand — four joints for each finger plus the wrist — about thirty times a second. The page reads those points to work out where each fingertip is, which fingers are extended, how the palm is tilted and whether you are pinching, then turns that into notes. No frame is ever sent anywhere; the model runs inside your browser tab.
What are the five play modes?
Air Harp gives you vertical strings that sound when a fingertip sweeps through them — the easiest and most expressive mode. Air Keys puts a keyboard strip in the lower part of the frame that you hold notes on. Chord Conductor uses your left hand to choose a chord and your right hand to shape it. Theremin maps left-right movement to pitch and up-down to volume, like the real instrument. Drum Pads gives you a nine-pad grid you hit in the air.
What does the pinch gesture do?
Touching your thumb to your index finger latches sustain, exactly like a piano pedal — notes keep ringing after your finger has left the string or key. Release the pinch to let them decay. The on-screen hand outline turns solid while sustain is held so you can always see the state.
What does making a fist do?
It silences that hand immediately. Because the app only follows extended fingers, a closed hand has nothing to trigger with — which means you can move your hand back across the frame without playing anything on the way, the same way a harpist lifts their hand clear of the strings between phrases.
How do I play more than one note at a time?
Use more fingers. Air Harp and Air Keys are fully polyphonic and follow all ten fingertips, so you can hold a chord with your left hand while a right-hand finger picks out a melody. Chord Conductor is polyphonic by definition. Theremin and Drum Pads are intentionally single-voice and percussive.
How do I choose which notes are available?
Pick a root note and a scale in the panel. Every string, key and pitch position is then locked to that scale, so wrong notes are impossible. There are twelve scales — Major, Natural Minor, Harmonic Minor, Dorian, Phrygian, Lydian, Mixolydian, Major Pentatonic, Minor Pentatonic, Blues, Whole Tone and Chromatic — in all twelve keys.
Can I play in a specific key to jam along with a track?
Yes, that is exactly what the root and scale selectors are for. Set the root to the key of the song and pick Major or Natural Minor to match, then everything you play will be in key. Minor Pentatonic and Blues are the safest choices for soloing over almost anything.
How does the Chord Conductor mode work?
Your left hand chooses the chord: one to five extended fingers select scale degrees I to V, index and pinky together give VI, and index, pinky and thumb give VII. Tilting the left hand outward flips the chord between its major and minor colour. Your right hand shapes it: the number of fingers steps through root position, first inversion, second inversion, a seventh and an added ninth, palm height sets the volume and tilt opens or closes the filter. A closed left fist stops the chord.
Can it see both hands?
Yes, two hands at once, twenty-one points each. Roles are assigned by where the hands appear on screen — the hand further left in the mirrored view is treated as your left hand — which is predictable and matches what you see. With only one hand visible the app falls back on the model's own left/right classification.
Why is the image mirrored?
Because a mirror is how you expect to see yourself: move your hand right and the on-screen hand goes right. Without mirroring, playing a rising scale would mean moving your hand the wrong way, which is genuinely disorienting. You can turn mirroring off in the panel if you prefer.
What do the smoothing and sensitivity sliders do?
Smoothing averages each landmark with its previous position. More smoothing means steadier visuals and fewer accidental triggers, but a slightly later response. Sensitivity sets how much movement counts as a deliberate hit — lower it if you are triggering notes by accident, raise it if gentle movements are being ignored.
How many notes can sound at once?
Sixteen. Each voice runs several oscillators, and without a cap a long sustained passage would gradually saturate the audio thread and start to crackle. When the limit is reached the oldest and quietest voice is faded out first, which is normally inaudible.
The sounds
How many instruments are there?
Fifteen: Warm Pad, Bright Lead, Retro Square, Electric Piano, Struck Piano, Music Box, Vibraphone, Nylon Guitar, Concert Harp, Bowed Strings, Church Organ, Brass Section, Flute, Choir Aah and Sub Bass. Each one is a different synthesis engine rather than a preset on the same sound, so switching genuinely changes the character of the instrument.
Are the instruments recorded samples?
No — every note is generated live with the Web Audio API. Pads and strings are subtractive (detuned sawtooth stacks through a filter with an envelope), the electric piano and vibraphone are two-operator FM, the piano and music box are additive with deliberately inharmonic partials, the guitar and harp use a Karplus-Strong plucked string model rendered per note, the organ is a drawbar stack and the flute is filtered noise. That is why the page weighs almost nothing and works offline once loaded.
Why do the guitar and harp sound like real plucked strings?
They use Karplus-Strong synthesis: a burst of noise is fed into a short delay line that is exactly one wavelength long, and each time round the loop the signal is slightly filtered and slightly quieter. That is a surprisingly accurate physical model of a vibrating string — the noise burst is the pluck, the loop is the string, and the filter is the energy the string loses to the air on every cycle. Each note is rendered into a buffer the first time you play it and cached from then on.
Can I use a MIDI keyboard or export MIDI?
Not here — this instrument is deliberately about the hands, and it records audio and video rather than note data. If you want MIDI export, the
Online Virtual Piano records what you play and downloads it as a standard .mid file, and the
Drum Loop Studio exports seamless loop WAVs.
Recording
Can I record what I play?
Yes. Press Record and the page captures two things at once: a WebM video of the canvas — camera view, hand skeleton, glowing strings and note ripples included — with the synth audio mixed in, and a separate uncompressed 16-bit WAV of just the audio. When you stop you get a preview you can play back in the page and two download buttons.
Why record a WAV as well as the video?
The video is for sharing; the WAV is for working with. WebM audio is compressed with Opus, which is fine for social posts but lossy. The WAV is the raw 16-bit stereo mixdown straight from the synth, so you can drop it into a DAW, layer it, chop it or master it without a generation of compression already baked in.
What format is the video and will it open on my phone?
It is WebM, which is what browsers can record natively. It plays in Chrome, Edge, Firefox and modern Android and macOS players, and it uploads fine to YouTube, Instagram and TikTok, which transcode on their side. Older iOS photo apps do not always preview WebM — if you need MP4, run the file through any converter, or use the WAV audio alongside a screen recording.
Is there a limit on recording length?
Only your memory. The recording is held in RAM until you download it, so a few minutes is comfortable on any machine and very long takes on a low-memory device may eventually stall. If you are recording something important, download it before starting the next take.
Does the recording include the sound of my room?
No. Only the microphone would capture your room, and the app never requests microphone access — it only asks for the camera. The recorded audio is the synth output alone, which is why it is completely clean.
Troubleshooting
There is a delay between moving my hand and hearing the note.
Some latency is unavoidable: the camera exposes a frame, the model analyses it and only then can a note start. Expect roughly 50 to 120 milliseconds on a typical laptop. To reduce it, use good lighting so the camera does not drop to a slower exposure, close other tabs that compete for the GPU, and avoid Bluetooth headphones — they add 100 to 200 ms on their own that nothing in a browser can remove.
The tracking is jittery or keeps losing my hand.
Almost always lighting. The model needs to see the outline of your fingers, so face a window or a lamp rather than having one behind you, and avoid a background the same colour as your skin. Keep your whole hand inside the frame — if fingertips leave the edge, the model guesses. Raising the Smoothing slider will also calm small jitter at the cost of a slightly softer response.
Privacy, practicalities & licence
Is my camera feed uploaded or recorded without me knowing?
No. The video stream goes from your camera straight into a canvas and a machine learning model inside the page. Nothing is uploaded, there is no server behind this tool, and the only recording that ever exists is one you start yourself with the Record button — which stays in memory until you download it.
How is this better than the gesture synth apps I have seen?
Most gesture instruments implement one idea — usually chord triggering — with three or four sounds. This has five genuinely different playing surfaces, fifteen synthesis engines, full scale and key control, ten-finger polyphony, and it records what you play as a shareable video with the hand overlay burned in and as a clean WAV at the same time. It is also fully self-contained: no account, no upload, no watermark.
What does fullscreen mode do?
It hides the page around the instrument so the camera view fills the screen, and it requests a screen wake lock so your display does not sleep while you are playing without touching the keyboard. Press F or Esc to leave. Fullscreen recordings look considerably better because the canvas is captured at a larger size.
Are my settings remembered?
Yes. Mode, instrument, key, scale, octave, string count, all the effect levels and the display toggles are stored in your browser's local storage and restored next time. Nothing leaves your device, and clearing your browser data resets everything to defaults.
Does it work offline?
After the first successful load, largely yes — the hand tracking model and the code are cached by the browser and all the sound is generated locally. The very first visit does need a connection to fetch the model.
Can I use my recordings commercially?
Yes. What you play is
your own work — no watermark, no attribution required, royalty free, commercial or not. The tool itself is free with no strings attached; a credit or a
coffee is appreciated but never required.
Is any of this stored or tracked?
No performance data, no video and no audio is stored anywhere but your own device. The site uses privacy-first analytics that set no cookies and record no personal data, purely to count page views.
Is it really free?
Completely — no account, no ads, no paywall and no watermark. It is one of the free browser tools at
jasperbernaers.com.