ZEHEN turns your laptop camera into a gesture controller for Windows and your voice into its command line. Say "Zehen" and it opens, clicks, types and answers out loud. Free, open source, no extra hardware.
Open palm brings up the launcher. Tap Notepad. Write “Hey” in the air and Windows types it. Jump to YouTube, scroll with two fingers, stretch both hands to zoom, swipe back.
Tracking runs in the background. A see-through HUD draws a glowing cursor ring; your screen stays yours.
Pinch and write in the air. Windows' offline handwriting recognition types it into any app, with “did you mean” fixes.
Hold an open palm: every open window, your favourite apps, search across all installed apps, system controls.
Stretch both hands to zoom like a photo. Swipe your other hand to flick between apps.
One Euro smoothing, debounced gestures and click-lock make a hand as precise as a touchpad.
Touch your real mouse and ZEHEN lets go. Make a fist to pause. Guided tutorial on first run.
One wake word, then plain language. It plans the steps, runs them, reads the result and keeps going until it is done. It learns how you say the wake word, so it wakes for you and nothing else. Ask it anything and it talks back.
Clicks buttons, tabs and list items by name through Windows UI Automation, types in any language, runs PowerShell for the rest.
A small translucent pill shows what your hand or voice is doing. Point at it and it opens, then folds back on its own. One button turns the camera off for voice-only use.
The wake word is detected on your PC and only counts at the start of a phrase, so ordinary conversation never wakes it. After it, the assistant plans the job, acts, looks at the result and answers out loud. Pick a request to see the loop. This is a recorded walkthrough, not a live model.
Vosk wakes on the first word. Your request is transcribed on the PC, or by Groq Whisper with a Groq key.
A language model answers in strict JSON. Anything outside the action list is dropped.
Apps, keys, UI Automation clicks, links, PowerShell. At most six steps per request.
Kokoro, an open source neural voice, reads the answer so you never need to look.
The camera frame is mirrored and handed to MediaPipe, never stored or shown.
21 landmarks per hand, both hands, every frame, fully on-device.
Finger poses, pinch distance scaled to hand size, debounced into a gesture.
Real mouse, wheel and keystrokes, plus the HUD so you always see what happened.
Cursor, clicks, scroll, two-hand zoom, launcher, air-writing, keyboard, tutorial.
Wake word, spoken replies, planning with a free cloud or offline model, UI Automation, PowerShell, glass panel.
Per-person calibration, custom gestures, a trained wake word, Hindi, sound feedback, multi-monitor, one-click installer.
Point at a window and say “maximize” or “send it to the left screen”.
Grab and throw windows between screens, rotate 3D models with two hands, push to press.
AR glasses, eye + hand targeting, smart home, gesture macros, plugin API.
The assistant needs a language model to plan. Use a free cloud key for speed, or let ZEHEN download a small model once and keep everything on your PC.
| Free cloud key | Offline brain | |
|---|---|---|
| Setup | Panel, Get key, sign in, Paste key | None. Downloads once, 1.1 GB |
| Model | Groq gpt-oss-120b, Gemini Flash or OpenRouter free | Qwen3 1.7B on your CPU |
| Skill | Fast and strong, handles multi-step plans | Slower and simpler |
| Cost | Free plan with limits the provider can change | Free forever |
| Leaves your PC | What you ask after the wake word, plus app and window names | Nothing |
# Windows 10/11 · Python 3.10+ · webcam + microphone
git clone https://github.com/priyanshusta/ZEHEN.git
cd ZEHEN
.\ZEHEN.bat
Fork it first if you want to change it: fork on GitHub. A guided tutorial runs the first time, then say "Zehen".
Yes. It is MIT licensed, has no paid tier and needs no account. The optional cloud brain uses providers' free plans; if one changes, switch keys or use the offline brain.
Yes. ZEHEN tracks both hands with MediaPipe through a normal laptop camera and drives the real Windows cursor, clicks, scrolling and zoom. No gloves, depth camera or other hardware.
That is the idea. Say the wake word, ask in plain language, and the assistant operates your PC and answers out loud. The wake word, speech engine and voice are open source and run offline.
Hand tracking, handwriting, the wake word and the voice are offline. Planning needs a brain: the offline Qwen3 model works with no internet after its one-time download.
The speech model has no word "zehen" and hears it as "zane". If your accent differs, add spellings to wake in .zehen.json, and say the word at the start of a phrase.
About 0.7 GB with the camera, hand tracking, voice recognition and the voice loaded.
Not yet. The window and app control, handwriting recognition and UI Automation are Windows specific. It is on the roadmap.