cjpais/Handy

▲ 84 stars today★ 31,651⑂ 2,878

A free, open source, and extensible speech-to-text application that works completely offline.

accessibilitycross-platformspeech-to-texttauri-v2
31,651Star
2,878Fork
97Watch
164Issue
RustLanguage
MITLicense
Open on GitHub ↗Homepage ↗
Created 2025-02-13 · last push 2026-09-15 · repository size 14402 KB · default branch main

README

Handy

Discord

A free, open source, and extensible speech-to-text application that works completely offline.

Handy is a cross-platform desktop application that provides simple, privacy-focused speech transcription. Press a shortcut, speak, and have your words appear in any text field. This happens on your own computer without sending any information to the cloud.

Why Handy?

Handy was created to fill the gap for a truly open source, extensible speech-to-text tool. As stated on handy.computer:

Handy isn't trying to be the best speech-to-text app—it's trying to be the most forkable one.

How It Works

1. Press a configurable keyboard shortcut: hold it to record and release to stop, or tap it to toggle recording on and off (Hold-only and Toggle-only modes are also available) 2. Speak your words while the shortcut is active 3. Release and Handy processes your speech using Whisper 4. Get your transcribed text pasted directly into whatever app you're using

The process is entirely local:

Quick Start

Installation

1. Download the latest release from the releases page or the website

Note: The Homebrew cask and winget package are not maintained by the Handy developers. 
     sudo apt install ./Handy_*.deb
Do not use dpkg -i unless the dependencies are already installed. If you already used it, run sudo apt --fix-broken install. 2. Install the application 3. Launch Handy and grant necessary system permissions (microphone, accessibility) 4. Configure your preferred keyboard shortcuts in Settings 5. Start transcribing!

Development Setup

For detailed build instructions including platform-specific requirements, see BUILD.md.

Integrations

https://github.com/cjpais/Handy/blob/HEAD/Install handy Raycast Extension

Control Handy from Raycast — start/stop recording, browse transcript history, manage dictionary, switch models and languages.

Source · by @mattiacolombomc

Architecture

Handy is built as a Tauri application combining:

Debug Mode

Handy includes an advanced debug mode for development and troubleshooting. Access it by pressing:

CLI Parameters

Handy supports command-line flags for controlling a running instance and customizing startup behavior. These work on all platforms (macOS, Windows, Linux).

Remote control flags (sent to an already-running instance via the single-instance plugin):

handy --toggle-transcription    # Toggle recording on/off
handy --toggle-post-process     # Toggle recording with post-processing on/off
handy --cancel                  # Cancel the current operation

Startup flags:

handy --start-hidden            # Start without showing the main window
handy --no-tray                 # Start without the system tray icon
handy --debug                   # Enable debug mode with verbose logging
handy --help                    # Show all available flags

Flags can be combined for autostart scenarios:

handy --start-hidden --no-tray
macOS tip: When Handy is installed as an app bundle, invoke the binary directly:
> 
> /Applications/Handy.app/Contents/MacOS/Handy --toggle-transcription

Known Issues & Current Limitations

This project is actively being developed and has some known issues. We believe in transparency about the current state:

Bluetooth Headset Microphones (macOS)

Using a Bluetooth headset microphone on macOS may temporarily reduce playback quality or volume while recording because Bluetooth switches to bidirectional audio. Keep your headphones as the output device and select your Mac's built-in or an external microphone in Handy to avoid this.

fn and Globe Key Shortcuts (macOS)

Shortcuts that include the fn (Globe) key only work on Apple keyboards — your Mac's built-in keyboard or an Apple external keyboard. They will never trigger on a third-party keyboard, even while it is connected to the same Mac.

This is a hardware limitation rather than a Handy bug. fn is not part of the standard USB HID keyboard specification: Apple reports it through a vendor-specific usage that macOS honors only from Apple devices, while third-party keyboards handle their Fn key entirely in firmware and send nothing to the computer. There is no event for Handy to listen for.

If you switch between a MacBook keyboard and an external one, pick a shortcut built from standard modifiers (ctrl, option, shift, command) or a regular key instead.

Major Issues (Help Wanted)

Whisper Model Crashes:

Wayland Support (Linux):

Linux Notes

Text Input Tools:

For reliable text input on Linux, install the appropriate tool for your display server:

| Display Server | Recommended Tool | Install Command | | -------------- | ---------------- | -------------------------------------------------- | | X11 | xdotool | sudo apt install xdotool | | Wayland | wtype | sudo apt install wtype | | Both | dotool | sudo apt install dotool (requires input group) |

Without these tools, Handy falls back to enigo which may have limited compatibility, especially on Wayland.

Other Notes:

| Distro | Package to install | Example command | | ------------- | --------------------- | -------------------------------------- | | Ubuntu/Debian | libgtk-layer-shell0 | sudo apt install libgtk-layer-shell0 | | Fedora/RHEL | gtk-layer-shell | sudo dnf install gtk-layer-shell | | Arch Linux | gtk-layer-shell | sudo pacman -S gtk-layer-shell | GNOME: 1. Open Settings > Keyboard > Keyboard Shortcuts > Custom Shortcuts 2. Click the + button to add a new shortcut 3. Set the Name to Toggle Handy Transcription 4. Set the Command to handy --toggle-transcription 5. Click Set Shortcut and press your desired key combination (e.g., Super+O)

KDE Plasma: 1. Open System Settings > Shortcuts > Custom Shortcuts 2. Click Edit > New > Global Shortcut > Command/URL 3. Name it Toggle Handy Transcription 4. In the Trigger tab, set your desired key combination 5. In the Action tab, set the command to handy --toggle-transcription

Sway / i3:

Add to your config file (~/.config/sway/config or ~/.config/i3/config):

  bindsym $mod+o exec handy --toggle-transcription

Hyprland:

Add to your config file (~/.config/hypr/hyprland.conf):

  bind = $mainMod, O, exec, handy --toggle-transcription
| Action | Trigger | | ----------------------------------------- | -------------------------------------------------------- | | Toggle transcription | pkill -USR2 -n handy or handy --toggle-transcription | | Toggle transcription with post-processing | handy --toggle-post-process |

Example Sway config:

  bindsym $mod+o exec pkill -USR2 -n handy
  bindsym $mod+p exec handy --toggle-post-process

pkill here simply delivers the signal—it does not terminate the process.

> Behavior change: older releases also accepted SIGUSR1 for toggling transcription with post-processing. WebKitGTK — the webview engine embedded in Handy on Linux — uses SIGUSR1 internally to coordinate JavaScript garbage collection, so listening for it caused phantom recordings and interrupted dictations every few minutes (#1660). Handy no longer listens for SIGUSR1 on Linux; the post-processing toggle is still available via handy --toggle-post-process. Remove any pkill -USR1 bindings: the signal is now delivered straight to WebKit's internal handler and can crash the app.

Overlay & Pasting Issues (Linux):

Platform Support

System Requirements/Recommendations

The following are recommendations for running Handy on your own machine. If you don't meet the system requirements, the performance of the application may be degraded. We are working on improving the performance across all kinds of computers and hardware.

For Whisper Models:

For Parakeet V3 Model:

Roadmap & Active Development

We're actively working on several features and improvements. Contributions and feedback are welcome!

In Progress

Debug Logging:

macOS Keyboard Improvements: Opt-in Analytics: Settings Refactoring: Tauri Commands Cleanup:

Verify Release Signatures

Handy release artifacts are signed with Tauri's updater signature format. The public key is stored in src-tauri/tauri.conf.json under plugins.updater.pubkey.

To verify a release manually, set ARTIFACT to the filename you downloaded, save the pubkey value from src-tauri/tauri.conf.json to handy.pub.b64, then decode the public key and matching .sig file from base64 and verify the artifact with minisign:

# Replace with the file you downloaded
ARTIFACT="Handy_0.8.1_amd64.AppImage"

python3 - "$ARTIFACT" <<'PY'
import base64, pathlib, sys


artifact = sys.argv[1]


pub = pathlib.Path("handy.pub.b64").read_text().strip()
pathlib.Path("handy.pub").write_bytes(base64.b64decode(pub))


sig = pathlib.Path(f"{artifact}.sig").read_text().strip()
pathlib.Path(f"{artifact}.minisig").write_bytes(base64.b64decode(sig))
PY


minisign -Vm "$ARTIFACT" \
  -p handy.pub \
  -x "$ARTIFACT.minisig"

On success, minisign prints:

Signature and comment signature verified

Do not use gpg for these .sig files.

Troubleshooting

Previous Clipboard Content Is Pasted Instead of the Transcription

If the transcription is correct in History but Handy inserts text you copied earlier, see issue #502. With the standard clipboard paste method, Handy restores your previous clipboard after a fixed delay. Under load, the receiving application may read the clipboard only after that restoration.

1. Open Handy's settings window and press Cmd+Shift+D (macOS) or Ctrl+Shift+D (Windows/Linux) to reveal Debug. 2. On macOS and Windows, try Reliable Paste (Beta) in Debug with a clipboard paste method selected. It uses clipboard read notifications to delay restoration instead of relying on the standard fixed delay. Test it in the application where the problem occurs; it is still experimental. 3. If Reliable Paste is disabled or unavailable, increase Paste Delay (After) in Debug and test again. This controls the wait before restoring your previous clipboard. Paste Delay (Before) controls the wait before sending the paste keystroke and addresses a different part of the operation. These delay settings apply to the standard paste path, not Reliable Paste.

If the problem persists, add your Handy version, operating system, receiving application, paste method, Reliable Paste setting, and before/after delays to the existing issue. Redact private dictated text before sharing logs.

Manual Model Installation (For Proxy Users or Network Restrictions)

If you're behind a proxy, firewall, or in a restricted network environment where Handy cannot download models automatically, you can manually download and install them. The URLs are publicly accessible from any browser.

Step 1: Find Your App Data Directory

1. Open Handy settings 2. Navigate to the About section 3. Copy the "App Data Directory" path shown there, or use the shortcuts:

The typical paths are:

Step 2: Create Models Directory

Inside your app data directory, create a models folder if it doesn't already exist:

# macOS/Linux
mkdir -p ~/Library/Application\ Support/com.pais.handy/models

Windows (PowerShell)

New-Item -ItemType Directory -Force -Path "$env:APPDATA\com.pais.handy\models"

Step 3: Download Model Files

Download the models you want from below

Whisper Models (single .bin files):

Parakeet Unified EN 0.6B (single .gguf file, recommended): Parakeet Models (compressed archives):

Step 4: Install Models

For Whisper Models (.bin files):

Simply place the .bin file directly into the models directory:

{app_data_dir}/models/
├── ggml-small.bin
├── whisper-medium-q4_1.bin
├── ggml-large-v3-turbo.bin
└── ggml-large-v3-q5_0.bin

For GGUF Models (.gguf files):

Place the .gguf file directly into the models directory, exactly like the Whisper .bin files above. Handy also picks up models already present in the shared Hugging Face cache (~/.cache/huggingface/hub), so a copy downloaded by another tool works without being moved.

For Parakeet Models (.tar.gz archives):

1. Extract the .tar.gz file 2. Place the extracted directory into the models folder 3. The directory must be named exactly as follows:

Final structure should look like:

{app_data_dir}/models/
├── parakeet-tdt-0.6b-v2-int8/     (directory with model files inside)
│   ├── (model files)
│   └── (config files)
└── parakeet-tdt-0.6b-v3-int8/     (directory with model files inside)
    ├── (model files)
    └── (config files)

Important Notes:

Step 5: Verify Installation

1. Restart Handy 2. Open Settings → Models 3. Your manually installed models should now appear as "Downloaded" 4. Select the model you want to use and test transcription

Custom Whisper Models

Handy can auto-discover custom Whisper GGML models placed in the models directory. This is useful for users who want to use fine-tuned or community models not included in the default model list.

How to use:

1. Obtain a Whisper model in GGML .bin format (e.g., from Hugging Face) 2. Place the .bin file in your models directory (see paths above) 3. Restart Handy to discover the new model 4. The model will appear in the "Custom Models" section of the Models settings page

Important:

Linux Startup Crashes or Instability

If Handy fails to start reliably on Linux — for example, it crashes shortly after launch, never shows its window, or reports a Wayland protocol error — try the steps below in order.

1. Install (or reinstall) gtk-layer-shell

Handy uses gtk-layer-shell for its recording overlay and links against it at runtime. A missing or broken installation is the most common cause of startup failures and can manifest as a crash or a hang well before any window is shown. Make sure the runtime package is installed for your distro:

| Distro | Package to install | Example command | | ------------- | --------------------- | -------------------------------------- | | Ubuntu/Debian | libgtk-layer-shell0 | sudo apt install libgtk-layer-shell0 | | Fedora/RHEL | gtk-layer-shell | sudo dnf install gtk-layer-shell | | Arch Linux | gtk-layer-shell | sudo pacman -S gtk-layer-shell |

If it is already installed and you still see startup problems, try reinstalling it (e.g. sudo pacman -S gtk-layer-shell again) in case the library files were corrupted by a partial upgrade.

2. Disable the GTK layer shell overlay (HANDY_NO_GTK_LAYER_SHELL)

If installing the library does not help, you can skip gtk-layer-shell initialization entirely as a workaround. On some compositors (notably KDE Plasma under Wayland) it has been reported to interact poorly with the recording overlay. With this variable set, the overlay falls back to a regular always-on-top window:

HANDY_NO_GTK_LAYER_SHELL=1 handy

3. Disable WebKit DMA-BUF renderer (WEBKIT_DISABLE_DMABUF_RENDERER)

On some GPU/driver combinations the WebKitGTK DMA-BUF renderer can cause the window to fail to render or to crash. Try:

WEBKIT_DISABLE_DMABUF_RENDERER=1 handy

Making a workaround permanent

Once you've found a flag that helps, export it from your shell profile (~/.bashrc, ~/.zshenv, …) or from the desktop autostart entry that launches Handy. If you launch Handy from a .desktop file, you can prefix the Exec= line, e.g.:

Exec=env HANDY_NO_GTK_LAYER_SHELL=1 handy

If a workaround helps you, please open an issue

More Today's Trending projects

1

alibaba / open-code-review

Go★ 28,122⑂ 2,023▲ 2,751 stars
→
2

debpalash / VoiceStudio

Python★ 30,697⑂ 3,683▲ 2,081 stars
→
3

JustVugg / colibri

C★ 33,539⑂ 3,522▲ 2,035 stars
→
4

tt-a1i / archify

JavaScript★ 63,354⑂ 4,208▲ 1,407 stars
→
5

cloudflare / security-audit-skill

JavaScript★ 4,982⑂ 310▲ 1,210 stars
→
6

Panniantong / Agent-Reach

Python★ 81,982⑂ 7,140▲ 1,094 stars
→