Synthalingua

Synthalingua

Note: The Synthalingua Wrapper code has been moved to a new repo here: https://git*hu*b.co*m/cyberofficial/Synthalingua_Wrapper
By using Synthalingua, you agree to use it responsibly and accept full responsibility for your actions. Let\’s keep it fun, safe, and positive for everyone!

• Real-time translation & transcription (stream, mic, file)
• Multilingual: Translate between dozens of languages
• Streaming & captions: HLS, YouTube, Twitch, and more
• Blocklist & repetition suppression: Auto-filter repeated or unwanted phrases
• Discord & web integration: Send results to Discord or view in browser
• Portable GUI version available

Quick Start

1. Install Python 3.12 from here and Git
2. Install FFMPEG (guide)
3. (Optional) Install CUDA 12.8 for GPU acceleration if you plan to use GPU features
4. Windows: Run setup.bat | Linux: See LINUX_SETUP.md for detailed setup instructions
5. Edit and run the generated batch/bash file, or use the GUI portable version:

ository: Synthalingua_Wrapper

Table of Contents

• About
• Documentation Wiki
• Quick Start
• Feature Highlights
• System Requirements
• Installation
• Command-Line Arguments
• Usage Examples
• Blocklist & Filtering
• Web & Discord Integration
• Troubleshooting
• Contributors
• Video Demonstration

Documentation Wiki

For comprehensive guides and detailed documentation, visit the information folder!

The wiki contains detailed guides for every feature:

Note: The Linux guide is not fully documented or tested. If you have trouble, please open an issue report on GitHub and I\’ll try to help.

The information folder is your go-to resource for in-depth help with any Synthalingua feature!

Documentation Wiki

For comprehensive guides and detailed documentation, visit the official Synthalingua Wiki:

https://gith**ub*.com/cyberofficial/Synthalingua/wiki

The GitHub Wiki contains detailed guides for every feature, including setup, usage, troubleshooting, and advanced options. Always check the wiki for the latest documentation and tips!

About

Synthalingua is a self-hosted AI tool for real-time audio translation and transcription. It supports multilingual input and output, streaming, microphone, and file modes, and is optimized for both GPU and CPU. The project is in active development and open source.

Legal & Friendly Disclaimer

Synthalingua is a self-hosted AI tool for real-time audio translation and transcription. It supports multilingual input and output, streaming, microphone, and file modes, and is optimized for both GPU and CPU. The project is in active development and open source.

Legal & Friendly Disclaimer

Hey there! Synthalingua is a fun and powerful tool for exploring languages, learning, and enjoying live translations. But just like any tool, it comes with a few important guidelines to keep things safe, legal, and friendly for everyone.

• Synthalingua is a tool, not a service. You run it on your own computer, and you are in control. It’s not a replacement for professional translators or interpreters.
• For fun, learning, and curiosity! Use Synthalingua to practice languages, understand foreign content, or experiment with AI audio. It’s great for hobbyists, students, and anyone curious about language tech.
• Not for official or critical use. Please don’t rely on Synthalingua for legal, medical, business, or other important communications. For anything serious, always consult a qualified human expert.
• Be kind and ethical. Don’t use Synthalingua to spread misinformation, harass others, or break the law. Respect the rules of any platform you use it with.
• Respect privacy and copyright. Only process audio or video you have the right to use. Don’t share or transcribe private conversations without permission.
• No warranty or liability. I built Synthalingua for the community, but I can’t take responsibility for how it’s used. You use it at your own risk.

? Examples of Good Use

• Translating a livestream for your own understanding
• Practicing a new language by listening to foreign media
• Making fun subtitles for a YouTube video you have rights to
• Learning how AI models handle different accents or languages

Please Don’t

• Use Synthalingua to translate confidential work meetings or private calls without consent
• Rely on it for medical, legal, or business decisions
• Use it to bypass paywalls, copyright, or platform rules
• Share or publish AI-generated translations as if they are 100% accurate or official

By using Synthalingua, you agree to use it responsibly and accept full responsibility for your actions. Let’s keep it fun, safe, and positive for everyone!

• Real-time translation & transcription (stream, mic, file)
• Multilingual: Translate between dozens of languages
• Streaming & captions: HLS, YouTube, Twitch, and more
• Blocklist & repetition suppression: Auto-filter repeated or unwanted phrases
• Discord & web integration: Send results to Discord or view in browser
• Portable GUI version available

Quick Start

1. Install Python 3.12 from here and Git
2. Install FFMPEG (guide)
3. (Optional) Install CUDA 12.8 for GPU acceleration if you plan to use GPU features
4. Run setup.bat (Windows) or setup.bash (Linux)
5. Edit and run the generated batch/bash file, or use the GUI portable version:

Feature Highlights

• Suppress repeated/similar messages: Prevents spam from repeated or hallucinated phrases in all modes
• Auto blocklist: Frequently blocked phrases are auto-added to your blocklist file
• Flexible input: Microphone, HLS stream, or file
• Captions mode: Generate subtitles for any audio/video file
• Web server: View live subtitles in your browser
• Discord integration: Send results to a Discord channel
• Customizable: Blocklist, language, device, RAM, and more

System Requirements

Supported GPUs:

• Nvidia: Most CUDA-capable cards (Kepler, Maxwell, Pascal, Turing, Ampere, Ada; e.g. GTX 750 Ti, 1050 Ti, 1660, RTX 2060/2070/2080/3060/3070/3080/3090/40xx, A6000, etc.)
• AMD: ROCm-compatible cards (Linux only, experimental)
• CPU: Supported (slower, but works for small models)

Notes:

• Nvidia GPU with CUDA is strongly recommended for best performance.
• AMD GPU support is experimental and Linux-only (see ROCm docs).
• CPU-only mode is available for testing or low-resource systems.
• Microphone is optional (use --stream for HLS input).
• For a full list of supported Nvidia GPUs, see the Official Nvidia List or Simple Nvidia List.

Installation

1. Install Python 3.12
2. Install Git
3. Install FFMPEG (guide)
4. (Optional) Install CUDA 12.8 for GPU if you plan to use GPU features
5. Run setup.bat (Windows) or setup.bash (Linux)
6. Edit and run the generated batch/bash file, or use the GUI

Command-Line Arguments

General

Model & Device

Input & Microphone

Streaming & File

Language & Translation

Output, Captions, and Filtering

Print SRT to Console

The --print_srt_to_console flag prints the final, fully combined SRT subtitles to the console after the SRT file is created (captions mode only). This is useful for quickly viewing the generated subtitles without opening the SRT file manually. It only prints the final combined SRT (not per-segment SRTs) and works with --makecaptions.

Example:

python transcribe_audio.py --makecaptions --file_input=\"C:/path/video.mp4\" --file_output=\"C:/output\" --file_output_name=\"MyCaptionsFile\" --print_srt_to_console

This will save the SRT file as usual and also print its contents to the console at the end of processing.

Web & Discord

Usage Examples

• Stream translation:

  python transcribe_audio.py --ram 11gb-v3 --stream_translate --stream_language Japanese --stream https://www.**twi*tch.tv/somestreamerhere

• Microphone translation:

  python transcribe_audio.py --ram 6gb --translate --language ja --discord_webhook \"https://di**scor*d.com/api/webhooks/1234567890/1234567890\" --energy_threshold 300

• Captions mode:

  python transcribe_audio.py --ram 11gb-v3 --makecaptions --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With word-level timestamps (may be slower):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --word_timestamps --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With vocal isolation (requires demucs):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --isolate_vocals --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With vocal isolation using specific model (skip interactive prompt):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --isolate_vocals --demucs_model htdemucs_ft --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With silence detection (faster processing for long silent periods):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --silent_detect --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With custom silence threshold for quiet speech (e.g., whispers):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --silent_detect --silent_threshold -45.0 --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With custom duration to ignore brief pauses (e.g., 2s minimum):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --silent_detect --silent_duration 2.0 --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # RECOMMENDED: Vocal isolation + silence detection (maximum efficiency and quality):
  python transcribe_audio.py --ram 11gb-v3 --makecaptions --isolate_vocals --silent_detect --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda

• Captions compare mode (all models):

  python transcribe_audio.py --makecaptions compare --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With word-level timestamps (may be slower):
  python transcribe_audio.py --makecaptions compare --word_timestamps --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # With vocal isolation (requires demucs):
  python transcribe_audio.py --makecaptions compare --isolate_vocals --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda
  # RECOMMENDED: Vocal isolation + silence detection (maximum efficiency and quality):
  python transcribe_audio.py --makecaptions compare --isolate_vocals --silent_detect --file_input=\"C:\\Users\\username\\Downloads\\file.mp4\" --file_output=\"C:\\Users\\username\\Downloads\" --file_output_name=\"outputname\" --language Japanese --device cuda

• Set microphone by name or index:

  python transcribe_audio.py --set_microphone \"Microphone (Realtek USB2.0 Audi)\"
  python transcribe_audio.py --set_microphone 4

Caption Generation Optimization

Advanced Caption Features

For the best caption generation experience, Synthalingua offers several advanced features that can be combined:

Silent Detection (--silent_detect)

• What it does: Intelligently skips silent regions in audio files
• Benefits: Faster processing, reduced resource usage, better transcription quality
• Best for: Podcasts, lectures, videos with long pauses or intro/outro music
• Usage: Only works with --makecaptions (caption generation mode)
• ️ Not supported: HLS/streaming modes or microphone input
• ? Customizable settings:
  • Threshold (--silent_threshold): Controls volume sensitivity
    • Default: -35.0dB (good for normal speech)
    • Quiet speech/whispers: -45.0dB or lower (more sensitive)
    • Loud speech only: -25.0dB or higher (less sensitive)
  • Duration (--silent_duration): Controls minimum silence length
    • Default: 0.5s (brief pauses treated as speech)
    • Ignore short pauses: 2.0s+ (only long silences count)
    • Detect quick breaks: 0.1s (very sensitive to gaps)

? Vocal Isolation (--isolate_vocals)

• What it does: Separates vocals from background music/noise using AI (requires demucs)
• Benefits: Cleaner transcription, better accuracy in noisy environments
• Best for: Music videos, podcasts with background music, noisy recordings
• Requires: pip install demucs or included in some distributions

⚡ Maximum Efficiency Combo (RECOMMENDED)

Combine both features for optimal results:

python transcribe_audio.py --makecaptions --isolate_vocals --silent_detect --file_input=\"your_file.mp4\"

Why this combination works so well:

1. Vocal isolation removes background noise/music, creating cleaner audio
2. Silent detection analyzes the cleaned audio for more accurate silence detection
3. Result: Only speech regions from isolated vocals are processed
4. Benefits:
   • ⚡ Fastest processing (skips silence and noise)
   • Highest accuracy (clean vocal-only audio)
   • ? Resource efficient (processes less audio overall)
   • ? Natural boundaries (respects speech patterns, no mid-word cuts)

Processing Workflow

1. Input: Original audio/video file
2. Vocal Isolation: Extracts clean vocals (if --isolate_vocals)
3. Silence Detection: Finds speech regions in vocal track (if --silent_detect)
4. Transcription: Processes only speech regions with natural boundaries
5. Output: High-quality SRT captions with perfect timestamps

Pro Tips

• Use --makecaptions compare with both flags to test all models efficiently
• Silent detection works on any audio, but is most effective with vocal isolation
• Both features maintain perfect timestamp accuracy in final SRT files
• Vocal isolation creates temporary files in temp/audio/ directory

Blocklist & Filtering

• Use --ignorelist to load a blocklist file (one word/phrase per line)
• Suppression: With --condition_on_previous_text, repeated or highly similar messages are automatically suppressed in all modes
• Auto blocklist: With --auto_blocklist and --ignorelist, phrases blocked 3+ times in the last 10 are auto-added to your blocklist
• Debug: Use --debug to print info about blocked/suppressed messages
• Blocklist and suppression logic applies to all output (console, Discord, web)

Web & Discord Integration

• Discord: Use --discord_webhook to send results to Discord (long messages are split, rate limits handled)
• Web server: Use --portnumber to launch a local Flask server and view subtitles in your browser

Accessing the Web Server

Once you\’ve launched Synthalingua with the --portnumber parameter:

1. Open your web browser and navigate to http://localhost:[PORT] (replace [PORT] with the port number you specified)
2. Example: http://**localho*st:8080 if you used --portnumber 8080
3. For accessing from other devices on your network, use your computer\’s IP address: http://[Y*OUR_I*P*]:[PORT]

Example usage:

# Start Synthalingua with a web server on port 8080
python transcribe_audio.py --ram 6gb --translate --language ja --portnumber 8080

# For remote HLS password protection
python transcribe_audio.py --ram 6gb --translate --portnumber 8080 --remote_hls_password_id \"user\" --remote_hls_password \"yourpassword\"

Troubleshooting

• Python not recognized: Add Python to PATH, restart, check version (should be 3.12)
• No module named \’transformers\’: Run pip install transformers in the correct Python environment
• Git not recognized: Add Git to PATH, restart
• CUDA not available: Install CUDA 12.8 (Nvidia only), or use CPU mode
• Audio source errors: Make sure a microphone or stream is set up
• For detailed troubleshooting: See the Troubleshooting Guide in the information folder
• Other issues: See GitHub Issues

Contributors

• @DaniruKun – https://wa*tso*nind*ustries.live
• @Expletive – https://evitelpxe.neo*ci*t*ies.org
• @Adenser

Video Demonstration

Command line arguments used: --ram 6gb --record_timeout 2 --language ja --energy_threshold 500

Command line arguments used: --ram 11gb-v2 --record_timeout 5 --language id --energy_threshold 500

Credits

Developed proudly in PyCharm IDE from JetBrains

JetBrains kindly provided an OSS license for this project, greatly improving development productivity. Learn more

More

• Wiki
• Official Nvidia List
• Simple Nvidia List
• FFMPEG Install Guide

Word-level Timestamps

The --word_timestamps flag enables word-level timestamps in subtitle output (sub_gen/captions mode only). This provides more precise alignment but may make subtitle generation a bit slower as it requires more processing power. If you notice any unusual slowdowns, try removing the flag next time you run this command.

Note: This flag has no effect in microphone or HLS/stream modes, and will show a warning if used there.

Vocal Isolation

The --isolate_vocals flag attempts to isolate vocals from the input audio before generating subtitles (sub_gen/captions mode only). This can improve subtitle accuracy for music or noisy audio, but may take additional time and requires the demucs package. If demucs is not installed, a warning will be shown.

Demucs Model Selection:

• By default, the program will prompt you to select which Demucs model to use for vocal isolation
• You can skip the prompt by using --demucs_model to specify the model directly
• Available models:
  • htdemucs (default): Latest Hybrid Transformer model
  • htdemucs_ft: Fine-tuned version for better quality (slower)
  • htdemucs_6s: 6-source separation (includes piano/guitar)
  • hdemucs_mmi: Hybrid v3 trained on expanded dataset
  • mdx: Frequency-domain model, MDX winner
  • mdx_extra: Enhanced MDX with extra training data
  • mdx_q: Quantized MDX (faster, smaller)
  • mdx_extra_q: Quantized MDX Extra (faster, smaller)
  • hdemucs: Original Hybrid Demucs v3
  • demucs: Original time-domain Demucs

Examples:

# Interactive model selection (default behavior)
python transcribe_audio.py --makecaptions --isolate_vocals --file_input=\"video.mp4\"

# Specify model directly (no prompt)
python transcribe_audio.py --makecaptions --isolate_vocals --demucs_model htdemucs_ft --file_input=\"video.mp4\"

# For best quality (slower)
python transcribe_audio.py --makecaptions --isolate_vocals --demucs_model htdemucs_ft --file_input=\"video.mp4\"

# For fastest processing
python transcribe_audio.py --makecaptions --isolate_vocals --demucs_model mdx_q --file_input=\"video.mp4\"

Note: This flag has no effect in microphone or HLS/stream modes.