KFaligner - Korean Forced Aligner

KFaligner is an HTK (Hidden Markov Model Toolkit) based Korean forced alignment system. It takes audio files and transcription text as input and outputs accurate time alignment information at the phoneme and word level in Praat TextGrid format.

Documentation updated: 2026-10-25. The deployed web application and GPU transcription environment have been verified with Python 3.12.

Item	KFaligner configuration
Website	kfaligner.tyoon.net
Whisper transcription	Open transcriber · medium, Korean, CUDA preferred
Existing WAV/TextGrid review	Open review
Alignment tiers	phone, word, syllable, utterance
Production service	kfaligner.service · Gunicorn 127.0.0.1:5011 behind Nginx
Python environment	/var/www/html/kfaligner/.venv

Log in before uploading or reviewing files. GPU acceleration applies to Whisper transcription; forced alignment uses HTK. The English counterpart is EFaligner.

Web workflow at a glance

1. Already have a transcript? Upload matching WAV + TXT/LAB files on the home page. Each aligned audio file gets its own waveform and TextGrid review card.
2. Need a transcript? Upload WAV files at /transcribe, edit the current transcript, then click Run alignment, to the left of 💾 수동 전사 저장. The latest editor text is saved and used immediately; a separate save is not required.
3. Need to check existing results? Upload matching WAV + TextGrid pairs at /review. Five pairs produce five review cards.
4. Need to fix only a region? Select it in the review card, correct the transcript for that region, and click 선택 구간만 재정렬.
5. Export results: download a TextGrid individually or a ZIP containing matching WAV, TextGrid and TXT/LAB files together, without subfolders.

📋 Table of Contents

• Key Features
• Usage
• System Requirements
• Installation
• Project Structure
• Web Application
• Technical Details
• Troubleshooting
• License

🎯 Key Features

• Automatic Phoneme Alignment: Precise alignment of Korean speech and text at the phoneme level
• Acoustic Models: This checkout includes a 16kHz model; 8kHz/11.025kHz selection requires the corresponding model directories
• Automatic Resampling: SoX converts unsupported sampling rates; 16kHz mono WAV is recommended
• TextGrid Output: Standard format ready for immediate use in Praat
• Automatic Dictionary Expansion: Automatic generation of pronunciation dictionary for unregistered words
• Web Interface: Flask web application with Whisper GPU transcription, direct transcript-to-alignment, multi-file WaveSurfer review and partial realignment
• Short Pause (sp) Handling: Separates pauses between words into distinct intervals
• Multi-Tier TextGrid: Generates phone/word/syllable/utterance tiers with Korean label support

📖 Usage

Basic Usage (Web Interface)

https://kfaligner.tyoon.net

Basic Usage (Command Line)

python3 align.py <audio.wav> <transcript.lab> <output.TextGrid>

Example:

python3 align.py test/mv01_t01_s01.wav test/mv01_t01_s01.lab test/mv01_t01_s01.TextGrid

Input File Formats

1. Audio File (.wav)

• Sample Rate: 16000 Hz (recommended)
  • 8000 Hz and 11025 Hz require matching acoustic models; this checkout includes model/16000/
  • Other sample rates automatically resampled to 16kHz
• Channels: Mono
• Bit Depth: 16-bit PCM

2. Transcription File (.txt or .lab)

• Encoding: UTF-8
• Format: Korean orthographic text (one line)
• Example: 기차도 전기도 없었다

Output File Format

TextGrid File (Praat compatible):

• phone tier: Phoneme-level alignment (gg, i, c, a, d, o, ...)
• syllable tier: Automatic syllable boundaries based on Korean syllable rules (CV(C))
• word tier: Word-level alignment with original Korean labels (when Korean input provided)
• utterance tier: Segments between sil combined as sentence units with Korean labels
• Boundary Markers: sil (silence), sp (short pause)

💻 System Requirements

Essential Requirements

• Python: 3.12 (verified web application and Whisper environment)
• HTK: 3.4.1 (Hidden Markov Model Toolkit)
• SoX: Audio conversion for HTK
• FFmpeg: Audio decoding for Whisper
• Praat: Desktop review is optional; the webapp parses TextGrids through praat-parselmouth
• OS: Linux (Ubuntu 20.04+ recommended)

Python Packages

Install the complete web dependencies from requirements.txt, including Flask extensions and praat-parselmouth. GPU transcription additionally uses requirements-whisper.txt, which pins Whisper and CUDA 12.6 PyTorch. requirements-lock.txt records the web runtime versions; install the Whisper requirements separately when reproducing this server.

🚀 Installation

1. Clone Repository

git clone https://github.com/exphon/kfaligner.git
cd kfaligner

2. Install HTK

HTK requires direct download due to license restrictions:

1. Download HTK-3.4.1 from HTK official website
2. Extract and compile:

tar -xvf HTK-3.4.1.tar.gz
cd htk
./configure --prefix=/usr/local
make all
sudo make install

3. Setup Python Environment

Run these commands from the repository root using Python 3.12:

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
# Needed for Whisper transcription on this CUDA server:
.venv/bin/python -m pip install -r requirements-whisper.txt

Put HCopy, HVite, SoX and FFmpeg on the runtime PATH. The production service already defines their locations. Use .venv/bin/python for command-line alignment unless the virtual environment is activated.

Resampling (Optional)

For non-16kHz audio files, convert using Praat script:

# Run Praat script
praat --run resampleTo16000.praat input.wav output_16k.wav

Adding Unknown Words to Dictionary

# Prepare sentence list file (flist.txt)
echo "새로운 문장들" > flist.txt

# Generate and add dictionary
./make_dict.sh flist.txt

This script uses tools in the bin/ directory to:

1. Convert Korean to Unicode
2. Generate phoneme sequences
3. Merge with existing dictionary

📁 Project Structure

kfaligner/
├── align.py                    # Main alignment script
├── __init__.py                 # Python package initialization
├── restart.sh                  # Server management script
├── make_dict.sh               # Dictionary generation script
├── resampleTo16000.praat      # Resampling Praat script
├── check_alignment.praat      # Alignment result verification script
│
├── bin/                        # Utility scripts
│   ├── make_kdict.py          # Korean dictionary generator
│   ├── add_dict.py            # Dictionary addition tool
│   ├── kdictmap.py            # Phoneme mapping
│   ├── han2uniconversion.py   # Korean-Unicode conversion
│   ├── convert_sentences_unicode.py
│   ├── dict                   # Pronunciation dictionary
│   ├── kdict0.txt             # Base Korean dictionary
│   └── kdict1.txt             # Extended dictionary
│
├── model/                      # HTK acoustic models
│   ├── dict                   # Main pronunciation dictionary (5,589+ words)
│   ├── monophones             # Phoneme list
│   └── 16000/                 # 16kHz models
│       ├── hmmdefs            # HMM definitions
│       ├── macros             # Macro definitions
│       └── config             # HTK configuration
│
├── test/                       # Test files
│   ├── *.wav                  # Sample audio files
│   ├── *.lab                  # Sample transcription files
│   └── *.TextGrid             # Output examples
│
├── webapp/                     # Flask web application
│   ├── app.py                 # Main application
│   ├── forms.py               # Web form definitions
│   ├── models.py              # Database models
│   ├── templates/             # HTML templates
│   │   ├── index.html
│   │   ├── login.html
│   │   ├── signup.html
│   │   └── results.html
│   └── data/                  # Upload and job data
│       ├── users.db           # User database
│       ├── uploads/           # Uploaded files
│       └── jobs/              # Processing job directory
│
├── tmp/                        # Temporary processing files
│   ├── tmp.mlf                # Input MLF
│   ├── aligned.mlf            # HVite output
│   └── *.mfc                  # MFCC feature files
│
└── examples/                   # Example files
    └── code.scp               # File list example

🌐 Web Application

Production deployment: kfaligner.tyoon.net

The production setup uses Nginx for HTTPS and a systemd-managed Gunicorn process on 127.0.0.1:5011. The configuration in deploy/ targets this server's /var/www/html/kfaligner checkout and the tyoon service account.

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements-lock.txt
.venv/bin/python -m pip install -r requirements-whisper.txt
sudo bash /var/www/html/kfaligner/deploy/install.sh

Before installation, .env must contain a private, randomly generated FLASK_SECRET_KEY (not the development default). Keep this file readable only by its owner (chmod 600 .env). Existing user and job data stay in webapp/data/. HTK must be installed in /home/tyoon/htk_install/bin, and sox, Nginx and Certbot must be available. DNS must point kfaligner.tyoon.net to this server, with inbound ports 80 and 443 reachable. The installer reuses the server's existing Let's Encrypt account.

The installer backs up configuration under /var/backups/kfaligner.*, blocks direct repository downloads through tyoon.net/kfaligner/, obtains a dedicated certificate, enables HTTP-to-HTTPS redirects, and enables the application and certificate renewal timer. Nginx reloads after successful certificate renewal. If certificate issuance fails, correct the reported cause and rerun the script.

Open https://kfaligner.tyoon.net after successful installation.

sudo systemctl status kfaligner
sudo systemctl reload kfaligner  # Reload code and .env settings
sudo systemctl restart kfaligner
sudo journalctl -u kfaligner -n 100 --no-pager
sudo certbot renew --cert-name kfaligner.tyoon.net --dry-run

restart.sh delegates to systemd once the service is installed. Gunicorn uses one synchronous worker because the existing aligner updates shared dictionary files. The Nginx upload ceiling is 128 MB; .env may set a lower application limit with MAX_CONTENT_LENGTH_MB. HTTPS sessions use secure cookies and trust the single local Nginx proxy.

Automatic transcription uses Whisper medium with Korean language selection. The server's .env selects WHISPER_MODEL_NAME=medium and WHISPER_DEVICE=cuda. CUDA 12.6 PyTorch runs the model on the first available GPU (currently an RTX 3090). The launch configurations reuse FFmpeg already installed at /home/tyoon/miniconda3/envs/espnet/bin/ffmpeg. Install the transcription runtime into the same Python environment as the webapp:

.venv/bin/python -m pip install -r requirements-whisper.txt
.venv/bin/python -c "import whisper; whisper.load_model('medium', device='cuda', download_root='.cache/whisper')"

The multilingual model is cached in .cache/whisper/medium.pt, shared by manual preloading and the production service. WHISPER_DOWNLOAD_ROOT can override the cache directory. GPU inference uses FP16, with CPU/FP32 fallback if CUDA cannot load the model. The first request after a worker restart loads the cached model into memory; later requests reuse it. Reload the service after changing Python packages or application code. Each new worker also rereads .env, so graceful reloads apply its settings. New transcription job manifests record the actual device in their device field. SMTP settings in .env are needed for password recovery emails.

Starting the Server Locally

# Development server (PORT from .env, default 5001; run from the repository root)
.venv/bin/python -m webapp.app

# Production server (Gunicorn + Systemd)
sudo systemctl start kfaligner
sudo systemctl enable kfaligner

# Using server management script
./restart.sh start|stop|restart|reload|status

Accessing Web Interface

https://kfaligner.tyoon.net

Whisper transcription to Korean alignment

Edit the current Whisper transcript and click Run alignment, to the left of 💾 수동 전사 저장. The server saves the submitted editor text, aligns that WAV, and opens its waveform/TextGrid review. Only the current file is aligned; other files and the original automatic transcription are retained.

• 💾 수동 전사 저장 saves text without running alignment.
• 전사 수정 화면으로 돌아가기 returns from the alignment result to the same transcription file/page, so you can continue through a multi-file job.
• If alignment fails, the corrected text remains saved for another attempt.
• The result ZIP contains the WAV, the exact TXT used for that alignment, and its TextGrid at the archive root. Transcription WAV/TXT downloads also retain their source files for subsequent editing and alignment.
• Refresh previously opened transcription pages after deploying a UI update.

Waveform and TextGrid review

After forced alignment, the result page displays one WaveSurfer player and all TextGrid tiers for every uploaded WAV pair. Existing WAV/TextGrid pairs can also be uploaded together at /review (or 파형·TextGrid 검토 on the home page). Files pair by basename; duplicate names and missing partners are reported. All pairs appear on one page, including an error card for an invalid pair.

Click an interval to play it, inspect its label and exact start/end times, change playback speed, or zoom and scroll the waveform and tiers together. Starting a player pauses other files. Long/short Praat formats, Unicode labels and point tiers are parsed by Praat.

To repair an alignment, enable 구간 선택 모드 and drag over the waveform, or click an existing TextGrid interval. Adjust the region handles or start/end fields, enter the transcript for that region, then click 선택 구간만 재정렬. The server crops only that audio region into a temporary mono 16 kHz WAV and runs HTK on the corrected transcript. It offsets the resulting phone, syllable, word and utterance intervals back into the original time axis. Original WAVs, annotations outside the selection and custom/point tiers are preserved. Intervals crossing a selected boundary are split there, retaining their original labels on the outside pieces. Select whole words with a little surrounding silence. A selection must be at least 0.1 seconds and lie within both the WAV and TextGrid; the partial-alignment process has a 180-second timeout.

Saving verifies that outside timestamps and labels are unchanged. Failed or outdated requests do not overwrite the current TextGrid. Successful edits are included in individual and flat ZIP downloads. 직전 수정 취소 restores the previous file byte-for-byte (one undo level), while 최초 TextGrid 다운로드 retrieves the file as it was before the first edit. Backups and correction text are stored privately under each job's .edits/ directory, outside ZIP exports. Original uploaded TXT/LAB files are not rewritten by partial TextGrid edits; exported TextGrids contain the latest saved alignment.

Review URLs can be reopened by the account that created the job. WAV requests support HTTP ranges. ZIP downloads now retain files so subsequent review and individual downloads continue working; there is no automatic retention expiry. ZIP archives place WAV, TextGrid and original TXT/LAB files together at the archive root, without results/ or uploads/ subfolders. Archives are rebuilt on download so existing jobs also receive this layout. WaveSurfer 7.12.12 and its BSD license are served from webapp/static/vendor/ without a third-party CDN dependency.

.venv/bin/python -m unittest discover -s tests -v

Other Features

• File Upload: Upload matching WAV + TXT/LAB pairs
• Batch Processing: Process multiple pairs in one request, with a card for each pair
• Result Download: Download TextGrid files
• User Management: Signup/login system
• Reopen Results: Save the review page URL to return to that job; no separate history list is provided

Environment Variable Configuration

Edit the project .env and reload kfaligner.service. Do not overwrite an existing secret key or commit .env. The application rereads this file when a worker starts. App defaults and production-specific settings are distinct:

Setting	Purpose
FLASK_SECRET_KEY	Private, randomly generated session key
WHISPER_MODEL_NAME=medium	Multilingual Whisper model used with Korean language selection
WHISPER_DEVICE=cuda	Prefer GPU; the app can fall back to CPU
MAX_CONTENT_LENGTH_MB	Combined request/upload size; app default 64 MB
MAX_FILES_PER_REQUEST	Total files per alignment/review upload; app default 500, so 250 pairs
TRANSCRIBE_MAX_FILES	WAV files per Whisper request; app default 20
ENABLE_CLAMAV_SCAN	Enable installed antivirus scanning on supported upload routes

The effective upload size is the lower of the application limit and Nginx's client_max_body_size (128 MB in the prepared KFaligner Nginx configuration). Whisper and HTK are separate stages; enabling CUDA does not make HTK GPU-based.

Relevant implementation and verification files

• webapp/app.py: transcription, manual-save/alignment actions and downloads.
• webapp/review.py: authenticated multi-file TextGrid review.
• webapp/partial_alignment.py: region merging, preservation checks and undo.
• webapp/static/js/review.js: WaveSurfer review and region selection.
• webapp/static/js/transcribe-actions.js: alignment button progress state.
• deploy/install.sh: HTTPS and systemd setup for this server.
• tests: parser, outside-region preservation, undo and transcript-to-alignment regressions.

.venv/bin/python -m unittest discover -s tests -p 'test_*.py' -v

As of the update above, 19 automated tests and an HTTPS browser workflow with real Korean Whisper transcription and HTK alignment passed. Browser checks also covered current-file selection, unsaved editor text, ZIP contents and mobile button placement. Runtime files stay under webapp/data/; this command's unit tests use temporary test directories.

🔧 Technical Details

HTK-Based Forced Alignment

KFaligner performs Viterbi alignment using HVite:

HVite -T 1 -a -m \
      -I tmp/tmp.mlf \
      -H model/16000/macros \
      -H model/16000/hmmdefs \
      -S tmp/test.scp \
      model/dict \
      model/monophones \
      > tmp/aligned.mlf

Key Options:

• -T 1: Trace level (debugging)
• -a: Output phoneme boundaries
• -m: Use models
• -I: Input MLF (Master Label File)
• -H: HMM definition file

TextGrid Multi-Tier Generation

• After separating word-ending sp in readAlignedMLF(), writeTextGrid() generates phone, syllable, word, and utterance tiers.
• _build_syllable_intervals() estimates Korean syllable boundaries based on romanized phoneme sequences.
• _build_utterance_intervals() groups intervals between sil as single utterances, using the original text converted from Korean transcription as labels.
• When Korean input is provided, _build_display_map() creates romanized/Korean mapping to output Korean labels in word and utterance tiers.

Short Pause (sp) Handling

Problem: HTK's sp is a tee-model (1-state) sharing the middle state of sil.

Solution (2025-10-09 update):

1. Dictionary: All words end with sp (5,589 words)
2. MLF: No sp insertion, only sil at sentence boundaries
3. Post-processing: readAlignedMLF() separates word-ending sp as distinct entries

# align.py Line 196-205
for wrd in ret:
    if len(wrd) > 1 and wrd[-1][0] == 'sp':
        sp_entry = wrd.pop()  # Separate sp from word
        separated_ret.append(wrd)  # Word only
        separated_ret.append(['sp', sp_entry])  # sp separately

Result: Pauses between words appear as separate intervals in TextGrid.

Phoneme System

Romanization-based Korean phonemes:

• Consonants: gg, dd, bb, jj, ss, k, t, p, c, h, g, d, b, j, m, n, ng, r, s
• Vowels: a, eo, o, u, eu, i, ae, e, wa, weo, wo, we, oe, wi, ui, ya, yeo, yo, yu, yae, ye
• Special: sil (silence), sp (short pause)

Supported Sample Rates

Sample Rate	Model Directory	Auto Convert
8000 Hz	model/8000/ (not included)	Resample to 16000 Hz before alignment
11025 Hz	model/11025/ (not included)	Resample to 16000 Hz before alignment
16000 Hz	model/16000/	-
Other	-	→ 16000 Hz

🐛 Troubleshooting

1. "dur<=0" Error

Symptom: HVite outputs ERROR [+8522] LatFromPaths: Align have dur<=0

Cause: sp in both MLF and dictionary causing tee-model conflict

Solution:

• Set between_token = None (no sp insertion in MLF)
• Keep sp only in dictionary

2. Duplicate sil Tokens

Symptom: sil sil appears at sentence start/end

Cause: -b sil option + sil in MLF = duplication

Solution: Remove -b option from HVite

3. Unknown Words in Dictionary

Symptom: Alignment fails for specific words

Solution:

# Automatic dictionary generation
./make_dict.sh flist.txt

# Or manual addition
python3 bin/make_kdict.py

4. Sample Rate Mismatch

Symptom: Degraded alignment quality or failure

Solution:

# Resample using Praat
praat --run resampleTo16000.praat input.wav output.wav

# Or use sox
sox input.wav -r 16000 output.wav

5. Web App Upload Failure

Symptom: "Maximum file size exceeded"

Solution:

# Edit .env, then reload kfaligner.service
MAX_CONTENT_LENGTH_MB=128

# Nginx configuration (when using proxy)
client_max_body_size 128M;

🙏 Acknowledgments

This project is based on the following technologies:

• HTK (Hidden Markov Model Toolkit): Cambridge University Engineering Department
• Praat: Paul Boersma and David Weenink
• Flask: Pallets Projects

👤 Author

Tae-Jin Yoon Sungshin Women's University © January 2016-2025

📧 Contact

For project inquiries or bug reports, please use GitHub Issues: https://github.com/exphon/kfaligner/issues

📄 License

A project-level LICENSE file is not included in this checkout. Third-party components retain their own licenses; the bundled WaveSurfer license is available in webapp/static/vendor/wavesurfer/LICENSE.

---

⭐ If you find this project useful, please give it a Star!