espnet/espnet

• ESPnet 202610.post2 — the command line has one name a task and two more of them: espnet phonemize reads the phones with POWSM, espnet align lines text up with the audio it was said in, and asr and tts became transcribe and synthesize (the old names still work); models published before June 2025 load again, after init: chainer was removed from the toolkit.
• ESPnet 202610.post1 — the command line grows espnet demo (the OWSM browser demo) and --live (the microphone, transcribed as you speak); one Speech2Text now loads either kind of OWSM checkpoint, with best_path() for CTC decoding without a search; espnet/espnet:inference-cpu-latest and -gpu-latest run a published model with nothing installed; three more demo Spaces (TTS, enhancement, speaker verification).
• ESPnet 202610 — one-line inference from the command line (pip install espnet && espnet asr audio.wav), two OWSM v4 demos as Hugging Face Spaces, a core install without the training stack (training is espnet[train]), batched beam search, PyTorch 2.11-2.14.
• ESPnet 202609 — ESPnet3 complete on egs3/librispeech_100 at ESPnet2 parity, CI rebuilt on a prebuilt image (compute per run halved), OpenBEATs pretraining, ten new recipes (ASR, TTS, SER, ST, audio SSL), Python 3.12-3.13.
• ESPnet 202604 — Docker-based CI, PyTorch 2.9.1 support, FastSpeech2 inference ~1.9x faster at batch 8, new recipes (Kinyarwanda, Emilia, kosp2e).
• ESPnet 202511 — parallel-processing primitives, refactored inference and evaluation pipeline, expanded SpeechLM support.
• ESPnet 202509 — Python 3.9-3.13, Debian 12 CI, the LID subsystem completed, multi-optimizer training (HybridOptim / HybridLRS).
• ESPnet 202506 — ESPnet3 groundwork (data organizer, trainer, model), LID training and task setup, codec1 recipes, USES2 speech enhancement, IPAPack++ S2T recipes.
• ESPnet 202503 — PyTorch Lightning trainer support, Hugging Face front-end, scaled dot-product attention, ML-SUPERB 2024 recipe.

pip install "espnet[all]"                       # training plus every task extra (except sds)
pip install git+https://github.com/espnet/espnet  # latest master

• Full setup (recipes, DNN training, Kaldi-style tooling): see the installation guide.
• Docker: see docker/ and the Docker docs.
• Task-specific tools live in tools/installers.
• ESPnet1 is no longer supported — use ESPnet2 (egs2/) or ESPnet3 (egs3/). See the ESPnet1 notice.

|system/pytorch ver.|2.11.0|2.13.0|2.14.0| | :---- | :---: | :---: | :---: | |ubuntu/python3.12/pip|||| |ubuntu/python3.13/pip|||| |debian12/python3.12/conda|||| |windows/python3.12/pip|||| |macos/python3.12/pip|||| |macos/python3.12/conda||||

Each badge is its workflow's aggregate status on master, where the full grid runs. Coverage is not uniform — some suites run on one pytorch only, and a pull request runs less than master does. What each column covers.

docker run --rm -v "$PWD:/data" -v "$HOME/.cache/huggingface:/cache/huggingface" \
    espnet/espnet:inference-cpu-latest asr /data/audio.wav

The second mount keeps the downloaded model between runs. With a GPU, use espnet/espnet:inference-gpu-latest, --gpus all and --device cuda; on a Linux host add --user "$(id -u):$(id -g)". The other images are in docker/.

| Demo | | Last run | | :-- | :-- | :-- | | Spoken dialogue — listen, think, speak | | | | Speech recognition, in any of 151 languages | | | | The words appearing as the audio arrives | | | | Speech translation — the same model, a different task symbol | | | | Text-to-speech, one voice and then 128 | | | | Speech enhancement, and what it did to the signal-to-noise | | | | Speaker verification — two recordings, one score | | | | Neural codecs — a waveform as a few integers a frame | | |

Each runs top to bottom on a CPU and pins the release it was checked against. The second badge is that notebook being executed cell by cell every Sunday, so a red one names the demo that broke rather than leaving you to find out by opening it. More, including the CMU course material: espnet/notebook.

cd egs3/librispeech_100/esp2_asr
train=conf/tuning/training_e_branchformer.yaml   # the config the model was trained with
python run.py --stages pack_model  --training_config $train --inference_config conf/inference.yaml --publication_config conf/publication.yaml  # -> exp/.../model_pack
python run.py --stages pack_demo   --training_config $train --demo_config conf/demo.yaml                # -> demo/
python run.py --stages upload_demo --training_config $train --demo_config conf/demo.yaml                # needs hf auth login

Run the packed app locally with python demo/app.py.

| Setting | ESPnet2 | ESPnet3 | | --- | --- | --- | | batch_bins (sequence batching) | Global budget for the whole distributed job; each sequence batch is split across ranks | Per-GPU budget by default; dataloader.train.batch_distribution: global opts into a global budget and splits each batch across ranks. Validation selects its mode under dataloader.valid. | | best_model_criterion | Each criterion is (phase, metric, mode); the number of models to keep is set separately with keep_nbest_models | Each criterion is (monitor, top_k, mode), so the number of checkpoints to keep is specified per criterion (for example, valid/loss, 3, min). | | seed | Set initially and reset to seed + epoch at the start of each epoch | Set once before training with Lightning's seed_everything(..., workers=True); random streams are not reset to seed + epoch by ESPnet3. | | Early stopping (patience) | patience and early_stopping_criterion are trainer options | Configure Lightning's EarlyStopping callback in trainer.callbacks (for example, its monitor, patience, and mode); no early-stopping callback is added by default. |

Existing ESPnet3 configs keep per-GPU behavior when batch_distribution is omitted. Changing the GPU count in that mode changes the effective global batch budget. Switching modes does not automatically rescale batch_bins or accumulation. Global mode requires shared batches with at least world_size items; unsupported iterator/sharding configurations raise an error.

See the migration guide and [dynamic batching guide](https://masao-someki.github.io/espnet_