wenet-e2e/wenet

WeNet

Roadmap | Docs | Papers | Runtime | Pretrained Models | HuggingFace | Ask WeNet Guru

We share Net together.

Highlights

• Production first and production ready: The core design principle, WeNet provides full stack production solutions for speech recognition.
• Accurate: WeNet achieves SOTA results on a lot of public speech datasets.
• Light weight: WeNet is easy to install, easy to use, well designed, and well documented.

Install

Install python package

``` sh pip install git+https://github.com/wenet-e2e/wenet.git

This pulls in torch>=1.13.0 with no upper bound, so pip may install a torch
build newer than your installed GPU driver supports. If you plan to use a GPU,
verify it after install:

sh python -c "import torch; print(torch.cuda.is_available())"

If this prints False on a machine with a GPU, torch silently falls back to
CPU with no error. Reinstall a torch build matching your driver's CUDA
version, e.g.:

sh pip install torch==2.4.0+cu121 torchaudio==2.4.0+cu121 --index-url https://download.pytorch.org/whl/cu121 --force-reinstall

Command-line usage (use -h for parameters):

sh wenet -m paraformer audio.wav

You can set -m with paraformer or firered or wenetspeech for chinese,
and set it to whisper-large-v3 or whisper-large-v3-turbo for english.

Note: this runs on CPU by default (--device defaults to cpu), even if
a GPU is available. Pass --device cuda explicitly to use the GPU (Python API:
wenet.load_model(model_name, device='cuda')). See
python usage for GPU/Turing (e.g.
Tesla T4) specific notes on dtype and attention-backend options.
Python programming usage:

python import wenet

model = wenet.load_model('paraformer') result = model.transcribe('audio.wav') print(result.text)

Please refer python usage for more command line and python programming usage.

Install for training & deployment
Clone the repo

sh git clone https://github.com/wenet-e2e/wenet.git

Install Conda: please see https://docs.conda.io/en/latest/miniconda.html
Create Conda env:

sh conda create -n wenet python=3.10 conda activate wenet conda install conda-forge::sox

Install CUDA: please follow this link, It's recommended to install CUDA 12.1
Install torch and torchaudio, It's recomended to use 2.2.2+cu121:

sh pip install torch==2.2.2+cu121 torchaudio==2.2.2+cu121 -f https://download.pytorch.org/whl/torch_stable.html

For Ascend NPU users:
Install CANN: please follow this link to install CANN toolkit and kernels.
Install WeNet with torch-npu dependencies:

sh pip install -e .[torch-npu]

Related version control table:

| Requirement  |      Minimum     | Recommend   |
| ------------ | ---------------- | ----------- |
| CANN         | 8.0.RC2.alpha003 | latest      |
| torch        | 2.1.0            | 2.2.0       |
| torch-npu    | 2.1.0            | 2.2.0       |
| torchaudio   | 2.1.0            | 2.2.0       |
| deepspeed    | 0.13.2           | latest      |

Install other python packages

sh pip install -r requirements.txt pre-commit install # for clean and tidy code

Frequently Asked Questions (FAQs)

sh

If you encounter sox compatibility issues

RuntimeError: set_buffer_size requires sox extension which is not available.

ubuntu

sudo apt-get install sox libsox-dev

centos

sudo yum install sox sox-devel

conda env

conda install conda-forge::sox

Build for deployment

Optionally, if you want to use x86 runtime or language model(LM),
you have to build the runtime as follows. Otherwise, you can just ignore this step.

sh

runtime build requires cmake 3.14 or above

cd runtime/libtorch mkdir build && cd build && cmake -DGRAPH_TOOLS=ON .. && cmake --build .

Please see doc for building
runtime on more platforms and OS.

Discussion & Communication
You can directly discuss on Github Issues.
For Chinese users, you can also scan the QR code on the left to follow our official account of WeNet.
We created a WeChat group for better discussion and quicker response.
Please scan the personal QR code on the right, and the guy is responsible for inviting you to the chat group.
|  ||  |
| ---- | ---- | ---- |
Acknowledge
1. We borrowed a lot of code from ESPnet for transformer based modeling.
2. We borrowed a lot of code from Kaldi for WFST based decoding for LM integration.
3. We referred EESEN for building TLG based graph for LM integration.
4. We referred to OpenTransformer for python batch inference of e2e models.
Citations

bibtex @inproceedings{yao2021wenet, title={WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit}, author={Yao, Zhuoyuan and Wu, Di and Wang, Xiong and Zhang, Binbin and Yu, Fan and Yang, Chao and Peng, Zhendong and Chen, Xiaoyu and Xie, Lei and Lei, Xin}, booktitle={Proc. Interspeech}, year={2021}, address={Brno, Czech Republic }, organization={IEEE} }

@article{zhang2022wenet, title={WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit}, author={Zhang, Binbin and Wu, Di and Peng, Zhendong and Song, Xingchen and Yao, Zhuoyuan and Lv, Hang and Xie, Lei and Yang, Chao and Pan, Fuping and Niu, Jianwei}, journal={arXiv preprint arXiv:2203.15455}, year={2022} } ```